Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

Future Blog Post

less than 1 minute read

Published:

This post will show up by default. To disable scheduling of future posts, edit config.yml and set future: false.

Blog Post number 4

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Blog Post number 3

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Blog Post number 2

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Blog Post number 1

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

portfolio

publications

Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models

Published in ICML, 2024

Recent research shows that using a pre-trained vision-language model like CLIP to align a query image with finer text descriptions from a large language model enhances zero-shot performance. We find that these descriptions align better with local image areas rather than the whole image. To leverage this, we introduce weighted visual-text cross alignment (WCA), which uses localized visual prompting to identify and align these areas with the descriptions, creating a similarity matrix. Our score function, based on weighted similarities, significantly improves zero-shot performance, achieving results comparable to few-shot learning methods.

Recommended citation: Jinhao Li, Haopeng Li, Sarah Erfani, Lei Feng, James Bailey, and Feng Liu. (2024). "Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models." Proceedings of the International Conference on Machine Learning (ICML).
Download Paper

talks

teaching