Jinhao Li

Jinhao Li

PhD Candidate in Machine Learning
School of Computing and Information Systems
The University of Melbourne

💬 About me

I am a Ph.D. candidate at AI research group, in the school of Computing and Information Systems, the University of Melbourne, Australia. My research interests lie in transfer learning of foundation models. I am particularly passionate about developing algorithms that align pre-trained foundation models (such as vision-language models and large language models) to downstream tasks with natural shifts. I am fortunate to be advised by Dr. Feng Liu, A/Prof. Sarah Erfani, and Prof. James Bailey.

📖 Research Interests

  • Alignment
  • Generalization and robustness VLMs
  • Parameter-efficient fine-tuning

📝 Selected Publications

[ICML 2024] Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models
[paper] [code] [poster]
Jinhao Li, Haopeng Li, Sarah M. Erani, Lei Feng, James Bailey, Feng Liu✉️.

Brief Introduction Recent research shows that using a pre-trained vision-language model like CLIP to align a query image with finer text descriptions from a large language model enhances zero-shot performance. We find that these descriptions align better with local image areas rather than the whole image. To leverage this, we introduce weighted visual-text cross alignment (WCA), which uses localized visual prompting to identify and align these areas with the descriptions, creating a similarity matrix. Our score function, based on weighted similarities, significantly improves zero-shot performance, achieving results comparable to few-shot learning methods.

🛠️ Experience

  • RA, School of Computing and Information Systems, University of Melbourne, 2023 - Present
    • Advised by Dr. Feng Liu
    • Working on cross-domain recommendation systems

💻 Services

Reviewer

  • Conference:
    • [2025] ICLR, AISTATS
    • [2024] NeurIPS, AJCAI, PRCV
  • Journal:
    • NEUNET
    • IJMLC

🏫 Teaching

Casual Academic

  • TA, COMP90051 Statistical Machine Learning, 2024 Sem. 2, The University of Melbourne
  • TA, COMP90051 Statistical Machine Learning, 2024 Sem. 1, The University of Melbourne
  • TA, COMP90051 Statistical Machine Learning, 2023 Sem. 2, The University of Melbourne

🎖 Awards

  • 2023.06, Melbourne Research Scholarship
  • 2022.12, Graduate with Distinction.
  • 2022.06, Unimelb Dean’s Honours List.

Contact

Feel free to reach out to me via email or connect with me on linkedin.