Jinyuan Li

About Me

I am a final-year master’s student at Tianjin University, under the mentorship of Prof. Gang Pan. Previously, I worked as a research intern at Baidu and explored multimodal document understanding. I am also fortunate to work with Prof. Jianfei Yu during my research journey.

My enthusiasm lies in exploring Multimodal Learning, Vision and Language. My recent series of work (MNER->GMNER->SMNER) explored how to unleash the potential capabilities of visual-language models in complex multimodal scenarios, how to build harmonious interaction and collaboration between multiple models, and how to construct image-text based knowledge augmentation methods in open-world scenarios. Additionally, I also maintain interest in several visual tasks (e.g., Blind Image Inpainting, Infrared and Visible Image Fusion).

I’m open to collaboration and discussions. Please feel free to contact me and explore possibilities together.

Research Interests

Vision and Language:

• Vision-Language Model Foundation and Application [RiVEG, SMNER]
• Multimodal Information Extraction [PGIM]
• Image Inpainting [AFAN] and Synthesis [DSTFuse]

News🔥

• [Jun. 2024] We release a new study proposing the Segmented Multimodal Named Entity Recognition (SMNER) task and constructing the corresponding Twitter-SMNER dataset. Datasets and Code will be released at here.
• [May. 2024] One paper is accepted to ACL 2024. See you in Bangkok!
• [Apr. 2024] Delighted to be onboard at Baidu as a research intern.
• [Feb. 2024] A new research about Grounded Multimodal Named Entity Recognition (GMNER) and Large Language Models has been released! see here.
• [Oct. 2023] One paper about Multimodal Named Entity Recognition (MNER) is accepted to EMNLP 2023.

Publications

Experience

• [Apr. 2024 - Jun. 2024] Baidu, Research Intern

Services

Conference Reviewer/Program Committee

• Annual Meeting of the Association for Computational Linguistics (ACL)
• Conference on Empirical Methods in Natural Language Processing (EMNLP)
• Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)
• International Conference on Computational Linguistics (COLING)
• ACL Rolling Review (ARR)
• ACM Multimedia (ACM MM)
• IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

Journal Reviewer

• IEEE Transactions on Multimedia (TMM)
• Elsevier Pattern Recognition (PR)

Teaching Assistant

• Advanced Computer Vision (Postgraduate), Tianjin University, Fall 2023

Awards and Honors

• [2022 & 2023] Second-class Academic Scholarship of Tianjin University
• [2021] Merit Student of Taiyuan University of Technology (Top 2%)
• [2020 & 2021] Academic Excellence Scholarship of Taiyuan University of Technology
• [2020] Provincial Second Prize in the National College Student Mathematical Modeling Competition
• [2020] Excellent Academic Progress Student of Taiyuan University of Technology
• [2019] Outstanding Student Cadre of Taiyuan University of Technology

Miscellaneous

When I’m not in research mode, I enjoy swimming (Swim one kilometer freestyle in less than 18 minutes) and violin (Fluent sight-reading skills). They have been with me for nearly twenty years. My favorite virtuoso is Ray Chen. His interpretation of the music always carries a distinct personal touch. I hope I can approach my research in the same way.

In the past, I demonstrated strong road cycling capabilities, maintaining a speed of 36 km/h for two hours. Additionally, I found being an anchor very engaging.