Man Luo (罗满)
Ph.D.
About
I am currently an AI Research Scientist at Intel Lab, working on large scale multimodal system pretraining and multimodal retrieval augmented generation (RAG).
My research dives deep into the realms of information retrieval and reading comprehension within natural language processing (NLP) and multimodal domains. The primary goal of my research is to develop advanced models that not only efficiently retrieve and utilize external knowledge for enriched comprehension and reasoning but also demonstrate superior generalization capabilities across unfamiliar tasks and domains. A few areas of my recent interest include:
- Knowledge Retrieval: How can we effectively retrieve and utilize external knowledge to not only enhance comprehension but also mitigate hallucination?
- Generalization: Designing models that can seamlessly adapt and perform across various tasks and domains without explicit training.
- Multimodal Understanding: Delving into the integration of textual, visual, and other modalities to bolster machine comprehension and response capabilities.
- Biomedical/healthcare application and innovation: Evaluate and innovate the LLMs to solve biomedical and healthcare challenges, such as long sequence processing, noisy data mitigation, data imbalance rectification, and enhancing interpretability.
Previously, I was a research fellow at Mayo Clinic, AZ, working closely with Dr. Imon Banerjee and Dr. Bhavik Patel. I earned my doctoral degree in 2023 from Arizona State University under the esteemed supervision of Dr. Chitta Baral. I am also privileged to collaborate with amazing industry researchers from Salesforce, Meta and Google during my internship.
News
2024
- [2024.10]: Our paper EMRank has been accepted to IEEE Big Data for Healthcare!
- [2024.10]: Our paper SA-OcrPaint has been accepted to WACV 2025!
- [2024.8]: Our Survey paper Ret-ICL has been accepted to TMLR!
- [2024.07]: New Paper SK-VQA is out! A synthetic generated VQA dataset improves multimodel content-based generalization capacity!
- [2024.06]: New Book Advances in Multimodal Information Retrieval and Generation is out!
- [2024.03]: New Paper SA-OcrPaint is out! A training-free framework to improve the Diffusion Model ability to generate visual text!
- [2024.03]: New Paper PCA-LLM is a domain specific LLM for Prostate Cancer, we train a GPT-2 type of model from scratch on Prostate Cancer clinical notes with domain specific vocabulary.
- [2024.03]: I am very excited to join Intel AI Lab, CA, as a AI research Scientist.
- [2024.02]: Call for Paper: Multimodal4health Workshop. Will be held at ICHI Conference in Orlando, Florida in June 3rd 2024.
- [2024.01]: Ret-ICL survey paper is out: In-context Learning with Retrieved Demonstrations for Language Models: A Survey.
2023
- [2023.11]: I will attend RSNA 2023 and give an oral presentaion and an invited course section on Nov.27th. Stop by and say Hi :)
- [2023.10]: LogiGLUE paper is out: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models.
- [2023.07]: A new multimodal-query retriever dataset is public available on github! Check the dataset and the paper.
- [2023.07]: Join Mayo Clinic as a Research Fellow!
- [2023.05]: Our work on the commonsense assumption of language models is available on arXiv, also check the dataset!
- [2023.05]: My Google internship work on Dr.ICL: Demonstration-Retrieved In-context Learning is available on arXiv, check it out!
- [2023.05]: Our work on end-to-end multimodal retriever has been accepted to ACL 23!
- [2023.05]: My Meta internship work on hybrid efficient retriever has been accepted to ACL 23, check out our paper!
- [2023.04]: Successfully defense my Ph.D. dissertation: Neural Retrieval and Reader for Information Retrieval and Question Answering [Slide]
- [2023.01]: Invited talk at SERUM @ WACV 2023, [Slide]
2022
- [2022.12]:2th O-DRUM Workshop will be back at CVPR 2023 with a new rhythm: Open-Domain *Reasoning* under Multimodal settings.
- [2022.08]: Start internship at Google Research.
- [2022.08]: Completed thesis proposal and became Ph.D. candidate.
- [2022.05]: Start internship at Meta Reality Lab working on efficient retrieval task.
- [2022.05]: 2 papers accpeted to NAACL 2022 SRW .
- [2022.04]: 1 paper accpeted to NAACL 2022 Finging.
- [2022.04]: 1 paper accpeted to ACL 2022 Spa-NLP workshop.
- [2022.02]: 1 paper accpeted to ACL 2022 Finding.
- [2022.01]: O-Drum Workshop will be held in CVPR 2022
2021
- [2021.12]: 1 paper accepted to AAAI 2021. Acceptant rate 15%.
- [2021.08]: 1 paper accepted to EMNLP 2021.
- [2021.06]: Yankai Zeng (mentor by me) passed his master thesis and now be a Ph.D. student in UTD.
- [2021.05]: Research intern at Salesforce and worked with Kazuma Hashimoto and Yingbo Zhou.
- [2021.04]: Finalist of 2021 Knowledge Mobilization Awards.
- [2021.03]:Invited talk at exploreCSR workshop (ASU).
- [2021.02]: 1 paper accepted to EACL 2021.
2019
- [2019.09]: 1 paper accpeted to ICLP 2019.
- [2019.09]: Presented at ICLP 2019 Doctoral Consortium
- [2019.06]: 1 workshop paper accepted to ASPOCP 2019
Professional Activities and Experience
Publications
End-to-end Knowledge Retrieval with Multi-modal Queries Man Luo, Zhiyuan Fang, Tejas Gokhale, Yezhou Yang, Chitta Baral ACL 2023 [Paper]
A Study on the Efficiency and Generalization of Light Hybrid Retrievers Man Luo, Shashank Jain, Anchit Gupta, Arash Einolghozati, Barlas Oguz Debojeet Chatterjee, Xilun Chen, Chitta Baral, Peyman Heidari ACL 2023 [Paper]
In-BoXBART: Get Instructions into Biomedical Multi-Task Learning Mihir Parmar, Swaroop Mishra, Mirali Purohit, Man Luo, M. Hassan Murad, Chitta Baral NAACL 2022 Finding [Paper] [Model in Huggingface]
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness Tejas Gokhale*, Swaroop Mishra*, Man Luo*, Bhavdeep Singh Sachdeva, Chitta Baral ACL 2022 Finding [Paper]
Improving Biomedical Information Retrieval with Neural Retrievers Man Luo, Arindam Mitra, Tejas Gokhale, Chitta Baral AAAI 2022 [Paper]
Strong equivalence for LPMLN programs Joohyung Lee and Man Luo ICLP 2019 [Paper]
Preprinted Papers
Can Transformers Reason About Effects of Actions? Pratyay Banerjee, Chitta Baral, Man Luo, Arindam Mitra, Kuntal Pal, Tran C Son, Neeraj Varshney arXiv preprint [Paper]