Man Luo (罗满)

Ph.D., Arizona State University

mluo26 [at] asu [dot] edu
Google Scholar / Github / LinkedIn / CV

About

I am currently an AI Research Scientist at Intel Lab, working on large scale multimodal system pretraining and multimodal retrieval augmented generation (RAG).

My research dives deep into the realms of information retrieval and reading comprehension within natural language processing (NLP) and multimodal domains. The primary goal of my research is to develop advanced models that not only efficiently retrieve and utilize external knowledge for enriched comprehension and reasoning but also demonstrate superior generalization capabilities across unfamiliar tasks and domains. A few areas of my recent interest include:

  • Knowledge Retrieval: How can we effectively retrieve and utilize external knowledge to not only enhance comprehension but also mitigate hallucination?
  • Generalization: Designing models that can seamlessly adapt and perform across various tasks and domains without explicit training.
  • Multimodal Understanding: Delving into the integration of textual, visual, and other modalities to bolster machine comprehension and response capabilities.
  • Biomedical/healthcare application and innovation: Evaluate and innovate the LLMs to solve biomedical and healthcare challenges, such as long sequence processing, noisy data mitigation, data imbalance rectification, and enhancing interpretability.

Previously, I was a research fellow at Mayo Clinic, AZ, working closely with Dr. Imon Banerjee and Dr. Bhavik Patel. I earned my doctoral degree in 2023 from Arizona State University under the esteemed supervision of Dr. Chitta Baral. I am also privileged to collaborate with amazing industry researchers from Salesforce, Meta and Google during my internship.

News

2024
  • [2024.10]: Our paper SA-OcrPaint has been accepted to WACV 2025!
  • [2024.8]: Our Survey paper Ret-ICL has been accepted to TMLR!
  • [2024.07]: New Paper SK-VQA is out! A synthetic generated VQA dataset improves multimodel content-based generalization capacity!
  • [2024.06]: New Book Advances in Multimodal Information Retrieval and Generation is out!
  • [2024.03]: New Paper SA-OcrPaint is out! A training-free framework to improve the Diffusion Model ability to generate visual text!
  • [2024.03]: New Paper PCA-LLM is a domain specific LLM for Prostate Cancer, we train a GPT-2 type of model from scratch on Prostate Cancer clinical notes with domain specific vocabulary.
  • [2024.03]: I am very excited to join Intel AI Lab, CA, as a AI research Scientist.
  • [2024.02]: Call for Paper: Multimodal4health Workshop. Will be held at ICHI Conference in Orlando, Florida in June 3rd 2024.
  • [2024.01]: Ret-ICL survey paper is out: In-context Learning with Retrieved Demonstrations for Language Models: A Survey.
2023
2022
  • [2022.12]:2th O-DRUM Workshop will be back at CVPR 2023 with a new rhythm: Open-Domain *Reasoning* under Multimodal settings.
  • [2022.08]: Start internship at Google Research.
  • [2022.08]: Completed thesis proposal and became Ph.D. candidate.
  • [2022.05]: Start internship at Meta Reality Lab working on efficient retrieval task.
  • [2022.05]: 2 papers accpeted to NAACL 2022 SRW .
  • [2022.04]: 1 paper accpeted to NAACL 2022 Finging.
  • [2022.04]: 1 paper accpeted to ACL 2022 Spa-NLP workshop.
  • [2022.02]: 1 paper accpeted to ACL 2022 Finding.
  • [2022.01]: O-Drum Workshop will be held in CVPR 2022
2021
  • [2021.12]: 1 paper accepted to AAAI 2021. Acceptant rate 15%.
  • [2021.08]: 1 paper accepted to EMNLP 2021.
  • [2021.06]: Yankai Zeng (mentor by me) passed his master thesis and now be a Ph.D. student in UTD.
  • [2021.05]: Research intern at Salesforce and worked with Kazuma Hashimoto and Yingbo Zhou.
  • [2021.04]: Finalist of 2021 Knowledge Mobilization Awards.
  • [2021.03]:Invited talk at exploreCSR workshop (ASU).
  • [2021.02]: 1 paper accepted to EACL 2021.
2019
  • [2019.09]: 1 paper accpeted to ICLP 2019.
  • [2019.09]: Presented at ICLP 2019 Doctoral Consortium
  • [2019.06]: 1 workshop paper accepted to ASPOCP 2019

Professional Activities and Experience

  • Guest Editor of PLOS Digital Medicine
  • Workshop orginzer for O-DRUM 2023, 2022
  • Reviewer for AAAI, NIPS, EMNLP, ACL, EACL

Publications

End-to-end Knowledge Retrieval with Multi-modal Queries
Man Luo, Zhiyuan Fang, Tejas Gokhale, Yezhou Yang, Chitta Baral
ACL 2023
[Paper]

A Study on the Efficiency and Generalization of Light Hybrid Retrievers
Man Luo, Shashank Jain, Anchit Gupta, Arash Einolghozati, Barlas Oguz Debojeet Chatterjee, Xilun Chen, Chitta Baral, Peyman Heidari
ACL 2023
[Paper]

In-BoXBART: Get Instructions into Biomedical Multi-Task Learning
Mihir Parmar, Swaroop Mishra, Mirali Purohit, Man Luo, M. Hassan Murad, Chitta Baral
NAACL 2022 Finding
[Paper] [Model in Huggingface]

Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness
Tejas Gokhale*, Swaroop Mishra*, Man Luo*, Bhavdeep Singh Sachdeva, Chitta Baral
ACL 2022 Finding
[Paper]

Improving Biomedical Information Retrieval with Neural Retrievers
Man Luo, Arindam Mitra, Tejas Gokhale, Chitta Baral
AAAI 2022
[Paper]

Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering
Man Luo*, Yankai Zeng*, Pratyay Banerjee, Chitta Baral
EMNLP 2021
[Paper] [Code]

‘Just because you are right, doesn’t mean I am wrong’: Overcoming a bottleneck in development and evaluation of Open-Ended VQA tasks
Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral
EACL 2021
[Paper] [Code]

Strong equivalence for LPMLN programs
Joohyung Lee and Man Luo
ICLP 2019
[Paper]

Preprinted Papers

A Simple Approach to Jointly Rank Passages and Select Relevant Sentences in the OBQA Context
Man Luo, Shuguang Chen, Chitta Baral.
arXiv preprint
[Paper] [Code]

Can Transformers Reason About Effects of Actions?
Pratyay Banerjee, Chitta Baral, Man Luo, Arindam Mitra, Kuntal Pal, Tran C Son, Neeraj Varshney arXiv preprint
[Paper]