Multimodal4Health Workshop @ ICHI 2024

Description

The practice of modern medicine relies heavily on the synthesis of information and data from multiple sources; this includes imaging pixel data, structured laboratory data, unstructured narrative data, and in some cases, audio or observational data. The integration of varied data modalities, ranging from the text in electronic health records and clinical notes to complex imaging like MRI, PET, histopathology images, and even video data from therapy consultations, and endoscopy, opens new avenues for groundbreaking patient care. For instance, in breast cancer management, combining patient history and physical examination data with mammography and genetic signatures can significantly enhance treatment planning. In the context of healthcare domain, the application of multimodal learning extends beyond just understanding the status of individual organs to understanding the status of multi-organ systems with their complex interactions. This presents a more holistic view of patient health that mimics a physician's workflow.

This workshop aims to tackle the unique and challenging fusion of diverse data types (e.g. text, high dimensional images, 1D signals) in healthcare. Unlike other domains, healthcare data demands measures to tackle extreme challenges related to data collection, missing data coding, ethical handling, particularly concerning patient privacy and data security, and the need for unparalleled accuracy in the downstream task with limited training data.

Recent successes in the generic computer science domain, such as large language models and stable diffusion image generation models, indicate the potential for transformative applications in healthcare. However, healthcare-specific challenges like handling high-resolution medical images without losing fine-grained details, addressing label imbalance and data sparsity, as well as managing long and time-sequenced data, present unique opportunities for innovation. Thus, the primary goal of this workshop is to bridge the `gap' between theoretical computer science advancements and practical healthcare applications, fostering interdisciplinary collaborations for holistic solutions. To achieve this goal, our workshop will feature leading researchers and experts from sub-disciplines including bioinformatics, natural language processing, computer vision, and multimodal representation learning. Through invited talks and panel discussions with industry and academia representatives, the workshop will explore the significant impact and challenges of multimodal learning in healthcare. By focusing on these critical aspects, the workshop is poised to set the stage for the next generation of healthcare solutions, driving forward the vision of AI in transforming healthcare through innovative multimodal learning approaches.