07-26, 11:25–11:33 (US/Eastern), Tomatis Auditorium
Unleashing the potential of large language models (LLMs) in healthcare is a hot topic of research amongst computational scientists and bioinformatics researchers worldwide. Large language models have the potential to assist in various aspects of medicine given their capability to process complex concepts. The constant development of new treatment regimens is daunting for healthcare professionals and the introduction of these models allows for professionals to be updated with the latest advancements which improves patient care. CancerGPT [1] , an LLM, employs zero-shot learning to extract knowledge from medical literature and then use it to infer biological tasks. With approximately 124 million parameters, it matches the scale of the larger, fine-tuned GPT-3 model with around 175 million parameters. The observations from the study demonstrated that CancerGPT provided accurate responses for seven different biological tasks. However, the study also highlights the vulnerability of LLMs to hallucination. To address hallucination, which is incorrect and nonsensical content of large language models , our research seeks to address this limitation of LLM by seeking answers to the following questions. Can these models be trusted to responsibly recommend personalized treatment plans for cancer patients thereby improving their health outcomes? Can they serve as reliable clinical predictor tools for tailoring treatment strategies? To answer these questions, we curate a dataset consisting of treatment outcomes for monoclonal antibody treatment regimens in head and neck squamous cell carcinoma (HNSCC), which is the seventh most common cancer worldwide. The treatment regimens [2] include Cetuximab, Pembrolizumab, Durvalumab, and Nivolumab prescribed from the National Cancer Control Programme (NCCP) and The Irish Society of Medical Oncology (ISMO) which has a total of thirteen different drug regimens for HNSCC. The treatment outcomes are based on RECIST [3] score for each of the four drug regimens. Ten clinical trials for each of the individual treatment regimens were incorporated in our study. We augment prompt engineering process by providing contextual information such as drug regimens for targeted therapy, regimen doses, targeted oncogenes, and clinical outcomes from the clinical trials used in our study. We use one shot learning during prompt engineering phase. We test the LLM responses by providing contextual and non-contextual information. The LLM responses are evaluated using true positives and true negative scores on ten patient profiles of similar and different cancer type on the same drug regimen. Preliminary results indicate that contextual information significantly reduces LLM’s from hallucinating. To make the models reliable and trustworthy our study encompasses age, gender and genomic profile of patients. By overcoming hallucination and incorporating multimodal features, we demonstrate that LLMs holds the potential to recommend personalised treatments for HNSCC in Irish patients.
References:
[1] Li, T., Shetty, S., Kamath, A., Jaiswal, A., Jiang, X., Ding, Y., & Kim, Y. (2024). CancerGPT for few shot drug pair synergy prediction using large pretrained language models. In npj Digital Medicine (Vol. 7, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41746-024-01024-9
[2]https://www.hse.ie/eng/services/list/5/cancer/profinfo/chemoprotocols/headandneck/
[3] Nishino, M., Jagannathan, J. P., Ramaiya, N. H., & Van den Abbeele, A. D. (2010). Revised RECIST Guideline Version 1.1: What Oncologists Want to Know and What Radiologists Need to Know. In American Journal of Roentgenology (Vol. 195, Issue 2, pp. 281–289). American Roentgen Ray Society. https://doi.org/10.2214/ajr.09.4110
Meghana Kshirsagar is an Associate Professor in AI/ML , in the department of computer science and information systems at University of Limerick, Ireland. She possesses rich experience of working as Associate Professor in the Computer science department at Government Engineering College Aurangabad, India. During 2019 -2022 she worked as postdoctoral researcher on the Science Foundation Ireland Project “Automatic Design of Digital Circuits” along with industrial partner Intel. From 2022-2023 she worked as a research fellow in “Limerick Digital Cancer Research Centre”. Her research interests include large language models, machine learning , digital twins , bioinformatics and blockchains. She is Principal Investigator for the National Challenge Fund , digital for resilience for the project “ALTER: Unleashing the power of Artificial Intelligence and Digital Twins for Emergency Care: A vision for the future”. She is Deputy Director of Biocomputing Developmental Systems Research Group at University of Limerick.She serves on the “Equity, Diversity and inclusion” committee. She has authored 80 papers in prestigious machine learning ,artificial intelligence and bioinformatics conferences and journals.She serves as both reviewer and guest editor in “Frontiers of Blockchain” journal. She has delivered workshops and webinars in high profile international events and workshops.