BioC2024

Unleashing the potential of large language models (LLMs) in healthcare is a hot topic of research amongst computational scientists and bioinformatics researchers worldwide. Large language models have the potential to assist in various aspects of medicine given their capability to process complex concepts. The constant development of new treatment regimens is daunting for healthcare professionals and the introduction of these models allows for professionals to be updated with the latest advancements which improves patient care. CancerGPT [1] , an LLM, employs zero-shot learning to extract knowledge from medical literature and then use it to infer biological tasks. With approximately 124 million parameters, it matches the scale of the larger, fine-tuned GPT-3 model with around 175 million parameters. The observations from the study demonstrated that CancerGPT provided accurate responses for seven different biological tasks. However, the study also highlights the vulnerability of LLMs to hallucination. To address hallucination, which is incorrect and nonsensical content of large language models , our research seeks to address this limitation of LLM by seeking answers to the following questions. Can these models be trusted to responsibly recommend personalized treatment plans for cancer patients thereby improving their health outcomes? Can they serve as reliable clinical predictor tools for tailoring treatment strategies? To answer these questions, we curate a dataset consisting of treatment outcomes for monoclonal antibody treatment regimens in head and neck squamous cell carcinoma (HNSCC), which is the seventh most common cancer worldwide. The treatment regimens [2] include Cetuximab, Pembrolizumab, Durvalumab, and Nivolumab prescribed from the National Cancer Control Programme (NCCP) and The Irish Society of Medical Oncology (ISMO) which has a total of thirteen different drug regimens for HNSCC. The treatment outcomes are based on RECIST [3] score for each of the four drug regimens. Ten clinical trials for each of the individual treatment regimens were incorporated in our study. We augment prompt engineering process by providing contextual information such as drug regimens for targeted therapy, regimen doses, targeted oncogenes, and clinical outcomes from the clinical trials used in our study. We use one shot learning during prompt engineering phase. We test the LLM responses by providing contextual and non-contextual information. The LLM responses are evaluated using true positives and true negative scores on ten patient profiles of similar and different cancer type on the same drug regimen. Preliminary results indicate that contextual information significantly reduces LLM’s from hallucinating. To make the models reliable and trustworthy our study encompasses age, gender and genomic profile of patients. By overcoming hallucination and incorporating multimodal features, we demonstrate that LLMs holds the potential to recommend personalised treatments for HNSCC in Irish patients.

References:

[1] Li, T., Shetty, S., Kamath, A., Jaiswal, A., Jiang, X., Ding, Y., & Kim, Y. (2024). CancerGPT for few shot drug pair synergy prediction using large pretrained language models. In npj Digital Medicine (Vol. 7, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41746-024-01024-9

[2]https://www.hse.ie/eng/services/list/5/cancer/profinfo/chemoprotocols/headandneck/

[3] Nishino, M., Jagannathan, J. P., Ramaiya, N. H., & Van den Abbeele, A. D. (2010). Revised RECIST Guideline Version 1.1: What Oncologists Want to Know and What Radiologists Need to Know. In American Journal of Roentgenology (Vol. 195, Issue 2, pp. 281–289). American Roentgen Ray Society. https://doi.org/10.2214/ajr.09.4110