PhD Scientific Days 2025

Poster Session III. - P: Health Sciences

Advancing Healthcare Application Development: Optimization Strategies and Limitations of Generative Artificial Intelligence-Based Language Models

Name of the presenter

Angyal Viola

Institute/workplace of the presenter

Semmelweis University Doctoral College, Health Sciences Division, Institute of Digital Health Sciences, Budapest, Hungary

Authors

Viola Angyal, MSc¹

1: Semmelweis University, Doctoral School of Health Sciences, Budapest

Text of the abstract

Introduction: The rapid advancement of Generative Artificial Intelligence (GenAI), powered by Generative Pre-trained Transformers (GPT), has revolutionized natural language processing. As GenAI models are increasingly integrated into healthcare applications, they offer new opportunities for improving workflows and patient engagement. However, optimizing these models for healthcare also reveals critical limitations.
Aims: Our primary objective was to explore the possibilities and limitations of Large Language Models (LLMs) and Small Language Models (SLMs) for specific tasks in healthcare applications. We also aimed to investigate customization techniques for GPTs by employing various prompt engineering strategies, fine-tuning methods, and the custom GPT builder provided by OpenAI (San Francisco, CA, USA).
Methods: We examined SLMs through scientific publications and utilized the GPT-4 model to test various methods for LLM customization. We explored the rapid custom-GPT builder, prepared a training dataset, and employed prompt engineering techniques for model customization.
Results: SLMs, with fewer than 1.5 billion parameters, are easier to use for small scientific domains and can operate offline. LLMs can be customized via Application Programming Interfaces (APIs) without developing new models. Fine-tuning, requiring 10 000–100 000 examples and advanced expertise, was the most costly, time-consuming, and least flexible method. In contrast, OpenAI’s custom GPT builder offered a user-friendly interface that needed only 10–50 examples. Prompt engineering, requiring no retraining or additional data, proved the fastest and most cost-effective technique. Zero-shot and few-shot methods were effective for LLMs with limited datasets.
Conclusion: Although custom GPTs cannot substitute consultations with healthcare professionals, these tools can expedite information access and support personalized care. Overall, based on our findings, prompt engineering emerged as the most cost- and time-efficient customization technique.
Funding: SUPPORTED BY THE 2024-2.1.2-EKÖP-KDP-2024-00002 UNIVERSITY RESEARCH SCHOLARSHIP PROGRAMME OF THE MINISTRY FOR CULTURE AND INNOVATION FROM THE SOURCE OF THE NATIONAL RESEARCH, DEVELOPMENT AND INNOVATION FUND