Shedding Light Inside the Black Box: Techniques for Explainable Artificial Intelligence in Healthcare

Abstract

Artificial intelligence techniques and systems are demonstrating their effectiveness in solving problems in many application areas, including healthcare. Over the years, several studies focused on leveraging machine learning and deep learning techniques to identify diseases in patients. However, while these techniques have demonstrated remarkable accuracy in diagnosis, they often operate as black box models, meaning they provide outputs without clear explanations of the rationale behind their decisions. This lack of transparency poses significant challenges, particularly in the healthcare domain, where decisions can have significant implications for the well-being of patients and treatment paths. In response to this challenge, explainable artificial intelligence (XAI) has emerged as a research area aiming at providing not only accurate diagnoses but also understandable and interpretable explanations for the decisions made by AI models. For providing explanations also Large Language Models (LLMs) may be exploited. This chapter provides an overview of cutting-edge XAI techniques for healthcare applications, which hold the promise of enhancing trust, enabling clinicians to better understand and contextualize AI results, and ultimately improving patient care. Nevertheless, XAI techniques in this domain produce explanations in a form that is not always easily understandable by healthcare professionals. Current conversational AI can play a crucial role in bridging this gap by transforming model explanations into a human-readable format, making them more accessible to clinicians and users. The chapter also presents a case study addressing the critical issue of disease detection using social media data, demonstrating how the integration of LLMs with XAI techniques allows to enhance model interpretability, facilitating efficient communication between the AI system and healthcare professionals.