Understanding disagreement between humans and machines in XAI: robustness, fidelity, and region-based explanations in automatic neonatal pain assessment.
Pirie, Craig; Antunes Ferreira, Leonardo; Coutrin, Gabriel de Almeida Sá; Carlini, Lucas Pereira; Moreno-Garcia, Carlos Francisco; Barros, Marina Carvalho de Moraes; Guinsburg, Ruth; Thomaz, Carlos Eduardo; Nobre, Rafael; Wiratunga, Nirmalie
Artificial Intelligence (AI) offers a promising approach to automating neonatal pain assessment, improving consistency and objectivity in clinical decision-making. However, differences between how humans and AI models perceive and explain pain-related features present challenges for adoption. In this study, we introduce a region-based explanation framework that improves interpretability and agreement between XAI methods and human assessments. Alongside this, we present a multi-metric evaluation protocol that jointly considers robustness, faithfulness, and agreement to support informed explainer selection. Applied to neonatal pain classification, our approach reveals several key insights: region-based explanations are more intuitive and stable than pixel-based methods — leading to higher consensus amongst explainer ensembles; both humans and machines focus on central facial features, such as the nose, mouth, and eyes; agreement is higher in "pain" cases than "no-pain" cases likely due to clearer visual cues; and robustness positively correlates with agreement, while higher faithfulness can reduce pixel-level consensus. Our findings highlight the value of region-based evaluation and multi-perspective analysis for improving the transparency and reliability of AI systems in clinical settings. We hope that this framework can support clinicians in better understanding model decisions, enabling more informed trust and integration of AI support in neonatal care.
Citation
PIRIE, C., FERREIRA, L.A., COUTRIN, G.A.S., CARLINI, L.P., MORENO-GARCÍA, C.F., BARROS, M.C.M., GUINSBURG, R., THOMAZ, C.E., NOBRE, R. and WIRATUNGA, N. Understanding disagreement between humans and machines in XAI: robustness, fidelity, and region-based explanations in automatic neonatal pain assessment. [2025]. To be presented at the 3rd World conference on eXplainable artificial intelligence 2025, 9-11 July 2025, Istanbul, Turkey.
Presentation Conference Type
Conference Paper (published)
Conference Name
3rd World conference on eXplainable artificial intelligence 2025
Start Date
Jul 9, 2025
End Date
Jul 11, 2025
Acceptance Date
Mar 24, 2025
Deposit Date
Apr 17, 2025
Publisher
Springer
Peer Reviewed
Peer Reviewed
Series Title
Communications in computer and information
Series ISSN
1865-0929; 1865-0937
Book Title
Explainable artificial intelligence
Keywords
Neonatal pain assessment; Human facial perception; Explainer disagreement
