Evaluating explainability methods intended for multiple stakeholders: an explainability framework for subject experts and non-experts in telecommunications.
Martin, Kyle; Liret, Anne; Wiratunga, Nirmalie; Owusu, Gilbert; Kern, Mathias
Professor Nirmalie Wiratunga email@example.com
Explanation mechanisms for intelligent systems are typically designed to respond to specific user needs, yet in practice these systems tend to have a wide variety of users. This can present a challenge to organisations looking to satisfy the explanation needs of different groups using an individual system. In this paper we present an explainability framework formed of a catalogue of explanation methods, and designed to integrate with a range of projects within a telecommunications organisation. Explainability methods are split into low-level explanations and high-level explanations for increasing levels of contextual support in their explanations. We motivate this framework using the specific case-study of explaining the conclusions of field network engineering experts to non-technical planning staff and evaluate our results using feedback from two distinct user groups; domain-expert telecommunication engineers and non-expert desk agent staff. We also present and investigate two metrics designed to model the quality of explanations - Meet-In-The-Middle (MITM) and Trust-Your-Neighbours (TYN). Our analysis of these metrics offers new insights into the use of similarity knowledge for the evaluation of explanations.
MARTIN, K., LIRET, A., WIRATUNGA, N., OWUSU, G. and KERN, M. . Evaluating explainability methods intended for multiple stakeholders: an explainability framework for subject experts and non-experts in telecommunications. KI - Künstliche Intelligenz [online], (accepted).
|Journal Article Type||Article|
|Acceptance Date||Dec 30, 2020|
|Deposit Date||Jan 7, 2021|
|Journal||KI - Künstliche intelligenz|
|Peer Reviewed||Peer Reviewed|
|Keywords||Machine learning; Similarity modeling; Explainability; Information retrieval|
This file is under embargo due to copyright reasons.
Contact firstname.lastname@example.org to request a copy for personal use.
You might also like
Learning to recognise exercises for the self-management of low back pain.
Human activity recognition with deep metric learners.
Learning to self-manage by intelligent monitoring, prediction and intervention.