Skip to main content

Research Repository

Advanced Search

A user-centred evaluation of DisCERN: discovering counterfactuals for code vulnerability detection and correction.

Wijekoon, Anjana; Wiratunga, Nirmalie

Authors

Anjana Wijekoon



Abstract

Counterfactual explanations highlight actionable knowledge which helps to understand how a machine learning model outcome could be altered to a more favourable outcome. Understanding actionable corrections in source code analysis can be critical to proactively mitigate security attacks that are caused by known vulnerabilities. In this paper, we present the DisCERN explainer for discovering counterfactuals for code vulnerability correction. Given a vulnerable code segment, DisCERN finds counterfactual (i.e. non-vulnerable) code segments and recommends actionable corrections. DisCERN uses feature attribution knowledge to identify potentially vulnerable code statements. Subsequently, it applies a substitution-focused correction, suggesting suitable fixes by analysing the nearest-unlike neighbour. Overall, DisCERN aims to identify vulnerabilities and correct them while preserving both the code syntax and the original functionality of the code. A user study evaluated the utility of counterfactuals for vulnerability detection and correction compared to more commonly used feature attribution explainers. The study revealed that counterfactuals foster positive shifts in mental models, effectively guiding users towards making vulnerability corrections. Furthermore, counterfactuals significantly reduced the cognitive load when detecting and correcting vulnerabilities in complex code segments. Despite these benefits, the user study showed that feature attribution explanations are still more widely accepted than counterfactuals, possibly due to the greater familiarity with the former and the novelty of the latter. These findings encourage further research and development into counterfactual explanations, as they demonstrate the potential for acceptability over time among developers as a reliable resource for both coding and training.

Citation

WIJEKOON, A. and WIRATUNGA, N. 2023. A user-centred evaluation of DisCERN: discovering counterfactuals for code vulnerability detection and correction. Knowledge-based systems [online], 278, article 110830. Available from: https://doi.org/10.1016/j.knosys.2023.110830

Journal Article Type Article
Acceptance Date Jul 20, 2023
Online Publication Date Jul 26, 2023
Publication Date Oct 25, 2023
Deposit Date Jul 27, 2023
Publicly Available Date Jul 27, 2023
Journal Knowledge-based systems
Print ISSN 0950-7051
Electronic ISSN 1872-7409
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 278
Article Number 110830
DOI https://doi.org/10.1016/j.knosys.2023.110830
Keywords Counterfactual explanations; Vulnerability detection; Explainable AI
Public URL https://rgu-repository.worktribe.com/output/2023452

Files





You might also like



Downloadable Citations