Bridging the gap between AI and explainability in the GDPR: Towards trustworthiness-by-design in automated decision-making

Can we achieve satisfactory explanations for complex machine learning models in high-risk automated decision making? How can such explanations be made admissible to a data protection framework safeguarding a right to explanation? We explore these questions, analysing from an interdisciplinary point of view the connection between existing legal requirements for the explainability of AI systems from the General Data Protection Regulation (GDPR) and the current state of the art in the field of explainable AI. We study the challenges of providing human legible explanations for current and future AI-based decision-making systems in practice, based on two case studies of automated decision making in credit scoring risks and medical diagnosis of COVID-19. These scenarios exemplify the trend in the usage of increasingly complex machine learning algorithms in automated decision making with growing dimensionality of data and model parameters. We highlight that current machine learning, and especially deep learning techniques, are unable to make clear causal links between input data and final decisions. This represents a limitation for providing exact, human-legible reasons behind specific decisions and makes the provision of satisfactorily, fair and transparent explanations a serious challenge. Therefore, we conclude that the quality of explanations might not be considered as an adequate safeguard for automated decision-making processes under the GDPR. Accordingly, we propose practical considerations regarding additional tools, such as algorithmic impact assessments, and other forms of algorithmic justifications based on broader principles of trustworthy AI, which should be considered to complement explanations. Although this article investigates AI explainability in relation to legal requirements, our analysis suggests that eventually all these principles need to be considered as a whole.

HAMON Ronan;  JUNKLEWITZ Henrik;  MALGIERI Gianclaudio;  DE HERT Paul;  SANCHEZ MARTIN Jose Ignacio; 

2022-02-03

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

JRC124175

1556-603X (online),   

https://ieeexplore.ieee.org/document/9679770,    https://publications.jrc.ec.europa.eu/repository/handle/JRC124175,   

10.1109/MCI.2021.3129960 (online),