Automated test generation tools often produce assertions that reflect implemented behavior, limiting their usage
to regression testing. In this paper, we propose LLMPROPHET, a black-box approach that trains LLMs on automatically generated regression tests using Few-Shot Learning to identify non-regression faults without relying on source code. By employing iterative cross-validation and a leave-one-out strategy, LLMPROPHET identifies regression assertions that are misaligned with expected behaviors. We outline LLMPROPHET’s workflow, feasibility, and preliminary findings, demonstrating its potential for LLM-driven fault detection.
RUBERTO Stefano;
PERERA Judith;
JAHANGIROVA Gunel;
TERRAGNI Valerio;
2025-08-21
IEEE Computer Society
JRC141000
2159-4848 (online),
https://doi.ieeecomputersociety.org/10.1109/ICSTW64639.2025.10962503,
https://publications.jrc.ec.europa.eu/repository/handle/JRC141000,
10.1109/ICSTW64639.2025.10962503 (online),
| Name | Country | City | Type |
|---|
This document is only visible at the Commission level.
You are not authorized to publish or distribute it outside the European Commission.
This is a public document. You can share this publication.
Datasets
| ID | Title | Public URL |
|---|
Dataset collections
| ID | Acronym | Title | Public URL |
|---|
Scripts / source codes
| Description | Public URL |
|---|
Additional supporting files
| File name | Description | File type |
|---|