From Implemented to Expected Behaviors: Leveraging Regression Oracles for Non-Regression Fault Detection Using LLMs
Automated test generation tools often produce assertions that reflect implemented behavior, limiting their usage
to regression testing. In this paper, we propose LLMPROPHET, a black-box approach that trains LLMs on automatically generated regression tests using Few-Shot Learning to identify non-regression faults without relying on source code. By employing iterative cross-validation and a leave-one-out strategy, LLMPROPHET identifies regression assertions that are misaligned with expected behaviors. We outline LLMPROPHET’s workflow, feasibility, and preliminary findings, demonstrating its potential for LLM-driven fault detection.
RUBERTO Stefano;
PERERA Judith;
JAHANGIROVA Gunel;
TERRAGNI Valerio;
2025-08-21
IEEE Computer Society
JRC141000
2159-4848 (online),
https://doi.ieeecomputersociety.org/10.1109/ICSTW64639.2025.10962503,
https://publications.jrc.ec.europa.eu/repository/handle/JRC141000,
10.1109/ICSTW64639.2025.10962503 (online),
Additional supporting files
| File name | Description | File type | |
|---|
