In today's era of information overload, the sheer volume of available data poses a significant challenge to effective knowledge management. The ability to synthesise knowledge from diverse sources in a timely manner, including academic, news, and web content, is crucial for informed decision-making. However, the complexity and scale of modern information ecosystems render manual synthesis increasingly impractical. Artificial intelligence (AI) offers a promising solution to this problem, enabling the automated synthesis of knowledge and facilitating more efficient information gathering. Many agentic tools are available today, but most focus on full automation, while in most cases scientist prefer a setup where they can intervene and steer the system instead of being passive users. We believe that this human-in-the-loop approach can contribute to user trust, accuracy and factuality. Furthermore, reliability and accuracy of AI-powered knowledge synthesis tools depend on rigorous validation and evaluation which are not publicly available so far.
To address these challenges, we present the Research ASSistant (RASS), an AI-powered prototype designed to support fast literature reviews currently in experimental use in the Joint Research Centre. We detail its development and capabilities and describe a tailored validation that may be of use for assessing AI-based knowledge synthesis systems in general. The framework operates across three levels, process, retrospective, and usability, and evaluates six dimensions: technical performance, content quality, domain relevance, methodological rigor, usability, and integration.
Our findings underscore the transformative potential of AI in supporting knowledge synthesis while highlighting the critical importance of rigorous, multidimensional validation to ensure the quality, relevance, and trustworthiness of AI-generated outputs.
CERESA Mario;
MEERSMANS Karen;
SPADARO Nicholas;
ZANI Alessandro;
2026-04-21
Publications Office of the European Union
JRC146034
978-92-68-39199-0 (online),
1831-9424 (online),
EUR 40696,
OP KJ-01-26-168-EN-N (online),
https://publications.jrc.ec.europa.eu/repository/handle/JRC146034,
10.2760/0363450 (online),
This document is only visible at the Commission level.
You are not authorized to publish or distribute it outside the European Commission.
This is a public document. You can share this publication.
