How to maximise event-based surveillance web-systems: the example of ECDC/JRC collaboration to improve the performance of MedISys
The mission of the European Centre for Diseases prevention and Control (ECDC) is to identify, assess and communicate current and emerging infectious threats to human health within the European Union (EU). The identification of threats is based on the collection and analysis of information from established communicable disease surveillance networks and from unstructured information mostly originating from non-health care sources, e.g. online news sites.
MedISys, an automatic real-time media monitoring and threat detection system developed by the Joint Research Centre (JRC) of the European Commission, is among the tools used by the ECDC for timely identification of potential public health threats from online information sources. In 2008, an ECDC internal analysis indicated that MedISys issued alerts faster than other human-mediated web-based systems. As timely detection is crucial to enable efficient response to public health threats, ECDC decided to further explore the potential of MedISys as an EU early warning tool. We analysed the functionality of the existing system in view of improving the usability of the web site, revising the sources and reducing the amount of irrelevant articles. We provided JRC with practical suggestions for the interface and asked public health experts at national level to assist in the revision of sources. To reduce the number of irrelevant articles, alternative search strategies for fifteen diseases were tested against the existing strategies using the positive predictive value (PPV) and the sensitivity to measure the performance of the system. Our intervention increased the PPV value (from 15.3% to 71.1%) and the sensitivity of the system.
We conclude that the best search strategies use a limited number of keywords weighted as positive (with weights adjusted below the alert thresholds) and an extended list of keywords weighted as negative. We recommend a high number of epidemiological terms within the keyword combinations.
The results indicate that user feedback is crucial to exploit the full potential of event-based surveillance systems such as MedISys. We will improve the detection of other infectious diseases and intend to cover all EU languages. Customized country versions will be set up in collaboration with JRC; ECDC will encourage the use of the system at national level in the EU member states.
MANTERO Jas;
BELYAEVA Evgenia;
LINGE Jens;
2011-04-27
Publications Office of the European Union
JRC63805
978-92-79-19755-0,
1018-5593,
EUR 24763 EN,
OP LB-NA-24763-EN-C,
https://publications.jrc.ec.europa.eu/repository/handle/JRC63805,
10.2788/69804,
Additional supporting files
| File name | Description | File type | |
|---|
