Evaluation of Epidemic Intelligence Systems Integrated in the Early Alerting and Reporting Project for the Detection of A/H5N1 Influenza Events

Web-based expert systems dedicated to epidemic intelligence were developed to detect health threats. The Early Alerting and Reporting (EAR) project, launched under the Global Health Initiative, aimed at assessing the feasibility and opportunity of pooling seven of those expert systems. A qualitative survey was carried out with EAR participants to document epidemic intelligence strategies and to assess perceptions regarding the performance of participating systems. Timeliness and sensitivity were rated with high scores illustrating the overall perceived value of all systems while weaknesses were underlined especially in terms of representativeness, completeness and flexibility. These findings were corroborated by the quantitative analysis performed on signals potentially related to influenza A/H5N1 events which occurred in March 2010. For the six systems for which this information was available; the detection rate ranged from 31% to 38%, and increased to 72% when considering the virtual combined system. The positive predictive values (PPV) ranged from 3% to 24% and the F1-score ranged from 6% to 27%. These low scores point out false positive signals related to varying abilities of the systems to efficiently sort-out information and reduce background noise. For the seven systems sensitivity ranged from 38% to 72%. An average difference of 23% was observed between the sensitivities calculated for human cases and epizootics, underlining the difficulties to develop an efficient algorithm or a single pathology. The sensitivity increased to 93% when the virtual combined system was considered, clearly illustrating the systems’ complementarities. The average delay between the detection of the A/H5N1 events by the systems and their official reporting by WHO or OIE was 10.2 days (CI95%, 6.7; 13.8). This work illustrates the diversity in implemented epidemic intelligence activities, differences in systems designs and the potential added values and opportunities for synergy: between systems, between users and between systems and users.

BARBOZA Philippe;  VAILLANT Laetitia;  MAWUDEKU Abla;  NELSON Noele;  MADOFF Larry;  LINGE Jens;  COLLIER Nigel;  BROWNSTEIN John;  YANGARBER Roman;  ASTAGNEAU Pascal; 

2013-08-12

PUBLIC LIBRARY SCIENCE

JRC73509

1932-6203,   

http://www.plosone.org/article/info:doi/10.1371/journal.pone.0057252,    https://publications.jrc.ec.europa.eu/repository/handle/JRC73509,