Title: Statistical Validation and Skill Assessment of Hyflux2 Model
Publisher: Publications Office of the European Union
Publication Year: 2016
JRC N°: JRC104895
ISBN: 978-92-79-64591-4
ISSN: 1831-9424
Other Identifiers: EUR 28338 EN
OP LB-NA-28338-EN-N
URI: http://publications.jrc.ec.europa.eu/repository/handle/JRC104895
DOI: 10.2788/371
Type: EUR - Scientific and Technical Research Reports
Abstract: The Joint Research Center (JRC) has developed extensive experience in tsunami early warning systems, using the JRC-SWAN finite difference code for wave propagation simulation and the JRC finite-volume HyFlux2 code for wave propagation and inundation modelling over the last years. Since 2011, NWP (Numerical Weather Prediction) atmospheric forcing terms have been included in the HyFlux2 code for simulating storm surge events. In the current work, the skill assessment of Hylfux2 is performed. A wide range of verification metrics has been utilised for both Hyflux2 model data sets namely NOF (raw forecasts with no adjustment) and YOF (post forecasts by applying an optimal type of offset). Investigating over typical metrics as bias, root mean square error (RMSE) and centred root mean square differences (CRMSD), inter-comparisons were possible versus another integrated storm surge forecast system namely KASSANDRA (KASS) of ISMAR-CNR. Referring to the ability of reproducing the variability of observations, inter-comparing over 10 common stations revealed that Hyflux2 YOF configuration although in the right direction, is not reaching the quality of KASS system for T+24-hour horizon. Hyflux2 normalised standard deviation manages to reach the 0.81 value compared to 0.97 value of KASS (with perfect score: 1.0). On the other hand, the most important message seems to be the one coming from the inter-comparison between CRMSD scores. Hylfux2 YOF forecasts appear to have a comparable CRMSD score (6.42 cm) to the score coming from KASS system (5.86 cm) for T+24 hours. Furthermore, there are stations (like Civitavecchia, Genova, Napoli and Palermo) over which Hyflux2 YOF forecasts score considerably better than KASS system, whereas the rest of YOF forecasts appear to have a lower (but still of high quality) correlation coefficient (0.80) score compared to the one coming from KASS system (0.89 cm) for T+24 hours. Another important area that special type of metrics was used (such as accuracy, frequency bias, hit rate, false alarm ratio, probability of false detection, success ratio, threat score, equitable threat score, true skill statistics, odds ratio and odds ratio skill score) has been the ability of Hyflux2 to provide useful (warning) forecast guidance in cases of high-intensity storm surge events. The selection of an optimal (95% percentile) threshold was made being high enough to be considered as extreme but also capable of providing enough cases for robust statistics. The main outcome of such an approach has revealed that 72% (T+72 hours) to 79% (T+12 hours) of all Hyflux2 forecasts were correct over central Mediterranean (CMEDI) for both NOF and YOF forecasts. The corresponding values for west Mediterranean (WMEDI) were reaching even higher values (80 - 81% to 88%) with similar skill values for both NOF and YOF configurations, but it should be stressed out that these results have considered a large number of correct negatives (referring to non-extremes events). Focussing over high-intensity events (that have been observed) Hylfux2 appears to have considerable forecasting limitations being able to capture only the 23% (T+72) to 34% (T+12) of events while missing more than 70% of the high-intensity events at T+48 hours. Such forecasting limitations become obvious during the in-depth analysis over two case study extreme events taken place over Ravenna (6 February 2015) and Venice (29 February 2016). The capabilities of both NOF & YOF forecasts based on ECMWF relatively low-resolution forcing terms to provide useful guidance in Ravenna case found to be limited even if both NOF & YOF managed to provide a relatively useful early warning for the extreme case of Venice. It appears that both NOF & YOF configurations (based on ECMWF forcing terms) have certain limitations to provide the best possible setup for detecting and simulating such high-impact events. On the other hand, HYflux2 YOF forecasts based on various COSMO model high-resolution forcing terms seem to do quite much better in capturing both events and providing useful (early) warning to the user. It seems that for such high-impact events higher-resolution forcing terms are necessary to correctly resolve the full extent and magnitude of the event. This higher resolution feature is most probably the reason why Hyflux2 based on COSMO model (run operationally by the Italian Air Force Weather Meteorological Service) high-resolution forcing terms provides much more useful guidance in cases of extreme events.
JRC Directorate:Space, Security and Migration

Files in This Item:
File Description SizeFormat 
statistical_validation_and_skill_assessment_of_hyflux2_model.pdf3.84 MBAdobe PDFView/Open

Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.