This publication is a Conference and Workshop report by the Joint Research Centre (JRC), the European Commission’s science and knowledge service. It aims to provide evidence-based scientific support to the European policymaking process. The scientific output expressed does not imply a policy position of the European Commission. Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use that might be made of this publication.

Contact information

Stefano Nativi (Ed.)

European Commission, Joint Research Centre, TP262, Via Fermi, 21027 Ispra (VA), ITALY

stefano.nativi@ec.europa.eu

Tel.: +39 0332 785 075

EU Science Hub

https://ec.europa.eu/jrc

JRC Mission

As the science and knowledge service of the European Commission, the Joint Research Centre‘s mission is to support EU policies with independent evidence throughout the whole policy cycle.

Artificial Intelligence at the JRC

1st workshop on Artificial Intelligence at the JRC, Ispra 23rd May 2018

1.1 Foreword

We first introduce AI (Artificial Intelligence) as a generic term that refers to any machine or algorithm that is capable of observing its environment, learning, and based on the knowledge and experience gained, take intelligent actions or propose decisions. Autonomy of decision processes and interaction with other machines and humans are other dimensions that need to be considered.

Although many of the methodological developments in AI date back more than 50 years, the reason why we now pay so much attention to AI in general and machine learning (ML) in particular is that the recent advances in computing power, availability of data, and new algorithms have led to major breakthroughs in the last years. The capacity to benefit from, and adapt to, the challenges and opportunities presented by this cognitive revolution is arguably one of the greatest strategic issues facing the European Union.

The Joint Research Centre, the European Commission’s science and knowledge service, provides support on several important and complex matters, but also, plays an instrumental role in several key initiatives of the European Commission. In particular, the emerging challenge of AI has been set up as one of the principal areas of interest for the Joint Research Centre.

1.2 List of authors

Amanda Anderberg

David Asturiol

Gianmarco Baldini

George Breyiannis

Melisande Cardona

Carlos Castillo

Vicky Charisi

Enrico Checchi

Panayotis Christidis

Iwen Coisel

Christina Corban

Jose Antonio Cotelo Lema

Raphael d’Andrimont

Giuditta de Prato

Blagoj Delipetrev

Wim Devos

Nicole Dewandre

Tatyana Dimitrova

Francesco Dottori

Néstor Duch Brown

Valentin Enescu

Pasquale Ferrara

Raul Fidalgo Merino

Lorenzo Gabrielli

Javier Galbally

Dimitrios Geneiatakis

Raimondo Giuliani

Emilia Gómez

Álvaro Gómez Losada

Jiri Hradec

María Iglesias

Camelia Ignat

Henrik Junklewitz

Milan Kalas

Thomas Kemper

Alexander Kotsev

Joachim Kreysa

Christophe Lavaysse

Guido Lemoine

Montserrat López-Cobo

Valerio Lorini

Bertin Martens

Marius Miron

Igor Nai Fovino

Domenico Nappo

Fabrizio Natale

Stefano Nativi

Bledi Nukaj

Mauro Pedone

Domenico Perrotta

Martino Pesaresi

Mauro Petrillo

Mikaela Poulymenopoulou

Apostolos Psyllos

Riccardo Righi

Peter Salamon

Sofia Samoili

Carlos Sánchez

Ignacio Sánchez

Vítor Sequeira

Pierre Soille

Spyridon Spyratos

Gary Steri

Daniel Tirelli

Jean Paul Triaille

Songul Tolan

Andrea Toreti

Aris Tsois

Ioannis Vakalis

Marijn van der Velde

Michele Vespe

Clemens Wittwehr

1.3 Abstract

This document presents the contributions presented at the first internal workshop on Artificial Intelligence (AI), organized by the Joint Research Centre (JRC) of the European Commission. This workshop was held on 23rd May at the premises of the JRC in Ispra (Italy), with video-conference to all JRC’s sites. The workshop aimed to gather JRC specialists on AI to share their experience, to identify opportunities for meeting the EC demands on AI, and explore synergies among different JRC’s working groups on AI.

The full-day session workshop was organized around three main topical strands entitled Policy support, New Initiatives and Technology Development. Contributions covered a wide range of areas, including applications of AI to Cybersecurity, Transport, Environment, Health and other specific issues. This report is structured according to those main topics of study.

According to the JRC Director General Vladimír Šucha: ”The workshop was very stimulating and interesting presenting a broad spectrum of activities and competencies across JRC. It gave a great opportunity to build a strong and hopefully useful position in the field of AI/ML”.

While the first part of the workshop was mainly informative, in the second part we collectively discussed about JRC priorities and the set-up of a Community of Practice (now available at https://webgate.ec.europa.eu/connected/groups/community-of-practice-ai-and-big-data) dealing with AI and Big Data. Finally, the preliminary results of the online survey were presented.

All colleagues were excellent in communicating their scientific activity in a flash and efficient way. All the presentations are available on Connected at https://connected.cnect.cec.eu.int/events/13938

1.4 Introduction

This report summarizes the contributions presented at the first JRC workshop on Artificial Intelligence (AI). The workshop, which was held on the 23rd May in Ispra with all the other JRC sites connected online, gathered JRC researchers of different fields of science and technology whose expertise was related to AI. The workshop agenda is included as Annex 1.

The main objectives of the workshop were:

To build knowledge and capacities and meet the European Commission scientific and technical demands on AI.

To identify the current scientific application areas on AI, as developed at the JRC.

To explore opportunities and synergies between different JRC’s working groups on AI.

The workshop was structured according to three strands: Policy support, New Initiatives and Technology Development. They are kept in this document and used to collect the diverse policy areas. The workshop presentations covered different policy areas (i.e. society domains), including:

Agricultural monitoring

Environment

Chemical risk assessment and eHealth

Corporate

Demography and migration

Earth observation

Economy

Ethics and legal aspects

Human-AI interaction

Information processing systems

Security and safety

Transport

In this report, a fact sheet summarizes each presentation (i.e. scientific activity). They are grouped according to the policy area that is considered more relevant. Each policy area is briefly introduced. Information covered by fact sheet deal with: AI Main Stream, Policy area (or Background), main Objectives of the AI application, applied Methodology, and useful References, when applicable.

Artificial Intelligence for policy support

2.1 Security and safety

The Joint Research Centre provides European policy makers with scientific and technological advice on safety, security and stability within the EU and beyond. Research, analysis and in-house developed tools support EU policies on border security, including maritime borders, the fight against organized crime and corruption, as well as implementation of measures against possible biological, chemical and explosive actions.

Modelling, simulation and response capabilities are studied to enhance the security and resilience of physical infrastructures and prevent or respond to cybercrime. Joint Research Centre develops warning systems in the EU to prevent, prepare and respond against all types of natural, technological or man-made disasters.

2.1.1 Machine learning for disaster risk management

George Breyiannis

Main AI Stream

Deep learning/machine learning

Policy areas and legal reference

Humanitarian aid and civil protection

Adapting to climate change

Objectives

To develop data-driven analysis of extreme multi-hazard events by leveraging the plethora of data and models available in the Joint Research Centre and the scientific community in general.

Methodology

Multiple datasets from models, remote sensing and in-situ measurements can be used to train Machine Learning algorithms to detect and recognize extreme events of various hazards.

Aiming to start with a minimum set of data and consider one hazard, the methodology follows a stepwise approach:

1) Collection and/or production of numerical model data for specific events with the highest possible accuracy,

2) Explore all available datasets for possible inclusion in the implementation,

3) Training of a number of Machine Learning algorithms,

4) Evaluation of the different approaches and cross-validation of the results.

The workflow will be based on open source software providing transparency and portability. The goal is to provide a reproducible solution that can provide updated results “on-demand” based on newly acquired data. Possible expansions include additional datasets as sources and application of the methodology to different areas of interest.

Scientific references

Schneider, T., et al., 2017. Earth System Modeling 2.0: A blueprint for models that learn from observations and targeted high-resolution simulations. Geophysical Research Letters 44(24): 12396-12417.

Jones, N., 2017. How machine learning could help to improve climate forecast. Nature 548(7668): 379-380.

Racah, E., et al., 2017. ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.

2.1.2 Social media for flood risk (SMFR)

Francesco Dottori, Milan Kalas, Valerio Lorini, Domenico Nappo, Peter Salamon and Carlos Castillo

Main AI Stream

Deep learning/machine learning, Natural language processing.

Policy areas and legal reference

Crisis preparedness and management, Humanitarian aid and civil protection:

Directive 2007/60/EC (OJ L 288, 6.11.2007) on the assessment and management of flood risks.

Objectives

To produce information that will be integrate into CEMS (Copernicus Emergency Management Services), in particular into EFAS (Early Flood Awareness system). This will allow:

To connect EFAS with soft social media information.

To augment the physical model information with social information about the situation on the ground.

To provide early information on real/observed impacts.

Methodology

Convolutional neural networks for text classification using proper classifiers. In case of languages without trained model, a language-aligned embedding for rapid modelling will be used.

Scientific references

Dottori, F., et al., 2017. An operational procedure for rapid flood risk assessment in Europe. Natural Hazard and Earth System Sciences 17: 1111-1126.

Castillo, C. Big Crisis Data: Social Media in Disasters and Time-Critical Situations. Cambridge University Press, New York, NY, USA, 2016.

Improve emergency preparedness and response to floods (JRC Exploratory Research-Activity Tier).

2.1.3 Image-based identification of tattoo for victims and perpetrators

Apostolos Psyllos

Main AI Stream

Deep learning/machine learning

Natural language processing

Computer vision/Image recognition

Policy areas and legal references

Combating organised crime, Security, Support for global security:

Directive 2011/92/EU (OJ L 335, 17.12.2011) on the assessment of the effects of certain public and private projects on the environment

COM/2015/0185 final: the European Agenda on Security.

JOIN/2013/01 final: Cybersecurity Strategy of the European Union - An Open, Safe and Secure Cyberspace.

Objectives

To apply AI methods and tools for enhancing the forensic evidence, crime investigation efficiency and supporting the law enforcement.

Methodology

Using machine learning & deep learning methods for enhancing the forensic evidence, crime investigation efficiency and supporting the law enforcement

Scientific references

Psyllos, A., Beslay, L. Tattoo Recognition Inception Report. JRC110284, 2018.

Di, X., Patel, V., 2016. Deep Tattoo Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, 2016.

Krizhevsky, A., Sutskever, I., Hinton, G., 2012. ImageNet classification with deep convolutional neural networks. Neural Information Processing Systems, 1: 1097–1105.

2.1.4 Machine learning in child abuse on-line investigation

Pasquale Ferrara

Main AI Stream

Deep learning/machine learning

Video automatic content recognition

Computer vision/Image recognition.

Policy areas and legal references

Combating organised crime, Security, Support for global security:

Directive 2011/92/EU (OJ L 335, 17.12.2011) on the assessment of the effects of certain public and private projects on the environment

JOIN/2013/01 final: Cybersecurity Strategy of the European Union - An Open, Safe and Secure Cyberspace.

Objectives

To use signal processing, biometric, computer vision and machine learning methods for enhancing the forensic evidence, crime investigation efficiency and supporting the law enforcement.

Methodology

Gaussian Mixture based approach for microphone fingerprinting of smart devices

Scientific references

Satta, R., Galbally, J., Beslay, L. State-of-the-art review: video analytics for fight against online child abuse. JRC85864, 2013.

Satta, R., Beslay, L. Camera fingerprinting based on sensor pattern noise as a tool for combatting child abuse on-line. JRC93821, 2014.

Ferrara, P., Beslay, L. Microphone smart device fingerprinting from video recordings. JRC110312, 2017.

2.1.5 Clustering and unsupervised learning for digital forensic

Henrik Junklewitz

Main AI Stream

Deep learning/machine learning

Video automatic content recognition

Computer vision/Image recognition.

Policy areas and legal references

Combating organised crime, Security, Support for global security.

JOIN/2013/01 final (7.2.2013): Cybersecurity Strategy of the European Union - An Open, Safe and Secure Cyberspace

COM/2015/0185 final (28.04.2015): the European Agenda on Security

Objectives

To apply deep learning to enhance the available forensic evidence, crime investigation efficiency and general support of European law enforcement.

To extract criminological investigation leads from large amounts of unordered digital evidence (e.g., audio, images, textual data, fingerprints).

Methodology

Clustering and unsupervised learning algorithms. Exploration of deep generative Bayesian and hierarchical models.

Dimensionality reduction techniques and data visualization. Testing variational autoencoders, Sparsity Methods and T-SNE visualization.

Scientific references

JRC Password Grammar Extraction for Context-Based Targeted Decryption (Limited Distribution). JRC110336, 2017.

JRC Camera fingerprinting from video recordings – Project AVICAO (Limited Distribution). JRC110280, 2017.

JRC Camera fingerprinting based on Sensor Pattern Noise as a tool for combatting Child Abuse on-line. JRC 93821, 2014.

2.1.6 Machine learning for targeted decryption

Iwen Coisel

Main AI Stream

Deep learning/machine learning

Natural language processing

Policy area and legal references

Combating organised crime, Security, Support for global security:

JOIN/2013/01 final (7.2.2013): Cybersecurity Strategy of the European Union - An Open, Safe and Secure Cyberspace

COM/2015/0185 final (28.04.2015): the European Agenda on Security

11th Security Union Progress Report (SUPR) (18.10.2017).

Objectives

To use machine and deep learning to enhance crime investigation efficiency and support digital forensics and law enforcement in general.

Methodology

To combine several machine learning algorithms (e.g., statistical natural language processing) for generating candidate passwords during a targeted decryption process.

Scientific references

JRC Context-Based Approach for Targeted Decryption – Phase 2 (Limited Distribution). JRC111907, 2018.

JRC Password Grammar Extraction for Context-Based Targeted Decryption (Limited Distribution). JRC110336, 2017.

JRC Context-Based Approach for Targeted Decryption – Phase 1 (Limited Distribution). JRC105256, 2016.

2.1.7 Artificial intelligence for customs risk management

Aris Tsois, Enrico Checchi, Raul Fidalgo Merino, José Antonio Cotelo Lema, Valentin Enescu, Mikaela Poulymenopoulou, Lorenzo Gabrielli, Mauro Pedone, Tatyana Dimitrova, Bledi Nukaj

Main AI Stream

Deep learning/machine learning.

Policy area and legal references

Combating organised crime, Fraud prevention:

Reg. (EEC) No 2913/92 establishing the Community Customs Code

Reg. (EC) No 450/2008 laying down the Community Customs Code (Modernised Customs Code)

Reg. (EC) No 515/97 as updated by Reg. (EC) No 2015/1525 on mutual assistance between the administrative authorities of MS and cooperation between the latter and the Commission to ensure the correct application of the law on customs and agricultural matters

Objectives

To extract knowledge on the transportation of goods into/from the EC Custom Union using large sequence of datasets from shipping containers logistic events.

The automatic detection of anomalies and risky behaviours (for security or financial fraud) in the cross-border transportation of goods.

Methodology

To follow a stepwise approach, namely: 1) data warehousing and collecting and preparing the data for analysis, 2) automated processing of all data streams for the extraction of key information using conditional random fields, frequent sequence mining, decision trees and deep learning, 3) usage of the extracted key information for the automatic detection of anomalies and risky behaviours, and 4) calculation of quality measures to estimate errors of the applied algorithms.

Scientific references

Poulymenopoulou. M., Tsois, A., 2017. Customs risk analysis through the ConTraffic visual analytics tool. 2017 European Intelligence and Security Informatics Conference (EISIC), Athens, Greece, pp. 107-114.

Tsois, A., et al. Using container status messages to improve targeting of high-risk cargo containers, in: Research Track at the World Customs Organization Technology and Innovation Forum 2015.

Chahuara, P., et al., 2014. Inferring itineraries of containerized cargo through the application of conditional random fields, in: Proceedings of the 2014 IEEE Joint Intelligence and Security Informatics Conference (JISIC 2014), The Hague, Netherlands, pp. 137-144.

2.1.8 Artificial intelligence for lightweight identification and authentication of electronic devices

Gianmarco Baldini, Raimondo Giuliani and Gary Steri

Main AI Stream

Deep learning/Machine learning

Policy areas and legal references

Support for global security, Fraud prevention:

COM/2015/0192 final (6.5.2015): A Digital Single Market Strategy for Europe

JOIN/2017/0450 final (13.9.2017): Resilience, Deterrence and Defence: Building strong cybersecurity for the EU

Regulation (EU) No 165/2014 (OJ L 60, 28.2.2014) on on tachographs in road transport, repealing Council Regulation (EEC) No 3821/85 on recording equipment in road transport and amending Regulation (EC) No 561/2006 of the European Parliament and of the Council on the harmonisation of certain social legislation relating to road transport

Objectives

To evaluate the application of Machine learning/Deep learning for the identification and authentication of electronic devices on the basis of their physical properties.

To overcome the limitations of authentication solutions based on cryptography.

To evaluate the feasibility of physical layer authentication in regards to reproducibility, robustness to environment effects (temperature, noise) and other factors.

Methodology

Physical properties of electronic devices can be used to authenticate them and complement cryptographic means (multi-factor authentication). Also, to replace them when cryptographic means are not applicable (too expensive or difficult to deploy).

Scientific references

Borio, D., et al., 2017. GNSS receiver identification using clock-derived metrics. Sensors 17(9): 2120.

Baldini, G., Giuliani, R., Cano Pons, E., 2017. An analysis of the privacy threat in vehicular Ad Hoc networks due to radio frequency fingerprinting. Mobile Information Systems. Article ID 3041749.

Baldini, G., Steri, G., 2017. A survey of techniques for the identification of mobile phones using the physical fingerprints of the built-in components”, in: IEEE Communications Surveys and Tutorials 19(3): 1761-1789.

2.1.9 Machine learning for password strength estimation

Javier Galbally, Iwen Coisel and Ignacio Sánchez

Main AI Stream

Deep learning/Machine learning

Natural language processing

Policy area and legal reference

Fraud prevention:

JOIN/2013/01 final: Cybersecurity Strategy of the European Union - An Open, Safe and Secure Cyberspace

Objectives

To develop a reliable and adaptable strength metric for passwords.

Methodology

A multimodal strength metric that combines several imperfect individual metrics to benefit from their strong points in order to overcome many of their weaknesses.

Scientific references

Galbally, J., Coisel, I., Sanchez, I., 2017. A new multimodal approach for password strength estimation, Part I: Theory and Algorithms. IEEE Transactions on Information Forensics and Security, 12:2829-2844.

Galbally, J., Coisel, I., Sanchez, I., 2017. A new multimodal approach for password strength estimation, Part II: Experimental Evaluation. IEEE Transactions on Information Forensics and Security, 12: 2845-2860.

Galbally, J., Coisel, I., Sanchez, I., 2014. A probabilistic framework for improved password strength metrics. IEEE 2014 International Carnahan Conference on Security Technology (ICCST), Rome, Italy, pp.1-6.

2.2 Migration

One of the main challenges to effective migration policies is working with traditional statistics, including census data that can often be outdated.

Social media platforms represent innovative sources that can provide up to date, dynamic information on migration, mobility trends and statistics. Estimates of social media users who are classified as expatriates can be a timely, low-cost, and almost globally available source of information for estimating stocks of international migrants.

2.2.1 Harnessing big data for migration

Michele Vespe, Fabrizio Natale, Spyridon Spyratos

Main AI Stream

Deep learning/machine learning.

Policy area and legal reference

Migration and asylum

Objectives

To harness the potential of big data and alternative data sources to fill migration data gaps

Methodology

Analyses of LinkedIn, Twitter, air passenger flows and mobile phones data as additional sources to census data have shown a significant value in complementing official statistics, in terms of timeliness and granularity.

This approach allowed for the timely capture of the increase of Venezuelan migrants in Spain and to extent national statistics to 1st Jan, 2018 (latest EU official statistics on migration date back to 1st Jan, 2017). Facebook advertising platform can be used to nowcast such figures almost in real time.

Scientific references

Spyratos, S., et al., 2018. Migration data using social media: a European perspective. JRC112310, 2018.

Rango, M., Vespe, M., 2017. Big Data and Alternative Data Sources on Migration: from Case Studies to Policy Support. Workshop summary report, 2017.

Vespe, F., Natale, F., Pappalardo, L., 2017. Data sets on irregular migration and irregular migrants in the European Union. Migration Policy Practice 7 (II): 26 - 33.

2.3 Agriculture monitoring

To meet the needs of the world’s growing population and taking into account changing consumption patterns, natural resources scarcity and the impact of climate change, food production must be increased in an economically, environmentally, and socially sustainable way.

Agricultural monitoring is carried out at the JRC mainly to distinguish, identify and measure the main crop production areas in Europe, estimate production early in the year and check the validity of farmers’ applications for European Union subsidies. The European Commission uses satellite earth-observation data as a cost-efficient way of gathering the necessary monitoring information.

2.3.1 Machine learning application on sentinel time series for Common Agricultural Policy (CAP) monitoring

Guido Lemoine, Wim Devos and Raphael d’Andrimont

Main AI stream

Deep learning/machine learning

Policy area and legal reference

Agriculture and rural development:

C/2018/2976 (OJ L 125, 22.5.2018) as regards modification of single applications and payment claims and checks

Objectives

CAP monitoring to confirm compliance with CAP (common agricultural policy) aid schemes.

Methodology

Sentinel-1 provides consistent time series for any EU location (120-250 images per year). These are combined with large agricultural parcels sets (~0.5-1 M features) with declared practice. Machine learning is applied to identify outliers and estimates cross-checked with other data sets (e.g. Sentinel-2).

AI technologies used: Google Earth Engine (data composition and reduction), Tensorflow (Machine learning on time series), and Python (pre- and post-analysis, cross-referencing and reporting).

Scientific references

Devos, W., et al. Discussion document on the introduction of monitoring to substitute OTSC. JRC108353, 2017.

2.4 Corporate

The Earth Observation and Social Sensing Big Data (EO&SS@BD) pilot project was launched in January 2015 as a response to the need for JRC to pursue a dedicated approach to Big Data and address the data flows originating from the EU Copernicus programme.

As a result, the setting up of a JRC prototype platform for big data storage, management, processing, and analysis was initiated in 2016. Its specifications were primarily based on the requirements of the JRC knowledge production units.

During the period 2017-2018, the Joint Observation Data and Processing Platform (JEODPP) was developed to serves the needs of JRC policy support activities requiring big data capabilities for analysing geospatial data. The platform is continuously developed in the framework of the JRC Big Data Analytics project.

2.4.1 The Joint Research Centre Big Data Platform (JEODPP)

The big data team(Pierre Soille)

Main AI Stream

Deep learning/Machine learning

Computer vision/Image recognition

Pattern analysis

Policy area and legal references

Information and Communication Technologies:

COM(2016) 178 final: European Cloud Initiative - Building a competitive data and knowledge economy in Europe

C(2016) 6626 final: Data, Information and Knowledge Management at the European Commission

Objectives

Processing of big data streams into policy relevant indicators and information.

Methodology

Distillation process is enabled by the JEODPP, a versatile multi-petabyte scale platform which offers a range of service layers for collaborative working as well as data and knowledge sharing.

Scientific references

Soille, P., et al., 2018. A versatile data-intensive computing platform for information retrieval from Big Geospatial Data. Future Generation Computer Systems 81: 30-40.

Kempeneers, P., Soille, P., 2017, Optimizing Sentinel-2 image selection in a Big Data context. Big Earth Data 1: 145-158.

Soille, P., 2008. Constrained connectivity for hierarchical image partitioning and simplification. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(7): 1132 – 1145.

2.5 Transport

Intelligent Transport Systems (ITS) are vital to increase safety and tackle Europe’s growing emission and congestion problems. They can make transport safer, more efficient and more sustainable by applying various information and communication technologies to all modes of passenger and freight transport. Moreover, the integration of existing technologies can create new services.

In the coming years, the digitalisation of transport in general, and ITS in particular, are expected to take a leap forwards. ITS will be used to improve journeys and operations on specific and combined modes of transport.

The EC also works to set the ground for the next generation of ITS solutions, through the deployment of Cooperative-ITS, paving the way for automation in the transport sector. Cooperative-ITS are systems that allow effective data exchange through wireless technologies so that vehicles can connect with each other, with the road infrastructure and with other road users.

2.5.1 Artificial intelligence for traffic management

Panayotis Christidis and Álvaro Gómez Losada

Main AI Stream

Deep learning/Machine learning

Policy area and legal reference

Transport themes:

Directive 2010/40/EU (OJ L 207, 6.8.2010) on the framework for the deployment of Intelligent Transport Systems in the field of road transport and for interfaces with other modes of transport

Objectives

To use vehicle-to-infrastructure data flows to improve traffic management.

To estimate impact of take up rates of autonomous vehicles in fleet.

Methodology

To use city level congestion data (see figure a.) and connected vehicle GPS data (see figure b.) to train a Machine learning model (see figure c.).

Scientific references

Final Report: Exploratory Research Project Autonomous Road Transport (ART): Project 209, WPK 4205 in JRC WP 2016/2017 (JRC 110276)

Gómez Losada, A. Data science applications to connected vehicles: Key barriers to overcome. JRC Science for Policy Reports. JRC 108572, 2017.

Condeco, A., Christidis, P., Dijkstra, L., 2016. Travel speed changes along the European core road network for the period 1960–2030: an application of octilinear cartograms. Journal of Maps 12(5): 1214-1217.

2.5.2 Autonomous cars and traffic safety

Panayotis Christidis and Álvaro Gómez Losada

Main AI Stream

Deep learning/Machine learning

Computer vision/Image recognition

Policy area and legal reference

Shifting to more efficient transport modes:

COM/2018/283 final (17.5.2018) on the road to automated mobility: An EU strategy for mobility of the future

Objectives

To evaluate accuracy and applicability of different deep learning approaches to estimate risks of autonomous driving levels.

Methodology

To use train & test Imagenet data (a) along with Benchmarking Deep learning models (b) and AV disengagement reports (c).

Scientific references

Final Report: Exploratory Research Project Autonomous Road Transport (ART): Project 209, WPK 4205 in JRC WP 2016/2017 (JRC 110276)

Gómez Losada, A. Data science applications to connected vehicles: Key barriers to overcome. JRC Science for Policy Reports. JRC 108572, 2017.

2.6 Digital economy

The rapid decline in computing costs, the emergence of the Internet as a communication tool, the rapid development of the mobile internet, the proliferation of day-to-day applications, and the increasing role of internet-based social networks and commercial platforms, have greatly affected the functioning of the economy and have profoundly affected businesses, public organisations, and personal life.

Online retailers are a growing force in consumer retail markets. Their share of sales continues to grow, prompting economists to wonder about their impact on inflation.

Much of the attention among central bankers and the press have been on whether the competition between online and traditional retailers is reducing retail mark-ups and putting downward pressure on prices.

2.6.1 Semi-supervised classification and forecasting methodologies in Amazon Marketplace

Álvaro Gómez Losada and Néstor Duch Brown

Main AI Stream

Machine learning

Data science

Policy area and legal reference

Digital economy and society:

COM/2018/ 237 final on Artificial Intelligence for Europe

Objectives

To support the European Digital Single Market (eDSM) research on online platforms and their use of data and algorithms.

To detect differences and common grounds of sellers using algorithmic pricing (if any) in United States versus Spain and Italy.

To understand the role of algorithms and data in shaping sellers’ strategies and consumer behavior.

To analyzing whether algorithmic pricing can have anti-competitive results, namely exclusionary practices such as collusion.

Forecast the likelihood of a seller to be displayed at the Amazon’s Buy Box.

Methodology

The main methodology components are:

Positive and unlabeled learning (PU learning) with statistical boosting algorithms. Data manipulation with Pandas module (Python).

Web scraping of Amazon marketplace in Italy and Spain.

One-class Support Vector Machine and other classifier algorithms.

Ensembles of classifiers (meta-learning).

Microeconomic theory.

References

An empirical analysis of algorithmic pricing on Amazon Marketplace. WWW’16 Proceedings of the 25th International Conference on Word Wide Web, 1339-1349. Montreal, Canada, April 11-15, 2016.

2.7 Information processing systems

Being able to read news from other countries and written in other languages allows readers to be better informed. It allows them to detect national news bias and thus improves transparency and democracy.

Existing online translation systems are thus a great service, but the number of documents that can be processed is restricted and submitting documents means disclosing the users’ interests and their (possibly sensitive) data to the service-providing company.

2.7.1 Statistical and neural machine translation

Camelia Ignat

Main AI Stream

Deep learning/Machine learning

Natural language processing

Policy area and legal reference

Information and communication technologies:

COM/2018/ 237 final on Artificial Intelligence for Europe

Objectives

To support innovative policy-making.

To perform real-time machine translation for news articles from a source language into English.

Methodology

Statistical machine translation main elements:

Find most probable sentence “e” in the target language given a foreign language sentence “f”.

Parameters from data – EM (Expectation-Maximization) algorithm.

Tuning using Minimum Error Rate Training.

Neural machine translation main elements:

Large artificial neural networks to predict likelihood of a sequence of words (entire sentences).

Deep learning and representation learning.

A single sequence model that predicts one word at a time.

Scientific references

Turchi, M., Atkinson, M., Wilcox, A., et al., 2012. ONTS: Optima News Translation System. Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. Avignon, France, pp 25-30

New initiatives

3.1 AI and human behaviour interaction

Over the last few years, thanks to an increase in data availability and computing power, AI techniques have been applied to different research problems related to computer vision, natural language processing, music processing or bioinformatics.

Some of these techniques are said to surpass human-level performance (e.g. image recognition) and model highly abstract human concepts such as emotion or culture. The practical exploitation of AI algorithms brings up a discussion on the impact of these algorithms into the ways human behave.

3.1.1 Human behaviour and machine intelligence (HUMAINT)

Emilia Gómez, Marius Miron, Songul Tolan, Vicky Charisi and Bertin Martens https://ec.europa.eu/jrc/communities/community/humaint

Main AI streams

Deep learning/Machine learning

Human-Robot interaction

Fairness, transparency and accountability of AI systems

Algorithm-supported decision making

Music and creativity

Background

HUMAINT is an interdisciplinary project within the Joint Research Centre’s Centre for Advanced Studies aiming to understand the impact of machine intelligence on human behaviour, with a focus on cognitive and socio-emotional capabilities and decision making.

Objectives

To advance the scientific understanding of machine and human intelligence.

To study the impact of algorithms on humans, focusing on cognitive and socio-emotional development and decision making.

To provide insights to policy makers with respect to the previous issues.

Methodology

The approach is interdisciplinary and is supported by the collaboration with an external group of experts.

Scientific references

Gómez, E. (editor), Assessing the impact of machine intelligence on human behaviour: an interdisciplinary endeavour, Proceedings of 1st HUMAINT workshop, Barcelona, Spain, March 5-6, 2018, edited by European Commission, Seville, 2018, JRC111773.

Fernández-Macías, E., et al., 2018. A multidisciplinary task-based perspective for evaluating the impact of AI autonomy and generality on the future of work. Workshop on architectures and evaluation for generality, autonomy and progress in AI, IJCAI-ECAI 2018, AAMAS 2018, AND ICML 2018.

3.1.2 How and why artificial intelligence cannot be disconnected from human2human relationships?

Nicole Dewandre

Main AI stream

Philosophy

Background

A human-centric approach to AI requires reconsidering this modern legacy and changing our entrenched conceptualisations of humans as rational subjects (individuals).

This can be achieved by embracing the alternative conceptualisation of relational selves (persona), which blends together human’s organic, rational and plural condition.

This reconceptualization makes room for grasping the political challenges raised by AI, notably the redefinition of the conditions enabling fairness and resilience in a hyper-connected society.

Objectives

To explore the mirroring effects between our representations of AI, on the one hand, and of humanness, on the other hand.

Methodology

The current approach to AI stems from the modern legacy of considering humans as rational subjects, either as uncompromising autonomous individuals or as mere functions or instruments.

Main methodological elements include:

Dialogue with policy-makers and AI practitioners.

Workshops and other forms of conversations (notably with the CAS projects on digital transformation).

SciArt partnerships.

Desk research, based on literature review and critical discourse analysis.

Scientific references

Dewandre, N., 2017. Humans as Relational Selves, AI & Society: 1-4.

Dewandre, N., 2015. The Human Condition and the Black Box Society. B2O: the online community of the boundary 2 editorial collective. [Internet: https://www.boundary2.org/2015/12/dewandre-on-pascal/]

The Onlife Manifesto: Being human in a Hyperconnected Era. Floridi, L. (Ed.) (https://www.springer.com/gp/book/9783319040929).

3.2 Legal issues

Ethics and law are inextricably linked in modern society, and many legal decisions arise from the interpretation of various ethical issues. AI adds a new dimension to these questions.

Systems that use AI technologies are becoming increasingly autonomous in terms of the complexity of the tasks they can perform, their potential impact on the world and the diminishing ability of humans to understand, predict and control their functioning. It is common to underestimate the real level of automation of these systems, which have the ability to learn from their own experience and perform actions beyond the scope of those intended by their creators.

3.2.1 Legal issues raised by Artificial intelligence

María Iglesias, Jean Paul Triaille and Amanda Anderberg

Main AI Stream

Regulation

Background

In the framework of the JRC collaboration with the European Institute of Innovation and Technology (EIT), Unit I.4 is working on a joint project to examine regulatory barriers in areas where a number of EIT start-ups are active.

The unit is also contributing to the JRC reports on the State of Play of Digital transformation and AI.

Objectives

To examine legal and regulatory implications raised by AI technologies in the field of e-health, automated cars and text and data mining.

To analyze the legal implications of the digital transformation and of AI in particular as regards the IP and data regulatory framework.

Methodology

The methodological approach elements include:

Desk research.

Survey and case studies on a number of AI start-ups and companies (active in self-driving cars, eHealth and Text and Data Mining).

Experts’ and stakeholders’ workshops.

Reference

Triaille, J.P., de Meeûs d’Argenteuil, J., De Francquen, A. Study for the EC on the legal framework of text and data mining, March 2014.

3.3 Chemical safety assessment

The use of animal experiments to advance science in general, and medicine and human safety assessment in particular, has been a contentious subject for many years. Although they have informed numerous scientific achievements, the objections to such studies on both ethical and scientific grounds have found strength with the increase of new alternative approaches which do not require the use of animals.

3.3.1 Toxicity prediction and risk assessment of chemicals

David Asturiol

Main AI streams

Deep learning/Machine learning

Computer vision

Image recognition

Background

Chemical safety assessment has traditionally been based on animal testing but the EU has been promoting for many years the replacement, reduction, and refinement of animal testing. Further it is also considered that traditional risk assessment approaches are insufficient to adequately predict the potential risk associated with any given substance, especially when considering normal life low-dose exposure.

Therefore new prediction models are needed in a new intelligent and more efficient safety assessment, based on in vitro testing in combination with computational modelling.

Objectives

To use AI and Machine learning to build new and more accurate toxicity prediction models that can be used in a chemical risk assessment context

To improve current in vitro models and to derive new knowledge.

Methodology

The methodological elements include:

Make use of newly created and recently available databases

New prediction models

Improve in vitro methods

Obtain knowledge from new results

Design new experiments

Scientific references

Luechtefeld, T., et al., 2018. Machine learning of toxicological big data enables read-across structure activity relationships (RASAR) outperforming animal test reproducibility. Toxicological Sciences 165: 198-212.

EURL ECVAM status report on the development, validation and regulatory acceptance of alternative methods and approaches (2017).

Asturiol, D., Casati, S., Worth, A., 2016. Consensus of classification trees for skin sensitisation hazard prediction. Toxicology in Vitro 36: 197–209.

3.4 JRC’s Institutional History Project (iHIP)

2017 marked JRC’s 60th anniversary and this presented an opportunity for JRC to celebrate its achievements in science for society and evidence-informed policy-making. To commemorate this milestone, JRC has created the JRC’s Institutional History Project (iHIP) which aims to capture JRC’s history by inviting all current and former JRC staff to share their own experiences in working for the organisation, but also, by creating the JRC historical archive.

In an innovative approach, historical archive will be digitised and subsequently archived with the help of automatic text mining and analysis, based on AI and machine learning. This growing corpus of digitised documents will be explored with innovative analytical techniques to identify and visualise interconnections and trends.

3.4.1 Artificial intelligence for Institutional History Project (iHiP)

Joachim Kreysa

Background

This project is primarily triggered by the internal interest of the Joint Research Centre to write down its own institutional History and for this to make use of its paper legacy. The latter is estimated to be 13 km of documents, ranging back to the early days of the JRC. It could only be indirectly linked to the EU policy interest to develop a European narrative, of which the role of the JRC could be a component.

Objectives

To use of JRC’s AI competences to solve the problem of how to handle and exploit its paper legacy.

Development, optimisation and implementation of a software pipeline for a (semi-) automatic archiving of digital documents, including deciding to retain or not the paper original. The software is expected to make use of deep learning and natural language processing.

Methodology

The methodological approach elements include:

Characterization of digital documents in line with e-Domec rules

Contextual content analysis to support application of retention policy of the Commission

Content analysis to ensure that documents of historic value are kept, even if not required by the Commission’s retention policy

Content analysis in support to the historic exploitation of the digitized JRC legacy

References

Call for expression of interest, available at: https://ec.europa.eu/jrc/sites/jrcsh/files/jrc-ihip-call-2016.pdf

3.5 e-Health

EU is facing common challenges related to the increase in chronic diseases prevalence. Besides, limited human and financial resources for ensuring sustainability of their health systems, and meeting the growing demands of ageing populations.

EU is also facing common challenges related to cross-border health threats. New opportunities are arising from big data and improved data analytics capabilities. Personalised medicine, use of clinical decision support systems by health professionals and use of mobile health tools for individuals to manage their own health and chronic conditions may benefit from these technologies. Their use on health and healthcare systems can increase their efficiency, improve quality of life and unlock innovation in health markets.

3.5.1 Big data + Artificial Intelligence + Health = eHealth

Mauro Petrillo

Main AI Stream

Deep learning/Machine learning

Natural language processing

Computer vision/Image recognition

Virtual personal assistants

Smart robots

Background

E-Health is a relatively recent healthcare practice supported by electronic processes and communication.

Big Data in Health refers to large routinely or automatically collected datasets, which are electronically captured and stored. It is reusable in the sense of multipurpose data and comprises the fusion and connection of existing databases for the purpose of improving health and health system performance.

AI approaches employing machines to sense and comprehend data like humans has opened up previously unavailable or unrecognised opportunities for clinical practitioners and health service organisations.

Objectives

To use of AI technologies for enhancing the standard of health and care.

To decrease citizen disease risks by improving prevention and treatment techniques.

Methodology

In the context of JRC Transversal Activity on Evidence-based Health, to analyse proper assessment of both which technology is used and how is used in the digital transformation of health.

Scientific references

Torrent-Sellens, J., Díaz-Chao, A., Soler-Ramos, I., et al., 2018. Modeling and predicting outcomes of eHealth usage by European physicians: multidimensional approach from a survey of 9196 general practitioners. Journal of Medical Internet Research 20(10): e279.

Novillo-Ortiz, D., Elsy , M.D., D’Agostino, M., 2018. Digital health in the Americas: advances and challenges in connected health. Health IT, systems and process innovations 4:123-127

3.5.2 Artificial Intelligence for Chemical Risk Assessment: AI4CRA

Clemens Wittwehr

Main AI Stream

Deep learning/Machine learning

Background

Challenges in Chemical Risk Assessment include the reliability of the models employed to get the risk data. Such models usually involve a large number of variables and deal with high amounts of uncertainty. Therefore, there is a strong need for a powerful tool to cope with that uncertainty.

Objectives

To find out how AI can support Chemical Risk Assessment, especially in the fields of Expert Knowledge Elicitation and Systematic Reviews.

Methodology

It is based on topics identified at a first workshop jointly organized with Aalto University (4-5 April, 2018; Helsinki, Finland):

Finding Experts

Facilitating Collaboration

Identifying Problems

Supporting Evaluation

Process Simulation

Evidence Gathering

Building Cognitive Models

Systematic Review

Knowledge Discovery

Next steps:

Explore topics off-line

Reconvene for targeted approach

Scientific references

Grabar, N., Wandji Tchami, O., Maxim, L., 2014. Machine learning-based detection of chemical risk. Studies in Health Technology and Informatics 205:725-729.

Ostrom, L.T and Wilhelmsen, 2012. C.A, Risk Assessment: Tools, Techniques and their Applications. John Wiley & Sons, NJ, USA.

Technology development

4.1 Pattern recognition in climate time series

A drought early warning system’s main purpose is to warn local communities when there is risk of a drought, improving preparedness and decreasing risks associated with crop and food loss.

This technology is particularly important for agriculture and water resource management. Effective warning systems require drought monitoring using appropriate drought indicators, meteorological data and forecasts, public awareness and education.

Developing a robust early warning system for drought events is a key challenge for modellers and forecasters. JRC provides practical support to European Commission with respect to monitoring, analysing and forecasting of droughts through the European Drought Observatory (EDO).

4.1.1 Using genetic algorithm in pattern classification to predict drought events

Andrea Toreti and Christophe Lavaysse

Main AI stream

Deep learning/Machine learning

Computer vision/Image recognition.

Policy areas and legal reference

Agriculture, Climate action and Global Safety and Security:

Directive 2007/2/EC (OJ L 108, 25.4.2007): establishing an Infrastructure for Spatial Information in the European Community (INSPIRE)

Objectives

To identify and classify large-scale atmospheric patterns associated with climate extremes and improve forecasting systems.

To improve the quality of meteorological data by identifying non-climatic change points.

Methodology

To use Genetic Algorithm (Hidden Markov Models) to identify when a system changes its status.

Scientific references

Lavaysse, C., et al., 2018. On the use of weather regimes to forecast meteorological drought over Europe. Natural Hazards and Earth System Sciences 18: 3297-3309.

Toreti, A., et al., 2012. A novel approach for the detection of inhomogeneities affecting climate time series. Journal of Applied Meteorology and Climatology 51: 317–326.

4.2 Improving data availability in spatial data

Monitoring with high resolution land cover is a key task that is required in a number of applications (e.g., urban planning, health monitoring and ecology). At the moment, some operational products from satellite are available to assess this information, but the frequency of updates is still limited despite the fact that more and more very high resolution data are acquired.

Deep learning is a class of machine learning algorithms that use a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. In deep learning, each level learns to transform its input data into a slightly more abstract and composite representation.

4.2.1 Deep learning for improving data availability in Spatial Data Infrastructures

Blagoj Delipetrev, Alexander Kotsev and Stefano Nativi

Main AI Stream

Deep learning/Machine learning

Computer vision/Image recognition

Policy area and legal references

European market for space data:

Directive 2007/2/EC (OJ L 108, 25.4.2007): establishing an Infrastructure for Spatial Information in the European Community (INSPIRE)

Regulation (EU) No 377/2014 (OJ L 122, 24.4.2014): establishing the Copernicus Programme and repealing Regulation (EU) No 911/2010

Objectives

To improve data availability with deep learning using Copernicus (Sentinel-2) and Infrastructure for Spatial information in Europe (INSPIRE).

Methodology

The main methodological approach elements are:

Improve data availability through the use of Deep Learning based on Copernicus (Sentinel 2) and INSPIRE

Compare the results of Deep Learning with traditional techniques (Random Forest).

Build own capacity in AI

Scientific references

Djerriri, K., Adjouj, R., Attaf, D. Convolutional neural networks for the extraction of built-up areas from Sentinel-2 images. Association for Geographic Information Laboratories in Europe 2017.

Lefebvre, A., Sannier, C., Corpetti, T., 2016. Monitoring urban areas with Sentinel-2a data: application to the update of the Copernicus high resolution layer imperviousness degree. Remote Sensing 8 (7): 606.

4.3 Machine learning for geospatial data

The JRC’s monitoring of agriculture using remote sensing started in 1988, with the aim of providing independent and timely information on crop areas and yields using emerging space technologies. The JRC focuses on anticipating and responding to the evolving needs related to agricultural monitoring. Through the years, JRC activities have led to several innovative developments such as control with remote sensing (CwRS), the Digital Land Parcel Identification System (LPIS) and parcel area measurement using Global Navigation Satellite System (GNSS) devices.

Symbolic Machine Learning is a spatial data mining technology for automatic processing, analytics and knowledge extraction from large amount of heterogeneous spatial data. It can be adapted to other application domains and is the core algorithm for the extraction of information for describing the status of human settlements and their dynamics, both in terms of built-up and population through automatic image information. The Symbolic Machine Learning framework shows some analogies with data analytics tools already developed and largely applied in other disciplines, such as the genetic association techniques.

4.3.1 Agricultural in-site data sourced from street-level imagery analysed with computer vision

Raphael d’Andrimont, Guido Lemoine and Marijn van der Velde

Main AI stream

Deep learning/Machine learning

Computer vision/Image recognition

Policy area and legal reference

Agriculture and rural development:

C/2018/2976 (OJ L 125, 22.5.2018) as regards modification of single applications and payment claims and checks

Objectives

To explore the capacity of crowdsourced geotagged street-level imagery to provide relevant and timely information along the growing season on:

– Crop type (e.g. maize, wheat, grassland).

– Crop phenology (e.g. emergence, tillering).

– Agricultural practices (e.g. mowing, plowing, fertilizing),

– Ecological focus areas (e.g. fallow land, field margins, hedges and trees).

To investigate the potential of deep learning algorithms to recognize and classify digital photographs according to crop type, phenology, landscape elements, and farm activity.

To unearth and provide a novel source of ground-truth and in-situ data whose derivation is easily applied to other sectors.

Methodology

Main methodological elements include:

The use of geotagged street-level pictures:

– On-going campaign in Germany and The Netherlands.

– Street view crowdsourcing (e.g., Mapillary, GSV).

– Social-media harvesting.

The development of Computer vision modules, by using:

– Transfer learning and Convolutional Neural Network.

– Retrain of MobileNet.

– TensorFlow GPU.

References

d’Andrimont, R., Lemoine, G., Van der Velde, M. Harnessing big data for agriculture monitoring: combining remote sensing. Open Access Data and Crowdsourcing, Big Data from Space 2017.

d’Andrimont, R., Lemoine, G., Van der Velde, M. Crop monitoring with Sentinels and crowd-sourced street level imagery. EGU General Assembly 2018.

d’Andrimont, R., Lemoine, G., Van der Velde, M., 2018. Targeted grassland monitoring at parcel level using sentinels, street-level images and field observations. Remote Sensing 10: 1300.

4.3.2 Big earth data analytics with symbolic machine learning

Martino Pesaresi, Christina Corban and Thomas Kemper

Main AI Stream

Deep learning/Machine learning

Policy area and legal reference

Information and Communication Technologies:

Regulation (EU) No 377/2014 (OJ L 122, 24.4.2014): establishing the Copernicus Programme and repealing Regulation (EU) No 911/2010

Objectives

To apply a new generic supervised classification framework based on symbolic Machine learning techniques that may contribute to solving geo-spatial big data analytical problems.

Methodology

The methodology makes use of symbolic Machine learning models and evidence-based techniques.

Scientific references

Pesaresi, M., Syrris, V., Julea, A.M., 2016. A new method for earth observation data analytics based on Symbolic Machine Learning. Remote Sensing 8(5):399.

Pesaresi M., Syrris, V., Julea A.M. Benchmarking of the Symbolic Machine Learning classifier with state of the art image classification methods - application to remote sensing imagery. Publications Office of the European Union, EUR 27518 EN, 2015.

Corbane C., Pesaresi M., et al., 2017. Big earth data analytics on Sentinel-1 and Landsat imagery in support to global human settlements mapping. Big Earth Data 1: 118-144.

4.4 Swarm robotics

Swarm robotics refers to study the design of robots, their physical body and their controlling behaviours. It is inspired but not limited by the emergent behaviour observed in social insects, called swarm intelligence.

Relatively simple individual rules can produce a large set of complex swarm behaviours. A key-component is the communication between the members of the group that build a system of constant feedback. The swarm behaviour involves constant change of individuals in cooperation with others, as well as the behaviour of the whole group.

Unlike distributed robotic systems in general, swarm robotics emphasizes a large number of robots, and promotes scalability, for instance by using only local communication. That local communication can be achieved by wireless transmission systems, like radio frequency or infrared.

4.4.1 Robust clustering for swarm robotics

Domenico Perrotta

Main AI Stream

Deep learning/Machine learning

Natural language processing

Smart robots.

Policy area and legal reference

Information and Communication Technologies:

COM/2018/ 237 final: Artificial Intelligence for Europe

Objectives

To make unsupervised methods robust to the presence of anomalies in the data using swarm robotics.

Methodology

Robust methods (robust clustering, in particular) will be used to make the coordination algorithms capable to produce reliable communications even in presence of a reasonable number of failures (even 50%).

Scientific references

Hamann, H. Swarm Robotics: A Formal Approach, Springer, New York, 2018.

Arvin, F., Murray, J.C., Licheng S. et al., 2014. Development of an autonomous micro robot for swarm robotics,” 2014 IEEE International Conference Mechatronics and Automation (ICMA). Tianjin, China, pp. 635-640

4.5 TES methodology in the PREDICT project

Europe should remain competitive and make the most of the opportunities offered by technology. In order to achieve this objective, there is a significant need for analytical methods based on objective data that will describe the changing technological ecosystems, and are capable of identifying gaps, weaknesses and strengths in European research and industry.

The Techno-Economic Segment Analytical Approach, developed by the JRC, responds to current needs and provided a synthetic overview of technological ecosystems and their dynamics.

4.5.1 TES – Techno-Economic segments analytical approach

Predict Team: Melisande Cardona, Giuditta de Prato, Montserrat López-Cobo, Riccardo Righi and Sofia Samoili

Main AI Stream

Deep learning/Machine learning

Natural language processing

Policy area and legal reference

Digital economy and society:

COM/2018/ 237 final: Artificial Intelligence for Europe

Objectives

Analysis of Information and Communication Technology enabled ecosystems.

Monitoring the EU competitiveness

Measuring Digital Transformation of the economy

Providing a timely representation of an integrated and very dynamic technological domain, not captured by official statistics or standard classifications

Answering to policy needs and supporting policy initiatives

Mapping EU & global hotspots (top countries, regions and cities)

Benchmarking the most relevant players

Capturing and exploring heterogeneous interactions in multiple dimensions

Acknowledgement of investment strengths/needs

Methodology

Main elements include:

Capturing the ecosystem (data collection and preprocessing)

Analysis of the ecosystem (technology & industry)

NLP approaches (unsupervised content description & representation)

Semantic representation (ontology based), using NN-based language model (word embedding model).

Topic modelling

Multidimensional representation: technological (sub)domains, methods, applications, tools, products and industry.

References

Samoili, S., Riccardo R., Lopez-Cobo M., De Prato, G., 2017. Modelling Emerging Topics in a Techno-Economic Segment (TES) Network, XII Workshop on Artificial Life and Evolutionary Computation, Venice (Italy).

PREDICT Team, 2018, Photonics Techno-Economic Segments (TES) Analysis – Selected TES Indicators (forthcoming).

PREDICT Team, 2018, Artificial Intelligence Techno-Economic Segments (TES) Analysis – Selected TES Indicators” (forthcoming).

4.6 Singularities in big data

Signals often get corrupted during transmission or processing, and a frequent goal in filter design is the restoration of the original signal. However, in some situations, certain ”nonlinear” features of the signal are more important than the overall information contents. Non-linear filters may be useful in these situations, especially in the big data context.

4.6.1 Unlinear filtering/decimation to recognize singularities in big data: improving compression and data processing

Daniel Tirelli

Main AI Stream

Deep learning/Machine learning

Computer vision/Image recognition

Video automatic content recognition

Policy areas and legal references

e-Infrastructures:

COM/2015/0185 final: The European Agenda on Security

JOIN/2013/01 final: Cybersecurity Strategy of the European Union: An Open, Safe and Secure Cyberspace

Objectives

Drastic reduction of the data without removing singularities.

Powerful automatisation of signal processing.

Methodology

According to the domain, the methodological approach includes:

non-integer decimation without smooth or cancellation of singularities (a)

automatic modal identification by un-linear filtering of data (b)

envelope definition by un-linear filtering of the data (c)

Scientific references

Tirelli, D., 2014. Vibration mitigation without dissipative devices. Shock and Vibration Article ID 135767: 1-14.

Marazzi, F., Tirelli, D. Combating earthquakes: designing and testing anti-seismic buildings, Science in School Issue (The European journal for science teachers) 15: Summer 2010.

4.7 Natural language processing for direct policy support

The topic of policy change is a widely researched area in public policy and political science. Policy bases can be very large and the relationships between policies can be complex. In addition, policy can change on a frequent basis.

Checking for gaps in policy or analysing the ramifications of changing policy is necessary to both identify and rectify gaps or unintended policy prior to the policy base being refined into requirements for a system.

4.7.1 Natural language processing in direct policy support

Jiri Hradec

Main AI Stream

Deep learning/Machine learning

Natural language processing

Policy Area and legal reference

Information and Communication Technologies:

COM/2018/ 237 final: Artificial Intelligence for Europe

Objectives

Extraction of information from the corpus of European legislation and technical reports.

Automated identification and extraction of policy relevant named entities (e.g., Water Framework Directive).

Automated creation of ontology to provide context to the named entity.

Temporal evolution of the named entities.

Automated extraction of named entity definitions (FIDELIO is a demand driven model).

Word sense disambiguation (is ”magnet” referring to a model name or to ferromagnetic material?).

Co-reference resolution (identification of named entities in sentences with indirect references).

Methodology

Main phases include:

Documents are harvested and their formats are converted to plain text

Thoroughly cleansed, sentences and phrases are extracted and a corpora by paragraphs and decades is created

A neural nets is trained and published

Scientific references

Řehůřek, R., Sojka, P., 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of LREC 2010 workshop New Challenges for NLP Frameworks. Valletta, Malta. pp. 46-50.

Honnibal, M., Montani, I., 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing (to appear).

Mikolov, T, et al., Distributed representations of words and phrases and their compositionality. Proceedings NIPS’13 Proceedings of the 26th International Conference on Neural Information Processing Systems, volume 2: 3111-3119. Lake Tahoe, Nevada. December 2013.

4.8 Data science for demography

Statistics on population change are increasingly used to support policymaking and to provide the opportunity to monitor demographic behaviour within political, economic, social and cultural contexts.

The Big data phenomenon makes realize that our world is now full of data. The question that needs to be addressed is: how can big data help measure more accurately and timely demographic phenomena? Can it be considered administrative data Big enough to use machine learning techniques with them?

4.8.1 Machine learning techniques to forecast population using Eurostat data: an exploratory study

Alvaro Gómez Losada and Néstor Duch Brown

Main AI Stream

Deep learning/Machine learning

Data science

Policy area and legal reference

Demography:

COM(2018) 237 final: Artificial Intelligence for Europe

Objectives

To assess the possibility of using small data sets and machine learning techniques to forecast population pyramids.

Methodology

Supervised classification using Random forest algorithm. Comparison with baseline projections. Arima and Exponential smoothing approaches.

Scientific references

Gómez Losada, A., Duch Brown, 2018. N. Machine learning techniques to forecast population using Eurostat data: an exploratory study. European Conference on Quality in Official Statistics (Q2018). Krakow, 2018.

Gómez Losada, A., Christidis, P., 2018. A novel approach to detecting pattern changes in time series from Eurostat. European Conference on Quality in Official Statistics (Q2018). Krakow, 2018.

4.9 Data science for air pollution

Europe cut significantly emissions in recent decades, but air pollution continues to damage human health and ecosystems, and remains a worldwide concern. Emissions of air pollutants come from many economic and societal activities, mainly represented by road transport, industry, agriculture and households.

A system for forecasting future air quality cannot, by itself, solve the problems described above. Forecasts, if they are reliable and sufficiently accurate, can however play an important role as part of an air quality management system. To produce air quality forecast helps people plan ahead, decreasing the effects on health and the costs associated.

The JRC supports the Commission, Member States and regional/local authorities in the field of air quality management. In particular, the JRC develops and/or applies tools to perform Integrated Assessment Modelling studies at a regional and local level.

4.9.1 Data science methodologies applied to urban air pollution

Álvaro Gómez Losada

Main AI stream

Deep learning/Machine learning

Data science

Data mining

Policy area and legal reference

Urban environment:

Directive 2008/50/EC (OJ L 152 11.6.2008) on ambient air quality and cleaner air for Europe

Objectives

To characterize different air pollution profiles in urban environments and forecasting high levels of pollution to prevent population from adverse health effects.

Methodology

Main methodological elements include:

Hidden Markov models for time series analyses.

Forecasting using supervised classification and pattern sequence.

Neural networks, support vector machines (linear and radial kernels), naïve Bayes, k-nearest neighbours, random forests, ripper rule learner, and decision trees.

Unsupervised classification.

Scientific references

Gómez-Losada, A., Pires, J.C.M., Pino-Mejías, R., 2018. Modelling background air pollution in urban environments: implications for epidemiological research. Special issue on Environmental Data Science. Environmental Modelling & Software 106: 13-21.

Gómez-Losada, A., et al., 2018. A novel approach to forecast urban surface-level ozone considering heterogeneous locations and limited information. Special issue on Environmental Data Science. Environmental Modelling & Software 110: 52-61.

Gómez-Losada, A., 2018. Forecasting ozone threshold exceedances in urban background areas using supervised classification and easy-access information. Atmospheric Pollution Research 9(6): 1052-1061.

4.10 Privacy analysis of mobile applications

While online users increasingly rely on the use of mobile applications (apps) for their everyday activities and needs, the processing of personal data through such tools poses significant risks to users’ security and privacy.

Such risks stem mainly from the variety of data and sensors held in mobile devices, the use of different types of identifiers and extended possibility of users’ tracking, the complex mobile app ecosystem and limitations of app developers, as well as the extended use of third-party software and services.

In the area of mobile apps and privacy there is still a serious gap between legal requirements and the translation of these requirements into practical solutions.

4.10.1 Artificial intelligence for automatic security and privacy analysis of mobile applications

Dimitris Geneiatakis, Gianmarco Baldini, Igor Nai Fovino and Ioannis Vakalis

Main AI Stream

Deep learning/Machine learning.

Policy area and legal references

Cybersecurity:

COM/2015/0192 final (6.5.2015): A Digital Single Market Strategy for Europe

Regulation (EU) 2016/679 (OJ L 119, 4.5.2016) on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

Directive 2002/58/EC (OJ L 201, 31.7.2002) concerning the processing of personal data and the protection of privacy in the electronic communications sector (Directive on privacy and electronic communications)

Objectives

To deploy AI techniques for assessing the invasiveness of a given app.

To analyze clustering apps behavior.

To provide easy-to-use instruments to empower users in understanding their exposure to privacy and security threats.

To propose options for policy support in terms of protecting users’ privacy and security.

Methodology

The methodology consists of three phases: Apps data collection and analysis, feature extraction, and model training and classification

Scientific references

Geneiatakis, et. al., 2015. A Permission verification approach for android mobile applications. Computers and Security 49(C): 192-205.

Geneiatakis, et. al., 2015. On the efficacy of static features to detect malicious apps in Android, in: Proceedings of 12th Trust, Privacy and Security in Digital Business CConference (TrustBus), Springer, 2015, pp. 87-98.

Geneiatakis, et. al., Towards a mobile malware detection framework with the support of machine learning, in: Proceedings of ISCIS Security Workshop, Springer, 2018.

4.11 Robust indoor localization in complex scenarios

Accurate indoor positioning technology provides location-based service for a variety of applications. However, most existing indoor localization approaches (e.g., Wi-Fi and Bluetooth-based methods) rely heavily on positioning infrastructure, which prevents their large-scale deployment and limits the range at which they are applicable.

Indoor positioning technologies based on the magnetic field have attracted considerable interest because magnetic sensors have become an essential sensor in most mobile devices.

4.11.1 Robust indoor localization in complex scenarios (RISE)

Carlos Sánchez, Pierluigi Taddei, Erik Wolfart and Vítor Sequeira

Main AI Stream

Deep learning/Machine learning

Computer vision/Image recognition

Policy area and legal references

Nuclear safeguards:

Council Regulation (Euratom) N0 1314/2013 on the Research and Training Programme of the European Atomic Energy Community (2014-2018)

Support the IAEA through the European Commission Cooperative Support Programme (ECSP).

Objectives

To contribute to research and development in the area of indoor localisation in infrastructure-free environments.

To explore recent achievements made in machine learning for indoor localisation and provide reference data sets that can be used by other organisations.

Methodology

To develop indoor localization using sensor data from mobile consumer devices. This uses NUVER’s laser-based mapping backpack and applies Machine learning techniques for place recognition. The outcome is a comprehensive, multi-temporal reference dataset for indoor localization.

Scientific references

Ceriani, S., et al., 2015. Pose interpolation SLAM for large maps using moving 3D sensors. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, pp. 750-757.

Sánchez, C., et al., 2015. Global registration of point clouds for mapping. Accepted for publication in the Proceedings of the 15th International Conference on Intelligent Autonomous Systems, May 2018, Baden-Baden, Germany.

Conclusions

The main conclusions from the presentations and the fruitful discussion are:

The quantity and diversity of JRC competences on AI is remarkable and the work in the different disciplines and policies makes JRC a quite unique institution.

AI requires a substantial use of computing power and data storage capacities. Since technology moves on very fast, computing infrastructures are becoming obsolete soon. We need to further understand what the needs are and ensure that the JRC ICT strategy is fully aligned with the emerging demands.

AI is pervasive. We need to build better capacities in house. It was proposed to have a summer school (in Ispra) with JRC scientists training colleagues. This School could be also open to colleagues from the Policy DGs to reduce the knowledge gap and better understand the AI opportunities and impact on the Society.

AI requires data to train learning-based models and contextualize them for specific applications. JRC data management policy and process deserve attention: for example, data contextualization (e.g. tagging) made by one Unit could be beneficial for others, streaming-data structuring activity can serve more applications, and datasets cleaning used by several units (e.g. ORBIS data) should be organized in a coordinated way to save resources. When private data are acquired multi-site license should be prioritized. The lack of a centralized data repository was raised, despite some efforts in this direction have already been initiated (e.g. JEODPP).

JRC Research Agenda on AI is needed. This should cover both the use, customization and adaptation of AI technological framework for specific purposes (i.e. applied research) as well as the understanding of AI related issues (e.g. transparency, robustness, accountability, liability, ethics, privacy, etc.). More in general, Digital Transformation understanding and AI uptake should be further explored in all their aspects (social, economic, legal, ethical, security, etc.).

We need to improve our internal capacity to share and re-use tools, avoiding to lose the outcomes of the activity made by temporary staff, when they leave JRC. An effort should be made to create and maintain a collaborative space.

The AI dichotomy must be addressed. We need to consider both the Narrow AI and the General AI. Today, Narrow AI is very successful and we need to measure its socio-economic impact. Vice versa, General AI is more speculative and can be investigated in future by the CAS projects, along with its futuristic issues (e.g. digital immortality, morphing with machine, etc.). We should focus on policy support and, meanwhile, develop our own competences on the more long term challenges to be able to understand and anticipate future developments.

A lot of knowledge is available in JRC but not fully shared. More than 100 colleagues filled the survey on AI. We need to further share our knowledge (e.g. by seminars) and create the conditions to further work together on AI and Big Data challenging topics such as security, ethics, transparency.

Working on AI, JRC scientist should not forget the EU values and rights. They should be central to our scientific activity, we should invest in training JRC staff on ethical, and legal matter to ensure that what they will develop respect existing legislation and ethical rules. In addition, training on cybersecurity could be beneficial to explain how to deal with AI solutions vulnerability.

The link between JRC internal work and AI Watch should be investigated. This is to increase JRC capacities on the systematic monitoring of AI related development. This could be a good opportunity to pilot new solutions and promote cross collaborations.

A Community of Practice (CoP) on AI and Big Data is needed. The primary function will be to help people in networking. The CoP should be active in: organising, workshops, conferences, seminars as well as creating working groups and task forces to address horizontal issue. The CoP should be moderated/coordinated to identify and focus on priorities (e.g. data and/or ICT infrastructures) and provide timely information when needed. Regular monthly seminars should be organised on a list of topics to be proposed by the CoP.

Openness to other DGs is needed in a governed way. We should distinguish between those discussions and actions restricted to JRC and other ones that could be beneficial for our colleagues in Brussels or Luxembourg. An Annual Event on AI should be organised by the CoP opened to all the Commission Services (with the possibility of joint presentations when our technologies are already used for supporting policies). Other ad hoc actions could address the requests from regulators and allow JRC scientist to better explain AI.

List of abbreviations

AI | Artificial Intelligence
CAP | Common Agricultural Policy
CAS | Centre for Advanced Studies
CERN | European Organization for Nuclear Research DL Deep Learning
EC | European Commission
ESA | The European Space Agency
EU | European Union
EUSC | European Union Satellite Centre
JRC | Joint Research Centre
ML | Machine Learning
MS | (EU) Member States
NLP | Natural Language Processing
OJ | Official Journal of the European Union
SML | Symbolic Machine Learning

Annexes

Annex 1. Workshop agenda

Morning session

Information exchange

9.00-9.10 | Opening (Vladimir Sucha)
9.10-9.20 | The Artificial Intelligence communication and the Digital Transformation & Artificial Intelligence project (Alessandro Annoni)
9.20-9.30 | Introduction and scope of the workshop (Stefano Nativi)
9.30-11.00 | Presentations

Session 1 - Policy support

George Breyiannis | Machine learning for disaster risk management
Valerio Lorini | Social media for flood risk
Apostolos Psyllos | Computer vision: automatic tattoo recognition
Pasquale Ferrara | Machine learning in child abuse on-line investigation
Henrik Junklevitz | Clustering and unsupervised learning for digital forensics Rudolf Haraksim Machine learning: biometrics and juvenile fingerprints.
Iwen Coisel | Machine learning for targeted decryption
Aris Tsois | Artificial intelligence for customs risk management
Michele Vespe | Harnessing big data for migration
Guido Lemoine | CAP monitoring by machine learning from Copernicus Sentinel-1 signatures
Panayotis Christidis | Artificial Intelligence in transport applications
Álvaro Gómez Losada | Studying the algorithmic pricing in Amazon market place: the Italian and Spanish cases

Session 2 - New initiatives

Emilia Gómez | Human behaviour and machine learning (HUMAINT)
Nicole Dewandre | ONLIFE
María Iglesias | Legal and regulatory implications of artificial intelligence
David Asturiol | Artificial intelligence in toxicology
Joachim Kreysa | iHiP as customer
Mauro Petrillo | Big Data + AI + Health = eHealth
Clemens Wittwehr | AI4CRA – Artificial Intelligence for Chemical Risk Assessment
11.00-11.15 | Coffee break
11.15-12.30 | Presentations

Session 3 – Technology development

Andrea Toreti | Pattern recognition & classification change point detection.
Blagoj Delipetrev | Deep learning and Copernicus for generation of new data in Spatial Data Infraestructures.
Raphael d’Andrimont | Agricultural in-situ data sourced from crowd-sourced street-level imagery analyzed with computer vision.
Thomas Kemper | Big earth data analytics with symbolic machine learning (SML).
Domenico Perrotta | Robustness issues in clustering.
Monserrat Lopez Cobo | TES methodology in the PREDICT project.
Daniel Tirelli | Un-linear filtering/decimation to recognize singularities in big data and to improve compression or data processing.
Alexandra Balahu | Machine learning for text mining – some use cases in I3
Jiri Hradec | Natural language processing in direct policy support.
Álvaro Gómez Losada | Application of machine learning to forecast population pyramids using Eurostat data: an exploratory study.
Álvaro Gómez Losada | Applications of data science to environmental research (air pollution).
Dimitrios Geneiatakis | Artificial intelligence-based automatic security and privacy analysis of mobile applications.
Pierre Soille | Earth observation and social sensing big data pilot.
Eric Wolfart | Robust sensor fusion for localization in complex scenarios (REFINE).
12.30-13.30 | Lunch

Afternoon session

Brainstorming and discussion

13.00-14.00 | Internal questionnaire on Artificial Intelligence at the JRCs: preliminary results (Stefano Nativi).
14.00-15.00 | Discussion on JRC priorities (Presentation and chairing: Alessandro Annoni).
15.00-16.00 | Discussion on the Artificial Intelligence & Big Data Community of Practice (Presentation and chairing: Stefano Nativi).
16.00-16.15 | Wrap-up and conclusions (Alessandro Annoni and Stefano Nativi).

Annex 2. References

Introductory texts at each section of the AI studied areas were obtained from the following sources:

Agricultural monitoring

https://ec.europa.eu/jrc/en/science-area/agriculture-and-food-security

https://ec.europa.eu/jrc/en/research-topic/agricultural-monitoring

Air quality

https://ec.europa.eu/jrc/en/research-topic/air-quality

Chemical risk assessment and eHealth

https://ec.europa.eu/jrc/en/research-topic/alternatives-animal-testing-and-safety-assessment-chemicals

https://ec.europa.eu/health//sites/health/files/ehealth/docs/com_2012_736_en.pdf

http://data.consilium.europa.eu/doc/document/ST-14079-2017-INIT/en/pdf

Earth observation

http://www.mdpi.com/2072-4292/8/7/606

Economy

https://ec.europa.eu/jrc/en/research-topic/digital-economy

https://www.kansascityfed.org/~/media/files/publicat/sympos/2018/papersandhandouts/825180810cavallopaper.pdf?la=en

Information and processing systems

https://radimrehurek.com/gensim/lrec2010_final.pdf

https://arxiv.org/pdf/1401.2943.pdf

Transport

https://ec.europa.eu/transport/themes/its_en,

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.402.6331&rep=rep1&type=pdf

Demography and Migration

https://gmdac.iom.int/iom-european-commission-launch-big-data-for-migration-alliance

https://statswiki.unece.org/pages/viewpage.action?pageId=77170622

Human-Artificial intelligence interaction

https://ec.europa.eu/jrc/communities/sites/jrccties/files/humaint-kickoffworkshop-proceedings-final.pdf

Ethics and Legal aspects

https://moderndiplomacy.eu/2018/04/24/the-ethical-and-legal-issues-of-artificial-intelligence/

Acknowledgement

We are particularly grateful to all the participants and contributors of the first workshop on Artificial Intelligence at JRC.

We also want to thank Karen Fullerton and Loizos Bailas for the important help in the workshop preparation and all the other JRC colleagues that made good suggestions for this successful event.

GETTING IN TOUCH WITH THE EU

In person

All over the European Union there are hundreds of Europe Direct information centres. You can find the address of the centre nearest you at: https://europa.eu/european-union/contact_en

On the phone or by email

Europe Direct is a service that answers your questions about the European Union. You can contact this service:

- by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls),

- at the following standard number: +32 22999696, or

- by electronic mail via: https://europa.eu/european-union/contact_en

FINDING INFORMATION ABOUT THE EU

Online

Information about the European Union in all the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en

EU publications

You can download or order free and priced EU publications from EU Bookshop at: https://publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/europeanunion/contact_en).

JRC117232

EUR 29798 EN

PDF    ISBN 978-92-76-08842-4    ISSN 1831-9424    doi:10.2760/705074

Epub   ISBN 978-92-76-14140-2    ISSN 1831-9424    doi:10.2760/102610

Luxembourg: Publications Office of the European Union, 2019.

The reuse policy of the European Commission is implemented by Commission Decision 2011/833/EU of 12 December 2011 on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Reuse is authorised, provided the source of the document is acknowledged and its original meaning or message is not distorted. The European Commission shall not be liable for any consequence stemming from the reuse. For any use or reproduction of photos or other material that is not owned by the EU, permission must be sought directly from the copyright holders.

All content © European Union, 2019, except: page 8 (Pixabay), page 18 (rawpixel.com from Pexels), page 20 (Gstudio Group-Fotolia.com), page 22 (Federal Highway Administration, Department of Transportation, USA), page 24 (ITS DataHub), page 27 (Needpix.com), page 29 (Mike Bird from Pexels), page 32 (Pixabay), page 35 (ibreakstock-Fotolia.com), page 37 (Sergey Nivens-Fotolia.com), page 41 (Photoman-Fotolia.com), page 45 (Pixabay), page 47 (Pixabay), page 49 (Mariusz Prusaczyk from Pexels), page 52 (Alex Knight from Pexels), page 54 (Lukas from Pexels), page 56 (Pixabay), page 58 (rawpixel.com from Pexels), page 60 (Kaique Rocha from Pexels), page 62 (AJ Nakasone from Pexels), page 64 (rawpixel.com from Pexels), page 66 (Pixabay).

How to cite this report: Nativi S. and Gómez Losada A. (Ed.), Anderberg a., Asturiol D., Baldini G., Breyiannis, G., Cardona M., Castillo C., Charisi V., Checchi E., Christidis P., Coisel I., Corban C., Cotelo Lema J.A., d’Andrimont R., de Prato G., Delipetrev B., Devos W., Dewandre N., Dimitrova T., Dottori F., Duch Brown N., Enescu V., Ferrara P., Fidalgo Merino R., Gabrielli L., Galbally J., Geneiatakis D., Giuliani R., Gómez E., Hradec J., Iglesias M., Ignat C., Junklewitz H., Kalas M., Kemper T., Kotsev A., Kreysa J., Lavaysse C., Lemoine G., López-Cobo M., Lorini V., Martens B., Miron M., Nai Fovino I., Nappo D., Natale F., Nukaj B., Pedone M., Perrotta D., Pesaresi M., Petrillo M., Poulymenopoulou M., Psyllos A., Righi R., Salamon P., Samoili S., Sánchez S., Sánchez I., Sequeira V., Soile P., Spyratos S., Steri G., Tirelli D., Triaille J.P., Tolan S., Toreti A., Tsois A., Vakalis I., van der Velde M., Vespe M., Wittwehr C., Artificial Intelligence at the JRC, EUR 29798 EN, Publications Office of the European Union, Luxembourg, 2019, ISBN 978-92-76-08842-4, doi:10.2760/705074, JRC117232.