Monitoring crop phenology with street-level imagery using computer vision

Street-level imagery holds a significant potential to scale-up in-situ data collection. This is enabled by combining the use of cheap high quality cameras with recent advances in deep learning compute solutions to derive relevant thematic information. We present a framework to collect and extract crop type and phenological information from street level imagery using computer vision. Each month in 2018, a fixed 200-km route was surveyed collecting one picture per second resulting in a total of 400,000 geo-tagged pictures. At 220 specific parcel locations detailed on the spot crop phenology observations were recorded. Classification was done using TensorFlow with a well-known image recognition model, based on transfer learning with convolutional neural networks (MobileNet). A hypertuning methodology was developed to obtain the best performing model among 160 models. This best model was applied on an independent inference set discriminating crop type with a Macro F1 score of 88.1% and main phenological stage at 86.9% at the parcel level. Potential and caveats of the approach along with practical considerations for implementation and improvement are discussed. The proposed framework speeds up high quality in-situ data collection and suggests avenues for massive data collection via automated classification using computer vision.

D'ANDRIMONT Raphael;  YORDANOV Momchil;  MARTINEZ SANCHEZ Laura;  VAN DER VELDE Marijn; 

2022-04-05

ELSEVIER SCI LTD

JRC128056

0168-1699 (online),   

https://www.sciencedirect.com/science/article/pii/S0168169922001831,    https://publications.jrc.ec.europa.eu/repository/handle/JRC128056,   

10.1016/j.compag.2022.106866 (online),