Title: TASK ALLOCATION IN HIGH PERFORMANCE PROCESSING OF GEOSPATIAL DATA
Authors: SYRRIS VASILEIOSRODRIGUEZ ASERETTO ROQUE DARIOSOILLE PIERRE
Publisher: Publications Office of the European Union
Publication Year: 2016
JRC N°: JRC98347
ISBN: 978-92-79-56980-7
ISSN: 1831-9424
Other Identifiers: OP LB-NA-27775-EN-N
URI: http://publications.jrc.ec.europa.eu/repository/handle/JRC100655
http://publications.jrc.ec.europa.eu/repository/handle/JRC98347
DOI: 10.2788/854791
Type: Conferences and workshops organized by the JRC
Abstract: In a sandbox framework for scientific computing, we deal with the task allocation problem when processing a high volume of geospatial data. A predefined meta-information catalogue guides the data selection and a profiling procedure based on data sub-sampling estimates the memory and processing requirements. The task allocation is formulated as an optimization problem conditioned by time dependencies and job execution priorities. The procedure can work as addon to any task scheduler that provides configuration options for computational resources allocation. Experiments demonstrate that the SLURM fine-tuning with the proposed method leads to better resource management and shorter schedules.
JRC Directorate:Joint Research Centre Corporate Activities

Files in This Item:
File Description SizeFormat 
syrris-etal2016bids.pdf713.27 kBAdobe PDFView/Open


Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.