TASK ALLOCATION IN HIGH PERFORMANCE PROCESSING OF GEOSPATIAL DATA
In a sandbox framework for scientific computing, we deal with the task allocation problem when processing a high volume of geospatial data. A predefined meta-information catalogue guides the data selection and a profiling procedure based on data sub-sampling estimates the memory and processing requirements. The task allocation is formulated as an optimization problem conditioned by time dependencies and job execution priorities. The procedure can work as addon to any task scheduler that provides configuration options for computational resources allocation. Experiments demonstrate that the SLURM fine-tuning with the proposed method leads to better resource management and shorter schedules.
SYRRIS Vasileios;
RODRIGUEZ ASERETTO Roque Dario;
SOILLE Pierre;
2017-01-17
Publications Office of the European Union
JRC98347
978-92-79-56980-7,
1831-9424,
OP LB-NA-27775-EN-N,
http://publications.jrc.ec.europa.eu/repository/handle/JRC100655,
https://publications.jrc.ec.europa.eu/repository/handle/JRC98347,
10.2788/854791,
Additional supporting files
| File name | Description | File type | |
|---|
