In a sandbox framework for scientific computing, we deal with the task allocation problem when processing a high volume of geospatial data. A predefined meta-information catalogue guides the data selection and a profiling procedure based on data sub-sampling estimates the memory and processing requirements. The task allocation is formulated as an optimization problem conditioned by time dependencies and job execution priorities. The procedure can work as addon to any task scheduler that provides configuration options for computational resources allocation. Experiments demonstrate that the SLURM fine-tuning with the proposed method leads to better resource management and shorter schedules.
SYRRIS Vasileios;
RODRIGUEZ ASERETTO Roque Dario;
SOILLE Pierre;
2017-01-17
Publications Office of the European Union
JRC98347
978-92-79-56980-7,
1831-9424,
OP LB-NA-27775-EN-N,
http://publications.jrc.ec.europa.eu/repository/handle/JRC100655,
https://publications.jrc.ec.europa.eu/repository/handle/JRC98347,
10.2788/854791,
| Name | Country | City | Type |
|---|
This document is only visible at the Commission level.
You are not authorized to publish or distribute it outside the European Commission.
This is a public document. You can share this publication.
Datasets
| ID | Title | Public URL |
|---|
Dataset collections
| ID | Acronym | Title | Public URL |
|---|
Scripts / source codes
| Description | Public URL |
|---|
Additional supporting files
| File name | Description | File type |
|---|