Ponencia
Multi-Objective Big Data Optimization with jMetal and Spark
Autor/es | Barba González, Cristóbal
García Nieto, José Manuel Nebro, Antonio J. Aldana Montes, José F. |
Departamento | Universidad de Sevilla. Departamento de Ciencias de la Computación e Inteligencia Artificial |
Fecha de publicación | 2017 |
Fecha de depósito | 2021-05-11 |
Publicado en |
|
ISBN/ISSN | 978-3-319-54156-3 0302-9743 |
Resumen | Big Data Optimization is the term used to refer to optimiza-
tion problems which have to manage very large amounts of data. In this
paper, we focus on the parallelization of metaheuristics with the Apache
Spark cluster ... Big Data Optimization is the term used to refer to optimiza- tion problems which have to manage very large amounts of data. In this paper, we focus on the parallelization of metaheuristics with the Apache Spark cluster computing system for solving multi-objective Big Data Op- timization problems. Our purpose is to study the in uence of accessing data stored in the Hadoop File System (HDFS) in each evaluation step of a metaheuristic and to provide a software tool to solve these kinds of problems. This tool combines the jMetal multi-objective optimiza- tion framework with Apache Spark. We have carried out experiments to measure the performance of the proposed parallel infrastructure in an environment based on virtual machines in a local cluster comprising up to 100 cores. We obtained interesting results for computational e ort and propose guidelines to face multi-objective Big Data Optimization problems. |
Agencias financiadoras | Ministerio de Educación y Ciencia (MEC). España Junta de Andalucía |
Identificador del proyecto | TIN2014-58304-R
P11-TIC-7529 P12-TIC-1519 |
Cita | Barba González, C., García Nieto, J.M., Nebro, A.J. y Aldana Montes, J.F. (2017). Multi-Objective Big Data Optimization with jMetal and Spark. En EMO 2017: 9th International Conference on Evolutionary Multi-Criterion Optimization (16-30), Münster, Germany: Springer. |
Ficheros | Tamaño | Formato | Ver | Descripción |
---|---|---|---|---|
Multi-objective big data optim ... | 513.4Kb | [PDF] | Ver/ | |