Presentation
Multi-Objective Big Data Optimization with jMetal and Spark
Author/s | Barba González, Cristóbal
García Nieto, José Manuel Nebro, Antonio J. Aldana Montes, José F. |
Department | Universidad de Sevilla. Departamento de Ciencias de la Computación e Inteligencia Artificial |
Publication Date | 2017 |
Deposit Date | 2021-05-11 |
Published in |
|
ISBN/ISSN | 978-3-319-54156-3 0302-9743 |
Abstract | Big Data Optimization is the term used to refer to optimiza-
tion problems which have to manage very large amounts of data. In this
paper, we focus on the parallelization of metaheuristics with the Apache
Spark cluster ... Big Data Optimization is the term used to refer to optimiza- tion problems which have to manage very large amounts of data. In this paper, we focus on the parallelization of metaheuristics with the Apache Spark cluster computing system for solving multi-objective Big Data Op- timization problems. Our purpose is to study the in uence of accessing data stored in the Hadoop File System (HDFS) in each evaluation step of a metaheuristic and to provide a software tool to solve these kinds of problems. This tool combines the jMetal multi-objective optimiza- tion framework with Apache Spark. We have carried out experiments to measure the performance of the proposed parallel infrastructure in an environment based on virtual machines in a local cluster comprising up to 100 cores. We obtained interesting results for computational e ort and propose guidelines to face multi-objective Big Data Optimization problems. |
Funding agencies | Ministerio de Educación y Ciencia (MEC). España Junta de Andalucía |
Project ID. | TIN2014-58304-R
![]() P11-TIC-7529 ![]() P12-TIC-1519 ![]() |
Citation | Barba González, C., García Nieto, J.M., Nebro, A.J. y Aldana Montes, J.F. (2017). Multi-Objective Big Data Optimization with jMetal and Spark. En EMO 2017: 9th International Conference on Evolutionary Multi-Criterion Optimization (16-30), Münster, Germany: Springer. |
Files | Size | Format | View | Description |
---|---|---|---|---|
Multi-objective big data optim ... | 513.4Kb | ![]() | View/ | |