Artículo
On exploring data lakes by finding compact, isolated clusters
Autor/es | Jiménez Aguirre, Patricia
Roldán Salvador, Juan Carlos Corchuelo Gil, Rafael |
Departamento | Universidad de Sevilla. Departamento de Lenguajes y Sistemas Informáticos |
Fecha de publicación | 2022 |
Fecha de depósito | 2022-04-11 |
Publicado en |
|
Resumen | Data engineers are very interested in data lake technologies due to the incredible abun dance of datasets. They typically use clustering to understand the structure of the datasets
before applying other methods to infer ... Data engineers are very interested in data lake technologies due to the incredible abun dance of datasets. They typically use clustering to understand the structure of the datasets before applying other methods to infer knowledge from them. This article presents the first proposal that explores how to use a meta-heuristic to address the problem of multi-way single-subspace automatic clustering, which is very appropriate in the context of data lakes. It was confronted with five strong competitors that combine the state-of-the-art attribute selection proposal with three classical single-way clustering proposals, a recent quantum-inspired one, and a recent deep-learning one. The evaluation focused on explor ing their ability to find compact and isolated clusterings as well as the extent to which such clusterings can be considered good classifications. The statistical analyses conducted on the experimental results prove that it ranks the first regarding effectiveness using six stan dard coefficients and it is very efficient in terms of CPU time, not to mention that it did not result in any degraded clusterings or timeouts. Summing up: this proposal contributes to the array of techniques that data engineers can use to explore their data lakes |
Agencias financiadoras | Ministerio de Economía y Competitividad (MINECO). España Ministerio de Ciencia e Innovación (MICIN). España Junta de Andalucía |
Identificador del proyecto | TIN2016-75394-R
PID2020-112540RB-C44 P18-RT-1060 US-1381375 |
Cita | Jiménez Aguirre, P., Roldán Salvador, J.C. y Corchuelo Gil, R. (2022). On exploring data lakes by finding compact, isolated clusters. Information Sciences, 591 (April 2022), 103-127. |
Ficheros | Tamaño | Formato | Ver | Descripción |
---|---|---|---|---|
1-s2.0-S0020025521012664-main.pdf | 7.329Mb | [PDF] | Ver/ | |