Artículo
Competitive Live Evaluation of Activity-recognition Systems
Autor/es | Gjoreski, Hristijan
Kozina, Simon Gams, Matjaž Luštrek, Mitja Álvarez García, Juan Antonio Hong, Jin-Hyuk Ramos, Julian Dey, Anind K. Bocca, Maurizio Patwari, Neal |
Departamento | Universidad de Sevilla. Departamento de Lenguajes y Sistemas Informáticos |
Fecha de publicación | 2015 |
Fecha de depósito | 2021-09-21 |
Publicado en |
|
Resumen | In order to ensure the validity and usability of activity recognition approaches, an agreement on a set of standard evaluation methods is needed. Due to the diversity of the sensors and other hardware employed, designing ... In order to ensure the validity and usability of activity recognition approaches, an agreement on a set of standard evaluation methods is needed. Due to the diversity of the sensors and other hardware employed, designing and accepting standard tests is a difficult task. This article presents an initiative to evaluate activity recognition systems: a living-lab evaluation established through an annual competition − EvAAL-AR (Evaluating Ambient Assisted Living Systems through Competitive Benchmarking − Activity Recognition). In the competition, each team brings their own activity-recognition system, which is evaluated live on the same activity scenario performed by an actor. The evaluation criteria attempt to capture the practical usability: recognition accuracy, user acceptance, recognition delay, installation complexity, and interoperability with ambient assisted living systems. The article also presents the competing systems with emphasis on two best-performing ones: (i) a system that achieved the best recognition accuracy, and (ii) a system that was evaluated as the best overall. Finally, the article presents lessons learned from the competition and ideas for future development of the competition and of the activity recognition field in general. |
Cita | Gjoreski, H., Kozina, S., Gams, M., Luštrek, M., Álvarez Garcia, J.A., Hong, J.,...,Patwari, N. (2015). Competitive Live Evaluation of Activity-recognition Systems. IEEE Pervasive Computing, 14 (1 (Jan.- Mar 2015)), 70-77. |
Ficheros | Tamaño | Formato | Ver | Descripción |
---|---|---|---|---|
Competitive Live Evaluation of ... | 1.151Mb | [PDF] | Ver/ | |