Artículo
ViolenceNet: Dense Multi-Head Self-Attention with Bidirectional Convolutional LSTM for Detecting Violence
Autor/es | Rendón Segador, Fernando José
Álvarez García, Juan Antonio Enríquez de Salamanca Ros, Fernando Deniz, Oscar |
Departamento | Universidad de Sevilla. Departamento de Lenguajes y Sistemas Informáticos |
Fecha de publicación | 2021 |
Fecha de depósito | 2021-09-08 |
Publicado en |
|
Resumen | Introducing efficient automatic violence detection in video surveillance or audiovisual
content monitoring systems would greatly facilitate the work of closed-circuit television (CCTV)
operators, rating agencies or those ... Introducing efficient automatic violence detection in video surveillance or audiovisual content monitoring systems would greatly facilitate the work of closed-circuit television (CCTV) operators, rating agencies or those in charge of monitoring social network content. In this paper we present a new deep learning architecture, using an adapted version of DenseNet for three dimensions, a multi-head self-attention layer and a bidirectional convolutional long short-term memory (LSTM) module, that allows encoding relevant spatio-temporal features, to determine whether a video is violent or not. Furthermore, an ablation study of the input frames, comparing dense optical flow and adjacent frames subtraction and the influence of the attention layer is carried out, showing that the combination of optical flow and the attention mechanism improves results up to 4.4%. The conducted experiments using four of the most widely used datasets for this problem, matching or exceeding in some cases the results of the state of the art, reducing the number of network parameters needed (4.5 millions), and increasing its efficiency in test accuracy (from 95.6% on the most complex dataset to 100% on the simplest one) and inference time (less than 0.3 s for the longest clips). Finally, to check if the generated model is able to generalize violence, a cross-dataset analysis is performed, which shows the complexity of this approach: using three datasets to train and testing on the remaining one the accuracy drops in the worst case to 70.08% and in the best case to 81.51%, which points to future work oriented towards anomaly detection in new datasets. |
Agencias financiadoras | Ministerio de Economía y Competitividad (MINECO). España |
Identificador del proyecto | TIN2017-82113-C2-1-R
TIN2017-82113-C2-2-R |
Cita | Rendón Segador, F.J., Álvarez García, J.A., Enríquez de Salamanca Ros, F. y Deniz, O. (2021). ViolenceNet: Dense Multi-Head Self-Attention with Bidirectional Convolutional LSTM for Detecting Violence. Electronics, 10 (13), 1-16. |
Ficheros | Tamaño | Formato | Ver | Descripción |
---|---|---|---|---|
ViolenceNet Dense Multi-Head ... | 4.547Mb | [PDF] | Ver/ | |