STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors
On-road behavior analysis is a key task required for robust autonomous vehicles. Unlike traditional perception tasks, this paper aims to achieve a high-level understanding of road agent activities. This allows better operation under challenging contexts to enable better interaction and decision-maki...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9779713/ |
| _version_ | 1811232006087901184 |
|---|---|
| author | R. Trabelsi R. Khemmar B. Decoux J.-Y. Ertaud R. Boutteau |
| author_facet | R. Trabelsi R. Khemmar B. Decoux J.-Y. Ertaud R. Boutteau |
| author_sort | R. Trabelsi |
| collection | DOAJ |
| description | On-road behavior analysis is a key task required for robust autonomous vehicles. Unlike traditional perception tasks, this paper aims to achieve a high-level understanding of road agent activities. This allows better operation under challenging contexts to enable better interaction and decision-making in such complex environments. In this paper, we tackle the problem of discriminating spatio-temporal features that capture the visual instants that require more attention. We propose a new approach called STAF (Spatio-Temporal Attention Framework) through Long Short Term Memory (LSTM) layers that uses a multi-head attention mechanism on its past cell state to focus on attributes that are relevant over time. Experiments have been carried out on two different scenarios over data from Joint Attention in Autonomous Driving (JAAD) and Honda Research Institute Driving Dataset (HDD), both datasets devoted to understanding the behavior of road agents. The evaluation and results obtained proof that the proposed “STAF” is outperforming state-of-the-art algorithms-based LSTM (Ramanishka, Rasouli, and LSTM-baseline). For example, STAF outperforms LSTM with mean Average Precision (mAP) of 13%. |
| first_indexed | 2024-04-12T10:56:02Z |
| format | Article |
| id | doaj.art-e3ba7d66532b47e594283b40be1bc679 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T10:56:02Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e3ba7d66532b47e594283b40be1bc6792022-12-22T03:36:06ZengIEEEIEEE Access2169-35362022-01-0110557945580410.1109/ACCESS.2022.31768619779713STAF: Spatio-Temporal Attention Framework for Understanding Road Agents BehaviorsR. Trabelsi0R. Khemmar1https://orcid.org/0000-0002-6230-2966B. Decoux2J.-Y. Ertaud3R. Boutteau4https://orcid.org/0000-0003-1078-5043ESIGELEC, IRSEEM, UNIROUEN, Normandie University, Rouen, FranceESIGELEC, IRSEEM, UNIROUEN, Normandie University, Rouen, FranceESIGELEC, IRSEEM, UNIROUEN, Normandie University, Rouen, FranceESIGELEC, IRSEEM, UNIROUEN, Normandie University, Rouen, FranceLITIS, INSA Rouen, UNILEHAVRE, UNIROUEN, Normandie University, Rouen, FranceOn-road behavior analysis is a key task required for robust autonomous vehicles. Unlike traditional perception tasks, this paper aims to achieve a high-level understanding of road agent activities. This allows better operation under challenging contexts to enable better interaction and decision-making in such complex environments. In this paper, we tackle the problem of discriminating spatio-temporal features that capture the visual instants that require more attention. We propose a new approach called STAF (Spatio-Temporal Attention Framework) through Long Short Term Memory (LSTM) layers that uses a multi-head attention mechanism on its past cell state to focus on attributes that are relevant over time. Experiments have been carried out on two different scenarios over data from Joint Attention in Autonomous Driving (JAAD) and Honda Research Institute Driving Dataset (HDD), both datasets devoted to understanding the behavior of road agents. The evaluation and results obtained proof that the proposed “STAF” is outperforming state-of-the-art algorithms-based LSTM (Ramanishka, Rasouli, and LSTM-baseline). For example, STAF outperforms LSTM with mean Average Precision (mAP) of 13%.https://ieeexplore.ieee.org/document/9779713/Deep LSTMattention mechanismSTAFroad scenesbehaviors understandingsmart mobility |
| spellingShingle | R. Trabelsi R. Khemmar B. Decoux J.-Y. Ertaud R. Boutteau STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors IEEE Access Deep LSTM attention mechanism STAF road scenes behaviors understanding smart mobility |
| title | STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors |
| title_full | STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors |
| title_fullStr | STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors |
| title_full_unstemmed | STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors |
| title_short | STAF: Spatio-Temporal Attention Framework for Understanding Road Agents Behaviors |
| title_sort | staf spatio temporal attention framework for understanding road agents behaviors |
| topic | Deep LSTM attention mechanism STAF road scenes behaviors understanding smart mobility |
| url | https://ieeexplore.ieee.org/document/9779713/ |
| work_keys_str_mv | AT rtrabelsi stafspatiotemporalattentionframeworkforunderstandingroadagentsbehaviors AT rkhemmar stafspatiotemporalattentionframeworkforunderstandingroadagentsbehaviors AT bdecoux stafspatiotemporalattentionframeworkforunderstandingroadagentsbehaviors AT jyertaud stafspatiotemporalattentionframeworkforunderstandingroadagentsbehaviors AT rboutteau stafspatiotemporalattentionframeworkforunderstandingroadagentsbehaviors |