HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos

Abstract : Taking advantage of human pose data for understanding human activities has attracted much attention these days. However, state-of-the-art pose estimators struggle in obtaining high-quality 2D or 3D pose data due to occlusion, truncation and low-resolution in real-world un-annotated videos. Hence, in this work, we propose 1) a Selective Spatio-Temporal Aggregation mechanism, named SST-A, that refines and smooths the keypoint locations extracted by multiple expert pose estimators, 2) an effective weakly-supervised self-training framework which leverages the aggregated poses as pseudo ground-truth instead of handcrafted annotations for real-world pose estimation. Extensive experiments are conducted for evaluating not only the upstream pose refinement but also the downstream action recognition performance on four datasets, Toyota Smarthome, NTU-RGB+D, Charades, and Kinetics-50. We demonstrate that the skeleton data refined by our Pose-Refinement system (SSTA-PRS) is effective at boosting various existing action recognition models, which achieves competitive or state-of-the-art performance.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03121883
Contributor : Di Yang Connect in order to contact the contributor
Submitted on : Tuesday, January 26, 2021 - 4:32:24 PM
Last modification on : Saturday, March 19, 2022 - 3:10:04 AM
Long-term archiving on: : Tuesday, April 27, 2021 - 7:09:10 PM

File

0599.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03121883, version 1
  • ARXIV : 2011.05358

Citation

Di Yang, Rui Dai, Yaohui Wang, Rupayan Mallick, Luca Minciullo, et al.. Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos. WACV 2021 – IEEE Winter Conference on Applications of Computer Vision, Jan 2021, Virtual, United States. ⟨hal-03121883⟩

Share

Metrics

Record views

178

Files downloads

29