Skip to Main content Skip to Navigation
Conference papers

Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos

Abstract : Taking advantage of human pose data for understanding human activities has attracted much attention these days. However, state-of-the-art pose estimators struggle in obtaining high-quality 2D or 3D pose data due to occlusion, truncation and low-resolution in real-world un-annotated videos. Hence, in this work, we propose 1) a Selective Spatio-Temporal Aggregation mechanism, named SST-A, that refines and smooths the keypoint locations extracted by multiple expert pose estimators, 2) an effective weakly-supervised self-training framework which leverages the aggregated poses as pseudo ground-truth instead of handcrafted annotations for real-world pose estimation. Extensive experiments are conducted for evaluating not only the upstream pose refinement but also the downstream action recognition performance on four datasets, Toyota Smarthome, NTU-RGB+D, Charades, and Kinetics-50. We demonstrate that the skeleton data refined by our Pose-Refinement system (SSTA-PRS) is effective at boosting various existing action recognition models, which achieves competitive or state-of-the-art performance.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03121883
Contributor : Di Yang <>
Submitted on : Tuesday, January 26, 2021 - 4:32:24 PM
Last modification on : Monday, March 1, 2021 - 2:34:32 PM

File

0599.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03121883, version 1
  • ARXIV : 2011.05358

Collections

Citation

Di Yang, Rui Dai, Yaohui Wang, Rupayan Mallick, Luca Minciullo, et al.. Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos. IEEE Winter Conference on Applications of Computer Vision 2021, Jan 2021, Virtual, United States. ⟨hal-03121883⟩

Share

Metrics

Record views

56

Files downloads

35