MSTCN-VAE: An unsupervised learning method for micro gesture recognition based on skeleton modality

Abstract:

We propose a novel unsupervised model for micro-gesture classification, called MSTCN-VAE, which follows the VAE structure by adding the Multi-scale TCN and hidden feature extraction block to the encoder, and the decoder is embedded with the Temporal Deconvolution block. The MSTCN-VAE model collects more temporal information from the input action sequences due to the advanced time series information integration method and thus exhibits better classification performance. By evaluation of the iMiGUE dataset, our approach outperforms the current state-of-the-art unsupervised methods in micro-gesture classification and is comparable to the accuracy of slightly earlier supervised models. Also, we validate the effectiveness of our model on the SMG dataset.

Resources

• Paper
• Code

Citation

@inproceedings{MSTCNVAE,
  author    = {Wenxuan Yuan and Shanchuan He and Jianwen Dou},
  title     = {MSTCN-VAE: An Unsupervised Learning Method for Micro-Gesture Recognition Based on Skeleton Modality},
  booktitle = {Proceedings of the CEUR Workshop on MiGA Workshop},
  series    = {CEUR Workshop Proceedings},
  volume    = {3522},
  pages     = {Paper 5},
  year      = {2023},
  url       = {https://ceur-ws.org/Vol-3522/paper_5.pdf}
}