PersonalRAC Personalized Few-shot Exercise Counting

Abstract

The task of Repetitive Action Counting (RAC) is an area of increasing research interest, with numerous technologies being developed in the field. However, existing state-of-the-art

methods, trained on currently available generic datasets, are not fit for recognizing personal- ized movements. Such capability has the potential to benefit application fields like physiother- apy and fitness by enabling the creation of unique tailored exercises for patients or clients and

tracking them with minimal effort. To address this issue, we introduce the Personalized Rep- etition Action Count (PersonalRAC) system, a novel method capable of counting actions in

low-data scenarios (i.e., implementing a few-shot learning approach). PersonalRAC integrates Few-shot Learning, Repetitive Action Counting, and Skeleton Action Recognition. Our system operates with minimal training examples on untrimmed videos by autonomously identifying start and end points, facilitating the easy registration of new exercises. To achieve this, we leverage the concept of salient poses, annotating a subset of Fit3D dataset for this capability

and proposing a few-shot division of it. The system processes videos of users performing ex- ercises and extracting skeleton information using MediaPipe. The information is processed to

make it more reliable for the next stage. The MotionBERT model for action detection analyzes

this processed information, and the output passes to a repetition counting module. Experi- mental results demonstrate the system’s effectiveness and robustness in accurately counting

repetitions across various exercise types. Our system achieves state-of-the-art performance in the few-shot and few-shot multi-cam settings on the Fit3D dataset, with respectively MAE of 0.33 (44.07% improvement) and an OBO of 0.64, and 0.22 MAE (53.19% improvement) and 0.71 OBO.