Posture Coach
Posture Coach is a group final-year capstone project for the Bachelor's of Engineering in Computer Science at Dayananda Sagar College of Engineering.
TL;DR
Posture Coach is a cross-platform mobile application that aims to help prevent injuries and improve the quality of people's workouts with just a smartphone. It uses the device's inbuilt camera to detect the user's exercise pose and provide personalized, detailed recommendations on how the user can improve their form in real time.
The Inspiration
Fitness is a vital component of our health and well-being. Due to the circumstances caused by the COVID-19 pandemic, most people did not have access to a gym or fitness equipment; many resorted to exercising at home. People became dependent on technology to receive appropriate fitness guidance and training.
Exercises such as squats, deadlifts, and shoulder presses are beneficial to health and fitness, but they can also be very dangerous if performed incorrectly. Most people need guidance to maintain correct posture while exercising. Technology can monitor and provide this guidance using Pose Estimation.
Demo
Design and Implementation
Pose Estimation
The PoseNet algorithm is a pre-trained model for pose estimation available in TensorFlow Lite and TensorFlow.js, making it straightforward to implement in mobile applications. It takes a processed camera image as input and generates 17 key points of body parts along with a confidence score between 0.0 and 1.0.
Keypoint Normalization
We built a custom parser for raw keypoint data and normalised the pose based on torso length in pixels to accommodate for varying body proportions, camera distance, and other factors. The average distance between the neck keypoint and the right and left hip keypoints is used to compute torso length. Distances are expressed as ratios of torso length.
Perspective Detection
For exercises like the bicep curl, we resolve ambiguity in camera viewpoint by analysing which keypoints are most apparent (left or right side) across all frames of the activity. We compare keypoint confidence scores for each side to determine which arm is conducting the exercise.
Evaluation
We perform exercise pose evaluation using a geometric approach. We determine metrics for each exercise, compute body vectors from keypoints of interest, and use personal training guidelines to design geometric heuristics that evaluate those vectors.
Feedback
The application provides real-time audio and text feedback as the user exercises. Using evaluation results, it delivers positive reinforcement ("Good work", "Well done") for correctly performed exercises and corrective suggestions ("lift your arm higher", "squat lower") for incorrect form.
Exercises
Currently implemented: Bicep Curl, Shoulder Press, Front Raise, and Squats.
Resources
The code is available at my GitHub repo.