Getting real-time feedback on your workout form usually involves a clunky setup. You either hire an in-person fitness expert or subscribe to one of those services where a trainer watches you remotely through a camera and guides you on your posture and the entire routine. Modern smartwatches can track your exercise stats, but they can’t offer posture corrections or insights into the whole activity without proper information about how your entire body is moving.
But what if your smartwatch could track your full-body movement without any additional hardware? Well, that’s exactly what researchers at Cornell University’s Smart Computer Interfaces for Future Interactions (SciFi) Lab have just accomplished. The team developed a system called EchoMotion that relies solely on ultrasonic sound waves, emitted and captured by the built-in speaker and microphone of a mass-market smartwatch, to work out a person’s body position.
It works much like sonar. On a commercial smartwatch, it’s an entirely software-driven system, and all it needs is the watch’s existing speaker and microphone to send out sound waves and pick up their echoes. Another huge benefit is that the whole system is developed atop a lightweight AI model and the whole package is less than 30MB in size. More importantly, it doesn’t take up system resources and can be easily deployed on Wear OS smartwatches, as per the team behind the project.
The broad objective of EchoMotion is to help users track their workouts more accurately with a realistic picture of how their body is moving, keep an eye on their daily activities, and, in a healthcare setting, assess their rehabilitation exercises.
EchoMotion solves one of the biggest challenges facing current-gen smartwatches. If you look at the Apple Watch or any other mass-market smartwatch from the likes of Samsung or Google, they rely on built-in accelerometers and gyroscopes to track motion. This information is used not just for workout tracking, but also to detect movement during sleep, which forms the foundation of features like sleep apnea detection. However, these sensors can’t provide a full-body picture of motion, whether a person is sitting, walking, or exercising.
EchoMotion’s speakers emit ultrasonic signals that surround the person in a sort of sound-wave bubble. The microphones then pick up the echoes that bounce off the wearer’s body, including their arms, legs, torso, and everything else. These echo patterns are passed through an AI model that reconstructs the person’s full-body pose, somewhat like the 360-degree camera rigs used to capture a surround view of an actor in films.
As part of the first test, the team had 10 people wear a prototype smartwatch and perform 18 types of physical movements. Once the activity data was collected, the researchers used the AI model they developed to predict the positions of 12 major body joints, all the way from the ankles up to the head.
The researchers then put the model on an unmodified Samsung Galaxy Watch 6 and tested it with 15 new participants. For users the system had never seen before, the average error per joint was just under 4 inches, without any calibration. However, once a person wore the watch and recorded their movements for just 12 minutes, that data was used for calibration, bringing the error margin down to about 2.4 inches.
And here’s the most interesting part. The team also deployed the same model on a Samsung Galaxy Watch 7 and a Google Pixel Watch 3 without any retraining, and EchoMotion worked on these devices too, with only a slight dip in accuracy. The broad takeaway is that the tech has a clear path to real smartwatches. If it can run on watches that are a few generations old, porting it to current models like the Samsung Galaxy Watch 9 or the Google Pixel Watch 5 shouldn’t be a stretch.
The one big caveat is people. When participants performed their movements in a busy building hallway, motion-sensing accuracy dipped because of interference from other people walking nearby. Interestingly, cluttered furniture and noise from a busy road barely made a difference.
Beyond smartwatches, the team is also exploring how EchoMotion could work alongside smart glasses. If that pans out, I think it could be a huge breakthrough. When it comes to XR wearables, tracking a person’s body for gesture input, especially in video games, is fundamentally important. So far, body tracking has proven to be a huge challenge, because it requires sophisticated sensors that add to the cost and bulk of XR devices, whether it’s a VR headset or a pair of smart glasses.
Another crucial win for EchoMotion is power efficiency. The ultrasound-based sensing on the prototype draws only about 12 milliwatts, roughly 40 times less than a wearable camera system. During testing on the Galaxy Watch 6, EchoMotion used only about 5 to 6% of the battery over a 30-minute tracking session.
The most promising aspect is that EchoMotion runs on an entirely unmodified, off-the-shelf smartwatch. It doesn’t require any custom hardware, such as pricey ultra-wideband or mmWave chips, or even an onboard camera. Everything happens using the built-in speaker and microphone, so it adds nothing to the bill of materials.
As far as benefits go, the researchers built an exercise-tracking demo on the Galaxy Watch 6 that recognized 20 gym workouts, such as dumbbell lateral raises, leg presses, chest presses, and squats, with 95.4% accuracy. Thankfully, it even worked in real gyms where background music was blaring at around 76 dB.
Another crucial benefit is that the tech doesn’t rely solely on wrist motion. Instead, it accounts for full-body joint positions and angles, which the researchers believe could open the door to workouts that need fine-grained feedback, such as Pilates, tai chi, and yoga.
The tech could also be a massive help for people recovering at home, especially patients doing rehabilitation exercises after serious injuries. Down the line, this kind of motion data could help healthcare experts provide timely interventions and suggestions to the people who need them.
“We have made significant efforts in making EchoMotion work on commercial devices, so that the adoption can be very easy,” said Saif Mahmud, lead author of the paper and a doctoral student at Cornell, noting that EchoMotion is a huge milestone for body tracking.
