Spending hours daily for more than five months, creating AI models, learning algorithms and conducting countless trials, junior Jony Li completed his mobility assessment app that won him second place in the recent Congressional App Competition, announced late Dec.. Jony hopes that SmartGait, his product consisting of a lightweight VR headset, smart insoles and video-based analysis, will revolutionize the medical field by eliminating factors that continue to limit patients with declining mobility from getting help and doctors from providing care.

The main driving factor behind SmartGait was Jony’s frustration with the current mobility assessment system. Witnessing his own grandmother’s decline in mobility, Jony decided to create a tool that would help decrease the large number of fall-related injuries, with over 14 million seniors falling in the United States each year and nine million resulting in injuries, often due to mobility issues or gait (walking pattern) problems.

“My grandma has experienced a severe mobility decline over the past year due to her aging. With my parents both busy at work, I often accompanied her to physical therapy. During this time, I had become frustrated with the therapists working with her,” Jony said. “They would often say that my grandma looked healthy, yet she had been rushed many times to the hospital due to fall-related injuries just weeks after the checkup.”

SmartGait is an app that can be downloaded on VR headsets and provides customizable environments and walking activities for patients with mobility struggles to be assessed. The smart insoles inside a patient’s shoes record foot pressure distribution. To complete the app, the video analysis feature, created using OpenPose, detects abnormalities based on joint angles and stride.

“As I looked deeper, I found out that infrequent clinic visits and expensive mobility assessment tools were the main cause,” Jony said. “Detecting anomalies during gait analysis was quite challenging. Walking is highly individual and influenced by fatigue, recovery and environment.”

Amidst the heavy workload and chaos of junior year, Jony invested lots of time in SmartGait. Before figuring out how to make his model detect the abnormalities that are causing patients mobility troubles, he had to establish his foundational knowledge of such AI models.

“I was not knowledgeable in AI anomaly detection models, so I spent a while learning the algorithm and conducting trials for it to work,” Jony said. “Research on mobility assessment had been conducted in the past, so I spent time reading through articles on their methodologies and the expense of it. I also consulted with my parents, who work in the technical field, and professionals to learn more. In the end, I chose to use smart insoles and video-based analysis to measure foot pressure and joint angles.”

The complexity of detection models was not the only thing that proved to be a challenge during the creation of SmartGait.

“The main challenge was defining what to measure,” Jony said. “My computer could only analyze so much data at once; it was definitely troublesome for me to find which factors were most important for mobility assessment.”

Jony’s younger brother, Ethan Li, saw his willingness to work through each challenge that came his way throughout the months-long creating process.

“Jony faced many challenges during the development process of his app, including debugging features and balancing the technical requirements with user experience,” Ethan said. “I am happy that he won second place at a prestigious competition. It is a huge accomplishment.”

So far, SmartGait has been successful. Jony plans on further enhancing his app over time and improving its effectiveness and convenience, sticking to his goal of allowing more senior citizens to more easily work through mobility obstacles that worsen with age.

“Early testing with a 71-year-old post-surgery showed that the system was intuitive and comfortable to use during daily activities,” Jony said. “Although the results are promising, there are many aspects of the project I still need to improve. I hope that in the future, I can further reduce the cost while also creating an algorithm precise enough to make a near-perfect analysis of a patient’s gait and mobility. [This isn’t] just my grandma’s issue; it is a national healthcare issue.”