Harvard smartphone app helps people who are blind navigate faster with fewer collisions

Show summary Hide summary

• What Mobilio does
• Designed around real needs
• How it performed in trials
• Why a smartphone-first approach matters
• Next steps

Researchers at Harvard have developed Mobilio, a smartphone app that uses AI and sensors to guide people with blindness or low vision along safe walking routes. Early trials suggest the system helps users travel faster and avoid more obstacles than typical navigation tools.

What Mobilio does

Mobilio combines a phone’s camera, GPS, inertial sensors and, when available, LiDAR to build a live understanding of the user’s surroundings. An on-device computer-vision model analyzes the video feed and generates real-time instructions tuned to the person’s actual walking gait.

The app communicates guidance through customized audio cues and short beeps, steering users toward a destination while warning them about hazards ahead. Its design aims to offer continuous, walk-focused assistance rather than the driver-oriented guidance common in many mapping apps.

Designed around real needs

Before building Mobilio, the team surveyed and interviewed more than 100 people with limited or no sight to identify practical priorities. That research highlighted three central requirements:

• Continuous guidance along straight paths, reducing the need for constant reorientation.
• Accurate, turn-by-turn navigation delivered in real time.
• Reliable obstacle detection to prevent collisions and dangerous detours.

“There’s a need for more data that comes from the perspective of humans walking along sidewalks, or walking outdoors,” said Raymond Liu, a PhD student and co-author of the paper in Nature Biomedical Engineering. Liu, whose older brother is blind, emphasized the importance of testing with actual users. “Having actual people test out your device is absolutely crucial,” he added.

How it performed in trials

In a controlled study reported in Nature Biomedical Engineering, 14 participants used Mobilio alongside a cane. Compared with participants who relied on a cane plus Google Maps, Mobilio users reached destinations about 13% faster and experienced 41% fewer indoor collisions.

The researchers note that conventional navigation apps are often built from a driver’s perspective and lack sufficient pedestrian-level detail for many visually impaired users. Mobilio’s semantic segmentation datasets, by contrast, were trained on images captured from a pedestrian viewpoint, which the team says improves its relevance for sidewalk and crosswalk navigation.

Why a smartphone-first approach matters

Rather than depend on specialized hardware, the project leverages devices millions already own. That lowers cost and increases accessibility, the authors argue, because users do not need to buy custom, expensive equipment.

“Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go,” said Patrick Slade, an assistant professor of bioengineering at Harvard SEAS.

Next steps

The Harvard team plans further testing in more varied, real-world settings to see how well Mobilio adapts to everyday routines and individual preferences. Those studies will help evaluate durability, usability across different environments, and how the system performs over longer periods.