If you're blind, finding your way through a new area can sometimes be challenging. In the future, though, you might just need to wear a headset. Caltech researchers have developed a Cognitive Augmented Reality Assistant (CARA) that uses Microsoft's HoloLens to make objects "talk" to you. CARA uses computer vision to identify objects in a given space and say their names -- thanks to spatialized sound, you'll know if there's a chair in front of you or a door to your right. The closer you are, the higher the pitch of an object's voice.
This could quickly become overwhelming, so the team created a trio of modes modes. The most intuitive option, spotlight mode, has objects speak their names only when you're looking at them. If you'd rather get a sense of what the room is like before moving, a scan mode will name everything in view of the HoloLens' cameras. And if you have somewhere specific to be, a target mode lets you focus on just one object.
This could quickly become overwhelming, so the team created a trio of modes modes. The most intuitive option, spotlight mode, has objects speak their names only when you're looking at them. If you'd rather get a sense of what the room is like before moving, a scan mode will name everything in view of the HoloLens' cameras. And if you have somewhere specific to be, a target mode lets you focus on just one object.
The technology is very young and not particularly accessible when it requires an expensive AR headpiece. Early results are promising, at least. Scientists conducted successful tests where blind volunteers found their way through a Caltech building using CARA to tell them where to turn, where the stairs were and even when there were hand rails. In the long-term, the goal is to make CARA-equipped headsets useful in many places. You could visit a store for the first time and navigate the aisles as if you're a regular.
