Archives: Sessions

Who knew that screen readers, unlike Web browsers, are not interoperable. Web site developers don’t worry about whether their code will work on Safari, Chrome or any other browser, but if they take accessibility seriously they have to test for JAWS, VoiceOver, NVDA and the rest. That’s about to change, thanks to the W3C ARIA-AT project.
(This session will be followed tomorrow by a live breakout session with King and Fairchild, as well as several other members of the W3C ARIA-AT team.)

The latest features in Microsoft’s Seeing AI app enable the app to recognize things in the world and place them in 3D space. Items are literally announced from their position in the room; in other words, the word “chair” seems to emanate from the chair itself. Users can place virtual audio beacons on objects to track the location of the door, for example, and use the haptic proximity sensor to feel the outline of the room.

All of this is made possible by combining the latest advances in AR, computer vision and the iPhone 12 Pro’s lidar sensor. And that’s only the start.

Apple’s iPhone and VoiceOver are among the greatest breakthroughs ever for accessibility but Apple never rests on its laurels. The next wave of innovation will involve what’s known as “machine learning” (a subset of artificial intelligence), which uses data from sensors on the phone and elsewhere to help make sense of the world around us. The implications for accessibility are just starting to emerge.

As AI-based computer vision, voice recognition and natural language processing race ahead, the engineering challenge is to design devices that can perceive the physical world and communicate that information in a timely manner. Amnon Shashua’s OrCam MyEye is the most sophisticated effort yet to merge those technologies in a seamless experience on a dedicated device.

With ever more powerful computer and data resources available in the cloud, Microsoft’s Seeing AI mobile app is destined to become a steadily better ally for anyone with vision challenges. Co-founder Saqib Shaikh leads the engineering team that’s charting the app’s cloud-enabled future.

When technologists design exciting new innovations, those designs rarely include blind people. Advocates urge us to employ a variety of strategies, Access to technology and information is a civil right. Yet when technologists design exciting new innovations, those designs rarely include blind people. Advocates urge us to employ a variety of strategies, from education to litigation, to ensure accessibility is baked into all future tech and information systems. Harvard Law’s first Deafblind graduate Haben Girma, disability rights attorney Lainey Feingold, and Chief Innovations officer with the DAISY Consortium, George Kerscher will discuss strategies for creating a future fully accessible to persons with disabilities, including those who are Black, Indigenous, People of Color, and persons with print disabilities.

It’s one thing for an AI-based system to “know” when it’s time to turn left, who came through the door or how far away the couch is: It’s quite another to convey that information in a timely fashion with minimal distraction. Researchers are making use of haptics, visual augmented reality (AR), sound and language to figure out the right solutions.

Map apps on mobile phones are miraculous tools accessible via voice output, but mainstream apps don’t announce the detailed location information (which people who are blind or visually impaired really want), especially inside buildings and in public transportation settings. Efforts in the U.S. and U.K. are improving accessible navigation.

Technologists like to imagine how their work affects people, but that’s no substitute for truly knowing the real impact on lives, or better yet, understanding what people, especially people with disabilities, really want from their surroundings and community. In her recent book, What Can a Body Do? professor and designer Sara Hendren’s “aim … isn’t to throw cold water on innovation; it’s to recenter the people, behind the tools, who must work with their surroundings, their adaptations at least as miraculous as the technology that helps them.” (Katy Waldman, in her New Yorker review)

For an AI to interpret the visual world on behalf of people who are blind or visually impaired, the AI needs to know what it’s looking at, and no less important, that it’s looking at the right thing. Mainstream computer vision databases don’t do that well — yet.