Stage: Main Stage

As homes become increasingly more technology-driven, inputs from multiple sources—teachable AI, multimodal understanding, sensors, computer vision, and more—will create a truly ambient, surround experience. Already, 1 in every 5 Alexa smart home interactions is initiated by Alexa without any spoken command. As Alexa develops an understanding of us and our home well enough to predict our needs and act on our behalf in meaningful ways, what are the implications for accessibility?

Thanks to mobile phones, GPS, and navigation apps, people who are blind or visually impaired can get around outdoors independently. Navigating indoors is another matter.

For starters, GPS is often not available indoors. Then there are the challenges of knowing where the door is, finding the stairs, or avoiding the couch someone moved. Combining on-phone and in-cloud technologies like inertial navigation, audio AR, LiDAR and computer vision may be the foundation for a solution, if product developers can map indoor spaces, provide indoor positioning and deliver an accessible user interface.

Today, instant access to the written word in braille is much less available to someone who is blind than the printed word is for someone who is sighted. Tools such as single line refreshable braille displays have been available for years, but a single line at a time gives the user a very limited reading experience. This limitation is especially felt when users are reading lengthy documents or when they encounter content such as charts and graphs in a textbook. The American Printing House for the Blind (APH), and HumanWare have teamed to develop a device capable of rendering multiple lines of braille and tactile graphics on the same tactile surface. Currently referred to as the Dynamic Tactile Device (DTD), this tool aims to provide blind users with a multi-line book reader, tactile graphics viewer and so much more. (This session will be followed by a live breakout Q&A session with Greg Stilson, head of APH’s Global Technology Innovation team, and HumanWare’s Andrew Flattres, Braille Product Manager.)

Who knew that screen readers, unlike Web browsers, are not interoperable. Web site developers don’t worry about whether their code will work on Safari, Chrome or any other browser, but if they take accessibility seriously they have to test for JAWS, VoiceOver, NVDA and the rest. That’s about to change, thanks to the W3C ARIA-AT project.
(This session will be followed tomorrow by a live breakout session with King and Fairchild, as well as several other members of the W3C ARIA-AT team.)

The latest features in Microsoft’s Seeing AI app enable the app to recognize things in the world and place them in 3D space. Items are literally announced from their position in the room; in other words, the word “chair” seems to emanate from the chair itself. Users can place virtual audio beacons on objects to track the location of the door, for example, and use the haptic proximity sensor to feel the outline of the room.

All of this is made possible by combining the latest advances in AR, computer vision and the iPhone 12 Pro’s lidar sensor. And that’s only the start.

Apple’s iPhone and VoiceOver are among the greatest breakthroughs ever for accessibility but Apple never rests on its laurels. The next wave of innovation will involve what’s known as “machine learning” (a subset of artificial intelligence), which uses data from sensors on the phone and elsewhere to help make sense of the world around us. The implications for accessibility are just starting to emerge.

As AI-based computer vision, voice recognition and natural language processing race ahead, the engineering challenge is to design devices that can perceive the physical world and communicate that information in a timely manner. Amnon Shashua’s OrCam MyEye is the most sophisticated effort yet to merge those technologies in a seamless experience on a dedicated device.

With ever more powerful computer and data resources available in the cloud, Microsoft’s Seeing AI mobile app is destined to become a steadily better ally for anyone with vision challenges. Co-founder Saqib Shaikh leads the engineering team that’s charting the app’s cloud-enabled future.

When technologists design exciting new innovations, those designs rarely include blind people. Advocates urge us to employ a variety of strategies, Access to technology and information is a civil right. Yet when technologists design exciting new innovations, those designs rarely include blind people. Advocates urge us to employ a variety of strategies, from education to litigation, to ensure accessibility is baked into all future tech and information systems. Harvard Law’s first Deafblind graduate Haben Girma, disability rights attorney Lainey Feingold, and Chief Innovations officer with the DAISY Consortium, George Kerscher will discuss strategies for creating a future fully accessible to persons with disabilities, including those who are Black, Indigenous, People of Color, and persons with print disabilities.

It’s one thing for an AI-based system to “know” when it’s time to turn left, who came through the door or how far away the couch is: It’s quite another to convey that information in a timely fashion with minimal distraction. Researchers are making use of haptics, visual augmented reality (AR), sound and language to figure out the right solutions.