Accessibility from the wheels up: The Waymo fully autonomous taxi

If people who are blind or visually impaired find Uber and Lyft liberating, imagine how they will feel summoning a fully autonomous ride from an app on their mobile phones. But wait, how exactly will they locate the cars and what happens when they climb in? Presenter Clem Wright is responsible for the fully autonomous taxi’s accessibility, and he will be joined by leadership from two organizations closely involved in that effort: The Lighthouse for the Blind SF and the Foundation for Blind Children.

[MUSIC PLAYING]

KIRSTEN KOROSEC: Well, thanks again for joining us today. And I want to start off with Brian and Mark. Let’s kick it off with you. Brian, you head up one of the oldest and largest blind organizations in the Bay Area. And Mark, your organization is in Chandler, both areas that are hotbeds of autonomous vehicle activity.

So I want to set the stage a bit. The emergence of ride hailing has provided a new level of accessibility for low-vision and blind riders. And I’m wondering, is ride hailing with humans behind the wheel enough? Or do you think that autonomous vehicles which would not have a human behind the wheel offers greater freedom of movement and accessibility? So Brian, what is the opportunity that autonomous vehicles provide that human-driven ride hailing cannot?

BRYAN BASHIN: Absolutely. It’s a great question. You know, in 2012, when TNC started in San Francisco, it was the biggest revolution of independent blind mobility probably since the invention of the white cane. But autonomous vehicles have some distinct advantages. You know, of course, the rides may be less expensive because there isn’t a human being in there.

But I want to be topical first and say that as a blind person myself, I can’t independently get into a car and drive. So right now, I have to get into TNC with a person who may or may not have a status with COVID. Autonomous vehicles offer the chance to have no COVID exposure. That’s going to be with us for a few years. So I think first and foremost, that’s what people think of. And there are a couple of other advantages.

Despite what we’d like to say, there’s still a lot of discrimination against the now almost 10,000 blind Americans who use guide dogs. Autonomous vehicles will not have that discrimination. Anyone gets in the vehicle. And maybe most profoundly, there’s a kind of deeper social equity.

Sometimes, we just don’t, as blind people, we don’t want another person to be in our business. We want to just go somewhere. We don’t have to answer questions. We just want to get there, be independent. And so the equity of being able to go, just like anybody else, on a mission, is really profound, and will be a huge advance.

KIRSTEN KOROSEC: Mark, I’m wondering what your viewpoint is. Are there accessibility barriers that you think that autonomous vehicles are in a unique position to remove?

MARC ASHTON: I think with all technology, when we had first text to speech software, it was clunky and what have you. But technology has brought it leaps and bounds over the last couple of decades. And I think with autonomous vehicles, the technology is going to move very quickly, and any problems that do come up with accessibility, I think that technology will build to solve.

I mean, the ability for just putting the features that some of these autonomous vehicles are doing, which is, like, a beeping– the ability of riders to beat the car’s horns so they can find a vehicle. For the Braille inside the vehicle. For the vehicle to explain to the rider where they are at any given time, how far they are, the distance. All that is progressive. So they keep adding these features as they get feedback from riders but also just from the community and in general.

And all those features, to be quite frank, are just as helpful for sighted people. Being able to find your car by touching a button on your iPhone is pretty global. And so [INAUDIBLE] great. And I think one of the [INAUDIBLE] things about autonomous vehicles is just the safety factor. Just the– you really aren’t getting in a car with a stranger.

And we all love ride hailing, and it’s been a great transformation. But you’re still getting in a car with a stranger. And some people, that’s very uncomfortable with. And they don’t know that person. They can’t, especially if they’re visually impaired, they can’t identify that person later. And so I think it’s a very — the non-driver is a big safety factor.

KIRSTEN KOROSEC: Clem, I promise to get to you in a moment. But I did want to kind of get a perspective from Brian because I know that, Brian, I know your organization isn’t working specifically with Waymo, but you have worked with other AV companies. And before we delve into Waymo’s process, I’m wondering, what companies, when you were working with them, what were the design features that they might have been overlooking that you’ve just seen across the industry in terms of specifically making their vehicles more accessible to blind users?

MARC ASHTON: It’s a great question. We at the San Francisco Lighthouse really love working with all the many dozens of autonomous vehicle companies in the Bay Area. There are a bunch of things. You know, when you take a driver out of a vehicle, suddenly, things that are driver used to do have to be thought of in a different way with technology. That’s a very exciting design challenge.

Let me just list a few. You drop off at Costco. OK. Where at Costco? An acre away from the front door? How do I tell the vehicle to get me to the front door? You know, I know using GPS, sometimes there’s a strip mall, and there’s one address for 30 different stores. When you get out of a vehicle, who will now tell me like a driver does, OK, the stairs are about 30 feet over there on the left.

Those things are practical for blind people. Will we be able to design a system so that when a blind person changes her mind and says, oh, I need I forgot. I’ve got to get a bottle of wine for this evening, that you can add a stop in the middle of the trip excessively. Or how do you tell the vehicle, wait here while I just dive in and pick something up. I’ll be right out in 3 minutes. How do I find this vehicle when the symphony lets out and there are hundreds of vehicles stretched out on the street?

And then some practical, physical things. Will the vehicle itself be such that I can open the trunk myself? How can I trust adjusts the heater and the air conditioner? Even in my current– I own a Forerunner– I can’t work the radio on that car currently.

What happens if I need to suddenly lock or unlock the doors inside the compartment? I don’t want to have to go and fool around with an app on my phone to do that. I need a direct way to do that. How do I put on emergency blinkers?

You know, blind people in general want to be active in this process. That’s what we tell designers. We don’t want to be passive spam in the can, as the old mercury astronaut said. We want to be able to do everything sighted people do in a vehicle, and I think clever designers are figuring out ways to do that.

KIRSTEN KOROSEC: Well, that’s the perfect tee up to Clem. So you just got a massive checklist. And I’m sure that you’ve hit all those marks. So why don’t you, Clem, walk us through the how Waymo has approach not just the design of its vehicle, but importantly, also the ride hailing app which is known as Waymo One in the Chandler area, to make it accessible to all user groups, and in this case specifically, for blind and low-vision users?

Sure. Yeah, that’s a great list, Bryan, and definitely a number of things that we’ve thought about that you and I have discussed as well.

BRYAN BASHIN: Yes.

CLEM WRIGHT: So we do think of– our overall approach is not to think of here are a couple of features that we’re going to build just for low-vision users or for users with other disabilities. It’s about thinking about the entire user journey from the first time you install the app and start onboarding with Waymo to getting dropped off at your destination, and every step in between. And thinking about what the needs are at those different points in the journey.

And different people are going to have different needs, and we want to make sure we meet those throughout. So a number of things that Brian called out like finding a car at pickup, finding the destination at drop off– really massive challenges. And we found that by doing user research with users with diverse abilities, we’re able to pick out certain needs and build certain features that can help everyone.

So for example, the honking the horn feature that Mark called out. This was something that we did build. Or more originally hey, this is something that we know that low-vision users are going to need. So we built this out, and we tested it. And we actually were running a separate usability study, a totally different feature, for all sighted users in a study, and they were trying to find the car pickup just as part of the flow.

And we accidentally left the honk horn feature on. It was just a prototype phase at that point. And the person saw it and was like, oh, I need to find the car. They honked the horn, and the car was pulled up around the back of the hotel where we were testing.

And it was it was a huge light bulb moment for me where it was like, oh, yeah. Obviously, this is going to be helpful for everyone. And we’ve seen that time and again in a number of different features we’ve built where we’ll think of more extreme needs that people with certain disabilities may face and brainstorm features around them, but then realized, hey, this is something that affects everyone.

KIRSTEN KOROSEC: So you bring up an interesting point, which is that luckily, you sort of kind of had this aha moment and sort of stumbled on it. But oftentimes, what can happen in tech companies is that you don’t have that aha moment because either your workforce doesn’t reflect a diverse user group or you’re not actively going out into the community.

So what is Waymo doing? I mean, do you have people on staff, for example, who are low vision or blind? Or are you actively working with folks in the community to be testing those features to think of– like, to do those– hopefully find those holes in the design?

CLEM WRIGHT: Yeah, I would put it our research into maybe three buckets. One is foundational research, where even before we’re thinking about a specific problem or a specific feature, just going on ride-alongs with people who are low vision or have different abilities and saying, how are they using Uber and Lyft today? How are they using trains? How are they getting around? Understanding some of those challenges. And that sort of builds up a bit of a foundation for some of our understanding of the problem that people are facing.

And then another bucket would be, once you’ve identified some of those problems, running focused user research studies on, hey. We’ve developed this feature, and we’d really love feedback to understand what’s working. And then the third is the ongoing testing.

And so we do have some folks at Waymo and some folks at Google who are testing our cars who have vision disabilities. And we also have folks in the Chandler area who have been part of past user research studies and are continuing to use our cars today. So we’re getting that ongoing feedback where they’re talking to our writer support agents or they’re leaving feedback in the app and helping us know that we’re on track.

KIRSTEN KOROSEC: So Mark, I wanted to ask you, you have a– you’re leading an organization that focuses on children, blind children. And so are there some design– on your checklist, what do you think autonomous vehicles should be thinking about– autonomous vehicle companies, excuse me– should be thinking about in terms of designing for children, specifically teens and young people, that might be different than the needs of an adult blind person?

MARC ASHTON: I think Brian said it best, is that we all had the memory of when we turned 16 and we got to get the keys to the family car, and take off, and leave home, and find freedom, right? And our kids want the same thing when they turn 16. Unfortunately, that’s not possible today with their vision loss. So they really just [INAUDIBLE] that freedom. That ability to rebel and disappear from being reliant on mom and dad like every other child out there, every other teenager out there.

And honestly, with the advent of the ride hailing and all these great organizations that are doing this now, kids younger and younger are going out and taking an Uber somewhere at younger and younger ages. And community families are more comfortable with that.

So I think this is obviously the next evolution of that. The autonomous vehicle allows 14-year-olds to go to the movies. Yes, our kids do go see the movies. Find the In-N-Out Burger. Do all those things that every other 14-year-old is doing. And it’s just that last technological advance I think that really gives our kids the same freedom as all those experienced with sight.

KIRSTEN KOROSEC: So it sounds like– sorry to interrupt, but it sounds like you see this more as an evolution and not necessarily a very specific design request that would address for the needs of blind children or blind teens, but more of that as these vehicles come into your city or community that people will just automatically go towards this. Do you see any other gaps specifically?

MARC ASHTON: Well, obviously, there’s. Going to be the adoption of new technology is going to be troubled and problematic, as it is with everybody. I mean, there were people who got the smartphones first, and then now, everybody’s grandma has a smartphone. It’s not a big issue.

But there is– the fun part that we’ve had, because Waymo’s doing it here in the Phoenix area. So every once in a while, they get all of us together, all of our students. And we have adult students as well as children and teenagers. And they just say, what’s your wish list? And when you get a group of people sitting around table and saying, ooh, it would be really cool if it did this. It would be really cool if it did that.

And then suddenly, Clem’s writing every note he possibly can, just like Brian gave him a full new list. And then they, I’m guessing, Clem, you send it out to your staff and say, can we do this? Or we should prioritize this. So it’s going to come. I mean, it’s just like the voice over opportunities, that voiceover was probably designed from blind and visually impaired, but everybody uses it. I mean, everybody talks to– hey, Siri. What time is it? Or hey, Siri. Remind me to call somebody at such and such time.

And that makes life for our blind and visually impaired community so much easier, and I think that’s what this autonomous vehicles going to do. It’s just going to be an evolution and hopefully– put it this way. I’m in Phoenix. Scottsdale is right next door. Scottsdale is the only town that still have horses hitches out front. It’s a Western town. But those were everywhere 100 years ago. Now, the technology has moved on beyond horse hitches, and that’s where I think autonomous vehicle’s are going to go. They’re going to be second nature very soon.

KIRSTEN KOROSEC: Right. We’ll have a different kind of hitch, right? So Clem, I’m wondering– and Bryan, I do have a question for you next, but I want to get to Clem first. Were there any design choices that Waymo initially made– I mean, we talked about one of the innovations, which was the sort of the honking horn.

But were there any design choices that Waymo started going down and realized after testing with different user groups that this was just missing the mark? And how did you correct it? Because I think it’s important in that when you see the end product, you understand sort of those challenges, hurdles, even sort of mistakes that were made along the way to get to this end point.

CLEM WRIGHT: Yeah, I think that’s a great question. There are trade-offs throughout. And one example is everyone wants to get from A to B as quickly as possible. Ideally, you have a nice, short trip. Sometimes, we can shave minutes off of your trip by pulling over to pick you up a half a block away or around the corner from where you are.

And because we don’t have to make that left turn, you don’t have to face additional traffic, maybe it’ll save you a bunch of time on your trip. For a lot of users, half a block, an extra block walk, not a big deal. Maybe crossing the street, not a big deal. And so we develop some systems to help make those optimizations.

And then in thinking through it, we didn’t get to the full point of rolling this out, but in thinking through it, we said, hey, actually, there are people who we’ve talked to through our foundational research that actually, walking across the street, even if it’s a short street, is a big challenge. Finding a car that’s a block away from where you expect it to be is a big challenge.

And so we built a setting in the app and gave that control to users to say, here’s an option. You can flip this on, and that’ll minimize your walk with the pick up and drop off, even if that’s going to add a couple minutes to your trip. And it sort of set things up so you have a bit of control to make that choice and that trade-off. And yeah, we’ve, seen situations like that drive development.

So Bryan, you gave this wonderful checklist, and you’re clearly working with a number of companies. So I want to get your perspective on whether the government, federal, state, or local, should have a role in dictating the design or operations of autonomous vehicles specifically as it relates to accessibility issues.

Or do you think that the way it’s working now, which is communication between organizations and the companies that are designing these products and programs like– I think of the DOT just funded the Inclusive Design Challenge, which was specifically focused around– it was an AV design competition. Is that enough? Where would you like to see things or where do you think things should move forward before you see widespread commercialization of AVs?

BRYAN BASHIN: So many good questions. Well, first of all, I really trust the smarts of people like Clem at Waymo who are asking the great questions and involving people with disabilities. You know the old phrase– nothing about us without us. Really critical.

Let’s see what industry invents. You can bet that industry will invent all kinds of things that government will never think of. But after a while, I think just as the web standards are being standardized now with WCAG as an international standard– there will be years from now. I’m sure there will be international standards.

But until that time, there are big questions that we need to put companies to chew on and work on. Questions like, guess what? Not everybody’s an English speaker. In California, half of– 48% of kids in school now speak Spanish at home. Will these AVs be able to be compliant in Spanish and Chinese and other major linguistic groups? Or do you have to be a member of the dominant culture to operate them?

There’s other questions like cognitive questions with the silver tsunami. Everybody’s microwave is flashing 12. People haven’t been able to figure out how to even get that technology right. Would your grandparent or grandmother be able to use this? Let’s not have a kind of technological apartheid.

Even requiring a smartphone. We’re not there yet. And so I know for the ride hailing, there are other groups. Go Go Grandparent comes to mind. But others, I want those groups not to have to mess with an app, no matter how elegant an app it is. I don’t want to shut them out of the promise of autonomous vehicles.

If we do it right, the blind and visually impaired, we’re going to be the biggest legislative advocate for early adoption. I don’t really think the big challenges here for autonomous vehicles are technical. I think they’re going to be fighting with insurance companies and fighting with perceptions of risk in state legislatures. And that’s why it’s really important that smart people like Clem dialogue with the blind community because we 5 million, 7 million of us who have no other independent way of driving will be in those state houses arguing that this will be the next revolution for blind independence. So I’m excited about the future that way.

KIRSTEN KOROSEC: You bring up an interesting point, which is as someone who writes for a tech-focused media outlet, it’s easy to forget that not everyone is comfortable with a smartphone or an app. And Clem, I’m wondering how Waymo is even thinking through that because the Wamyo One app is a big piece of communicating and hailing a vehicle that does not have a human behind the wheel. So have you given any thought to that? And have you considered even ways of hailing the vehicle– ad I know this is early days, but hailing a vehicle without a smartphone?

CLEM WRIGHT: Yeah, it’s a great question. I think, to make very clear, it is still early days here. We still we know there’s so much more that we have to do as Waymo to deliver on the vision that we’ve been talking about of someone feeling like they really have the freedom to go about and be on their own. Where we are right now is using an app to hail, and there are a bunch of ways we can help people ride in our cars with that.

We definitely are excited about the future, and it’s funny how it almost goes full circle of the technology we love to get to is you can walk out of the street and wave your hand and a taxi pulls over and picks you up. There are a lot of benefits to being able to [? hail ?] ahead of time and those things that an app affords. But I think it’d be really exciting to see that.

We’ve definitely talked about it. Right now, we’re really focused on, hey. We have people taking rides in fully autonomous vehicles. No driver in it, right today, right now, in the Phoenix area. And we’re working to make sure that the tools we’re giving them are helping them.

KIRSTEN KOROSEC: So one of, for those who have never ridden in an autonomous vehicle or specifically the Waymo driverless in that’s going around Chandler, so there’s no human safety operator behind the wheel. I always say that because a lot of testing is being done with a human operator behind the wheel. But in this case, for those who haven’t been in that, you walk in the minivan door of the Chrysler Pacifica. Hybrid minivan door opens, and there are these screens that are on the back of the head rest of the front seats that show your ride progress.

And so Clem, how is that communicated to a blind user? And that they know that they’re in the right vehicle? There might be a time in the future where there will be multiple autonomous vehicles lined up. So can you get specific about the user interface once you’re in the vehicle about how that’s communicated? I’m assuming voice, but maybe there are some other ways of communicating that Waymo has employed within these vehicles.

CLEM WRIGHT: So you’re right. You get into the vehicle, and there are passenger screens. We’ve worked really hard to duplicate as many of the actions as we can that are core to the experience that are available on the passenger screens or available through physical buttons in the car in the app as well.

So you can start the ride from the app. You can, if you feel for some reason you need to pull over, you can pull over from the app. You can then resume your ride. We’re excited about in the future, as Brian mentioned, letting you play music and enjoy the ride in that way from the app.

Voice does play a key role. So for people who can’t see the screens, we’ve made sure to duplicate the key messages that show up on the screen playing through the in-car audio. We also have a setting that users can enable that gives them even more assistive audio, we call it. So if it’s your first ride, you can enable the setting before you get in, and the audio will play. And it will do things like you said of confirming your identity and saying, like, hey, Clem. You’re in the right car. Wait a minute. I’m not Clem.

It will explain where the buttons are to start the ride. It looks like some of those key things. And also, as you’re going, there are moments you encounter while driving– the car may be yielding for a pedestrian to cross, or turning onto a big highway.

If you’re blind, you can’t see that. Don’t have that context. The assistive audio helps provide that for you as you’re go along. We also, of course, I’ll just add, we have– rider support is always available. So from the app or from physical buttons in the car, you can speak to a human agent. And they can help explain situations for you as well.

KIRSTEN KOROSEC: So this came up earlier today but also in a previous conversation, this idea of pick up and drop off points. And that seems to be sort of a critical point. And I want to ask. Clem, you about it in a moment. But Marc, I wanted– you were the one who originally brought this up in a previous conversation. And especially when you’re dealing with potentially young people who are ensuring that they really get from point A to point B without an adult with them.

And so I’m wondering what your perspective is on that. And specifically, do cities have a role in designating drop off and pick up points? Not necessarily that the industry always has to adhere to, but those areas where maybe there is going to be a lot of kids using it, for example? Do you have any perspective on that, drop off, pick up point?

MARC ASHTON: Yeah, Kirsten. It’s a great thought because we do this already. Cities already have handicapped parking spots, for example. So they’ve already designated part of the parking lot or part of the sidewalk or whatever that is very specific for people of needs. And so that may be an evolution here is that the purple zone is for riderless autonomous vehicles drop off and pick up points. And that may be where businesses and/or I can imagine a hotel or anything like that would say, OK. I’ve got to paint, as part of my development, I got to paint this curb purple, and I got to do the yellow one for drop off [INAUDIBLE] people. Then I got to do a red one for fire.

So it could be as simple as that. And all that takes time. The thing about all this is that innovation is evolutionary. It takes time to, A, discover, and B, for ideas to come up. And I think that I’m not– I don’t want government to be part of the innovation. I want government to be helpful in the protection.

But I also want them to protect on the back end. Let us try at first and then say, OK. Here’s an idea. We’ve got to protect. We’ve got to start painting curbs purple. It’s just something like that, it’s the afterthought. And because I think if we try to answer all the questions ahead of time, we’re never going to turn it on. Right? We’re never going to get there.

KIRSTEN KOROSEC: We’re almost out of time. But Bryan, I wanted to give you a chance to weigh in. Is there a role in just the design of how our cities are designed on the infrastructure side to allow for AVs not just to navigate more easily, but specifically on that drop off, pick up points?

BRYAN BASHIN: Absolutely. In fact, in San Francisco, there’s work to be done. There’s work evolving now for specific ride hailing drop off points. So many of the things we love in cities, the density and the fun, involves a real congestion. And for a blind pedestrian, knowing this is where, if you stand here, the vehicle will come here, that’s amazing security.

In the early days of the chaos of all this other stuff, just like the chaos of those scooters when they were deployed until they actually were legislated into some kind of system by the cities, that’s what we’re facing. And designated pickup and drop off points were definitely one of the many solutions involved.

I think the bottom line here is blind and visually impaired, we want to be co-designers. As we heard Clem say, we want to be at the get-go so that we don’t have this vehicle and then say, oh, what should we do with it? How should it work?

No, like it or not, these things will become part of public utilities. And as public utilities, we’ll need to start legislating ways in which they operate in the wider world. So it’s exciting. You know, we’re inventing the future here. Henry Ford said, if I asked people what they wanted, they would have told me to invent a faster horse.

No. We’re not going backwards here. We’re inventing how cities should be, how vehicles should be. What the relationship between the demographics of disability and technology should be. I can’t imagine a more exciting time to live.

KIRSTEN KOROSEC: OK, Clem. I’ll give you the very final, very briefest of words. How is Waymo thinking about pick up and drop off points? And specifically, because Bryan brought this up, has Waymo considered doing sort of a very specific geolocation, something similar to what a company like what Three Words does where it’s not a physical address, but actually a very specific point?

CLEM WRIGHT: Yeah. I mean, pick up and drop off is such a complex problem. We bring in as much data as we can to figure out where our server side system can suggest, OK. We think this is going to be where the rider, given where their GPS is, where they expect to be picked up, where their destination is, where they expect to be dropped off. We try to bring in information about the access points of the building. Handicap parking location. All sorts of information.

We give we give that information to the car. The car that needs to arrive on the scene and figure out, OK. Given that information, actually, that spot that you told me you go, there’s another car parked there right now, or there’s a fire truck, or there’s a pedestrian there, and I actually need to scoot around.

So there’s so many levels to this. And it would be wonderful to have dedicated places where we know we can reliably always find a spot, but we’re building our system to be robust to all those things that can go not according to plan and make sure our user experience can meet them on the other side of that and get people to where they’re going.

KIRSTEN KOROSEC: Great. Well, thank you, Bryan, Marc, and Clem for joining us today, and we’ll kick it back to Will.

[MUSIC PLAYING]