The Screen Reader Interoperability Gap – A Hidden Systemic Barrier

At Sight Tech Global two years ago, we unveiled the ARIA AT initiative, which aimed to address the frustrating, damaging reality that screen readers are not interoperable on the Web, unlike their cousins for sighted users – browsers like Chrome and Safari. In other words,any developer that takes accessibility seriously has to test their code on JAWS, VoiceOver, NVDA and the rest. In this session, the people advancing the ARIA AT project  are back with a refresher, progress to report, and a call to action.

[VIDEO SOUNDTRACK PLAYING]

SPEAKER 1: On a vintage computer, web pages cycle through. Some work and some show incompatibility errors.

SPEAKER 2: Back in the 1990s, web browsers were all over the place. A website that worked in Netscape didn’t always work in Internet Explorer or the other way around. Only big companies with dedicated teams could afford to make websites work in multiple browsers.

SPEAKER 1: On a split screen, several hands type, and one uses a mouse.

SPEAKER 2: To fix this, thousands of people from dozens of internet companies worked for over a decade to standardize the web. It was a massive investment of effort that improved the state of browser interoperability. But there were still lots of compatibility bugs.

SPEAKER 1: Back to the monitor, an error message reads, this page best viewed in Netscape.

SPEAKER 2: So, in 2012, a group of people started the web platform test project, or WPT for short. Today, you can go to wpt.fyi and see the results of millions of small tests. That run every day to make sure websites render consistently across all platforms.

SPEAKER 1: On the WPT website, a cursor moves over a browser interoperability graph.

SPEAKER 2: When you use a website, and it works the way you expect, it’s because we have web standards and automated testing technology. The reason we’re here today is because assistive technology users were not included in this push for interoperability.

SPEAKER 1: An animation of a giant sphere pushing clusters of shapes into the far corner of the screen. Louis, a light tan-skinned blind man smiles at us. LOUIS: My name is Louis. Pronouns, he/him/his. I am from Southern California.

SPEAKER 1: Palm trees sway in the wind, a close-up of someone strumming an acoustic guitar. LOUIS: I play guitar in a musical duo. And I am proud to be an accessibility tester. Behind me is a desk on which is a technology setup. Keyboard connected to a desktop, a monitor, and a laptop. The experience for a screen reader user can vary dramatically from one screen reader to the next. In practical terms, this means that I can spend a morning trying out different screen reader and browser combinations to overcome whatever inaccessible content I’m dealing with that day. Or sometimes depending on how the content is rendered, I can’t get past that. And I’m somebody with the tools to experiment and figure out these accessibility issues. Now I’m going to show you how two screen readers voice the web differently. AUTOMATED VOICE MALE: Delicious pizza that you can get from an accessible site when you need a quick blank. List of pizza options. Region checkbox, not checked. Pepperoni, checked, not checked, checked. LOUIS: Now I’ll be using the screen reader on the laptop. AUTOMATED VOICE FEMALE: Pepperoni off. You are currently on a switch on, off, on, on. LOUIS: Screen reader logos appear on screen. Voiceover, TalkBack, Jaws, Narrator, NVDA, VoiceView, and Orca.

SPEAKER 2: There are many screen readers with hundreds of interpretation differences like the one Louis just shared. These differences can exclude people.

SPEAKER 1: An animation of shapes dropping onto a conveyor belt and moving right.

SPEAKER 2: Developers who can afford it test across screen reader and browser combinations. But the way screen readers and browsers interpret web code changes all the time.

SPEAKER 1: Shape clusters jump in and out of their place in line.

SPEAKER 2: To make the web truly equitable and inclusive, assistive technologies need to work with web code in predictable ways. This is why we started the accessible rich internet applications and assistive technologies community group. ARIA-AT for short. Here we are on the ARIA-AT website, in ARIA-AT we are bringing together a community of accessibility experts, user advocates, assistive technology vendors, and web developers to create what we call assistive technology interoperability.

SPEAKER 1: Several different shapes intersect with each other to form one big shape. The big shape shifts around trying out different combinations.

SPEAKER 2: We are testing common ARIA patterns and collaborating with screen reader companies to resolve inconsistencies. In the future, we envision developers will be able to focus on accessible experience design. And assistive technology users will finally enjoy a web platform designed for them.

SPEAKER 1: The shape grows bigger and envelops the screen.

SPEAKER 2: Visit us at aria-at.w3.org to read our interoperability reports, learn more about our automated testing technology, or even join the group.

SPEAKER 1: A person uses their laptop at a dining room table, they click on join community group and a webcam turns on.

SPEAKER 2: We’d love to have you contribute your expertise.

SPEAKER 1: Big spheres and streams of smaller shapes all flow organically in the same direction.

SPEAKER 2: Let’s make sure that web platform reliability includes assistive technology users. We build better when we build for everyone.

[MUSIC PLAYING]

CAROLINE DESROSIERS: Thanks, Alice. And happy to be here at Sight Tech Global. And welcome, everyone, to our session titled, The Screen Reader Interoperability Gap – A Hidden Systemic Barrier. And hopefully, that intro video provided all of you with a sense of what the work of the ARIA-AT community group is all about. Today, we’re going to be catching up with some of the members of the ARIA-AT community group and get an update on the progress they’ve made and where they’re going next. The ARIA-AT community group officially launched at Sight Tech Global about two years ago in 2021. So really excited to hear about all of the progress since then. So let’s do a quick round of intros. And if you could all please tell us what is your day job and what is your connection to the ARIA-AT project. Isabelle, let’s start with you.

ISABEL DEL CASTILLO SOLIS: Hi. I’m Isabel. I work for Prime Access Consulting, PAC for short. And basically, my day job is to create and write test plans for the ARIA-AT project. And I also do some baking on the side.

CAROLINE DESROSIERS: Great. And Brett.

BRETT LEWIS: Hi. Really happy to be here today as part of this panel. I’m a senior software engineer at Vispera working on Jaws. And I’ve really been interacting with the ARIA-AT group trying to make sure that Jaws supports everything that is required for these tests and providing some feedback on how the tests and their recommendations have been received.

CAROLINE DESROSIERS: All right, that leaves us with, Matt, over to you.

MATT KING: Thank you, Caroline. Yeah, I’m a technical program manager at Meta on our central accessibility team. And I’m one of the chairs of the ARIA and assistive technology community group. I help get it off the ground.

CAROLINE DESROSIERS: Wonderful. Well, happy to have you all collected here as a group to really dig into what’s going on with ARIA-AT. So let’s get into some questions. What exactly is screen reader interoperability? And why is it so essential to digital inclusion? And, Matt, maybe we’ll start with you.

MATT KING: Yeah. I would like to start by talking about how people who do not need a screen reader can use the web and what they have that people who need a screen reader don’t have. So what most people can do is go to just about any website using any browser. And it basically works. And the people who made that website, they don’t really care if different people are using different browsers because they can safely assume that things are mostly going to work. They are benefiting from what is called browser interoperability. Inter means between or among. And so the concept of browser interoperability is simply that users can move from one browser to another browser. And the website is still operable. This is a really big deal. It’s really important. Because before browser interoperability was a given and just part of the air that we breathe, developing a website that worked for everyone was really complicated. It was pretty expensive. And not everybody could do it. So the web was a lot less open, and free, and certainly, a lot less equitable. Unfortunately, those of us that are screen reader users, we don’t have access to that same equity because we don’t have interoperability. It makes a big difference which screen reader, a web developer uses when they’re developing their website. And this has major negative ramifications for web developers and screen reader developers, and most importantly, for users. Sites that we would consider accessible, in many cases, they’re really only accessible to what I call the elite disabled. And for people who are blind, that means people who have multiple devices, multiple screen readers, and most importantly, the skills and the training, knowledge, and experience to be able to work around the problems that are posed by the lack of interoperability.

CAROLINE DESROSIERS: Right. So screen reader interoperability extremely important for usability, for accessibility. Everyone from developers to users are paying a price because screen reader behavior is just not sufficiently predictable. So let’s try to dig into this and understand a little bit more. Isabel, in your work, you help clients build accessible websites. And in your experience, what are the everyday consequences that website developers experience in today’s world where screen reader interoperability is not a given?

ISABEL DEL CASTILLO SOLIS: Firstly, there’s an impression of hampered creativity. And sometimes the impression is accurate. Designers and developers feel that they can’t use or just don’t know if they can use certain patterns because they might not be compatible with other screen readers. It makes for a negative impression– a perception of accessibility. It feels like a burden. It’s not adapting to how products are really made. And then there are additional costs of training, testing, maintenance, and so on. People are having to learn and stay updated on how ARIA and other specifications are supported by different screen readers, understand the nuances of how various screen readers work, determine if what they’ve developed is meeting user expectations. And they have to create fragile workarounds. It’s a lot and mostly misplaced. Testing is not being done by the disabled people, with decades of informed experience of how this tech should work, but by people who are just trying their best to understand the basics. This could be an argument for more employment among disabled users who rely on this access technology every day. But in practice, that’s not happening. And many disabled people don’t really want to be shoehorned into accessibility work. Anyway, it takes too much time, too much money. And all of this contributes to accessibility being sidelined and deprioritized. Quite often, disabled users can’t expect an equitable experience at launch– sorry, a product or feature launch time because too much of the testing and fixing is happening afterwards. It’s the classic domino effect. You spend time remediating something that’s gone before, which takes resources away from the new things being built.

CAROLINE DESROSIERS: Thank you, Isabel. So the benefits of interoperability to site developers are pretty clear. There are definitely ways that to save time, save resources, and also pave the way for innovations to happen. Brett, it is not very obvious how a world where you know your competitor’s screen reader behaves in ways that are much the same as your product. And that’s advantageous to you. Could you explain that a bit more?

BRETT LEWIS: And this is a really, really interesting question. So at Vispera, one of the things we are hoping to do with every release of Jaws is provide innovation in how we support things, customization. Our goal is to let the users have as much opportunity to tailor their screen reader experience to what works for them. Do they want to hear something? Do they not want to hear something? Are there ways we can make it easier for them to access information in a way that’s just efficient and not time-consuming? We offer customization for things like scripting. With Jaws, you can do things that you can’t do with other screen readers and so on. So all of these things allow us to differentiate ourselves from maybe one of our competitors. Isabel touched on this a little bit earlier in her response in when she was talking about the costs of testing. So the big thing that I think all of this ARIA-AT effort allows us to do as a screen reader developer is to rely on testing for basic functionality. So work that we’ve always done with every release of jaws to make sure that we’re not regressing. Something hasn’t broken accidentally can easily be done by this suite of ARIA-AT tests. We can say to ourselves with confidence. We know that we have met the basic requirements of this. And we don’t necessarily have to do separate testing, which now is a manual process. But in the future with ARIA-AT, will be an automated process. So we can take those results and rely on them. What this means is that we can free up resources that would have been spent on all of this testing internally to really providing this enhanced experience for our users. The thing I talked about at the beginning. So the innovative new features, efficiency, customization, all of those things. So really should enable us to provide more for our users, if we have this basic testing provided by ARIA-AT.

CAROLINE DESROSIERS: Right. So this standardization of testing also helps you on your side with saving time and resources and also innovating as well. So it sounds like everything just comes together under a unified effort. Let’s switch gears a little bit and talk about users. All three of you are screen reader users. And given how important this technology is to you, I’d imagine you have some thoughts on why interoperability would be so important to the individual user. And Isabel, let’s start with you.

ISABEL DEL CASTILLO SOLIS: Well, uncertainty is a big factor with this website work tomorrow when my browser installs an update. I mean, will I be find myself needing an alternative screen reader or assistance to complete a task I could do just fine before? Not everybody has the choice and the tools to deal with those things. And even if they do, it’s not necessarily welcoming.

CAROLINE DESROSIERS: Great. And Brett, how would you respond to this question?

BRETT LEWIS: It’s funny. I had written down an answer to this. And then I was looking at some of the things we’ve actually fixed as part of the ARIA-AT project. And it reminded me of what I think the biggest benefit that I’ve seen. And that’s one of the things that’s really hard for a screen reader user to know is things that they haven’t heard. So in other words, where there’s an error in your screen or leaves something out, it’s often really difficult as a screen reader user to know maybe I’ve missed something. And one of the things that I think has been the benefit is we now know that, for example, if you navigate into a radio group, you’re always going to hear the text of the radio group, the name of the radio group associated with that. So that ability to not miss things I think is really what it does– is really done the most so far and I think has the potential to do in the future for me.

CAROLINE DESROSIERS: Matt, any comments on this one?

MATT KING: Yeah, I would say, for me personally, besides the equity, big equity gap problem that we talked about earlier, the effect of the lack of interoperability on usability is really personally very frustrating. Ironically, a lot of the work that web developers do to try to improve the usability of their websites actually makes it a lot worse for people who are using screen readers. So a really good example of this that’s pretty common is that there is code that web developers can put into the web page, that tells the screen reader how a table is sorted. But very often, because the way that screen readers don’t all respond to that code in the same way, web developers will go ahead and put in what they think are super helpful comments or put in extra sentences that tell you for each column, how it’s sorted, and how you can change the sort, and what you need to do in order to activate the button that changes the sort. And all of that extra wording that’s in the design of the site is stuff that you can’t turn off. So it can make reading the table a lot harder. And so I’m hoping that in the future, when screen readers behave more consistently, they won’t feel like they need to do this kind of thing so much, and we should be able to have much more usable experiences across the board.

CAROLINE DESROSIERS: Yeah, so it sounds like these developers who are taking steps to improve accessibility with the best of intentions would really welcome something like this because they have an exact framework and guideline to follow to achieve that result. Great. So let’s learn a bit more about the actual work that needs to be done to make this whole thing a reality. And I know this project has been in the works for actually close to five years. So changing one product is hard enough as we know. Changing the behavior of products across an industry is clearly a big challenge. And I understand that a first step is to define the behavior as we want through some standardized testing. And the question is, how do we exactly do that? And Isabel, can you speak to this one?

ISABEL DEL CASTILLO SOLIS: In some ways, we are doing the work that uses unwillingly take part in every day. Assessing current screen reader support levels or patterns, and finding the gaps, and asserting what people need in the real world. I suppose the difference is that via this project, we have an opportunity to exhaustively establish and formalize those things in an effort to solve many of the problems we’ve already covered. It always starts with the creation of a test plan targeting a specific functionality pattern. That’s a collection of testing tasks. Representing how users are likely to interact with it. The commands, they might use. And the results, they should be able to expect. Those expectations now are rated based on the likely user impact between things that access technology must do to create an accessible experience. The things AI ideally should do. And extra things that it may do to help a user out with extra convenience. It doesn’t stop there though. Everybody who then works through that test plan to record results may have ideas on how to add to it or make it better and more applicable to users. It’s a lot of work. And not only to thoroughly anticipate and catalog what users need, but to comprehensively evaluate the degree to which they’re currently being met. And also, record the results. Because integrity is important, we want our data to stand up to scrutiny, as well as being useful.

CAROLINE DESROSIERS: Thank you. Brett, as someone working on screen reader code, what do you then do with the tests that Isabel and the community group are delivering? Why don’t you take that question?

BRETT LEWIS: Sure. So they’ve not only delivered us the tests. But they’ve delivered us some of the test results. And so when we first started getting the test results, we took those and basically fixed the Jaws. There were areas where we just weren’t necessarily meeting all of the requirements. And so we would create issues internally and address those issues and get fixes out as soon as possible. One example of this was I mentioned the radio groups earlier. One of the things we weren’t doing was announcing the name of the radio group when navigating into it using a quick nav key. And this was a pretty important improvement. We were announcing it if there was a focus change. For example, if a user tab’s in there, but we weren’t otherwise. And so it really helped us improve the experience for all of our users right there. One of the other things that we’ve found is that sometimes we take the test results, and we say here’s a gap that we need to fill in Jaws. And we implement what the tests recommend. And we roll it out to our users. And what we find is that it’s an experience that for some unanticipated reasons, it doesn’t work for our users as a requirement. So an example of this, we had a disclosure pattern where we were supposed to speak list items as you navigate into them using quick nav keys. And so we made this change and we rolled it out to our beta testers. And we found that on the audible website, for example, when you do a search, all of the individual search results were inside a list item. So as you navigate through the page, every single time you’d navigate to a search result, you’d hear too much information. So we sent this feedback back to the ARIA-AT group. And they revised the tests because their screen reader users. They understand the complexities and why this might not work as a requirement. So really both of those things implementing fixes based on what the tests have identified and also helping us provide feedback to the ARIA-AT group on what tests work and what should be required. And maybe an optional addition to the test.

CAROLINE DESROSIERS: So it sounds like we end up with some tests that reflect a reasonable set of basic behaviors. And if those tests are covering screen reader behaviors across whatever is possible for site developers to build, sounds like that would be a lot of data. So, Matt, how much data is there exactly and how are you managing it?

MATT KING: Yeah, it’s a lot of data and is only getting bigger. We have currently draft test plans for 35 examples of common UI patterns. We get these examples from the ARIA Authoring Practices guide. And for each of those 35 test plans, they may have 10 to 40 tests in them. And so that’s hundreds of tests across the plans. And then for each test in every plan, we have to map the commands of the screen reader that can be used to execute that command. So we end up with thousands of assertions just with the amount of data that we have right now. Over time as we cover more of the AGP– and right now, we’re only covering three screen readers. And we’re going to cover more screen readers. And then in the future, we hope to cover other kinds of assistive technologies. So it’s not just screen readers. So the amount of data is going to grow a lot. And we have been building the infrastructure that it takes to manage all this data and manage the consensus process for all these tests when we get the feedback that Brett and Isabel were talking about on the test for members of the community group. And from screen reader developers– In order to do all that, it’s a lot of work. It doesn’t come for free. We’ve been contracting with Bocoup to build the infrastructure. They are right people for the job because they are some of the same people who have also worked on browser interoperability infrastructure.

CAROLINE DESROSIERS: And how do you keep all of this current in terms of the test plans and the reports? Don’t you need to rerun over and over again as screen readers change?

MATT KING: Yeah, that’s exactly what we need to do. So, initially, everything has to be run by human and validated by a human. There’s no getting around that. But once we have defined expected results for every single test, there isn’t any reason why a machine couldn’t rerun that. Except for the fact that there was no standard way of doing this. And so one of the things that we’re doing is developing something called AT driver, which is a standardized API for driving screen readers. And once we have that, the same code can be used to run tests for any screen reader that implements that API. Recently, we’ve made quite a lot of progress in this space. And we have started to rerun some of the tests using automation and generate some of the reports that way.

CAROLINE DESROSIERS: So automation, so, so important. So the project is in year five at this point. And you clearly have made gains in test writing and automation. What other big wins do you have? And what is the outlook for 2024? Matt, over to you.

MATT KING: Yeah, so I would say the other biggest win we had this year is we launched 80 support tables in the Authoring Practices guide. So now when somebody goes to an example of how to code a pattern, for some of them, they can see what is the level of support for that pattern. And this is a really big deal for us because it exposes the data in ways that we hope are going to be generally useful to a broader audience.

CAROLINE DESROSIERS: Great. And how can people help at this point? Isabel, maybe you can comment on that.

ISABEL DEL CASTILLO SOLIS: Yes, well, we’ve already talked about the level of investment faced by individual companies and product creators. And essentially, this project is taking that on at scale. We are always in need of experienced screen reader users to review and run tests, people to propose improvements to those tests, and the tools developed by the project. And generally, looking to expand a reach and diversity of viewpoints. We really encourage you to reach out if you have any interest in what we’re trying to do. Even if you’re not exactly sure what you could offer. Because there is plenty to be done. As Matt said, we are hoping to get a lot more data into those tables.

MATT KING: Yeah, in 2024. We’re hoping to get several dozen of those AT support tables out there and incorporate more of the feedback we’ve received from the community. So there is a lot of work to do in that space.

CAROLINE DESROSIERS: Certainly sounds like it. And yes, please get in touch with this group. It’s certainly an important mission that you all are pursuing here with this project. So, yes, please get in touch. I’d like to do just a quick round of final thoughts from all of you about what’s next for the ARIA-AT project or maybe additional calls to action that you’d like to make. And Brett, let’s start with you.

BRETT LEWIS: I think the promise of this project is really hard to underestimate particularly with the impending automation. I think we’re really going to see an explosion in the benefit from this as automation becomes a reality, as the number of tests increases. It’s really going to provide benefit for all screen reader users. Vispero is very committed to this in the long run. It’s one of the things I have to hear about all the time, which is great. But probably at my performance review as well. But there will be long-term commitment for this project with Jaws and the Vispero. And I think the benefit is just going to increase.

CAROLINE DESROSIERS: Thanks. And Isabel, final thoughts.

ISABEL DEL CASTILLO SOLIS: Yeah, I just want to say that the aim of this project is not to make all screen readers the same or have websites. And apps be carbon copies of each other. Quite the opposite. We want uniqueness to always shine through while being backed by consistent support. That way, experiences can be created that legitimately stand out while being inclusive and better for everyone.

CAROLINE DESROSIERS: Matt, what’s next for the future?

MATT KING: Well, I just can’t stop dreaming of the day when it really is possible for web developers to build experiences that are truly usable for people with disabilities. And accessibility is really inclusive of all blind people. And I don’t think that’s possible. Until screen reader interoperability is part of the air we breathe just like browser interoperability. And when we have that, then I think it will be possible for accessibility to not just be limited to the elite disabled, it should be equally available to everyone.

CAROLINE DESROSIERS: Well, thank you so much to our panelists for a really great session and everyone in the audience for attending. So exciting to hear about the progress that’s been made clearly a lot of work ahead. But thank you all for sharing a bit more about your work. I’m sure everyone has questions for our panelists. So please bring those to the Q&A session. Thanks again and back to the team at Sight Tech Global.

[MUSIC PLAYING]