[MUSIC PLAYING]
WILL BUTLER: Hello, everyone, and welcome to our discussion with Greg Stilson from APH. My name is Will Butler. You all know me from Sight Tech Global. But Greg, thank you so much for talking to me today about this exciting new product to come out of APH.

GREG STILSON: Oh, thanks for having me on Sight Tech, Will. This is great.

WILL BUTLER: To clarify, this is a R&D project. But when are we going to actually see this thing come out, Greg?

GREG STILSON: Our timeline is going to be, hopefully, the end of 2023. We'd like to make sure that we have field testing in process well into 2023. So we're looking at still a couple of years out. But we want to start getting the word out now, just because of the gravity of what we're trying to do.

WILL BUTLER: It is exciting. And we'll get a little peek at it at the end of our session today. So stay tuned, folks, to see how this tactile display actually works. But before we dive into that, Greg, help me sort of define the dream here a little bit. Why tactile graphics? Maybe, as a kid coming into a learning setting, what is that experience like for a kid trying to get access to a math textbook or another type of learning material?

GREG STILSON: It's a good question. And honestly, it really depends on the situation that the school is in, whether that textbook's been created in the past. So one example I'll give is an Algebra 2 textbook, for example. We produced an Algebra 2 textbook this year. And a textbook, to kind of put it in perspective, can take up to 30 to 40 volumes of Braille and tactile graphics all combined into these bound massive books, and can take over a year to produce. And part of that is because you're engaging a lot of people with a lot of expertise, from tactile graphics artists to Braille transcriptionists. And as a student, if that book has not been produced, and let's just say that the book order hasn't come in that far in advance to when you need it. I specifically remember an instance in college when I needed a calculus book. And it didn't actually arrive until I was three weeks from the end of the semester. So making sure that you receive that content timely is really a challenge at times.

WILL BUTLER: I definitely remember all the energy and effort it took to get my books converted into alternate formats. So it can make a huge difference if this could just happen automatically.

GREG STILSON: Exactly, yeah.

WILL BUTLER: Talk a little bit about impromptu learning too, because I think it's one thing to have a textbook translated. But in a school setting, or just in a work setting or wherever you are, things are being thrown at you all the time, a hand out, a print out, an image up on a screen. How do TVIs and other people in the community currently deal with impromptu learning?

GREG STILSON: Yeah, and this is where I truly admire the creativity of the TVIs and the paraprofessionals and things like that and the people like that in our field, because you are, no pun intended, blindsided with a lot of different content that teachers, professors, just forget to provide the student or the teacher in advance. And so they're doing these amazingly creative acrobatics to produce tactile graphics from things like common things that we provide at APH. And we have a ton of different impromptu learning tools that can create things like shapes or geometric concepts or things like that, all the way to things like Wikki Stix or these rolling wheels that basically create a tactile graphic on a Braille paper. So they're using household items or anything like that, just to produce very vague images, so that the student has an understanding of what the teacher, the class is looking at.

WILL BUTLER: I remember when I first learned about the thermoform paper that allows you to kind of sketch something out and then basically bake it.

GREG STILSON: You got it.

WILL BUTLER: And turn it into a tactile image. You are using really, really creative ways to kind of roughly guess at what an image should look like.

GREG STILSON: Right, exactly. Yep, yep.

WILL BUTLER: So enter APH, you lead innovation at APH. It's the 21st century. What is the solution? What's the dream solution for tactile graphics and learning combined?

GREG STILSON: So in 2020, we put out a request for information, RFI, to basically say, look, our dream here is to create a tactile device that is capable of producing both tactile graphics and standard Braille in the same tactile array. And it's been something that has been tried by a number of organizations, academic projects, and things like that. But nothing's really come to fruition and come to prime time. And one of the things here at APH is we also, we are one of the largest textbooks producers of textbooks here in the United States. And so our vision has always been to produce textbooks on a device like this, to basically create, essentially, a tactile Kindle, so that students will be able to receive a textbook, gain access to it, and then basically read it as if they're reading a paper textbook, having access to all the Braille questions, the literary content. But then also, if there's a tactile graphic, being able to click into that and view that in its detail as much as possible as well. So we put out this request for information. And we got a lot of responses and a lot of varying technologies. So no two technologies we're alike. And we also, in addition to looking at technology, wanted to look at the business as well. And APH really does a lot of our projects with partners today. So we recognize our strength is in knowing the field of education of students with visual impairments and our research. And most importantly, I think the design of our user experience. But we wanted an equal partner in this, because we knew that this project wasn't just going to be challenging from a technology perspective. It's also going to be tremendously expensive. And we wanted a partner who was willing to share in this project, both financially and with the collective brainpower as well. So APH and the company Humanware partnered together to basically be equal partners in this project to try to bring this dream of this concept of what we're calling sort of the Holy Braille into a reality.

WILL BUTLER: I love that. And in true innovator format, you're trying to put yourself out of business in order to stay in business.

GREG STILSON: You got it, man. It's almost like you just got to kind of shoot yourself into the 21st century and say, all right, we got to move and rip off the Band-Aid quick.

WILL BUTLER: Right, right. OK, so what have you learned from past iterations of attempts at this hybrid device? And what are the goals? What do you want to be able to accomplish with this dynamic tactile device?

GREG STILSON: So we saw-- we've been involved in a number of tactile projects. We've worked with the past graffiti project, which was a research project that we undertook back in, starting in, I think, 2015 something like that. And we learned a lot. We learned that the market needed more than just a tactile display for graphics. And Braille has significant value when people put their hands on a device like this. Don't get me wrong. There is value, a ton of value in a graphical display. But access to standard Braille, the possibilities really multiplied when you put forth that possibility. And as past iterations were able to create things like jumbo Braille, it didn't create that transferable skill to reading standard text or standard Braille. And you can do so much more with graphics when you're able to add labels. One thing I can tell you, as a blind person myself, is it's very challenging to look at a graphic and understand what it is. And so having Braille labels or potentially audio labels can really help guide the user in grasping the content that they're looking at.

WILL BUTLER: So you've got you've got this unnamed dynamic tactile device that you guys made. You've got it in your hands. What does it feel like? What does it look like? We're going to look at it in just a few minutes. But describe it to me.

GREG STILSON: Yeah. And like I said, what I'm going to show today is really the underlying technology. This is not the product that we will be releasing. This is not even something that looks like the product that we're going to be releasing. But what I have here is a pad, basically, of 10 Braille lines that is capable of showcasing what the technology feels like when you touch, what it looks like when you create graphics. And being in the virtual world that we are, we're going to do the best that we can with the camera technology that I've got. So I'm going to already apologize for the sort of fuzzy graphics that you are going to see. But all in all, what I want to get across is that the Braille that you touch feels like standard refreshable Braille display that you'd feel today. The difference is that these pins do not go down. So as you touch them, they're locked in place, which is, I think, really good, especially for people who have neuropathy or struggle reading. And quite honestly, I can tell you as a very attention-challenged student myself when I was in school, I remember with refreshable Braille displays, I would push those piezoelectric cells up and down and up and down in the classroom as sort of a fidget type of thing. These can't do that. Once the pin is up, it's up. And so what you'll feel is lines of Braille that feel like a traditional refreshable Braille display line of Braille. But then you'll also feel tactile graphics. These pins are designed to produce evenly-spaced tactile graphics as well.

WILL BUTLER: And these little equidistant dots, are these different than the standard dots people are used to feeling? What makes these dots unique?

GREG STILSON: Yeah, I can't really speak to the underlying technology. But they are not the same as piezoelectric. But the part that we really focused on with this technology is refining the way the pin feels to the fingertip. And what I can tell you is that it feels very similar to a refreshable Braille display pen that you're used to. One of the challenges that we looked at when we're working on this project is we're changing the way that people access refreshable Braille, to a point where it's never really been done before. And one of the things that we looked at was, we can't change the way that the pin actually feels to somebody. We can't change the way Braille feels. It's just too much to change. It's too much to accept. So we're like, OK, the way I look at it is like if you took the way a print font looks, if you took away the way of a specific print font looks to a sighted person, the adoption rate would be far lower. So I wanted to make sure that we really refined the pin and the way the pin feels so that people are comfortable as soon as they lay their fingers on it.

WILL BUTLER: It's like if you open up your operating system one day and it was all Comic Sans. You would feel differently about using the device. Tell me about the software, though. Is it just going to run your software? Or is there a vision for others to be able to develop with this device?

GREG STILSON: It's a great question. So yeah, we recognize, we talked about the primary goals of this. Number one being textbooks, being able to gain access to textbooks. Number two is to be able to gain access to impromptu learning things, so being able to take graphics that exist in the world today, and whether they're on a computer or a smartphone or something, being able to essentially use this as a tactile monitor. So being able to send that to the tactile display. And in this case, yes. Number three is be able to develop an ecosystem where developers and partners of APH are able to create apps. So our goal is to create an API so that these apps can be created by third party developers or, potentially, academics, who want to create experiences that we can't even dream of right now. Our goal is just, let's figure out how to create multi-line Braille displays and have graphics displayed. There's a lot smarter people out there than I am to create these amazing experiences. And as a former low vision kid, who's really lost my vision over time, I was a big, big fan of video games. I've got these dreams of tactile games for these kids to be able to play.

WILL BUTLER: Wow.

GREG STILSON: So who knows? The sky's the limit, I think. We need to build that foundation. But like I said, I think with opening it up to potentially third party developers that are sort of partners of APH, I'm beyond excited to see what gets created.

WILL BUTLER: You started working on this just a little over a year ago in the midst of a global pandemic. And I'm wondering, how on Earth did you do user research and testing for this? Because for a physical device, I imagine many others who are watching have also confronted these same issues over the past year.

GREG STILSON: It was not, I would say, an optimal climate for user research. We did some really creative things. And I am so proud of the team for really not getting discouraged, because it would have been so easy to just be like, no, we've got a few blind people in house here at APH. We know the field pretty well. We can make assumptions, and that'll be enough. And that's 100% the wrong answer.

WILL BUTLER: Sounds dangerous.

GREG STILSON: It really does, and it really is. So what we did is we reached out to schools for the blind, to teachers that we know in the field, and really asked them to get their students together. And some schools, we're seeing, or some teachers were seeing students in person. Some teachers were doing everything virtually, so trying to figure out where we could actually get fingertips on graphics and things like that. But quite honestly, the level of creativity that we got to, the number one thing that we were looking at is, what is the optimal size of the tactile space? How many Braille characters is right? What's the right aspect ratio to do this? Because without that knowledge, the rest of the device doesn't come to fruition, because that's the part that's going to take the largest chunk of the device. So that was our first focus is, what is the right size of the tactile array, because what we learned is if you get too narrow, then the graphic looks distorted. If you get too long and not tall, but wide, then it looks different the other way. And then you chime in with making sure that you're providing enough Braille that brings value. If somebody's just reading a multi-line textbook, for example, and no graphics. So there's a lot of variables here. So what we ended up doing was we sent out, literally envelopes, of pieces of paper that we embossed tactile graphics and Braille, different Braille examples, and surveyed a number of teachers and students who were either with or virtually with their students, and then collected feedback. And we were very happy with the results that we got, and most importantly, very happy with the patience that these wonderful people had. But with something like this, because this is such an exciting project, I was really enthusiastic to see, just the excitement that people had with this. And they wanted to participate. They wanted to provide their feedback.

WILL BUTLER: Yeah.

GREG STILSON: And I think it was also a breath of fresh air amongst the whole COVID pandemic quietness that was happening.

WILL BUTLER: Well, and that's why I think, it speaks to the importance of why someone like APH could be an important part of making this, because you have the ability to go analog like that and use creative solutions. But it can be done. And I think user research is still very much an important piece of this. I want to know, though, what's the business model here? So who will be buying this device? I presume it's not cheap. But how are people going to get their hands on them?

GREG STILSON: So the one thing is that we are continuing to look. It's not just going to fall on the federal quota system. So here in the United States, schools get designated federal dollars to purchase educational materials directly from APH. Yes, some of those dollars will be able to go to purchasing these dynamic tactile devices when ready. But we recognize that you can't blow your entire chunk of money on one device like this. You're then giving up tools that could benefit other students, because this will cost significantly more than other devices from APH. So we're looking for complementary funding. We're already engaging the federal government, because this is a new initiative to really modernize our textbook distribution. And make no mistake, one of the things I want to be clear about is that this is not going to eliminate the need for Braille transcribers. What our goal with this, and initially especially, is just to reduce, it's the metric that I've created in my head called the time to fingertips. So if we can reduce, one example that I told you is this Algebra 2 book that we produce last year. This algebra two book cost $30,000, nearly $30,000 to produce, and took almost 13 months to complete. And so if we can reduce the time to fingertips, for the time that the order comes in to the time that it gets the student's fingertips, even semi-significantly, it's still a huge value add. We're also going to reduce the cost of the binding, packaging, embossing, shipping, any of that kind of stuff. We still, the tactile graphics artists and the Braille transcribers are absolutely going to be needed, because a device like this is also going to require an entirely new Braille format, electronic Braille format, because something like this for textbooks has never been created before. And so Braille transcribers are still going to be transcribing books. We're just going to be distributing them electronically.

WILL BUTLER: And outside of education, I imagine this would also have a lot of applications as a work accommodation for various industries, right?

GREG STILSON: You got it. Yeah, and I can tell you that even just the little bit we've talked about this so far, one use case that has really come to light is for blind software engineers. There's been a large excitement about blind people seeing the UI that they actually create, touching the UI that they create, being able to really hone their skills on UI creation that may not be as accessible as it possibly could be. And so there's that. There's also creating tactile maps and a number of different things. Like I said, I don't even think we've dreamt of half the uses that this thing this thing will have. But yes, in the United States, our primary market is going to be the education market. I'd love to see some professionals being able to use this. And then of course, internationally, Humanware's got a fantastic international distribution network. And so we're going to work with them to be able to help to distribute this internationally as well.

WILL BUTLER: So one last question before we get to the demo. The device is not done yet. Where do you still need help? And is there-- we have all these wonderful technologists and developers and folks who work in assistive technology watching. Where do you want to take some steps forward in order to get this thing shipped?

GREG STILSON: I appreciate the question. And it's not even close to done. So we've got a lot of work to do. I really look at three things that we need help with. Number one is user feedback. We're going to be looking for, next year is kind of going to be pandemic pending. Hopefully, my tour of being able to go to various locations with prototypes and actually get hands-on feedback, we're going to have some experiences that we're going to try to put people through, and hopefully get feedback on exactly how our UX should be. And what is the right way? Because quite honestly, like I said, we're going to be creating things that have never been created. So what's in my head is most likely not the right way to do it. So we need to get we need to get feedback, number one. Number two is, one of the biggest things that this will be able to do is take a graphic and basically do that impromptu learning experience, or be able to render that on the tactile display. And I'd love for AI to play a huge factor in that. And we don't have that expertise. And I am a firm believer that there is a lot of work being done in photography with regards to filtering and identifying scenes and focus items and things like that. One of the things that we know is that, with a blind person, sometimes less is more with tactile graphics, and really identifying what is the intended focus of this graphic, so that that can be highlighted automatically, and the rest can kind of be faded into the background. I think there's a lot of need for creating a data model based on tactile graphics, that really is focused on the needs of a blind user and not such a focus on a sighted photographer or something like that. So that is really the second. The last piece is, we're going to be creating this new, we're calling the EBRF, which is the new electronic Braille ready format that is really designed for a multi-line experience. And this is sort of this new Braille electronic format for the 21st century. And a part of what we're going to need to do is create an advocacy group around this, so that it gets adopted as an international standard. And so we'll be circulating a white paper on that as well.

WILL BUTLER: Can folks reach out? Is there an email address or something?

GREG STILSON: Yeah we're going to, DTD, dynamic tactile device, DTD@aph.org is the best place to get us.

WILL BUTLER: That's pretty easy to remember. DTD@aph.org. Greg, will you show us what you got? Will you pull back the curtain for us?

GREG STILSON: Absolutely. Like I said, with the Zoom gods at our disposal, we'll see. One of the things that you will see, I'm going to switch over my camera in a second. But due to lighting, what we're looking at is it may look like a bit of a trapezoid when you look at it. It is a square device, just to let you know. It's the way the angle is. I've got it angled so that the reflection makes it a little bit more visible. But what I'll be showing here in a second is some graphics and some Braille examples, just to show you that what I'm showing really works. We've seen a lot of things that have kind of come through and then disappeared. And my biggest point that I want to leave you with today is this is real, and this is something that does work today. So I'm going to show you a few examples now.

WILL BUTLER: Yeah, I see some-- I'm zooming in here. I see some arrays.

GREG STILSON: You got it. As long as the screen changed away from my head, that's all I care about.

WILL BUTLER: Yep, looks good. So Greg, walk me through this for those who are listening. What have we got here?

GREG STILSON: Sure. So what you're seeing is basically a bunch of Braille cells. And what I'm going to show now is all the pins are down. So what I'm going to show you first is just an example of a typical bar graph. So this is just going to be an example of a standard bar graph. So I'm going to load that up now. It's now loading. It takes about two seconds to load. And now, what you visibly should be seeing is a bar graph. On the left side of the bar graph is the y-axis. And what I have is 0, 2, 4, 6, 8, 12. On the bottom, I have the x-axis, which says cats, pigs. For some reason, somebody put ants and dogs. So you're seeing which is the most popular. And I can tactically feel with my fingertips that the bar is far higher for dogs. And if I go all the way across, it goes to 12. I'm not sure, this is just an example of a graph.

WILL BUTLER: With what limited sight I have, it looks like the graph is just perfectly rendered, exactly like it would be in a--

GREG STILSON: Yeah, from a tactile perspective, Will, as I'm touching it, it feels just like a typical graph you'd feel in a textbook.

WILL BUTLER: It's pretty crisp, yeah.

GREG STILSON: Yep. And like I said, these pins are the height that you would feel on a standard refreshable Braille display. So the learning curve, it's not like you're relearning to read Braille. Another example I'd like to use here is a typical pie graph. So this one, you're going to-- and it may be hard to see, but there will be sort of a Braille key at the top, followed by two pi charts. So I'm going to load that up. It's now loading up. So a couple of seconds later, I've now got two pie charts filled in both on the left and the right. And then at the top, you have sort of a key to show. It says carbohydrates, proteins. And then here on the left, I can see that 65% is proteins. And on the right, we can see that 50% is proteins. I'm not really sure what that all means, but just to show the capability. One of the features that I like to show here, though, is our dream is that you'd be able to look at these in, let's just say, a zoomed out version like we are today. So they're zoomed out. You see two pie charts right now. But let's say that I want to zoom in on the left one. Well, our dream here is that I'd be able to just double tap on the left pie chart, and it will load up zoomed in a much bigger full screen kind of view, and actually see that it says protein, 65%. And then it's a much larger version of the pie chart at that point.

WILL BUTLER: Wow, that is so cool. You paint a very compelling image of the future, Greg.

GREG STILSON: I want to show you that, I think my favorite one to show is a typical math problem. And this is one example that you see in, let's just say, a geometry book right now. So I'm loading this up. At the top, you see sort of a word problem written in Braille. And at the bottom, you see a right triangle. And at the top, it says number one, determine the measure of B to the nearest tenth of a degree. And so here I can feel that the left side says it's 14 centimeters. I have the right angle here, because there's a little box, if you can see that. But then if the user wanted to actually zoom in to the triangle, they'd be able to double tap on that triangle and zoom in. And then at this point, the triangle's totally zoomed in, and you're seeing it in much more detail, making it larger and things like that. So this is sort of what our vision for a textbook looks like that, is that you'll be able to read the literary portion of the textbook. But if you need to interact with the tactile graphic, you essentially could double tap on that graphic or graphic symbol and then see it in a full screen graphic viewer at that point. So like I said, there's so much work to do on this. But this is where I wanted to kind of at least instill confidence that we're dealing with the underlying technology that really works.

WILL BUTLER: And is the vision that TVIs and others would be able to read the device visually alongside a learner?

GREG STILSON: You got it, yep. So the device will be able to, that's part of our specifications, will be able to cast to a monitor or connect with an HDMI cable to something like that so that the teacher will be able to see, in print, what the student's doing, or be able to at least see the visual representation of a Braille display as well.

WILL BUTLER: It's incredible. You can think of so many applications and different uses, a software engineer, both being able to check their code and check their interface on the same device.

GREG STILSON: Yeah, exactly. And the last thing I'll leave you with is kind of a fun one. We always play, when I demonstrate this, we always play name that name that graphic. And this, at APH, we have a library called the tactile graphics image library, which is a bunch of tactile graphics that are just collected that are produced by tactile graphics artists. And the cool thing is that we are already able to, with very little modifications, render almost every graphic in our tactile graphics image library. And this is one that I know the sighted audience will probably recognize instantly. But just to show you the gravity of what we can do here, this is just a graphic that we pulled off of the tactile graphics image library. If I activate it now, it loads up. And Will, I'm not sure how good your vision is there.

WILL BUTLER: I'm probably not going to be able to do it.

GREG STILSON: Having said that, our sighted audience should be able to recognize, that's a bicycle. It just loaded up instantaneously. You've got your two wheels here.

WILL BUTLER: Yeah.

GREG STILSON: And you've got your handlebars. That just came off of our tactile graphics and image library, just like that. We barely had to do any modifications to make that work.

WILL BUTLER: Wow.

GREG STILSON: So we're incredibly excited about the underlying technology. And there's just, there's so much work to do. So we really appreciate any excitement or willingness to support this project.

WILL BUTLER: That's amazing, Greg. What a cool device. And folks, email DTD@aph.org if you're as excited about this as I am. And join in the conversation.

GREG STILSON: Yeah, please. And if you are interested in this, we are going to be doing a breakout session following this session here. It'll be myself and then Andrew Flatres, the Braille and tactile product manager from Humanware teaming up to answer any questions that you may have, or try to go a little bit deeper if you do have more interest in this. So definitely, please reach out and join that session as well. Super excited to talk to you.

WILL BUTLER: Wonderful. Thank you so much for joining us as Sight Tech Global, Greg. And I'd love to check in with you next year and see how the progress is going on this.

GREG STILSON: Absolutely. We'll already schedule it. Love it.

WILL BUTLER: All right, take care.

GREG STILSON: Take care.

[MUSIC PLAYING]