The Timing for APH's Monarch tactile display could not be better

[MUSIC PLAYING]

JOSHUA MIELE: Thanks, Alice. Really wonderful to be here at Sight Tech Global. Today, I am thrilled to be speaking with Greg Stilton From the American Printing House for the Blind. Greg, you lead global technology innovation at APH. But just in case anybody doesn’t really know who you are and what you do, and maybe even what APH is, why don’t you give us a little background on your role there at APH and APH’s position in the world?

GREG STILSON: Yeah, absolutely. Thanks, Josh. We are the American Printing House for the Blind. We’re one of the longest running non-profit organizations. We are federally funded. And the American Printing House for the Blind is a resource that provides materials, products, and resources for students and teachers, students, who are visually impaired and the teachers who teach them. I always say we do everything from the lowest of the low tech, things like Braille paper, to some manipulatives, a lot of kits, and things like that to high-tech products like refreshable Braille displays, low-vision magnifiers, and this Monarch tactile display that’s coming down the road. I’ve been there since 2020. Not new to the assistive tech world. I ran product at Humanware for a little over 12 years and also did some work with IRA, as well, when they were in the infancy– of their infancy of their startup as well.

JOSHUA MIELE: You’ve been doing great stuff for a long time. And it’s really great to be talking to you today about– you mentioned the Monarch. And there’s so much to say about that. But for those who have not been oriented to it, could you just give us a quick overview, maybe a 10,000-foot view of what the Monarch is and does, and maybe just– I think you’ve got a unit there to show us or maybe a video of it. That’ll be really good for people to actually see a demo maybe.

GREG STILSON: Yeah, absolutely. So the Monarch has been this long-standing dream that many of us– I’m blind as well. And we’ve all had this dream of a tactile display that’s capable of showing both Braille and tactile graphics on the same surface. And this is a project that’s really come to fruition because of partnership. The American Printing House for the Blind knew that we could not do this on our own. And so we partnered with the aforementioned Humanware, who’s a manufacturer of assistive technology. They’re also doing some of the software that’s running on this. We also partnered with the National Federation of the Blind to make sure we had the voice of the nation’s blind doing testing and providing feedback alongside. Along with really one of the foundational pieces of this technology is the dot technology that goes into the tactile display. And so this utilizes the Braille technology or the Braille cells, provided by Dot Incorporated out of South Korea. And really, it is a self-contained device. It is a tool that has very specialized applications on it that you’ll be able to see. I’m actually going to do a quick demonstration of what it does. But the goal here is that this product will be a tool that a student would essentially be able to use in every class that they’re going to encounter in their educational career. So being able to tackle the Braille literacy aspects along with things like geometry, mathematics, science, and as you mentioned, Josh, maps, as well, geography. So what you see on the screen here– I’m going to share my screen– is you’re seeing the main menu. I’ve got it connected to my computer as a monitor. So visually, what you’re seeing is the Home menu of the device. And you see the applications that we have. This is an early beta device. So there’s a limited number of applications that you’re seeing. But you see Braille editor, tactile viewer, a word processor, a file browser, the Victor reader book reading application that we’ll get into in a little bit, and then an All Applications option. So the first thing I want to just quickly show is this is just simulating. What I’ve done is press the Enter key on the book reading application. And I’ve opened up the book Alice in Wonderland. And what you’re seeing here is the actual Braille dots of Alice–

JOSHUA MIELE: Not our Alice. Not Alice, our host, but–

GREG STILSON: Not Alice. Correct.

JOSHUA MIELE: Different Alice.

GREG STILSON: The OG, if you will.

JOSHUA MIELE: Although, it could be like Sight Tech Global wonderland.

GREG STILSON: Exactly.

JOSHUA MIELE: Next year, we’ll call it wonderland.

GREG STILSON: There you go. So at this point, what I’m looking at is the Chapter 1 of Alice in– this is a BRF. And we’ll get into why that means something here shortly. But this is just reading an original BRF that is on this device. I see Chapter 1. And I see Alice was beginning to get very tired of, blah blah, blah. And I can pan this display using the Page Up and Page Down buttons that are to the left and the right of the tactile display. So I’m panning this up and down. And able to see things like indentations, center alignment, things that you would traditionally see on a piece of Braille paper. For our sighted audience, I can actually invoke, if I go into the Options menu here, I can turn on a visual Braille representation. So our sighted audience or our sighted teachers would be actually able to see what our students are actually looking at, which is really powerful in that I can turn on this visual Braille representation. And because we have a touch sensor on this device, they can actually even see where I’m touching on the tactile display. So you’d be able to see of a red highlight of where my fingers are touching at this point. So that’s just one quick overview of the Braille portion. I wanted to quickly jump in to show one tactile graphic. And that is if I go into the tactile viewer here, one of the things that we’re really excited about with this is that it actually opens up the concept of dynamic tactile graphics. So here, I don’t have to run a graphic off on an embosser to look at it. I could type in– I’m connected to the APH tactile graphics image library over our wireless connection. And I can type in the 50 states. I’m just doing a search right now on our tactile graphics image library. And I can type in 50 states. And it’s now showing me search results of the 50 states that are available.

JOSHUA MIELE: That is super cool.

GREG STILSON: At this stage, I’m going to now hit the Enter key on the 50 states tactile graphic that exists already.

JOSHUA MIELE: That’s a graphic that’s in your library. It’s been vetted. It’s known to be a quality tactile graphic.

GREG STILSON: Exactly. So now, what you’re looking at is you’re looking at the overview mode. And what this does is it takes the tactile graphic that was created, but for a 11.5 by 11 sheet of paper. And it’s scrunched it down into our tactile display. But what I can do is using our touch sensors of the display, I can actually zoom in. So if I want to find the state of California, for example, I can zoom in by just positioning my finger on Cali– where I believe California would be. And now, it’s zoomed in to where California and the Northwest region of the US is. And I’m using my arrow keys that are to the right or the left of the display to now pan around, and see the full display of California, where it meets Oregon and all those types of areas as well. But this is where it’s really powerful because I can zoom back out. And if I want to go to the North East region, I can go to the top right corner of the display, find where that is, and just double click my finger. And now, I’ve zoomed in on the top right corner of the US here. And I see New York and those type of states as well.

JOSHUA MIELE: You’re really pandering to the crowd there. The California, New York. Sorry, middle America.

GREG STILSON: Well, I’m from Wisconsin. So we could jump in the middle, if we want, Josh.

JOSHUA MIELE: I have a ton more questions. So I don’t want to rush you through your demo. But I do want to move on. Is there anything else you wanted to point out?

GREG STILSON: No, I think that covers the Braille and the graphics. So that’s great.

JOSHUA MIELE: So the next thing I want to talk to you about is, this is super cool. I’m a tactile graphics nut. I love and have always been a huge advocate for tactile graphics and their use. This is a technology that, I think since I was about eight, has been almost available. Everybody’s been showing demos and prototypes of what I have come to call, and, I think what has generally been called the Holy Braille. So why now? Why is this the right time? What’s changed that allows us to really now think that this is going to happen, finally?

GREG STILSON: It’s a great question. And quite, honestly, when we started this whole project, Josh, the first year was really spent. We called it our Prove It tour. And it was really going around to the major organizations, the main players in this space and showing demonstrations of why this is the real deal. And I think it’s a combination of a number of things. Number one is the technology finally exists. The dot technology is the real deal. It has the ability for us to do this tactile array in a fashion that allows us to do both graphics and Braille on the same surface. That’s number one. Number two is that those partnerships have to exist for this to really come to fruition. There’s no way that APH. As its own entity, could do this on its own. So we had to establish both a manufacturing partnership along with a user organization partnership with the National Federation of the Blind. And then I’d say, lastly, is funding. As APH, we do have this connection with the federal government. This is not going to be a cheap device. We are very clear on that. But we are also very clear that we are going to the federal government and saying, in the same fashion that you fund textbooks for the sighted population, and for every kid in America, you need to fund this device, which will be every textbook the kid reads along with so much more capabilities as well. And we’ve received very, very good feedback from the federal government amongst all the turmoil that exists today in the federal government as well.

JOSHUA MIELE: Sure. Let’s think about this from the student’s perspective. How is this likely to impact our students? What ramifications do you think it has or do you hope it has for Braille and tactile literacy?

GREG STILSON: So for Braille literacy, when a student utilizes– and I know you use a single line Braille display, Josh. I use one as well. They’re fantastic in making sure that Braille stays at your fingertips. But as for the teaching aspect of teaching Braille, they don’t teach you to read correctly. So being able to read with two hands, using what they call the butterfly method to make sure that you’re reading all the way across and tracking with one hand while you’re reading, finishing the line with the other. That’s not how they teach you to read on a single line Braille display. And I’m really excited to see what potential we see both in speed and comprehension, when they’re reading on a multi-line display like this. And when we look at textbooks and Braille transcription, being able to deliver these textbooks or just leisure books wirelessly, digitally so that we’re not waiting for volumes to come in the mail. And as somebody who was a computer science person, I waited for my math book.

JOSHUA MIELE: We’re old. We’ve been there.

GREG STILSON: Yeah. You’re waiting for your volumes of Braille to come way after the rest of the class has already been to those volumes of Braille, or they’ve been to those chapters. So our hope here is that we’re going to be able to greatly speed up textbook delivery, but also speed up access to tactile graphics, and being able to do what we showed earlier, access the tactile graphics image library. So if a student says, what is a pyramid look like? We can pop up a 3D graphic of a pyramid on the device.

JOSHUA MIELE: Tactile graphics, that is my jam. I am super excited. I love Braille, love Braille literacy. But I think that folks have really– blind folks have really been operating in a world really almost free of tactile graphics, many of us. And it’s essential for STEM learning, for looking at timelines, for history, for maps. There’s so much that graphics will enable. And the idea of having an electronic device, the mixed Braille and graphics is super cool. But what are the other graphics opportunities? I’m thinking about– obv– I was thinking about physics simulations, games. Talk to us a little bit about some of the wilder side of tactile graphics that this enables.

GREG STILSON: Being able to connect– and this is something that we’re already in conversations with the mainstream screen reader providers and things like that. But using this as, essentially, a tactile terminal for your other devices, being able to immediately transfer information from a computer or from a smartphone to a tactile display on demand. I think AI is going to play a huge factor into that. Being able to– today, we don’t have the AI or the existing AI to be able to strip a tactile graph or a graphic down into its necessary components for a blind student to access what they need. We all know that’s going to be coming. And I think we share the dream that you’d be able to take a graphic and say, make this tactile readable. And it’s able to strip it down to its necessary components at that point. The last thing I’ll say is that we are– one of the things that when we look at the challenges in the classroom is the impromptu learning challenges. The teacher forgets to get the TVI, a graphic, or a component that they’re going to be going over in the classroom ahead of time. And now, what ends up today happening is the TVI is using household things like rubber bands and wiki sticks to be able to put that graphic together.

JOSHUA MIELE: Yeah.

GREG STILSON: One of the things we’re working on is a mobile device drawing tool for the TVI to essentially sketch out what that graphic is on their smartphone, or their tablet, or their computer. And then hit a button to transfer that in real time to the Monarch so that they can continue to learn, even though that wasn’t provided ahead of time.

JOSHUA MIELE: I always talk about the three E’s, education, employment, and entertainment. And I think the final E– it’s super important for all kinds of reasons. And it often gets lost in the scramble of dealing with making sure that the first two, education and employment, are accessible. Have you made any games for the Monarch yet?

GREG STILSON: We are working on two right now. One of them is a start up game to teach how you do the pointing, and clicking, and zooming, and stuff like that. It’s a take on the old Mario paint flyswatter game. That was fun. So we’re trying to swat the fly. Essentially, track it and swat it. But we’re also working on tactile chess, which I’m super excited about. And that’s something that Humanware and APH have really worked on heavily is this software development kit. Our hope is to build our own games. But we want others to be able to build games for this as well. So really excited to say more on that here, probably in the next year. But I would say, right now, we know gaming is a huge part of education as well. The best way you can teach kids is when you– they don’t think they’re learning something, but they are.

JOSHUA MIELE: You bet.

GREG STILSON: Definitely fired up about that.

JOSHUA MIELE: So you mentioned AI a couple of minutes ago. And I just want to– the hardware is super cool. The fact of a tactile display. But you also mentioned BRFs. And VRFs are hard-coded, formatted Braille files that require a particular page size, a particular line length. What else needs to change in order for this Monarch dream to be realized? What are the software parts of this and the file standards parts that need to change?

GREG STILSON: Yeah. So we call this the Holy Braille Highway. If the Monarch is the Holy Braille, we look at that as the car. But you need the road for it to drive on. And that’s really where, as you said, the file standards had to change. You have BRFs that are designed for a physical piece of paper. And now, this is the– we’re hoping– the first tactile display of this nature. But at the same time, we know there’s going to be more. And we also know that single line displays are not getting all they could out of a BRF file. You have more intelligence that you could put into a book like that or a file like that to make it easier to navigate. And so the functionality that we have in this device is only as good as the file type that we’re opening. And so today, when you have a BRF file, it has zero navigation components to it, other than the find command.

JOSHUA MIELE: I just noticed.

GREG STILSON: Yeah, exactly. So being able to take a new file standard. And we partnered with the DAISY consortium to really shepherd this through. But the e-Braille standard is this combination of hard coded Braille, so formatted Braille. But also, marked up in a similar fashion to an EPUB file. So being able to jump by table, jump by heading, use your table of contents. But most importantly, include your tactile graphics in the book as well. So utilizing links to link to tactile graphic files that are in the book itself. Today, when a book is created at APH, the graphics are always created separately. And then bound in at the last minute. And today, with an e-Braille file, you’d be able to see. It would say something like, see figure B. But that would be a link. You could double click on that link. And it would take you right to that graphic within your book at that point. This, I can tell you, would not be moving forward, if we didn’t have the e-Braille project happening in parallel. And that’s something that we’re equally as excited about.

JOSHUA MIELE: And I think it has many other implications as well. The idea of reflowable Braille means that not only will it be able to be put on a Monarch regardless of line length, but you can– we’ll have more flexibility in embossing files as well. And so just the e-Braille standard, I think, has been a long-time coming. And I think it’s great that the Monarch has pushed that forward. I know that we’ve only got a few minutes left. And I’ve got so many questions. But real quick, Greg, give me a– paint a picture, if you will, with words of how this is going to get used in the classroom. What’s your vision for that? And also, how are students going to get the unit?

GREG STILSON: Good question. So within– so we see a world where students are going to use this in almost every class from their English class to reading literature, all that stuff. But most importantly, in the STEM subjects. And that’s really where we’re partnering with Desmos to create a tactile graphing calculator as well. So being able to do on-the-fly graphs, and be able to see where intercepts are happening and things like that without having to go to an embosser and take the time to emboss it on the fly. But as we mentioned, that impromptu learning in virtually every class being able to show a right triangle when you need to, that type of stuff. But I think there’s going to be more, as we get into the learning management systems of being able to and starting to dream up these worlds, where you could take a visual graphic and turn it into a tactile graphic on the fly. Accessing things like Google Classroom, if a teacher has a graphic there that you need to look at, being able to download that into your Monarch and have it apply to a tactile filter, if you will. So I think those are just some of the early foundational– I’ve got a dream of being able to draw on this thing as a blind kid. I would love to see blind students doing a little bit more in art. But then you asked how–

JOSHUA MIELE: Not just art, but all kinds– drawing for communication.

GREG STILSON: Exactly.

JOSHUA MIELE: If you want to teach them– if you’re blind and you want to teach one of your sighted colleagues, classmates about some geometry that you understand that they don’t, it’s nice to be able to draw.

GREG STILSON: Exactly. You got it. I think that’s in the future as well. But you mentioned, how are they getting it? So we are about to start– so right now, we’re in the midst of field testing. One of our obligations to the federal government is that we actually put this in classrooms and report out on the data. So we have it in 45 classrooms around the country. And we start something that is really new for APH, but it’s something that we believe needs to happen. So we’re funding through our Center of Assistive Technology training starting in January. We’re starting regional teacher training sessions, where teachers can apply to be included in these regional in-person two-day trainings, where they will learn, as a TVI, how to both utilize and teach this product. And at the conclusion of those trainings, as they have finished them, they will be given a Monarch of their own to practice on, to report out on, how they’re creating content for, things like that. So that we know this is a tech barrier for some TVIs. And this is a way that we’re really introducing it early before the product gets in the hands of kids. So that the teachers can eliminate that fear factor of getting it in the hands of the students down the road.

JOSHUA MIELE: And I assume that teachers can just reach out to APH to get on that– to connect with that list. That’s great. Don’t mean to rush you, but we don’t have as much time as I wish we did so. But here come the hard questions. How much is it going to be? And when can people actually get it not in a beta program?

GREG STILSON: Yeah. So our hope is to have this ready by school year– 2024-2025 school year. So the fall of next year. And it’s not going to be cheap. As I said, we’re still looking at anywhere from $18,000-ish type of dollars. And it’s not cheap. This is where the federal government is really going to come into play. So it will be available at minimum on the Federal Quota system in the same fashion that you receive your Federal Quota dollars for your schools and things like that. But our hope is to get additional funding from the federal government that does not eat into the Federal Quota dollars as well. And so that is something that is still ongoing. Call to action, at least, for our listeners here is if you are in the United States, please contact your local representative. And tell them why a tool like this will help blind students in the United States, and, ultimately, the world.

JOSHUA MIELE: It has been such a pleasure to speak with you today. I’m so thrilled that this product is coming from APH, who really understands how to make it– and the partners– it’s really phenomenal. I’ve got my fingers crossed. I can’t wait to have one of my own. And it is always great to speak with you. And I look forward to seeing your progress, as the project moves forward. So thank you for speaking with us today. And thanks for being part of Sight Tech Global. And back to you, Alice. Thanks.

[MUSIC PLAYING]