Inventors Invent: Three New Takes on Assistive Technology

Inventors have long been inspired to apply their genius to helping blind people. Think of innovators like Mike Shebanek (Voiceover, Apple) or Jim Fruchterman (Bookshare, Benetech), to name just two. Today, innovators have a nearly miraculous array of affordable technologies to work with, including LIDAR, computer vision, high speed data networks, and more. As a result, innovation is moving ahead at a dizzying pace. In this session, we will talk to three three product innovators on the forefront of turning those core technologies into remarkable new tools for people who are blind or visually impaired.

[MUSIC PLAYING]

NED DESMOND: Thank you, Will. And thanks, everyone, for joining us today. Welcome to Inventors Invent– Three New Takes on Assistive Technology.

Now, inventors have long been inspired to apply their genius to helping blind people. Think of innovators in recent years like Mike Shebanek, who invented VoiceOver at Apple, or Jim Fruchterman, who worked so hard to deliver Bookshare from Benetech. And those are just two people. There are many of them.

Today, innovators have a nearly miraculous array of affordable technologies to work with, including LiDAR and computer vision and high-speed data networks and much more. In this session, we’re going to talk to three [AUDIO OUT] of turning those core technologies into remarkable new tools for people who are blind or visually impaired.

So it’s with a great deal of respect and admiration that I am happy to introduce our panel of creators. So we have three gentlemen today. First is Cagri Hakan Zaman. He’s the co-founder of Mediate, an innovation lab based in Boston, which is the creator of the Supersense app for the visually impaired and blind.

Cagri is an expert in spatial computing intelligence who aspires to create AI systems that enable and empower individuals in physical and digital spaces. Prior to establishing Mediate, Cagri completed his PhD at MIT, and his dissertation was spatial experience in humans and machines, which offered an investigation into the computational aspects of human spatial perception and reasoning, with the aim of developing spatially aware machines.

Louis-Philippe Massé is the Director of Product Management at HumanWare. Dr. Massé has a doctorate in physics. He worked for more than 20 years in the industries of fiber optics, 2D and 3D sensor design, as well as in product development in the field of metrology and product inspection. He started at HumanWare as the Director of Product Management. He’s currently Vice President of Product Innovation and Technologies, where he is responsible for product management, development, as well as [AUDIO OUT] Soon he is overseeing the launch of Stellar Trek, which is what we’re going to talk about today.

Finally, Kursat Ceylan. Kursat was born in 1986, and he was born blind. Kursat studied psychological counseling in university, and during his years in university, he was selected to attend the YGA leadership program as one of 50 people out of 50,000 applicants. He volunteered on projects for the socioeconomic development of the visually impaired.

And after his graduation, he started work at Roche in Istanbul. During this time, he was producing and hosting a Turkish award-winning radio show called Exploration of Senses that are Suppressed by Sight. Kursat won many awards for his WeWALK cane project, where he is co-founder. And we’re going to talk about WeWALK today.

So welcome, gentlemen. Thank you for joining us today.

So I’m going to begin with Kursat. We’re going to just quickly step through some of the product features associated [AUDIO OUT], which is a cane and it’s an app. And perhaps you could give us the very quick rundown on what the WeWALK project is all about and how it helps people who need assistance.

KURSAT CEYLAN: So WeWALK is a smart cane developed for the visually impaired people. And it alerts visually impaired user to haptic feedback about upper-body obstacles such as trees and poles, and gives turn-by-turn navigation, informs about restaurants, cafes, stores while I’m passing by. The most importantly, it gains new features by integrating with cutting-edge smart city solutions. And so this is my WeWALK cane. And you can see the obstacle detection sensor here. And it has a touchpad here. And also–

[VOICE ASSISTANT SPEAKING INDISTINCTLY]

–it has a microphone speaker. Also it has gyroscope, accelerometer, compass in itself so visually impaired people, while they are walking, they can– on WeWALK’s voice menu, by swiping–

VOICE ASSISTANT: Explore. Transport.

KURSAT CEYLAN: –through touchpad.

NED DESMOND: I see. Kursat, does it work independently of an app? Or does it require an app to work along with it?

KURSAT CEYLAN: Actually, both of them. So you can keep using it as a standalone device. And you can get benefits from its obstacle detection feature. But when you pair with your smartphone through Bluetooth module, you can reach navigation and other smart features of WeWALK.

NED DESMOND: I see, I see. And what are the key technologies involved in the cane? What technologies are you using to deliver these features?

KURSAT CEYLAN: I think, most importantly, we use IMU sensors, so which help us to provide more accurate navigation experience to visually impaired people. As [AUDIO OUT] are walking on the street, the navigation technologies makes our lives really easier. But we have to hold our smartphone always. So it means our two hands are occupied, holding white cane, holding smartphone. And it’s a distractive situation.

So thanks to WeWALK’s inbuilt sensors, we can put into our pockets our smartphones. And then I can keep getting navigation through WeWALK’s touchpad. And we provide more accurate navigation experience thanks to WeWALK’s inbuilt sensors.

NED DESMOND: So you’re detecting above-ground obstacles with sonar, right?

KURSAT CEYLAN: Yes, you are right.

NED DESMOND: So that gives a little more safety to the person using the cane.

KURSAT CEYLAN: Exactly.

NED DESMOND: And then there’s one hand free, which is a big advantage, of course. So you’ve got your phone in your pocket and you’re using your cane.

KURSAT CEYLAN: This is so important, Ned. It was around three years ago, I was in New York to give a speech at the United Nations. And while I was going to my hotel, I was using my traditional white cane and holding it with my one hand. At the same time, I was holding my smartphone with the other. And I was pulling my luggage with these three fingers. And I was trying to manage all these things.

And not surprisingly, oof, in this chaos, I bumped into a pole. So that’s why you can see some scars on my forehead.

So as you may know, visually impaired people are trained at the schools, at the blind organizations, during the mobility trainings, how to use their empty hand. However, because of holding the smartphone, we cannot apply these methods to protect ourselves to increase [AUDIO OUT].

NED DESMOND: Right. And you’ve also built in a custom navigation. So how custom is it? Are you building on top of other navigation systems? And how have you put that together?

KURSAT CEYLAN: Yeah, of course. We use Google infrastructure. But we built our own technology on top of it to provide more accurate navigation experience.

Because, as you guess, visually impaired people need more accurate navigation comments. For example, it tells me, turn right in 100 meters. OK, I will turn right. But there should be two different streets on my right. So which one should I– in 1 o’clock direction or 3 o’clock direction? So we provide clockwise accuracy navigation experience for visually impaired, the technology that we developed.

NED DESMOND: I see. And so you also have an interface, a voice-activated interface. Is that the predominant way that the user interacts with the cane and the app?

KURSAT CEYLAN: Yes, you’re right. As I showed at the beginning, WeWALK smart cane has a voice menu. So visually impaired people navigate through by swiping on WeWALK’s touchpad. So this is one way of interaction.

At the same time, we have just released our voice assistant interaction as well. So it is beta right now. But we believe in the potential of voice assistant interaction through WeWALK smart cane. And we will empower it day by day.

NED DESMOND: Right. [AUDIO OUT] as there are more sensors and more data and more complex algorithms providing information to the end user, the interface gets more complicated. How do you convey this information in a timely way to the user? What have you learned along the way about that problem now that your device is in use with many people?

KURSAT CEYLAN: So you are right, we need to provide some more simplified user experience. And that’s why we started with voice menu to WeWALK’s touchpad. But we observed that there are many visually impaired people who interact with their smartphones through voice assistants. You know, I have met many visually impaired people who even don’t use VoiceOver, they are just using Siri.

So that’s why I think voice assistant approach can make our lives easier as well as, of course, our users’ lives easier, and can solve this too many data process.

NED DESMOND: Yeah. And what about haptic? Do you think haptic has a long-term role in interaction with the end user?

KURSAT CEYLAN: I think it will take place in our lives. But we will learn how to use it more appropriate way. And also, currently, we alert our users to haptic feedback about obstacles, but we got some demands from our users. They want to hear sound as well. In these days, we are working on adding sound as well.

But it doesn’t mean that haptic is not a suitable way to communicate with users. But I think we should find a better way of using it, maybe applying different patterns or it can be more understandable for visually impaired users. And we can build. And it will be so helpful for visually impaired still.

NED DESMOND: Great. And as you look to the future on your product roadmap, what would you say, what is the most important thing that you’re thinking about? What stands out as the really important technology or feature that you know your audience, that your customers, really want?

KURSAT CEYLAN: So it’s not difficult to guess. We keep developing our technology. And we always push our boundaries. And we want to make WeWALK a personal hub for visually impaired people where they can reach all sorts of information and technology about their mobility experience.

And we feel ourselves so lucky because we are not alone in this journey. So WeWALK’s AI studies was funded by Microsoft. Because WeWALK is one of the Microsoft AI for Accessibility startups. And also we are partnering with Imperial College London to [AUDIO OUT] our technology more. And we are making a research and development partnership with them.

And currently we are working on mobility training feature of WeWALK. So as you know, visually impaired people are trained how to use white cane. But unfortunately there is no follow-up mechanism. So with WeWALK’s mobility training feature, teachers will have a chance to follow their students’ mobility improvement. And this is the technology that we are building together with Microsoft right now.

NED DESMOND: So you’ve mentioned that to me in the past. That’s a very impressive system. Because trainers can then see the data [AUDIO OUT] even when the trainer is not with them. So that would be a pretty remarkable advance.

So how many people are on the team right now?

KURSAT CEYLAN: So we are a team of 23. And we are headquartered at London, UK. And we have subsidiaries in Turkey and the USA as well.

NED DESMOND: Great. And the price of the cane?

KURSAT CEYLAN: So it costs $600 currently. And it’s available. And so far, we reached thousands of users in 59 different countries.

NED DESMOND: Great, great. Well, congratulations on your success so far.

KURSAT CEYLAN: Yeah, thank you.

NED DESMOND: And let’s turn to Louis-Philippe Massé and talk about the Stellar Trek. So Stellar Trek’s a very ambitious device because it’s coming in on the heels of another very successful device, which is the Trekker. Is that right, Louis-Philippe?

LOUIS-PHILIPPE MASSÉ: Yeah, exactly. And actually we have entered the navigation device market in the assistive technology field since [AUDIO OUT] with our first device. And it was pretty successful. So of course, we have big shoes to fit. But we’re pretty confident that this new model will be pretty well received and pretty useful with the new features we will introduce.

NED DESMOND: Right. I mean, the ambition level is really pretty stunning. Because in the words of your colleagues, it’s going to provide tools at every step of our users’ everyday lives. That’s a huge step up from where Trekker is, right? Trekker had a more limited feature set.

LOUIS-PHILIPPE MASSÉ: Exactly, before, when somebody wanted to go from point A to point B, well, point B had to be on the sidewalk. So now we want the point B to be really at your friend’s house, or the restaurant, or you’re meeting someone, or somewhere else.

So that’s why we kind of introduced what we call [AUDIO OUT] not just navigation on a larger scale, like what Google Maps or other application could give you, but first, of course, our microlevel navigation is really optimized for blind users.

But then, after, the new features we’ll introduce is use the camera that are on the device to actually locate the door that you actually want to go to. So if you’re going to a 12 St. John Avenue, when you are in front of that address, you take the device, which I have a prototype here, and the user just scans, broadly, the scene, and the device will locate potential doors and eventually identify if it’s the corridor, and give you precise micronavigation directions to go to [AUDIO OUT] that door.

NED DESMOND: That’s really pretty remarkable. Can we see the device again? You gave us a–

LOUIS-PHILIPPE MASSÉ: Very short one. Don’t freeze the image.

NED DESMOND: [LAUGHS]

LOUIS-PHILIPPE MASSÉ: That’s a prototype that is used, actually, for drop test and durability test. So there is no logo on it.

NED DESMOND: But is that approximately the size of what the production model will look like?

LOUIS-PHILIPPE MASSÉ: It’s exactly the same physical size. And the only difference is that it’s going to have markings and logos and stuff like that. But you see it’s actually a little smaller than the usual smartphones. It’s a little thicker, because we have much more powerful antennas than you could find on a smartphone. So that’s why it’s a little thicker there.

NED DESMOND: Well, that leads me to the tough question a lot of people are probably going to bring up, which is a separate piece of hardware. So in the case of WeWALK, the cane is separate and it’s working with an app. In this case, the device is replicating a lot of what is in a smartphone, but doing other things as well. Could you explain the advantages of having a separate device and not using one of the [AUDIO OUT] platforms?

LOUIS-PHILIPPE MASSÉ: Well, there’s many reasons to that. Of course, we recognize that there’s pretty cool applications that you can get them for free on your phone, and it’s working pretty well. But it’s really going from really the whole journey that you want to go on. That’s one thing.

And also, just on the hardware side, we really have optimized the hardware. For example, the antennas are much more sensitive than the usual smartphone antennas to make sure that the GPS reception is always optimal. And we will also introduce, in this model, the L5 GPS reception. So there is the precision on the positioning that is submeter. So it’s like 3 feet or better in terms of resolution. So it’s still not so common on most smartphones you can get there.

And of course [AUDIO OUT] we have dedicated hardware to process the images and so on. And finally, one of the advantages is that there’s physical buttons. So the interface is really made for people who are not necessarily comfortable touching on a smartphone screen. So it’s all these combinations and also build on our past experience, that we wanted this product to evolve gradually. So this that’s why we haven’t embraced the old smartphone bandwagon so far.

NED DESMOND: I see. So I guess, then, there are usability advantages of developing your own hardware, the physical buttons. And then you can control the trickier electronic elements to amplify the technology where you need to amplify it– better antennas, greater sensitivity.

Have you been advantaged by all the things that have happened in the technology marketplace in the past few years? The rise of very sophisticated GPU sensors, and the presence of cloud compute, and other things like that? Because in a way, sitting in Silicon Valley, as I do, people would say, that’s crazy, why would you develop your own hardware when you’ve got an unbelievably sophisticated low-cost platform on a smartphone. What is your response to that?

LOUIS-PHILIPPE MASSÉ: Well, there’s, again, many reasons. When everything is optimized to work together, it has many advantages. Smartphones are wonderful machines. And we’re not competing against them. Actually most of our users also have an Android or Apple [AUDIO OUT] iPhones. And they’re quite happy about them.

But a navigation device such as ours, it’s really something our users really depend on. So for example, the battery life, it’s something critical. If you’re going to some area you don’t know, you better be sure that, if you’re alone, that you can depend on something.

So our battery is made to last much longer than smartphone batteries when it’s in use. Because if you use a navigation application on most smartphone, well, the battery will deplete quite fast compared to our device. It’s more than a factor a factor of five or six. Because our device is really optimized for this.

But in the broader sense, we actually benefit a lot from all the new technologies that appears in the last few years. Just the processing power that permits all these calculations. [AUDIO OUT] So that’s why our first navigational device was actually a backpack. It was a big device. But now it fits in a small package. And it has enough power that it can localize and recognize the door that you want to go. And it guides you there. So that’s it.

NED DESMOND: And to what extent are you are you drawing on cloud services, for instance, to provision the features, for instance? Is your computer vision your own proprietary system? Or are you using another platform?

LOUIS-PHILIPPE MASSÉ: So that’s a pretty good question. And actually, we’re still debating this. We wanted the device to be self-sufficient. So we want to make sure that even if you are in an area where, for example, you couldn’t get any Wi-Fi or cellular signal, you’re still able to use the satellite to guide you to your destination, and let the device [AUDIO OUT].

So we will have offline capabilities, even for the AI part. So we have the no-role computation that is based, in part, on the device. We’re looking how to extend that, of course, because we believe that, as we move in the future, internet connectivity is going to be pretty well universal, especially in the urban areas.

So we want, actually, to benefit from cloud computing and the capabilities that it adds to the device. So we don’t see this as a competition. We have the baseline, completely offline calculation. But we will add, later on, capabilities to use the cloud.

NED DESMOND: So that’s probably going to be a pretty significant factor in the future, letting the user know, perhaps, that some features may [AUDIO OUT] depending on whether it’s on-device or in-cloud, that sort of thing. Does that come up as an issue?

LOUIS-PHILIPPE MASSÉ: Well, you can make the same parallel when you’re using a smartphone and you go to a remote area, for example. You obviously will not have 5G reception if you go to the deepest countryside. So some of the fancy application, you use a lot of data, will not work very well there, or not at all. So it’s kind of the same.

In urban area, in the big cities, getting internet coverage, it’s not even a question. It’s there. And so of course we’ll benefit when we have it. But when we don’t, we’ll still have enough capability in the device and the features that will guide the users safely to their destination.

NED DESMOND: Will the device require a subscription to a cellular network?

LOUIS-PHILIPPE MASSÉ: At the beginning, we will not introduce these cloud computing [AUDIO OUT] for now. It’s actually something we are kind of debating internally. Some people are kind of allergic to these subscriptions. So we might have a subscription-based service or a one-time fee that will cover the expected lifetime of the product. So we’re actually debating this. So we’re not very clear yet on the exact model for that.

NED DESMOND: Great. Well, I think it’s terrific to hear you talk this way. Because I think a lot of people aren’t familiar with how difficult it is to figure out a product [INAUDIBLE] you have to address, and trade-offs, and all the rest. It’s very [INAUDIBLE].

LOUIS-PHILIPPE MASSÉ: And I’m speaking for the others as well, that since we have an additional layer of complexity is that we need to design a device that is used by people who don’t have sight. So it’s even more. So that’s why we have these challenges. But that’s why we’re here, the three of us.

NED DESMOND: HumanWare has a great track record of building great products. So I’m sure that this will be interesting.

I wanted to dive into one particular feature set which sounds super attractive. My own wife is blind. And I know that the last few feet of her getting around, finding the right doorway, or indoor navigation, locating the stairwell, something like that, can be especially tricky and even a little dangerous.

Your device is ambitious in that respect. It really wants to try to close that gap. How are you doing that?

LOUIS-PHILIPPE MASSÉ: Well, as I was explaining earlier, when you’re on the side of the side of the street, you know you’re not far away from that building. But you’re still at risk of walking randomly into a car that is crossing [AUDIO OUT] or a bicycle path, or some rocks that could be on your path. So that’s why we want to do– I’m sure your wife know about the FFF, the Final Forty Feet, which there’s also, sometimes, a fourth F letter that sticks into that acronym. Anyways.

NED DESMOND: Yes, that’s right.

LOUIS-PHILIPPE MASSÉ: [LAUGHS] And that’s what we want to de-risk. There’s always risk. We don’t pretend that we will remove all the risk of blind pedestrian navigation.

But we want to give additional help such that it will be a little like having a friend helping you going to those final 40 feet. So we will use the AI and the [AUDIO OUT] to eventually locate potential threats. And sometimes, when you’re on the sidewalk, the path to that door is not a straight line. So we will say, OK, you will have to go at 10 hour for 40 feet, and then turn to 2 hours, and so on, something like that. So that’s the direction we go. Sorry.

NED DESMOND: And the interface will be audio? voice? [INAUDIBLE].

LOUIS-PHILIPPE MASSÉ: It’s audio. Now, all the feedback from the device is audio. We have basic voice command. But since we will have a neural engine on the device, we’ll also introduce natural language understanding such that you can use the device as a conversational agent to actually help you, more like a personal assistant, to go from your destination.

So you will say, to the device– we haven’t given the device a cute name yet– but suppose you say, hey, Ned, guide me to the door of the Starbucks. And then the assistant will [AUDIO OUT] this way, but be careful, there’s a telephone pole in the way. So things like that. Like if you were with a friend.

NED DESMOND: Yeah, I’m used to that question.

LOUIS-PHILIPPE MASSÉ: [LAUGHS]

NED DESMOND: [AUDIO OUT] domestic life. Great. And what is the price going to be on the device?

LOUIS-PHILIPPE MASSÉ: Final price has not been announced yet. It’s going to be above $1,000 US. Probably, as I was saying earlier, we’re looking at different models in terms of if some of the service will be subscription-based or not. So this will be refined in the next few months.

We’re planning to launch the product around April/May. So before that of course, we’ll have the final price. Sorry?

NED DESMOND: Next year.

LOUIS-PHILIPPE MASSÉ: Yes, of course. I’m sorry. In April/May 2022.

NED DESMOND: So that’s very soon.

LOUIS-PHILIPPE MASSÉ: Yeah, yeah, we feel the pressure.

NED DESMOND: Sure. [AUDIO OUT] working on the development of this device?

LOUIS-PHILIPPE MASSÉ: We’re pretty lucky at HumanWare. We have a pretty nicely-sized team of about 48 people in my R&D team. So we have a lot of people who are working on different aspects of the product, the hardware, of course, but also the software, the AI, but also all the tests that need to be done.

And we are in that phase now. We’re really on the testing phase, to make sure that we catch all the potential issues. Because especially for a navigation device, it’s not like a recreational audio player or something like this. We know that people will depend on this and their security will depend on this, at least in part. So we’re really in a beta testing phase big time now until the launch of the product.

NED DESMOND: Great, great. Well, thank you for that very deep and candid discussion about the development of Stellar Trek.

So let’s turn to our final participant in this panel, Cagri Hakan Zaman, who straddles the world of academia and product development. That’s a fun place to be, I’ve always thought.

Could you tell us a little bit about Mediate, which is the company that you co-founded, and what you’re trying to accomplish with Mediate, and how it led you to create the Super Lidar app family, Super Lidar and LiDAR Sense, I think is the second one, right?

CAGRI HAKAN ZAMAN: Yeah, the Supersense. Sure. So Mediate stemmed from my PhD research at MIT, which is on spatial experience and perception in humans and machines. I was always interested in how humans make sense of the environment around us, how we find where we are, how we navigate, how we understand what’s going on [AUDIO OUT] to develop AI systems that can replicate our skills.

And with that, I basically applied for an accelerator at MIT to build a company around these technologies. And the Supersense, our first product, was sort of a natural ally for this type of task. Because one of the communities that need the most spatial awareness is the visually impaired and blind community.

So we devised the app Supersense. And Mediate’s vision is to enable people in both physical and digital spaces and empower them with new technologies using AI and augmented reality.

NED DESMOND: And it’s a remarkable app. It’s also a crowded field. So how do you– on the one hand, I can see it as a really cool proof of concept in a lot of ways. But where do you think that app or its successors, down the road, can stand out? I always got the impression that this is a step down a road, not necessarily the [AUDIO OUT].

CAGRI HAKAN ZAMAN: Yes, exactly. So we are following a strict user-centered design process with whatever we do.

So the reason Supersense ended up having some of the features that the competitors have is that our user base was demanding these to be included. So we wanted to listen to them and provide them while actually developing our technology and solutions, using Supersense as our testbed. And which, interestingly, grew to be our major app. It was loved by people. And we kept improving it.

But what we think is the most crucial thing in an app that is using a lot of technologies in a smartphone is to be task-centered, not feature-centered, meaning it should try to provide a solution as fast as possible for the relevant problems that people have instead of giving them a lot of toolkits. [AUDIO OUT] relevant tasks and thinking about what is the most efficient and quickest solution to these tasks so people will not spend a lot of time trying to find the solution.

I think one of the sort of distinctive feature of Supersense is its UX. You can do anything only by two swipes or two taps. You will never spend time trying to find what are you looking for, like from a toolbox. We spend a lot of time improving and designing our user experience so that it provides the most efficient way to find a solution to a problem.

NED DESMOND: Right, right. That all makes a lot of sense. And as you look down the road, I was intrigued when you launched Super Lidar. And I know that you have a big interest in AR.

And across many of the panels at the show this year, [AUDIO OUT] and audio augmented reality is a big subject of conversation. We have a session on indoor navigation. The Seeing AI team at Microsoft is very interested in audio-based AR. How are you seeing that shape up? Because the launch of LiDAR sensors, of course, on the iPhone, created all kinds of new possibilities. But it feels like the possibilities are so broad. And it’s hard to know where to begin. And you’re a pioneer. So we’d love to learn how you see the LiDAR possibilities and what that app has taught you.

CAGRI HAKAN ZAMAN: Of course. Our first feature in Supersense was what we called then the object finder. It was the first of its kind, if you believe. This technology was there for a while, but none of the other apps apparently didn’t pick up the very key thing that people need to find things.

So the first thing we actually came up with was [AUDIO OUT] contextually relevant task finder and object finder. So we started with the intent to provide a general task solver that is relevant for navigational and spatial awareness-related tasks. So when the LiDAR came out, it was a no-brainer for us. Because we have been developing all the technologies. And all our R&D was focused on using computer vision technologies to parse the environment and find out information in it.

So we quickly developed what we already knew, the Super Lidar, which is a prototype, which is being developed right now with actual users, that gives a sense of what is around you by converting distances into sound and rhythms that gives you a sense how big a room is and how you orient in it.

And I think there’s a big potential there, of course. This is early days for LiDAR. We are still using our computer vision. And down the road, I think technologies will blend, vision and LiDAR technologies, to create this type of solutions as well.

NED DESMOND: As [AUDIO OUT] researcher in this field, when you look at where we are today and your close connection to the next generation of products, what’s most exciting or what do you wish you could get your hands on, I guess, in order to build something that you really know people would love?

CAGRI HAKAN ZAMAN: I think, this year, one of the sort of converging areas is digital twins and what people fancifully say, metaverse, the ability to collect and construct 3D environments and access the information in it. It requires a lot of infrastructure. But what we are headed towards and what I wish we had already is a system where people can freely navigate and independently navigate indoor and outdoor environments, combine some of these technologies that are already there, [AUDIO OUT] already developing great technologies.

But what we are missing is an encompassing ecosystem that can provide and collect all this information from people. If a blind individual goes into any market or any shopping mall, and they don’t need to really ask around where to go, just pull up their device and the information is already there, I think that will be the ideal solution in terms of navigation.

NED DESMOND: Is that about algorithms? Or is it about data? or both?

CAGRI HAKAN ZAMAN: Both. The ability to efficiently encode and reconstruct environments, store maps, and then share them over the networks, and using smartphones or other devices to access that, localize a person indoors, these are all dependent both on the development of technologies and the algorithms and infrastructure. Especially the cloud computing will be one of the key elements in this.

NED DESMOND: Right. Because this can’t be on-device, at least no device that we can imagine today.

CAGRI HAKAN ZAMAN: Actually, we have our, like, R&D work that allows you to record. This is one of the upcoming features in Super Lidar [AUDIO OUT] that is going to be in Super Lidar that allows you to record any environment you are in. And then, later on, someone can just localize themselves in the same environment and follow the path you created for them.

This could be used, for example, for a hotel, to let people find reception or other places, you need to just store it once, locally, and then you can share that information with someone. But this is, again, just localized and isolated solutions. For it to be a general sort of task-solver, we need more connected and cloud computing.

NED DESMOND: The very cheerful and helpful staff at Trader Joe’s would really welcome this, at least when my wife walks in the door. So a great advantage.

That’s really fascinating. And I promised, at the start of the show, of this panel, that I’d let each of you ask one question of one of the other panelists or both panelists. And this has to be the lightning round, because we’re out of time. But why don’t you go first, [AUDIO OUT] ask that. Who has a question that they’d love to ask?

LOUIS-PHILIPPE MASSÉ: I can break the ice.

NED DESMOND: OK, thank you.

LOUIS-PHILIPPE MASSÉ: Cagri, I’m very curious, because I know that the indoor navigation is definitely something we look on our side. And I see, actually, potential good discussion we could have together for sure. Is it something that, for example, you would consider– because you were talking about multiple devices to get the multiple sensors and so on, and multiple environments– is it something that your solution could be used on different platforms?

CAGRI HAKAN ZAMAN: Exactly. One of the projects that is supported by Veteran Affairs this year, we called it [INAUDIBLE]. The idea is that we would like to create the standard for recording the indoor environments and adding information to them so that [AUDIO OUT] interacting with that data can use it. This could be a device by a human wearer or a smart home device, a Google Assistant, anything.

So we want to get started with this type of infrastructures. And we are definitely interested in getting many people to get onto the same ship so that we can actually grow an ecosystem of it.

LOUIS-PHILIPPE MASSÉ: And it’s really like crowdsourcing.

CAGRI HAKAN ZAMAN: Yeah, it’s partly crowdsourcing. Or once the standard is there, once the data collection method is there, I think there will be different methodologies. For us, we we’ll make it for an individual to use for their own purposes now to see– actually it’s an experiment too. What else can you add to that representation you have in your hand?

You are thinking of building infrastructure. Maybe you want to be able to keep track of the changes in this physical infrastructure so you can actually share this with contractors.

LOUIS-PHILIPPE MASSÉ: With others.

CAGRI HAKAN ZAMAN: With others, yeah.

NED DESMOND: Cagri, any questions?

CAGRI HAKAN ZAMAN: I wanted to ask– can I ask?

NED DESMOND: Take it away, take it away.

CAGRI HAKAN ZAMAN: All right. I want to ask to Kursat, actually. Kursat, this is an exciting work and I think it’s the first time I see, in your hand, how the device is.

So you mentioned that people are learning how to use the white cane. Are there difficulties for people to adapt to WeWALK because of its differences? Or are you planning to let people adapt WeWALK to a different type of orientation training as well?

KURSAT CEYLAN: So that’s why we built our technology on top of standard white cane. Because we want to provide a technology to visually impaired people which is similar to their standard tool, which is white cane. So that’s why it doesn’t need [AUDIO OUT] adaptation period. But since it’s a technology, like every technology, it needs some adaptation period.

And currently we provide special training sessions for our users. But also we are working on developing automated training session. So the smart cane will introduce itself and will orient itself to the user, and make the adaptation period much more easier.

CAGRI HAKAN ZAMAN: Great. That’s what I thought. Because when you mentioned the orientation mobility feature, I think this is one of the good directions, instead of relying on what is there, you are innovating the way people interact with the [AUDIO OUT].

NED DESMOND: Kursat, do you have any questions you’d like to ask?

KURSAT CEYLAN: Yes, I have one question to Louis. Actually, he showed the device. But since I am blind, I couldn’t see.

LOUIS-PHILIPPE MASSÉ: Sure, I’m sorry.

KURSAT CEYLAN: If you could describe it. Because the ergonomics is so important for visually impaired. Because they will hold it in their hands while they are walking on the street. So I’m curious about its dimensions, weight, et cetera.

LOUIS-PHILIPPE MASSÉ: Sure. Sure, no problem. I will describe it. So it’s about the size of a smartphone if you look at the footprint. But it’s a little thicker, I would say about 2 centimeters thick, maybe. Or in inches, it’s a little less than an inch. It’s kind of rounded. And in the front, there is the keys that are used to enter directions or control the device and so on.

And it’s a pretty intuitive interface. It’s pretty similar to our older devices [AUDIO OUT]. We didn’t redefine everything because people were attached to that physical interface.

What’s on the underside of the device, two high-resolution cameras that are used for the doors recognition and eventually the object recognition as well. So that’s why having your thumb controlling the keypads, if you put the device in front of you, if you scan left and right, that’s where you will localize potential candidates of a door.

So then, through audio cues, it will say, we think we have located a door straight in front of you, for example. Then we take a high-resolution image to do the actual AI door recognition and also recognize the door number, the civic number of the door or the address.

I don’t know if it’s good enough in terms of description.

KURSAT CEYLAN: It helped me a lot. Thank you. So can I ask one more question, Ned?

NED DESMOND: Yes, please.

KURSAT CEYLAN: So which operating system do you use? Do you build your own operating system? Or are you using standard–

LOUIS-PHILIPPE MASSÉ: This one is Linux-based. It’s a Linux-based device.

KURSAT CEYLAN: OK, thank you.

NED DESMOND: And how many physical buttons are on the front of the device?

LOUIS-PHILIPPE MASSÉ: Well, four. And there is, in the middle, this kind of joystick. It’s not a joystick, but there’s up, down, left, right arrows with a center push button too. That is kind of the Enter command. So it’s pretty simple. There’s not thousands of buttons.

But since– I’m not blind, but I can easily, without looking at the device, recognize the buttons because they have all different shapes and different locations. And they’re all accessible from your thumb. So you don’t need to have two fingers, two hands, to operate the device. So [AUDIO OUT] they enter their destination, they put the device in their pocket, because they have the cane. And when they arrive close to their destination, then they pull it out to take the image of the door, to localize, to do that those final 40 feet.

NED DESMOND: Well, thank you for that. Gentlemen, I wish we could talk for another hour, because I have a long list of questions I’d like to ask [INAUDIBLE]. Absolutely fascinating. And I hope our audience appreciates the incredibly interesting and difficult work you do to create the next generation of fabulous assistive tech. So thank you very much.

LOUIS-PHILIPPE MASSÉ: Thanks. Thanks a lot, Ned.

KURSAT CEYLAN: Thank you.

[MUSIC PLAYING]