Hands on with Seleste

Rapid advances in phones, data networks, and hardware miniaturization always seem to be converging on the concept of that super useful, affordable, unobtrusive assistive device. Seleste launches this year with a pair of tech-enabled glasses that mark an important waypoint on that journey.

[MUSIC PLAYING]

BRIAN: Hands On with Seleste Moderator Jennison Asuncion. Speakers Shubh Mittal and Smit Patel.

JENNISON ASUNCION: Thanks, and welcome. I’m really excited about this particular session because I’m originally from Canada, and this invention is a Canadian invention. And we’re going to get into it with the coinventors or cofounders of the Seleste glasses. Shubh Mittal and Smit Patel. So let’s start with you, Shubh, and why don’t you tell us a bit about yourself and your story to getting to where we are with the Seleste glasses?

SHUBH MITTAL: Yeah, thanks, Jennison. It’s great to be here today at Sight Tech Global. My background is in computer science. And before starting at Seleste, I worked at companies like Deloitte and Amazon. I initially got into this space because I was really interested in the smart glasses technology. And I did a lot of research in the industry and basically came to the conclusion that a lot of smart glasses on the market just weren’t useful to people. So I was thinking of ways that you can make smart glasses that would actually be useful. And I thought, OK, maybe this is something that can help people with vision loss. And so I called a friend of mine in Victoria, who’s blind. And I talked to him about his challenges and his life and about the idea. And he just gave me really good feedback. And so I just kept talking to more and more people with vision loss and continued to get really great feedback on the idea. And I thought, OK, maybe this is something that’s worth bringing to life. And what’s really cool is that the first friend I called in Victoria also studied computer science. And we kept talking about how to make the technology work and actually develop these glasses. And I’m really proud to say now that he’s the CTO at Seleste, and his name is Smit.

JENNISON ASUNCION: Great lead in. Smit, why don’t you tell us a little bit about yourself and what was the passion for you to get involved with this project?

SMIT PATEL: Mentioned I studied computer science at University of Victoria. actually, I was the first blind student to study computer science at U Vic. So one of the biggest challenges with me being in a new place. Like, I was studying at U Vic, so I moved to a new city. And one of the challenges was finding different things, grocery stores, getting around the campus. And so one of the biggest challenge for me with that is using your phone. Sometimes, I get on a FaceTime call. And it’s difficult to point my camera and with holding a cane. Then, essentially, my hands are occupied and I cannot do much. So the idea Shubh brought up when he brought up the smart glasses with me, it was a really interesting idea. And I really liked the fact that it would give me more hands-free experience and more freedom to move around and just make things very easier for me.

JENNISON ASUNCION: I got to tell you, Smit, that’s one of the reasons I got excited about this when Ned Desmond first approached me to participate in Sight Tech Global was, I had seen a couple of wearables out there in the market over the years. And I was a little hesitant because, typically, the ones I’ve seen are big and bulky, and they don’t allow me to use my cane. And just using the cane and the phone, like you said, can be a little cumbersome. And I mean, I’m holding the glasses in my hand right now. And I can tell you, as someone who’s blind, they feel sleek, they’re lightweight, and there’s nothing about them that feels that much different than a, quote unquote, “regular” pair of glasses. The lenses are a little thicker. But nothing that would prevent me, certainly, from trying them out. So I like what you folks went with the design and the architecture of the glasses. Shubh, you kind of alluded to this a little bit. I wanted to see if you wanted to say more. There have been a number of wearables that have come out there for the blind and visually impaired market over the years. And I’m just wondering, what made you decide on the glasses? How do you feel these differentiate themselves in what’s available out there now?

SHUBH MITTAL: Yeah, a great question. I mean, it’s exactly like you said, when we started off, we were looking into the assistive technology industry. And a lot of the products were bulky. They were hard to use. And they were incredibly, incredibly expensive. I mean every single person that we talked to had a scenario where they wanted to buy a piece of assistive technology but didn’t because of how expensive it was. I actually talked to the sales director at one of these big assistive tech companies about the price of their products and how they expect people to pay for the product because he knew the unemployment numbers. He knows that there are a lot of people out there with vision loss that struggle with finding jobs. And what he said was that they’re mainly targeting people that are retired and have a ton of money or people that are covered by huge government grants. And that’s crazy because that’s such a small percentage of the visually impaired community. Most people aren’t covered by government grants, and most don’t have a ton of money lying around. And oftentimes, it’s those people that need access to this technology the most. So our mission is to change the narrative that assistive technology has to be bulky and expensive. And our goal is to revolutionize the smart glasses industry.

JENNISON ASUNCION: That’s awesome. And we’ll get into a little bit– in the next question after the next one I have for you folks about the audiences that you folks envision. But let’s get to the neat stuff. Smit, in terms of the glasses themselves and the technology that’s in them, either you Smit or Shubh, what differentiates these glasses technologically from others that might be out there on the market that you want to comment on?

SMIT PATEL: OK, for sure. Yeah, so as you and Shubh were having the conversation about it, firstly, our glasses are much slicker. They have a really great form factor. They are really indistinguishable from a regular pair of glasses. So the glasses have an autofocus camera. Then we are the first company to have a dual bone-conducting speaker system.

JENNISON ASUNCION: I love– I love that. I love that by the way. The sound, yeah, that sound is amazing. Sorry, keep going. Keep going.

SMIT PATEL: Thank you, yeah. So that’s one of our differentiating factors. And we have a microphone obviously. And mostly, the other companies wearable devices, they do AI features such as OCR, object recognition, et cetera. We have those features on top of it. We also do video calling. Like a user who is a blind person or a partially sighted visually impaired person can call a Seleste volunteer or a person in their family, friend, via network. And basically, they can get help to a video call through the glasses. And we also, as I previously just mentioned, can expand. The user can read text, recognize objects, get scene descriptions, and other AI features to our glasses.

JENNISON ASUNCION: That’s amazing. Shubh, did you want to add anything else?

SHUBH MITTAL: Yeah, well, I mean, also about– I’m holding the glasses in my hands as well. And yeah, we have all of these amazing AI and video calling. And the glasses weigh 55 grams.

JENNISON ASUNCION: That’s– yes. These are very– I’m also holding my set here. I’m here in the Silicon Valley. You’re out in Vancouver. And see, we both have our glasses here. I have to say that the one thing I found interesting was that you’ve built in gestures into the glasses themselves. So I could flick up to for the volume and flick forward to take the picture. Now, in the interest of full disclosure, and we’ll talk about this again in a subsequent question, but there’s a mobile app that goes along with the glasses. But I’m just wondering, is it possible right now to remap the gestures? So if I wanted the flick forward feature to do something other than take a photo, is that possible yet? Or is that something that I could put in as a feature request?

SHUBH MITTAL: Yeah, no that’s absolutely something we’re working on. We haven’t exactly figured out the exact integrations like what button does what and what a swipe should do what feature. That’s something– like, right now, we’re testing the glasses. And so we’re going to change what that maps to as we test it. And then, yeah, ideally, we can also give that to the user because some people might really like the AI features, and so they’ll want to map the buttons to AI. Whereas, other people really want to call, and they might want to call certain people every time. And so allowing people to create custom mappings on the glasses will really help. So yeah, it’s something we’re working on.

JENNISON ASUNCION: Amazing. Let’s switch focus for a minute. We kind of alluded to this a little bit before. And I’ll let you two decide who wants to take a first crack at this question. But who are the different groups of blind or visually impaired users that you envision would be using these Seleste glasses?

SHUBH MITTAL: Yeah, well, I mean, firstly, we have just the people that have preordered the glasses so far, we have lawyers and artists and students and software developers and Paralympic athletes and so much more. And so we have a lot of features that are just helpful for everyone. Like being able to read text is a challenge whether you’re in school or you’re retired. And being able to video call is helpful for everyone. But that being said, we think there’s a lot of use cases, specifically, to help visually impaired people in school and in the workplace.

JENNISON ASUNCION: Oh, absolutely. I know, Smit, you were talking in your introduction about just getting around Victoria and getting around a new place. I could totally see these glasses being helpful for someone who’s going for a job interview. And it’s a nervous it’s a nerve wracking enough. And it’s not like you could have an opportunity to kind of get orientation and mobility to a brand new location on demand. But to have the glasses and whether it’s a friend or family member or another service provide guidance and with the autofocus camera and just not having to worry about lugging a big heavy device around to help you with directionality, I could totally see that as being helpful. So I think that employment situation is neat, which is kind of ironic, given, Shubh, you mentioned that the person that you talked to in the assistive technology industry was talking about retired people who had lots of money. And here we are, we’re trying to get people jobs, right? So. And I also think in the campus environment, I remember having to learn a campus, I would go with a friend ahead of time to just learn where classrooms were. I’m completely blind myself. But sometimes, classrooms change at the 11th hour, and it’s just nice to know that this is available. And similarly, in an office situation, you could have to change a conference room location sometimes, so. But I’ll stop there. I don’t know. Smit, did you want to add anything further about the different groups that you hope will take advantage of these glasses?

SMIT PATEL: For sure, yeah. I’ll just add, there are so many different opportunities for different people in lots of professions that they can use the glasses. Or even imagine at your house when we get a new product and trying to figure out how to use the product. That’s also a very interesting use case. Or you can help while cooking. We already touched on navigation, et cetera. So these are some of the use cases that I was thinking. And as one other thing as well, we just want to provide more independence to the user. So yeah, so the glasses would be able to do.

JENNISON ASUNCION: No, that’s great.

SHUBH MITTAL: Yeah. Exactly. I mean, going back to the example of school, like right now, a lot of students need to have someone beside them that’s helping them take notes from the board or helping them to use inaccessible physical worksheets they have to do. So with the glasses, you can kind of do that stuff yourself. You can read the board yourself. You can read worksheets yourself. Same with the workplace, there’s all of these documents, I’m sure, as you know, Jennison.

JENNISON ASUNCION: Yeah, and not only documents. But I remember the days when I was in the classroom, and like you said, a professor put something up on the display. But in the corporate environment, during training courses, people put up PowerPoints. And not only that, if I want to go to a live cultural event, like a concert or a play or something like that, there’s built-in audio description if I’m wearing the glasses. I’m assuming I can just keep the volume low enough to not disturb others but to still get information, which is great. I alluded, also earlier, to the fact that– because this is all cutting edge. And you folks were good enough to be able to ship me glasses so I could at least see the glasses. I know you’re finalizing the mobile apps. There’s an iOS and Android mobile app to go with these glasses. Shubh, I’m wondering if you wanted to take a first crack and just talk to folks tuned in as to what– how does the mobile app work with the glasses?

SHUBH MITTAL: Yeah, absolutely. So when you’re using glasses, you’ll have to download our app, which as you said, is on iOS and Android. And it’s essentially through this app that enables all of the functionality that we’ve talked about so far. When you open the app, you’ll see a list of the contacts that you’ve added which you can click on, and it’ll start a call with them through the glasses. Through the app, you can choose a volunteer. You can choose one of the AI features. Right now, it’s all controlled through the app. But as you said, we’re adding other ways to interact with the device because the whole point is that it’s hands free, right? And so right now, it’s through the app. But we want to add ways that you can use voice commands to the glasses or–

JENNISON ASUNCION: Oh, wow.

SHUBH MITTAL: –buttons on the glasses like you’ve tested.

JENNISON ASUNCION: Yes.

SHUBH MITTAL: Or even like in real life gestures. Like what if you could just point at something? Like you’re holding a piece of paper and you point at it, and it’s able to read it out to you. And so a lot of these cool features is what we’re working on so you don’t necessarily have to use the app every time. But yeah, so the way it works with the app and the reason you need it is because you need to connect the glasses to your phone’s hotspot. And this also means that you can use the glasses wherever you go. You’re not tied down to only at home with Wi-Fi. You initially set up the glasses. And after you do that, they’ll just automatically connect to your phone’s hotspot or your phone’s Bluetooth so you don’t need to set that up every time. And yeah, that’ll allow you to control all the features. You can also use the glasses as just Bluetooth headphones. So all of your audio, like music and voiceover, can go through the glasses. And I use the glasses just on my bus ride home just to open up Spotify and listen to music. You can pick up calls through the glasses. And those kind of things can do without the app.

JENNISON ASUNCION: Neat. No, I think that’s great that there is functionality with and without the mobile device. And I look forward to trying out the mobile device and pairing them with the glasses. Smit, was there any other functionality in the mobile apps that Shubh may have missed, or do we have good coverage there?

SMIT PATEL: I think, yeah, we have pretty good coverage, yeah, with that.

JENNISON ASUNCION: OK, Smit, why don’t I, for this last question, why don’t I start with you? What does the future of the Seleste glasses look like? And most importantly, for folks tuned in, when can they get hold of a pair of the glasses and try out the mobile app as well?

SMIT PATEL: For sure, yeah. So let’s talk a little about the future first, and we can talk about how folks can get the glasses.

JENNISON ASUNCION: Absolutely.

SMIT PATEL: So for our future, we, as Shubh touched on it initially a little bit, we want to change how people interact with technology. What I mean by that is, right now, if we want to use a piece of technology, we have to either pull out our phone or our laptop, and we have to exactly give it a command or tell it what we need it to do, right? But what we want to do is, as Shubh said, maybe it could be a light gesture. Can be as simple as pointing at a box and the glasses recognizing that and it easily reading the content from the box. So that’s one thing. As I previously touched on a little bit, we want to make the user more independent and provide them with more freedom. So we are actually planning to add navigation and explore features on our glasses. The glasses would be able to give turn-by-turn directions to a user. Let’s say, if I was exploring a new city, and I’m walking down a street near my hotel. So the glasses would be able to tell me if I pass a pizza place or a store, et cetera. We also want to add indoor navigation. And what I mean is if– yeah, we touched a little bit, you and me, when we were at school, we had to go to the campus earlier and know where the classes are. At work, we need to know where the conference rooms are. And with the indoor navigation, the glasses would be able to give us accurate directions, even inside a building as well. And yeah, we know we have lots of AI features and video calling. But all of them use data. So if the user is worried they might be low on data or be in an area with not good network connection, then we are also making offline AI for those use cases. So users would still be able to read tags, recognize objects through our offline AI features.

JENNISON ASUNCION: Oh, that’s amazing.

SMIT PATEL: We also want to– yeah– to add more. We also want to integrate with more of our favorite assistive apps like GoodMaps, Soundscape, Seeing AI. Yeah, there are lots of great assistive apps. And we want to integrate with them on our platform as well.

JENNISON ASUNCION: Perfect.

SMIT PATEL: Yeah.

SHUBH MITTAL: I’ll just add– I’ll just add one thing to that, which is we’re working on all these great features for the next coming months. But what’s really cool about what we’re doing, and again, talking about the role of the mobile app, is all these features won’t require a new set of glasses. It won’t be version 2.2, 2.0, 3.0, that OK, now you can have navigation. What it’ll really be like is just an update on the App Store. And so it’s just– all of these features are available for free. And it’ll just be available to you when you update the app. You don’t need to go get new hardware every time. You’ll just always have access to the latest features that we develop.

JENNISON ASUNCION: Great. And Shubh, did you want to comment on– is it too early to give a solid date on which folks who are tuned in might be able to try the glasses themselves? Are we talking months, six months, or is it still too soon?

SHUBH MITTAL: No, yeah. So where we are right now with development– and actually, Smit, maybe you can answer this.

SMIT PATEL: Yeah, absolutely. Right now, we are actually delivering the glasses to our early preorders in the next two weeks. And we will be slowly flushing out the product and developing new features as I mentioned. And users can expect to get the new glasses sometime mid next year. Might be early April or June. And yeah, so right now, we are just doing– users can book their glasses by paying $50 for now. And yeah.

JENNISON ASUNCION: So just because you cracked out a little bit there, you said $50?

SMIT PATEL: Yeah.

JENNISON ASUNCION: That’s great.

SHUBH MITTAL: So typically how it works is, like Smit said, we’re going to release– we’re getting feedback from users, and we’re going to kind of polish the product and make improvements based on that feedback. And then we’ll release the glasses in the summer. And they’ll cost $950. But the way we did it last time is, we had, unfortunately, just a limited quantity that we could sell. And so moving forward, we’ll have more glasses. But we still predict that it’ll be limited. And so if people are interested, they can just go to our website, which is Seleste.co– Seleste dot– S-E-L-E-S-T-E dot C-O. And on our website, there’ll be a checkout form where you can pay $50 and put down a deposit for the glasses so that when they come out, you’ll get a discount on the glasses. And you’ll kind of reserve your spot for when we fully release them.

JENNISON ASUNCION: Wow. This is amazing stuff. I want to thank Shubh and Smit for taking time and for joining me here at Sight Tech Global to discuss the Seleste glasses. I’m looking forward to– I just moved to a new city, so I’m looking forward to trying out some of these new features as they’re rolled out. So thank you very much for your time. And good luck.

SHUBH MITTAL: Thanks, Jennison. It was great talking.

SMIT PATEL: Thanks, Jennison, and thanks Sight Tech for calling us on. It’s great to be here.

[MUSIC PLAYING]