Exploring Envision's Revolutionary AI-Powered Glasses

In this Ctrl+Listen Podcast episode sponsored by Octopart, hosts James and Nora interview Karthik Kannan and Vanessa Vigar, co-founders of Envision, an AI tech company specializing in vision-based technology to aid the visually impaired. Karthik recounts their realization of the challenges faced by the blind and low-vision community during a school talk in India, leading to the inception of Envision. The AI-powered Envision glasses and app offer features like text and object recognition, color detection, and more. Karthik highlights the transformative potential of AI in accessibility, foreseeing it as a groundbreaking force comparable to the advent of electricity. Vanessa Vigar, a low-vision user of the Envision glasses, shares personal experiences, emphasizing their impact on independence and time-saving. Envision offers three editions of its glasses and a subscription plan, aiming to revolutionize accessibility through AI.

Listen to the Episode

Watch the Episode

Episode Highlights

• The Inspiration for Envision
• How Envision Glasses Work
• Can the AI recognise Emotions?
• What is the Envision Ally?
• What has Vanessa's Experience Been with the Envision Glasses?

Links and Resources

• Learn More about Vanessa Vigar: https://www.linkedin.com/in/vigar/?or...
• Learn More about Karthik Kannan: https://www.linkedin.com/in/karthikk-...
• Learn More about Envision: https://www.letsenvision.com/
• Watch a user review of how the glasses have changed their life: https://www.youtube.com/watch?v=6f8Nf...

Transcript

James Sweetlove: This is James from the Ctrl+Listen podcast brought to you by Octopart. Today we have some very special guests for you. I'm here with my co-host Nora but we have Karthik Kannan and Vanessa Vigar, they are here from Envision which is a really fantastic company working in AI Tech but through vision based technology to help low vision and blind people actually navigate the world and it's fantastic that they're doing this so we're really happy to talk to them thanks for coming on the show today.

Karthik Kannan: Yeah thank you for having us James and Nora.

James: Yeah thanks great so just before we start Karthik you're one of the co-founders right?

Karthik: Yeah I'm one of the co-founders of Envision.

James: And how did you sort of have the idea to create something like this?

Karthik: Yeah sure I think um the idea you know uh both me and my co-founder we don't have people in our lives who are blind or low vision uh so you know like rest of the world I think um you know we were completely oblivious um to to accessibility to you know how people with visual disabilities lead you know uh lead lives and so on uh but it all changed you know when uh me and my co-founder went to give a talk um at this school in India so this was actually a school for people who are you know uh blind in low vision uh and it's uh and you know the topic that we were called to talk upon was what exactly was it did it mean to be a designer or an engineer right um because you know the school was basically you know it was full of high school students who were just going to be passing out of high school going into college and into the real world and uh a lot of them were were not very clued into what were what were the possibilities uh what what kind of vocational possibilities existed outside of the usual engineer and Doctor uh kind of a thing that you have in India and so when we went there uh you know when we just told them that a researcher or a designer is just somebody who solves problems for a living right uh it was supposed to be a 30- minute talk um you know just peppered with a couple of questions here and there I think one of the questions that I asked was uh you know what kind of problems would you like to solve when you grow up because you know I was just telling these kids that you know being a researcher or a designer is just you know waking up in the morning and going through a to-do list of problems to solve and hopefully at the end of the to-do list uh you are on to something useful for society as a whole right so uh when I asked the question about what these kids would like to do when they grow up I you I expected something along the lines of you know uh going out and colonizing Mars or you know curing cancer or usually when you talk to high school students they're just so full of possibilities right it's just they have this very idealistic vision of the world so I was expecting something like that but the answers that I got were just remarkably different you know uh it was things like being able to go out out more independently being able to live independently being able to read a book independently so it was all these mundane things and it was at that moment that it actually hit me and my co-founder that these kids would spend a huge chunk of their lives trying to overcome some of the most basic hurdles that cited people take for granted right just mundane existential work that they will have to like overcome you know uh over and over again uh and that's something that deeply B Ed us because you know being a researcher and designer we were supposed to be the people out there looking to solve problems and this seemed like a problem that that should be solved and uh it was also around that time coincidentally that artificial intelligence was starting to really take off right um you know we were in at a time when a lot of advances in artificial intelligence um you know even 7 years ago uh you know in certain areas like being able to read text or being able to detect objects in an environment AI was outperforming humans in these tasks and so what we thought was okay we could use artificial intelligence as a bridge between the world as it is and the world as it should be so for you know we realized very early on that uh you know we can't expect the world to become accessible overnight you know you know it's a it's impractical to expect us to put a braille display on every single you know piece of information out there in the world and uh here is this remarkable technology that takes uh you know that doesn't expect the World to Change that all it requires is an image and uh with some clever tricks we'll be able to pull out as much information as possible from those uh from those images so that was the Genesis for Envision uh that was the very beginning and uh we've been on that Journey ever since for the last 6 seven years uh building tools that help people with the visual impairment to live more independently using artificial intelligence.

Nora: Wow that's fantastic and what tools exist assisted before what was like the status quo for um people that suffer from low vision?

Karthik: Well you know uh they were all tools of the 19th or the early 20th century um so for example let's take being able to read text you know before you know artificial intelligence and Envision came on the scene uh people had to really work with these huge bulky scanners uh that would cost anywhere between $3 to $5,000 that you know you had to put on a desk and you had to connect to uh you know power source and uh you can't really carry it around and it wasn't versatile it was just very basic stuff that could read printed documents you know like a brochure and all that so it couldn't read things like handwritten text and it couldn't read uh uh you know languages other than the usual Latin languages and things like that right so it was bulky was super expensive and it just did one thing if you wanted to be able to you know go out and and read your uh you know sence in the shop you had to buy a completely different device for that and and hope and pray it worked right so you were basically uh you know um having this arsenal of tools that were overpriced and you know just were completely clunky to use and people had to put up with that so that was what um it was in fact you know I keep telling people that a you know um uh people in the blinded low vision Community are expected do to 21st century work with 19th century tools right that's basically what the status quo was before artificial intelligence came on board you know

Nora: Wow

James: And what can you explain a little bit about how your Tech Works what are the features how does it operate?

Karthik: Sure so uh Envision is basically just a suite of tools that uh you know you can think of as a Swiss army knife kind of a thing right so we have a tool that helps people read text for example and you know when I say read text you could read text of any surface could be a curved surface could could be a flat surface you can read text on a printed document you could read a digital display like an ATM machine or a computer monitor you could read text in over 100 different languages including handwritten text in all those 100 different languages you can feed it any type of document in the world could be a magazine article a newspaper article you know like a timetable or a menu card and the glasses and the app will be able to read it right so that's one thing Envision does the other thing is uh you know being able to recognize objects in your surroundings you know you could be outside the glasses will be able to tell you what object it is and where it is and how far it is uh you could use it to detect you know people's faces you could go ahead and use it to ask questions of the documents you just scanned uh you could use it to detect color currency it's about 15 different things you could do with Envision and uh the way people prefer to use Envision is Through The Envision glasses right so that's because you know people just wear the glasses on on their face you know if people watching the video can actually look at Vanessa she's actually wearing a pair of Envision glasses right now and uh the glasses basically have a camera on the right hand side which captures images around you and then there's artificial intelligence that literally runs on the glasses that help extract information from those images people take and then there is a speaker on board the glasses that speak out the information to the person right so it's basically converting the visual word into audio right and uh and and we can we work today with every type of visual information there is in the world be text objects faces currency color um and so on right so that's basically what Envision is and envision is also available as a smartphone app um and so for people who uh can't afford to use the glasses or or don't want to use the glasses they can always use the Envision app it's a completely free app that we built it doesn't have all the features of the Envision glasses uh but it has around 80% of it and it does a great job at at all the things I just described so these are the two ways that people use Envision and that's basically a brief description of what it does as well

Nora: I I have a question about how how does the the AI mitigate subjectivity and for instance like if I'm wearing the glasses and I'm looking at something like a drawing with a uh with a mauve circle how does it know that it's that color cuz like so like blue or purple that's a that's a a color that humans often like well that's blue actually no that's purple like how does how does the AI um uh distinguish yeah distinguish between things that humans often uh you know misinterpret or or it's up to the individual?

Karthik: Yeah no that's that's a great question actually so there are two parts to this right one is a very specific thing to Colors um you know colors are notoriously the you one of the most difficult things to crack in computer vision because colors like how humans perceive colors depends so much on lighting you know uh if you have an overhead lighting on a blue image it looks almost Mau and if you have something of you know from a different angle it looks purple and and so on right and uh just like how we humans get tripped up uh you know cameras and and and computer vision is no uh you know this one either so it basically doesn't get colors very easily uh unless it's on a digital screen so if you're sending it sending a screenshot of you know of a UI to Envision and ask it to describe it then it gets the colors right because there's no subjectivity there right it's just knows that this is it but if you're capturing an image from a camera then it does become tricky for for the for the classes to handle um and for that matter pretty much any tool out there that recognizes color just on the basis of visuals it gets very tricky to do that and on the other hand with subjectivity in other areas of computer vision for example let's say you're trying to scan a handwritten note and if it's uh and if you know God forbid you have to scan a handwriting like mine which is just outright horrible right sometimes you read the RS as P's and n's as you know M's and so on right so it doesn't get it and at that time what we do is we always give people the option to call a human being and you know get a get a more um I would say nuanced opinion on what actually is going on right um there is a very very nice feature in the glasses actually one of the most popular features is not the AI features it's actually a feature called call and Ally where you can actually make a video call directly from the glasses to a friend or a family member and it is precisely made for such situations where you don't know know whether the AI is actually leading you on the right direction or not or sometimes the AI just flat out says I can't help you with it or I'm not able to give you the right answer and people can bring a human you know the glasses user can bring a friend or a family member or a trusted person in the loop and get feedback get help immediately without having to like you know reach out for their phone and things like that right so these are the two ways we work with this

Nora: So Ally is basically a feature on the glasses that you press and it calls uh one person in your group your inner circle wow that's amazing so it's it's uh it's basically like a shortcut for support.

Karthik: Yeah it's like uh it's like a FaceTime on the glasses right that's how you can think of it you want to just uh you can add any person as an ally as a contact friend or a family member could be a colleague uh could be anybody and uh you know they're always available in your contact list and at any point in time if you're are stuck outside or get need some help uh you know from a human being you can just straight up you know hit the button and say hey I'd like to go ahead and talk to a friend or you know talk to this person or that person and the glasses just places a call directly uh as long as Internet is available the glasses can just place a call and uh people can actually ask questions.

Nora: And how does the AI read other humans so if I'm encountering Vanessa when she's wearing the glasses and I look forlorn or I look excited like how do is the AI going to interpret that or because that's also really subjective too.

Karthik: True you know uh that's also been one of the big holy Grails in computer vision is trying to better understand the nuances of human emotion and uh that's something that I would say has you know over the past couple of years at least there's been tremendous progress on that front uh you know if you asked me this question a couple of years ago I would have just told you flat out that an AI can never come close to describing the complexity of a human emotion but 2 years later uh you know just I I've seen the remarkable progress uh not just from the whole chat GPT side of things but also other you know um uh companies or other developers working in the space and putting stuff out in the open um we've come quite some you know quite some distance where AI does really well is in describing a person's you know physical characteristics like you know what color uh what color is their hair are they having a beard um you know what color their eyes and so on it gets the physical characteristics of it really right in terms of emotions you know um pronounced emotions are very easy for AI to capture right if you're really like giving me a nice white smile then you know AI can actually understand that oh this is actually you know like a nice white smile right whereas if it's a smirk or if it's something more subtle then AI has a very difficult time picking it up if from a visually impacted person's perspective right this is um very important important problem to solve because for a lot of people they're very interested in knowing what the other person is wearing and how they look like and you know um and so on right I think if it's especially someone who is uh who's a loved one I think they'd like to know uh on any given day what are they wearing how do their hair look like and try to get an idea of uh of of the person in front of them right trying to try to paint a picture with words and uh in on that front AI has made remarkable progress over the last couple of years and I and I can definitely say um in the coming years it's just going to get more and more better right I've seen some of the very early research in this area and it's super promising and  you know um we've also implemented a lot of that research uh wherever we we can possibly you know back it up with uh with proper data we've we've implemented that within the Envision glasses as well Alum 365 lets you hold the fastest design reviews ever share your designs from anywhere and with anyone with a single click it's easy leave a comment tagging your teammate and they'll instantly receive an email with a link to the design anyone you invite can open the design using a web browser using the browser interface you're able to comment mark up cross probe inspect and more comments are attached directly to the project making them viewable within alting designer as well as through the browser interface give it a try and get started with Alim 365 today

James: And Vanessa as someone who is low vision and uses the glasses how have they impacted your life?

Vanessa: Well I think I'm kind of lucky because I'm how do I put this it's all glass half full glass half empty depending on how you look at it I have around between 40 and 50% Vision in a in a lit room like this I'm I might be clumsy but I'm I'm okay and I walk outside and it's now evening I'm I'm useless I can't find anything so from me I still function pretty much you know in a visual way um I don't really use um new or or special assistive Technologies um apart from the accessibility features that are built in and I'm I'm pretty much a Mac User so all the zoom functions I've got a giant cursor if you could see my screen it's like huge um I zoom everything in and and um I I use that and for now that's good but in the future you know I'll probably use screeners and everything when I use the vision glasses I really do use them for when I get tired um with the ask and vision feature that we now have which is this uh GPT AI um powered piece which is super exciting I always call it a virtual visual assistant it's a bit of a mouthful but it really helps so you know my scenarios and user cases I think are very similar to other people's um go to a restaurant you got a huge menu card and I'm squinting at the thing and and looking at it and I don't really want to be be there for hours when everyone else has already decided so I'll use the glasses to um with scant text and then in scant text I'll ask it to summarize if there's anything there that's got you know whatever I'm feeling like you know just tell me if there's something there that's got beetroot in because I Happ to love that or if I feel like having some fish um or anything under sort of 20 20s whatever you are or any desserts or something drinks menu is really candy you know want R me out the cocktails that kind of thing um and then I use it at work as well um where I've got just a lot of documents to get through and although I'm in the UK today I'm normally based in the Netherlands so it's cartic normally um that's our base and I get tax bills through in Dutch and I speak Dutch but reading and writing it takes me so long so I'll use ask Invision to scan again I'll scan pages of my tax bill which is never a nice thing to do but what it will do is it will summarize that text for me it will translate it into English it will tell me how much I need to spend and what the deadline is and if there's any contact information so these new features for me are even more powerful because they're just time saving and they can actually and that's the thing of losing sight it just everything takes so much longer at least for me and you know I'm you know we're all busy but you know work full-time we've got a lot going on in EN vision and I don't want to waste time with those kind of things and and everything and the video calling the Ally calling is is cool I do use it um the glasses work like 20 Vision so where you know a sighted fully sighted person can function sometimes I can't and you know if I'm kind of getting stuck and getting a bit stressed I might make a video call uh from the glasses um and I think the other thing is kind of we've added clock orientation which is if if you could imagine just if you're not blind or low vision you close your eyes and someone says to you okay the bed is over there or the chair is over here behind you or yeah they don't say that people say to you it's over there or or they don't actually give you a clear orientation what you can now do is find the chair at 2:00 or find the dog at 3:00 or find your cane or what you know phones and things so it's just really practical time-saving stuff for me I'll be honest with you um and I I think they look quite cool as well.

James: Yeah I think so too.

Nora: Yeah I guess I'm wondering how this technology has like affected how your friends and family can be allies for you like has has anyone around you heard what the glasses how they're describing um visual uh things around you and been like oh that's how I should describe it to you or like does it does it kind of like offer um a template on how to best um yeah an ally or assist a low vision person?

Vanessa: It can but nothing beats real eyesight sure you know there's context there's information there I think that's all like using the tech it's also learning to give visual information to somebody who can't see it takes practice it takes um honest feedback you know so you know there's actually a a wonderful community of uh blinds and low vision influencers bloggers just like you know people who are cited or hard of he it doesn't matter but and there's there's a dark humor that goes with it right so you know we we we kind of and I kind of choose which side I want to be on the fence with depending on the joke but you know that thing of yeah you know where's where's where's the pubo it's over there you know it's like okay so tell me where over there so you know you ask a lot of questions and so it's like with ask inv Vision if you ask it a lot of questions it you'll get the information it may not be the first time but you'll get the information and it can surprise you as well you know um you know I where I live I always have a a view of where things are and what I'm doing and suddenly it will give me a lot more information which actually can be quite confronting because you think you're seeing your your your world as it is but actually realize you're actually missing out stuff and so personally that for me was also quite just something to get used to I guess um but again I'm not a good user case cuz I'm in the middle but for people who have no vision this is a lifechanger and and there is actually today a lovely lovely lovely customer of ours and he's a friend of Envision he's one of our beta testers called Wes he's based in Canada and he's on his way on a train right now to Toronto to do an interview for us he loves his glasses and he has told me he can sit on the train and he was really emotional because he was using describe scene to tell him what was around him and the colors and and the descriptions were so rich for him that he never had that level of information and he's he he just he couldn't he was really choked up it was really beautiful and and Wes is kind of like a really technical kind of guy he's quite you giv me the information but I even I could hear that this was some new level of information and and Independence was just what we've been working for really.

James: Wow that's great we're kind of over time but it's totally fine because uh but I just Karthik I know there was one topic you really wanted to talk about which was uh the role of AI in space of accessibilit.

Karthik: Yeah no I think um you know we've been talking about it more or less throughout the whole thing I think Envision is a very good example of AI having a huge impact in the accessibility space I think when we talk a lot about AI especially today um it's more about along the veins of oh it's going to be replacing humans it's uh it's going to be used for XY z uh industrial use case uh it's it's going to be used with drones it's going to be so there's a lot of topic around you know chatter around AI uh and it's it's positive effects overwhelmingly um on like you know areas like industrial uh uh applications and and medical applications and so on but I think the biggest I would say impact that AI has is in the area of accessibility right um just being able to say you know take uh a like take a piece of information and then pass it through an AI and then all of a sudden that that piece of information becomes accessible to someone you know who it it wasn't accessible to earlier right and I'm not I'm not just talking about the vision space uh there's huge advances that are happening in the in the areas of speech recognition for example uh which is really helping people who with site you know hearing loss uh deal with with the world more easily right so I think AI more than any other industry I would say AI has the biggest impact on accessibility and uh it's going to open up a whole new world uh for people with disabilities uh than what they were than what the world was you know I would say like 5 years ago right I think the next five years uh we're going to see this tremendous tremendous explosion of of productivity of of General well-being improvements uh for people with disabilities because of AI that uh today I think the general population doesn't really see that much right because so much of the headlines is around Sensational news um but I think AI is going to become sort of the the the I would say probably the most important invention uh for people in you know with disabilities right um and for people in the accessibility space or uh people working in assist of Technology uh AI is going to probably be like the dawn of electricity right that's what I I truly believe in and uh that's also the future that we are working towards at with Envision right to to be that force uh majure in the whole you know for people with visual disabilities and and I know there are companies that are also working in along the similar lines uh for people with other types of disabilities like hearing laws and so on.

Nora: Yeah I also love that just you know these conversations around technology are bringing um issues about um accessibility uh into more into the discourse like into the into the public sphere because you know they're topics that should be talked about by everyone not just specific people that um you know it affects.

Karthik: Yeah yeah definitely and it's um and you know there have been many areas of of you know software uh you know and the internet in the last 10 15 years though there have been some improvements made uh with accessibility and making digital you know goods and and products more accessible I think we still have a long way to go and again there AI is going to play a huge role because all of a sudden you know with uh with AI you don't have to deal with you know pesky menu items and buttons and and clicking and tapping and so on uh you can have a normal you know a natural conversation with it just like how you uh and I are having one right now and uh again that will get rid of so many layers of complexity and crft for people with with any kind of disability that uh it's just going to be a world that's unimaginable uh you know like 5 years later people are going to be like wait how did we actually you know have why did we have to put up with all that stuff right I mean how did we even put up with that stuff so that's what I think uh AI is going to be and and I think we're just grateful to play a part in it.

James: Last question for you before we wrap things up uh do you want to just tell people about the different tears that you offer with the glasses this uh the different plans I guess or models.

Karthik: Yeah sure so the Envision glasses um you know have three different editions that's what we'd like to call it uh the first is the read Edition right uh and the read Edition you know retails at uh $1,899 and what it offers is basically offers the functionality of The Envision classes so you're going to be able to be read short pieces of text completely offline you can go ahead and scan documents with it or you can also scan like entire books with it right so you know you can scan multi-page documents or books with the glasses so the read functionality is there for anybody who just wants to use the glasses primarily as a reading tool then there is the home edition of the glasses right um which basically offers people all the functionalities of The Envision glasses right uh and there is the and that retails for $24.99 and then there is the professional edition of the glasses which offers all the functionality of the read Edition sorry the home edition of the glasses right so that's about 12 to 15 different functionalities uh and so on and it also offers an extended warranty for the classes so apart from just a one year warranty you have a two-year warranty uh on the glasses and we also offer a choice of frames so um you know people can go ahead and add their own lenses uh to the to the glasses if they want to and that retails at $3,499 now one key difference again with all these three editions uh more more so with the other two versus the professional Edition is the we offer updates uh to the glasses for the first year in the read and the home Edition for free right so after the first year people can go ahead and choose to receive updates on the classes I mean I I have to also mention one thing here is that people can still use the glasses even if they choose not to pay for updates what people are paying for is just for the new features so all the features that they have you know been using so far they still get to keep it they still get to use it regardless of whether they they pay uh you know a fe a fee for updates or not so people can purchase these you know yearly updates um you know we call them passes and uh that is what we offer in the read and the home edition where after the first year people pay a yearly you know uh fee to get access new updates and in the case of the professional Edition we offer the updates free for a lifetime so people don't have to pay anything additional uh you know with the professional addition it's just a onetime cost people pay that onetime cost they get the glasses they get it extended warranty period they get uh access to the frames and they also get access to a lifetime you know uh updates uh along with the unit so these are the three additions of the glasses that we have and like I mentioned apart from the glasses we also have the Envision app which is available on both IOS and Android and people can just you know uh go ahead and install that for free um on an iPad or an iPhone or an Android device or an Android tablet and so on and uh early next year we're also going to be pushing out a version of Envision on the desktop as well so people could use it on their browsers uh or their desktop if they choose to please.

Vanessa: I have to add I have one more just one more thing to add we actually launched earlier it's in Tri it's a subscription to the Envision glasses so not the software but the actual whole thing for $150 per month and that's actually in the US only at this stage so we're trying that and actually it's been quite successful for people who may not want to make the big cash outlay up front and and just sort of pay for a few months um but also you know we do do this this sort of 30-day trial by and if you don't like it you can send it back because you know not all technology suits everybody and and you've got to sort of are these things um so it's just that other subscription Edition which is um also available in the US only wow really cool.

James: So just to to wrap things up if people want to check out your product or stay up to date with what you're doing um what are the best places to do that.

Vanessa: Yeah check out our website it's let envision.com so that's L E T S E N V I S IO N.com um and then there's a lot of it's totally of course an accessible webs it lots of information but get on to any social media platform and look for um our handle there at let's en vision and we're we try to be active there and we do a monthly webinar as well which is kind of cool we Market that everywhere um and that's meant to be sort of more than just a sort of product update even though we go under the hood with Karthik and he always gets a sneak peek into what's coming down the down the pipe but we also have some nice updates and and guests and things like that on there so I think out of all that um yeah those are probably the easiest options.

James: Great and social media?

Vanessa: Yes social media so we're on uh Facebook Twitter Instagram LinkedIn and we with our own channels but we also and YouTube um but we also have a community uh on those platforms as well as the groups.io and those are all under let's Envision or Envision AI.

James: Great amazing well thank you so much for taking the time to talk to us it's been absolutely fascinating.