Blind Pedestrians and AI Autonomous Cars

Anticipating blind pedestrians is an edge problem for AI self-driving cars.

By Lance Eliot, the AI Trends Insider

I was driving along Ocean Boulevard and Pacific Coast Highway (PcH) in Malibu and Santa Monica, California when I came to a stop at a crosswalk that had no traffic signal. It was dicey that this major crosswalk has no traffic signal since it is commonly used by pedestrians that are trying to cross from the “inland” side of the street over to the ocean side of the street (putting them nearly onto the sand of the beach).

 During the summer months it is a continual herd of pedestrians trying to make use of the crosswalk. Usually, it takes at least one or more daring souls to start the herd into crossing since the cars are zooming along and generally pretending to ignore the crosswalk and won’t acknowledge the people that are wanting to make use of it (the pedestrians tend to bunch-up on the curb and when there is a sufficient number or when they are exasperated at waiting, they then in solidarity march into the crosswalk to get across the street).

 I bring up this particular occasion because of something that caught my attention.

 In the pack of pedestrians there was an elderly man with a white cane. He was moving the cane back-and-forth and tapping it on the ground. By glancing at his face, I’d guess that he was blind (though he hard dark and heavy sunglasses on, and I couldn’t directly see his eyes), and he was opting to head over to the beach with the other pedestrians. Using this crosswalk would seem especially dangerous for him since it was not controlled by a traffic signal. He would pretty much need to rely on the judgment of the other pedestrians to know when to go ahead and enter into the crosswalk.

 If he entered into the crosswalk on his own, I have my doubts whether the crazy drivers would honor the aspect that he was blind. It would seem self-evident by the use of his cane that he presumably was blind, but I am just saying that the SoCal drivers are so focused on themselves that they probably either would not notice that he was blind or (worse still) not care. I can imagine some drivers that would have kept going and possibly even cursed at him for boldly using the crosswalk on his own. His best bet would be to hope that the other bunched-up pedestrians made a wise choice to cross, and he would cross along with them.

 By the way, I was tempted to call this the story of “The Old Blind Man and the Sea.” Or, since I was watching him, maybe “The Old Blind Man and the See.” 

My humblest apologies to Ernest Hemingway.

The man was moving relatively slowly and generally was in the middle of the pact of pedestrians that were crossing the street. The flow of the pedestrians was faster than his walking speed and so many of the pedestrians were moving around him to proceed forward. They gave him some extra space inside the pack, especially since his cane was being aimed a foot or two beyond his width, alternating from left to right of his body.

 It was probably handy that there were a lot of pedestrians crossing since otherwise I’d have gauged that he would have ended-up maybe half-way across and the rest of the pedestrians would have completed crossing by then. A large enough swell of pedestrians was sufficient to cover him throughout his crossing. If there were just a few pedestrians, they would have helped to likely curtail traffic for the first part of his crossing, but then he’d have been on his own. One can only hope or assume that if traffic had come to a halt to let the bulk of the pack across, they would have the patience to let him finish crossing (again, it’s a toss-up here if that’s what the drivers would actually do).

 I’m sure you are wondering whether or not California has a driving regulation that says you are supposed to stop and let a blind pedestrian cross the street. Yes, we do. I’m sure you then are thinking, well, if it’s the law, shouldn’t this blind man have no qualms about crossing the street in the crosswalk, whenever he wishes, whether alone or with others? I certainly agree that the theory is that the drivers here would abide by the law, but I can tell you that the drivers here aren’t necessarily that law abiding to begin with (they readily driver over speed limits, they drive excessively fast in school zones, etc.).

 I’d earlier in my career had a chance to get a glimpse, as it were, about the world as perceived by a blind person. 

When I was a university professor, I taught a class on software engineering that involved a lot of programming. On the first day of one of my classes in the Fall semester, a student came up to me before class started and explained to me that he was legally blind. He said that he was telling me not because he expected any special treatment but due to giving me a heads-up about his status.

He indicated that I did not need to do anything other than what I normally do when teaching my class. I asked him if it would help that when I wrote on the whiteboard that I perhaps carefully say out loud what I am writing. He said this wasn’t needed because he had a volunteer aid that went with him to his classes and sat next to him to let him know what was being written on the board. He also told me that for the exams, since I was likely to handout written exams, he would have the same aid there to tell him the questions. The aid would not interact with him about the answers and only tell him what the questions were. If I had any concerns about the exams and whether the aid might help him “cheat” in some manner, he said that there were official and trusted TA’s (Teaching Assistants) that were ready to do the same as his “interpreter” for the written exams.

 I was excited to have him take the class and it got me curious as to how he would do. Could he do the intense programming that was required for the course? I tried to imagine that my eyes were unavailable and could not fathom how I could do such programming. It seemed to me that he would have to somehow keep in his mind the various large passages of code that were involved and try to run through the lines of code purely mentally. I normally look at my computer screen and scroll back-and-forth at the code, using my eyes to refresh my mind and try to punch out the code.

 Anyway, he did a wonderful job in the class. He spoke-up as other students did. He got the programming work done. He did quite well on the exams. I would not have known he was blind in terms of his classroom efforts, other than his having let me know before the class got underway. I did discover that his mental model of the programs was especially impressive, and I believe was more extensive than most of the other students in the class. I wasn’t sure whether this was because he happened to have a greater mental capacity for it, or whether by necessity he had to keep it in his mind in this manner.

 Legally Blind Aspects

You might find of interest that there are an estimated 1-2 million legally blind people in the United States, and perhaps 7-8 million people all-told that have some kind of “visual disability” that renders them relatively blind (some prefer the phrase “visually impaired”). There is a somewhat strict definition for the phrase “legally blind” in that it means you need to have 20/200 or less vision. The states that have the most people with a visual disability or impairment include California, Texas, New York, and Florida.

There is also a large segment that is over the age of 65. 

This would seem logical in that as you get older the chances of having eye or vision issues is generally more likely than when you are younger. The elderly blind man that was using the crosswalk was somewhat “typical” in that part of Los Angeles as there are a significant number of retirees living in that area of town.

I had mentioned earlier that I was doubtful that many drivers here would have even noticed that he was blind. I’ll call these in-observant drivers. 

Drivers And Blind Pedestrians

The in-observant drivers aren’t watching to see whether someone is blind. When I’ve asked such drivers why they don’t pay attention to this, they often tell me that they don’t want to be discriminatory towards someone that is blind, so why not just forget about looking for blind pedestrians and treat all pedestrians the same way.

I suppose you can somewhat understand their logic. But, at the same time, it seems to me that it belies the point that the driving code specially cautions that drivers should be extra careful when driving near to someone that is blind. It’s the law. I don’t have much belief that these in-observant drivers are really so concerned about being non-discriminatory and that instead it is a laziness and lack of care for others that really motivates their lack of attention.

 According to the driving code, one aspect that in-observant drivers often do, and which can make life especially harder for a blind pedestrian, involves coming to a stop part-way into the crosswalk. You’ve likely done the same, whereby you meant to come to a stop before you reached the crosswalk line but didn’t well-estimate the braking needed and so ended-up slightly protruding into the crosswalk. I’m not saying you’ve done this when a blind person was using the crosswalk per se, and only suggesting that it is an easy enough driving action to do.

 People that are sighted can presumably readily see that your car is protruding into the crosswalk and walk around it. This can be troubling at times for even sighted pedestrians, particularly when there is a large number and it isn’t easy for them to flow around the front of your car that is intruding into their crosswalk walking space. I’ve seen many drivers that realized they made a faux pas by entering into the crosswalk and so they try to back-up. Unfortunately, this then can create troubles for the car immediately behind them, and you suddenly have several cars all trying in a daisy chain way of backing up while having already come to a stop at a traffic signal or crosswalk (backing up because the driver at the front of the pack goofed-up).

 A blind person might not so easily detect that a car is protruding into the crosswalk. They are right to assume that the crosswalk is supposed to be unencumbered. I’m sure that anyone that is blind and has used the crosswalks here would know over time that there is going to be a relatively high frequency of instances involving cars that have not stopped properly in-advance of the crosswalk. I was thinking that part of the reason the blind man was moving his cane back-and-forth somewhat widely was perhaps due to his wanting to detect that kind of circumstance.

 Besides the in-observant drivers, there are also drivers that try maybe too hard to help a blind pedestrian. These eager beavers want to make sure that the blind person has ample room and also try to protect them from other drivers. It’s a nice sentiment, but sometimes goes over-the-top.

 For example, I’ve seen some drivers that opted to honk their horn to warn others that a blind person was crossing the street. The logic seems to be that by honking their horn it gets those inobservant drivers to become observant. I believe it is also assumed by the horn honking driver that it will let the blind person know that they are being acknowledged.

 My understanding is that the horn honker is generally being misguided by their attempts to be thoughtful. The honking of the horn will likely startle the blind person and they might not have any clue as to why you are honking your horn. Furthermore, if the blind person is actively making use of their listening skills, doing so to try and make-up somewhat for the blindness, it is conceivable that the horn honking might be especially startling to their open ears. I also wonder whether other drivers will comprehend why the horn has been honked. Some might think that the horn honker is being rude and maybe it could spark some kind of road rage.

 For my article about road rage, see:

 For my article about pedestrians, see:

 For my article about defensive driving, see:

 I’ve seen some drivers that get locked into a kind of stalemate chess move with a blind pedestrian. The driver decides to come to a stop, doing so several feet in-advance of the crosswalk. The blind person does not necessarily at first know that the car is even stopped, since it is so far from the crosswalk. The driver, though well intentioned, perhaps starts to get impatient that the blind person isn’t readily taking the opportunity to cross, and so the driver starts to creep forward. But, the blind person that has by then detected that there might be a car there, now becomes bewildered by what the driver is trying to do. They then each wait for the other to make the next move.

 Being a pedestrian already have a number of challenges. Try to imagine that you are blind and look around you the next time you go for a walk where there is traffic. It can become quite apparent as to how tricky and scary it could be to deal with the nutty drivers that don’t care about blind pedestrians and likely too don’t especially care about pedestrians overall. As mentioned, even drivers that do care about pedestrians can go over-board and create situations that though well-intended can end-up being confusing and potentially dangerous.

 AI Autonomous Cars And Blind Pedestrians

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. One aspect of robust AI software for driving is that it considers how to deal with pedestrians, including the case of pedestrians that are blind.

 Allow me to elaborate.

 I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the auto makers are even removing the gas pedal, brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

 For self-driving cars less than a Level 5, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

 For my overall framework about AI self-driving cars, see my article:

 For the levels of self-driving cars, see my article:

 For why AI Level 5 self-driving cars are like a moonshot, see my article:

 For the dangers of co-sharing the driving task, see my article:

 Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

 Here’s the usual steps involved in the AI driving task:

  •         Sensor data collection and interpretation
  •         Sensor fusion
  •         Virtual world model updating
  •         AI action planning
  •         Car controls command issuance

 Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too. There are some pundits of AI self-driving cars that continually refer to a utopian world in which there are only AI self-driving cars on the public roads. Currently there are about 250+ million conventional cars in the United States alone, and those cars are not going to magically disappear or become true Level 5 AI self-driving cars overnight.

 Indeed, the use of human driven cars will last for many years, likely many decades, and the advent of AI self-driving cars will occur while there are still human driven cars on the roads. This is a crucial point since this means that the AI of self-driving cars needs to be able to contend with not just other AI self-driving cars, but also contend with human driven cars. It is easy to envision a simplistic and rather unrealistic world in which all AI self-driving cars are politely interacting with each other and being civil about roadway interactions. That’s not what is going to be happening for the foreseeable future. AI self-driving cars and human driven cars will need to be able to cope with each other.

 For my article about the grand convergence that has led us to this moment in time, see:

 See my article about the ethical dilemmas facing AI self-driving cars:

 For potential regulations about AI self-driving cars, see my article:

 For my predictions about AI self-driving cars for the 2020s, 2030s, and 2040s, see my article:

 Returning to the topic of blind pedestrians, most of the auto makers and tech firms would tend to say that this is an edge problem. An edge problem is one that is considered at the edge or corner of what you are otherwise trying to solve. You tend to delay dealing with edge problems. In the case of the auto makers and tech firms, they are primarily focused on getting the AI to drive a self-driving car such that it does the usual kind of things like drive properly and not hit people, but this often does not include the trickier situations that might be considered rare in their book.

 Admittedly, the odds of an AI self-driving car coming upon a blind pedestrian is going to be overall somewhat rare, since it all depends on where the AI self-driving car is actually driving. If an AI self-driving car is being used in a state that has almost no blind people, the odds are that the AI is not going to encounter a blind pedestrian. If the AI self-driving car is driving in a state that has an abundance of blind people, but the AI is not driving in areas for which blind people are more likely to be found, once again the AI might not have much chance of encountering a blind pedestrian.

 In that sense, the auto maker or tech firm would argue that worrying about driving aspects for coping with blind pedestrians is low on the priority list.

 They would also tend to argue that if their AI self-driving car is already programmed to deal with pedestrians in general, there is really not a need to do anything special regarding blind pedestrians. In the minds of those AI developers, they assume that if the AI driving legally, such as stopping properly in-advance of a crosswalk, it is sufficient as a means of handling any situations involving a blind pedestrian.

 I consider such AI developers to be rather myopic in their views. I think that they perhaps cannot put themselves into the shoes of the blind pedestrian to understand how the aspects of the AI can potentially help or hinder a blind pedestrian (I refer to such AI developers as being egocentric in their designs).

 For my article about edge problems, see:

 For my article about egocentric AI developers, see:

 For how AI developers might reach a point of tossing in the towel and not devote attention to special cases, see my article:

 For my article about how groupthink can lead AI developers down a bad path, see:

 Right-of-way And Blind Pedestrians

As per the driving code here in California, a car is supposed to give the right-of-way to a pedestrian that is blind. 

How can you or the AI figure out that someone might be blind?

As a human, you would look to see if the person had a white cane, and/or possibly a guide dog. You might also do a kind of facial recognition, trying to look at the face and eyes of the person, if possible. You might also observe how the person walks and moves. These are all tell-tale clues about whether the pedestrian might be blind.

For today’s sensory systems on AI self-driving cars, by-and-large the depth of analysis of pedestrians is relatively shallow. Pretty much the goal of most AI systems of today is to just determine that an “object” is a pedestrian versus being say a fire hydrant or street post. The pedestrian is treated internally in the AI system as a kind of stick figure. The stick figure is at this position on the street, so many feet away from the self-driving car. The stick figure is moving and going toward the AI self-driving car or moving away. And so on.

 Trying to figure out if the pedestrian has a cane is not included in many of today’s AI sensory data interpretation routines. It’s a hard thing to spot. It is thin and might be hidden by being next to a person’s body, thus the body of the person and the cane seem to be one blob. Once the cane starts moving, it becomes perhaps even harder to detect due to the often-rapid motion of moving back-and-forth and while it is in close proximity to the person’s body.

 Many of the sensory systems don’t even try to computationally pick apart a group of pedestrians, such as a pack that is standing at the curb or walking across the street in a crosswalk. It would take a lot of processing cycles to figure out each person in the pack, including what is a person versus not a person, which arms and legs go with which person, their direction and pace, and the rest of those separation aspects is all computationally hard to calculate in quick time.

 A human driver can usually glance at a scene and be able to recognize that there are say five people in a pack of people, including there are two women, two men, and a small boy. You can likely gauge their ages and something about them by how they look, how they dress, how they carry themselves, and so on. You likely are able to notice readily where they are looking, and you can often do a reasonably good job of predicting what they might do next. A pedestrian that is hunched forward and appears eager to cross the street is more likely to get onto your watchlist, meaning that you are anticipating they might jump the gun on the crosswalk crossing and go sooner than the others.

 Many AI developers of self-driving cars would argue that this kind of processing is just not in the cards right now. They would wave their arms and say that it is arduous to develop AI to be able to deal with this kind of sensory data. The sensory data itself is not pristine and so you need to contend with lots of noise and partial images or other kind of data issues. Many would say that just let the Machine Learning (ML) deal with this, but that’s not yet a silver bullet on these matters either.

 For idealism about AI self-driving cars, see my article:

 For humans helping to teach AI self-driving cars via Machine Learning aspects, see my article:

 For learning and robomimicry, see my article:

 For more about Machine Learning and AI self-driving cars, see my article:

 Based on the toughness of being able to discern a blind pedestrian, and along with the presumed rarity, there isn’t much effort going toward trying to craft specific capabilities for this purpose. This means that blind pedestrians need to do their best to be no different than other pedestrians, which, places the burden somewhat more so on their shoulders, when in fact we should be hoping for AI self-driving cars that can do a better job than human drivers in coping with blind pedestrians.

 That’s why we are working on AI components for this purpose.

 One might even predict that there could be more difficulties about AI self-driving cars and blind pedestrians since it is anticipated that most AI self-driving cars are more likely to be Electrical Vehicles (EV’s) rather than gasoline powered cars. This makes a difference in that an EV is often much quieter than a gasoline powered engine. You’ve probably had an EV sneak-up on you and got surprised that the car got so close to you without you realizing it.

 Imagine EV’s that come up to a crosswalk and come to a halt. With a gasoline powered engine, you have a likely greater chance of detecting where it is, doing so by listening for the motor. If you are blind, and you’ve honed your listening skills, you can likely pluck out of the cacophony of street noises the sound of an engine that is a few feet away from you. If AI self-driving cars are mainly EV’s, it will mean that for blind pedestrians they have one less lowered chance of detecting the car than if it were a nosier gasoline powered car.

 Does this imply that an AI self-driving car should do things like honk its horn to alert that a blind pedestrian is nearby? I don’t think we want AI self-driving cars to do what some humans do about blind pedestrians, even if those humans think they are doing the right thing. No, honking is not likely a good idea.

 For some EV’s, the auto makers are generally adding features to be able to purposely make a small noise such as hum or chime to let people know that the EV is nearby, which is being done generally to let pedestrians know of the presence of an EV. That feature would likely be helpful too for the case of blind pedestrians.

 For more about EV’s, see my article:

 For my article about AI self-driving car conspicuity, see:

 For aspects about ridesharing and AI self-driving cars, see my article:

 For human driving foibles to be avoided, see my article:

 Ridesharing Might Increase The Chances Of Encountering Blind Pedestrians

One aspect for AI self-driving cars that might increase the chances of possibly getting near to or somehow entangled with blind pedestrians involves the advent of ridesharing via AI self-driving cars. It is widely predicted that AI self-driving cars will be extensively used for ridesharing. Imagine that you can at any time of the day and any day of the week be able to readily get an AI self-driving car to come and pick you up. No need to deal with human drivers.

This also means that those AI self-driving cars are going to be pulling up to the curb in all kinds of varying situations. 

Today, we seem to have human drivers that are ridesharing drivers that will stop anyplace at all to pick-up a passenger. Will AI self-driving cars do the same? I ask partially because it can be startling as a pedestrian that all of a sudden, a car is hugging the curb just inches from you as you are walking along on the sidewalk.

This brings up too a different topic that relates to blind pedestrians, namely if a blind pedestrian wants to take a ride in an AI self-driving car, will the self-driving car and the AI be able to accommodate this kind of pedestrian? 

You would hope so.

The AI will need to ensure that it converses with the passenger in a sufficient manner to keep them informed about the driving journey, perhaps more so than a sighted person. This can get tricky too as to where to drop-off a passenger, since a sighted person might readily look out the window and tell the AI that the spot chosen is not a good one, but a blind person is unlikely to be able to make that same kind of judgement due to lack of sight.

 For the responsibility aspects of AI self-driving cars, see my article:

For AI self-driving cars operating non-stop 24×7, see my article:

For conversing with an AI self-driving car to give driving commands, see my article:

 For the socio-behavioral aspects of humans instructing AI self-driving cars, see my article:

 I’ve not yet mentioned the role of guide dogs in this matter. 

First, similar to the limitations of today’s sensory systems to identify pedestrians, the capability of AI for identifying dogs and particularly guide dogs is very primitive and not at all akin to what a human driver might be able to discern. You might be surprised to know that there are actually aren’t that many official guide dogs in the United States, perhaps 10,000 or so, and thus the odds are that a blind pedestrian will probably not have a guide dog and would be using a cane alone instead.

In any case, we’ll eventually want the AI to be able to discern dogs and other pets that are on the sidewalks and in our roadways, along with being able to figure out whether the animal has a particular purpose. In the case of the blind pedestrian, the presence of a guide dog could further aid the AI in ascertaining that the pedestrian might be blind (of course, it could be a sighted person training the guide dog, etc.).

 Another factor of the AI involves it contending with other drivers that have detected a blind pedestrian. I mentioned before that some human drivers will go to extraordinary lengths to deal with a blind pedestrian. For example, suppose a human driver suddenly decides to back-up, because they intruded into a crosswalk. If there is an AI self-driving car directly behind that car, the AI needs to also back-up, if feasible. As also mentioned, there will still be lots of human drivers on our roadways during the time that AI self-driving cars are emerging, and so this is another aspect that the AI needs to deal with about human drivers.

 Currently, many of the auto makers and tech firms are treating the aspects of what is behind the self-driving car as relatively unimportant. The number and types of sensors at the back of a self-driving car are rather limited and slim. The amount of processing devoted to analyzing what is behind the AI self-driving car is also relatively slim.

 I mention this aspect because when an AI self-driving car has to from time-to-time backup, it could be that a blind pedestrian has made their way around the back of the AI self-driving car. The other day, when a car that had intruded into a crosswalk opted to back-up, the pedestrian decided to go behind the car, which as you might guess was a bit of an issue since you now had a pedestrian that had decided to go around the car, but the human driver had decided to back-up so as to presumably not have the pedestrian need to go around the car. It was nearly an accident that occurred in front of my eyes.

 AI self-driving cars also need to be able to make right turns with great care, in case a pedestrian opts to step off a curb early or might be standing just off the curb. Likewise, making left turns needs to be done with care. A pedestrian that is perhaps in a crosswalk that involves where the self-driving car is trying to make a left turn might be unaware that the car is making a left turn. The AI needs to be looking for pedestrians that are in crosswalk and maybe taking extra time to get through it.

 For my article about AI self-driving cars looking behind themselves, see:

 For more about the danger of left turns, see my article:


Blind pedestrians are already considered by the driving code and laws to be warranted for special treatment by drivers. 

It is incumbent upon the auto makers and tech firms to make sure that AI self-driving cars not just treat blind pedestrians as any kind of pedestrian but include extra capabilities and care for detecting and ensuring that blind pedestrians do not inadvertently get hit or run over by the AI self-driving car. 

Let’s not turn a blind eye to that important challenge.

Copyright 2019 Dr. Lance Eliot 

This content is originally posted on AI Trends.

This UrIoTNews article is syndicated fromAITrends