Traffic Lights And AI Autonomous Cars

Once AI self-driving cars become more prevalent, it will be a life or death matter as to whether they can handle traffic signals. Credit: Getty Images

By Lance Eliot, the AI Trends Insider

Quick quiz for you. 

There are approximately 300,000 of these in the United States alone. 

You see them every day. 

You notice them, but you don’t notice them. 

They save lives, but are generally unheralded for what they do. Sometimes you curse them. Sometimes you thank them. 

Can you guess what I am referring to?

Answer: Traffic lights.

Yes, as human drivers you are surrounded by traffic lights.

Most major intersections have them. You watch for that green light so that you can zoom through the intersection. When you see a yellow light, you need to judge whether to hit the brakes before the light turns red, or maybe hit the accelerator to try and make it before the light turns red. It’s a game.

It’s actually a deadly game. 

There are an estimated 700 deaths per year due to red-light running crashes. That’s sad and preventable.

Equally horrific, there are an estimated 126,000 injuries due to red-light running. That’s also sad and preventable. 

The next time you see some nut rush through a red light, I hope you’ll realize the crazy danger involved and perhaps adjust your own driving to no longer try to rush through a traffic signal.

Considering The Nature Of Traffic Lights

When you think about it for a moment, you’ll realize that traffic signals are kind of a scary mechanism since they don’t do anything to actually physically stop anyone from driving crazily. 

It is all a voluntary system. 

It is our collective belief in the traffic signal that makes it real. The traffic signal does not toss down a huge net to stop cars that disobey the light. It does not spring forth spikes from the roadway to stop cars that are going to potentially ram into another car. Instead, it is just some lights that we as humans have generally agreed to abide by.

Imagine though the chaos without the traffic signals. 

We would either all be having continual near misses, or the intersections would need to have stop signs or some other control mechanism, which would likely stall traffic and make our driving times longer. You could put traffic officers at intersections, which is the way things used to be, prior to the advent of traffic signals. A traffic officer would stand in the intersection and direct traffic. At one point, New York City had something like 6,000 traffic officers to direct traffic. The traffic signal put a lot of those traffic officers out of the job of directing traffic, though they were still needed for various other traffic related duties.

When traffic signals first began, they were often a sign board that said the word Go and the word Stop. 

The Go signboard would pop-up, and after a bit would be pushed down. The Stop word would pop-up and then after a bit be pushed down. Human operators at first stood nearby and controlled these signboards. Eventually, it was quasi-automated. Traffic signals were at first only with two lights, one that said Go and one that said Stop, or had the red light and green light similarly. Ultimately it became clear that human drivers would not judge things well when there were only two modes, and thus a third light was born, a yellow light, which helped to reduce the frequency of intersection crashes. The yellow light became a handy warning to drivers that the red light would be coming soon.

Since the red light is perhaps the most important of the three lights (I don’t want to get into a debate about that here, but I think you generally agree that the red light is quite important!), it is normally placed at the top of the three lights that are on a traffic signal. Next is the yellow light, at the middle, and logically in sequence of what then comes next, namely the green light at the bottom. 

Not all countries do things this way. But, it is pretty common and makes sense that you would want them to at least have the traffic signal in a Red-Yellow-Green or a Green-Yellow-Red sequence. A Yellow-Red-Green or a Yellow-Green-Red would seem counterintuitive and likely confusing, even if we all agreed to it.

How long does each color light get lit? 

I realize that if you’ve ever sat at traffic signal for what seemed like an inordinately long time, you might claim that the red light seems to be at times glowing for an hour or more. 

Well, the reality is that many traffic signals are established with pre-timed intervals but usually for a duration of just some set number of seconds (not hours). The green goes for X number of seconds, the yellow goes for Y number of seconds, and the red goes for Z number of seconds. This is the easiest way to “program” a traffic signal. What should the values of X, Y, and Z be?

For yellow lights, there is a rule-of-thumb that many use. The rule-of-thumb is that the yellow light should be set for 1 second for each 10 miles per hour of the posted speed nearby. Thus, if the posted speed is 40 miles per hour, the yellow light should last for 4 seconds. It doesn’t have to be set for that interval of time, and some locations use something like 3 seconds to maybe 6 seconds for all of their yellow lights (rather than having to figure out per each intersection what the nearby posted speed is).

 The setting of the time for the green light and the red light is trickier. 

If you arbitrarily pick a number of seconds, it could be that it hampers traffic. A street leading into the intersection that has a lot of traffic should presumably have a longer green light, so as to allow more traffic to get through the intersection. Sometimes you get to a traffic signal that is “dumb” and it seems to allow the same amount of time for both the red light and the green light in each direction, but this bottles up the roads leading in that have a lot of traffic, while it infuriates those drivers too when the light is green for the mildly traveled road that has no traffic streaming through the intersection.

Traffic engineers will sometimes study traffic patterns and then advise that a traffic signal should be setup in somewhat savvier timings. This can help with the flow of traffic through the intersection. The problem here though is that for most “dumb” traffic signals they can only set one value for X, one value for Y, and one value for Z. This then means that regardless of the time of day, it is going to stay with the same interval times. This explains why at midnight you sometimes sit at an intersection when there is no other traffic around. It has a pre-timed setting that does not vary.

 At some intersections, they put pressure plates in the street to try and detect car traffic. The pressure plate is activated by the weight of the car, and the plate then informs the signal that a car is waiting. Rather than a pressure plate, it can be a magnetic detecting plate that is activated by the metal in your car. In any case, the point is that the traffic signal is coordinated with something that tries to detect traffic. For the plates, it is mainly that a car is sitting still and waiting. The plates aren’t usually tracking traffic per se.

 In the early days of traffic signals, the traffic signal would ring a bell to indicate when the light was either changing colors or when it was red. Today, very few traffic signals make any such sounds. We are all inside our car cocoons and it would seem unlikely that having a sound would help us particularly, and it would be a likely irritant to those that live or work near the traffic signal.

 AI Autonomous Cars And Traffic Lights Aspects

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars and consider the traffic signal aspects as a crucial capability for self-driving cars.

 Some automakers and tech firms consider the advanced traffic signal problem to be an “edge” problem in that it is not core per se of the act of driving a car. Allow me to qualify that point. Certainly we all agree that being able to detect a traffic signal is at the core of the driving task. But, this can be done in a rather simplistic manner, or it can be done in a more advanced manner. For some, once the simplistic version has been figured out, they move on to other self-driving car capabilities and consider the traffic signal problem entirely solved.

 Let’s take a deeper dive into the topic of traffic signals.

 First, how do you know that a traffic signal exists?

 I am sure you are thinking that my question seems kind of silly and the answer is obvious. You look out your windshield and you see the traffic signal. Duh. 

Well, as humans, we have an incredible ability of our eyeballs and our ability to see. We can look at a scene and find the gorilla that’s hidden over there behind the stack of boxes. Similarly, we can look at the street scene up ahead and “know” where the traffic signal is.

The sensors on the AI self-driving car have to put in a bit more work. 

The cameras capture images of what’s ahead of the self-driving car. The image needs to be analyzed by the system. The image might contain not only a traffic signal, but perhaps there’s a plane flying through the sky that can be seen behind the traffic signal, maybe there’s a few birds resting on the traffic signal, maybe there’s rain coming down and the traffic signal is partially obscured by the heavy rain. And so on.

It’s not such an easy thing to find the traffic signal in a picture. Yes, the traffic signal is likely the same kind of shape nearly all of the time. It’s usually on a post of some kind. It’s got the three lights. It’s hanging over the intersection. These are all valuable clues. I’m not saying it is rocket science per se to find the traffic signal, but it is more work than you think.

 So, the first step involves capturing images via the sensors and analyzing those images to find the traffic signal. You want to find the traffic signal and also not be fooled by something that might resemble a traffic signal. There could be other nearby lights such as for a lit-up billboard or maybe lights on the exterior of a building. Those might be red lights, yellow lights, green lights, and so you cannot just look for a particular color of a light.

 You also might be faced with the circumstance of an intersection that does not have a traffic light. If the image analysis says that it cannot find a traffic signal, does this mean for sure that there isn’t one there? Maybe yes, maybe no. It could be that the image analyzer capability could not find the traffic signal. If the sensor analysis reports to the sensor fusion that there isn’t a traffic signal, and if turns out there is one there, the result could be catastrophic.

 Let’s consider the stages of the AI self-driving car processing:

  •         Sensor data collection and analysis
  •         Sensor fusion
  •         Virtual world model updating
  •         AI action plan preparation
  •         Car commands control issuance

 See my framework for further info at:

 For AI self-driving car defensive driving capabilities see my article:

 What Happens During Detection

Let’s pretend then that the visual sensor and analysis did not find a traffic signal, but that the traffic signal does exist. The sensor feeds its results into the sensor fusion. The sensor fusion compares each of the sensory device indications to try and triangulate and make sure that no one sensor is misleading or maybe has failed to do its job.

Would the radar on-board of the AI self-driving car detect the traffic signal? This can be tricky in that the radar might not have enough of a profile to bounce a radar signal off the traffic signal and detect it. Would the LIDAR detect it? This depends on whether there is even LIDAR available (for Tesla’s they aren’t using LIDAR). 

Anyway, the point is that a traffic signal could exist but not be detected by any of the sensors of the AI self-driving car.

Imagine then that the self-driving car believes that it can just barrel through the intersection. 

This could be wrong to do and perhaps there is a red light and the cross traffic is going through the intersection. Crash!

Now I realize you might object and say that the AI self-driving car has hopefully detected the other cars that are going through the intersection and so would realize that something is afoot. Furthermore, if there is other traffic next to the AI self-driving car, it would presumably be slowing down and stopping, due to the presumed red light, and so the AI self-driving car should notice that the other cars nearby it are stopping and so it should consider stopping too.

This brings up my earlier point about simplistic traffic signal capabilities versus more advanced traffic signal detection capabilities. If the AI and the self-driving car is solely programmed to visually find the traffic signal, this can be a significant risk as to being able to properly determine when a traffic signal is present or not. Some AI systems for self-driving cars only do the visual detection. Adding the sensor fusion and the other sensors data is considered “more advanced” – and likewise, comparing to the traffic situation is even more advanced. In other words, if you are only thinking of the traffic signal as just an object, you would only care whether you detected that particular object or not.

 We know though as humans that we use all sorts of other clues to figure things out. 

Edge Cases Upon Edge Cases

I’ll add more twists to the traffic signal problem. 

Suppose the traffic signal is there but it is not working? 

Maybe the power is out, maybe it has had a failure, etc. I know you might think that it low odds that a traffic signal exists but is not working – it is though a possibility.

It is a very real possibility that you might not even be able to see a traffic signal such as suppose you are driving behind a big truck as you come up to an intersection. Your view is blocked by the truck. The AI of the self-driving car has to be advanced enough to deal with these various scenarios. Keep in mind that this is all life-or-death stuff. A wrong move by the AI self-driving car can harm the human occupants of the AI self-driving car, and harm other humans by hitting other cars or maybe hitting a pedestrian, etc.

 Another variant to deal with involves the visibility of seeing the traffic signal. 

Suppose the camera has dirt on the lenses and only gets a partial image. The lighting near the traffic signal can also impact detecting the status of the traffic signal. Have you ever driven up to an intersection and the sun was directly in your eyes? You could barely see the traffic signal lights. You knew that there was a traffic signal there, but it was nearly impossible to see if the light was red, yellow or green.

Indeed, you’ve undoubtedly noticed that most modern traffic signals make use of hoods to help protect the light and make the light stand-out. Often, a traffic signal light is aimed at a particular angle to try and make it more visibly apparent. These elements can help the image processing. They can also make the image processing more difficult, depending upon how much the hood hides the light or that angle of the light is a kilter of where the camera on the car is.

 Future Of Traffic Lights Is Not Here Today

One hope is that the future traffic signals will be “smart” instead of dumb. 

The smart traffic signal has more advanced capabilities than the traditional “dumb” one does. 

For example, a smart traffic signal might emit an electronic signal indicating the status of the traffic light. In that case, the AI self-driving car can potentially receive an electronic signal rather than relying only on a beam of light. This can significantly aid the detection of the traffic signal.

Those that are looking further into the future would even say that the traffic signal as we know it today will ultimately no longer exist. If we later on opt to get rid of all human driven cars, and we had only AI self-driving cars, presumably the use of lights to signal the status of the traffic signal is no longer needed. It could be just an electronic signal. Also, there would not be a need to have the large pole that currently houses the traffic signal. You could put the traffic signal electronic emitter in a squat box near the intersection instead.

 I like to point out to those futurists that it will be a long time before we have only AI self-driving cars. 

For the foreseeable future, we will have a mix of human driven cars and AI self-driving cars (there are around 200 million conventional cars today in the United States alone). As such, we’ll need to keep the light emitting traffic signals for those “darned” human drivers. There are experiments currently involving sending an electronic signal to your smartphone, thus, in theory, we might be able to have human drivers that rely upon their smartphones to let them know the intersection status, rather than looking at a traffic light. I don’t think that’s going to hold-up though and we are likely to continue with the traffic signals as they are.

Where we will likely have advances towards smart traffic signals will be the ability of the traffic signal to adjust the timing of the lights based on more informed traffic analytics. A city might have traffic detecting sensors throughout the city, some being on buildings, some embedded in the roadway, some collected via flying drones, and so on. These sensors will collect traffic info. The traffic info will be analyzed and fed to the traffic signals so that the traffic signals can adjust their timing. This could greatly reduce gridlock.

 Your AI self-driving car might also get connected to these traffic analytics. 

Perhaps your AI self-driving car realizes that the traffic for the next six blocks is slowed due to the traffic signals at the intersections straight ahead. It might be able to predict that if the self-driving car makes a right turn up ahead and then goes onto another street, the self-driving car can avoid the bottlenecks up ahead. The AI could either opt to take the alternative route, or speak to the human occupant and ask permission first whether the alternative path is OK with them.

V2I Will Eventually Occur

There will be the advent of V2I (vehicle to infrastructure) for AI self-driving cars. 

This means that AI self-driving cars will be able to electronically communicate with the street infrastructure, including traffic signals, and with bridges (which can indicate how crowded they are), and tolls, etc. All of that will aid in navigating traffic signals. There is also going to be V2V (vehicle to vehicle communications), involving self-driving cars electronically communicating with other self-driving cars. This will be handy akin to my earlier example of cars around you that are coming to a stop at an intersection – not only would the AI self-driving car presumably directly detect this, but the other self-driving cars might be sharing with your AI self-driving car that they have detected that the intersection up ahead has a broken traffic signal so they are all coming to a stop.

Smart traffic signals might have some unintended consequences. 

Suppose a hacker was able to connect to a traffic signal and force it to show whatever red/yellow/green the hacker wanted to display. Suppose the traffic signals are all interconnected so as to allow for timed activity across the entire city, but then the hacker can opt to control all of those at once. There is also the privacy aspects that maybe a traffic signal is observing traffic and capturing license plates or other info about the cars passing through the intersection – would this be a good thing or a bad thing? It might help when trying to find criminals, but it might be used for nefarious violations of privacy.

Here’s one that might be somewhat chilling too. 

Should your AI self-driving car be allowed to run a red light or not? 

You might argue that the AI should never be allowed to carry out an illegal act. But, suppose that you are bleeding to death as a human occupant in an AI self-driving car and there are no other cars nearby, wouldn’t it be OK to run the red light so as to get to a hospital? There are numerous scenarios involving situations of these kinds. Also, if your AI self-driving car does run a red light, should it tell the police? Should it issue you a ticket?

See my article about the ethical aspects of AI self-driving cars:


There’s a famous joke about traffic signals and green/red lights. 

Here’s a short version of it. 

Let’s pretend you get into a ridesharing car by Uber or Lyft and the human driver heads to your destination. 

At an intersection that is a red light, the driver goes through a red light without stopping. You are startled by this. 

You ask the driver why didn’t he stop at the red light? 

He says that his brother never stops at red lights and it works well for him. 

While you are thinking about this, another red light nears and he drives through that one too. 

Catching your breath, all of a sudden as the driver approaches a green light he jams on the brakes. What, you ask, why in the world did he stop at a green light? He turns to you and says, because I thought my brother might be coming along through the intersection from the other road.



But with AI self-driving cars we need to make sure they are well versed in dealing with traffic signals. The simplistic approach might be OK for the moment, when we have only a few AI self-driving cars on the roadways, and while they are used only in carefully mapped and geo-fenced areas. Once AI self-driving cars become more prevalent, it will be a life or death matter as to whether they can handle traffic signals. This means that the AI needs to become more advanced. 

I’m giving a green light for that to happen.

Copyright 2019 Dr. Lance Eliot 

This content is originally posted on AI Trends.

Note: One aspect of robust AI software for driving is that it considers how to deal with pedestrians, including the case of pedestrians that are blind. See my column Blind Pedestrians and AI Self-Driving Cars, published August 13, 2019.

This UrIoTNews article is syndicated fromAITrends