Tesla FSD will be pure vision and not rely on radar use

Elon tweeted that v9 of the FSD beta would remove its reliance on radar completely and instead determine decisions based purely on vision. Humans don’t have radar after all, so it seems like a logical solution and tells us Tesla is feeling much more confident in their vision AI.

Radar and vision each have their advantages, but radar has thus far been much more reliable in detecting objects and determining speed. If you’ve ever noticed your Tesla being able to detect two vehicles in front of you when you can only see the one directly ahead of you, that’s radar at work.

In this situation the radio waves from the radar sensor are bouncing underneath the car in front of you and are able to continue traveling and detect that there is another object ahead even though it could never “see” it.

It really is one of those wow moments where you can feel the future and the ability for AI-powered cars to drive better than humans one day. It’s baby steps and slowly we’ll see more and more of these situations where the vehicle simply sees or does something we could never do.

There’s no doubting that more sensors could provide a more reliable and accurate interpretation of the real world as they each have their own advantages. In an ideal world a vehicle with radar, lidar, vision, ultrasonic sensors and even audio processing would provide the best solution. However, more sensors and systems come at a price, resulting in increased vehicle cost and system complexity.

After all humans are relatively safe drivers with two “cameras” and vision alone. If Tesla can completely solve vision, they’ll easily be able to achieve superhuman driving capabilities. Teslas have eight cameras, facing in all directions. They’re able to analyze all of them concurrently and make much more accurate interpretations then we ever could in the same amount of time.

Tristan on Twitter recently had some great insight into Tesla vision AI and how they’re going to replace radar. Here’s what Tristan had to say:

"We recently got some insight into how Tesla is going to replace radar in the recent firmware updates + some nifty ML model techniques

From the binaries we can see that they've added velocity and acceleration outputs. These predictions in addition to the existing xyz outputs give much of the same information that radar traditionally provides (distance + velocity + acceleration).

For autosteer on city streets, you need to know the velocity and acceleration of cars in all directions but radar is only pointing forward. If it's accurate enough to make a left turn, radar is probably unnecessary for the most part.

How can a neural network figure out velocity and acceleration from static images you ask?

They can't!

They've recently switched to something that appears to be styled on an Recurrent Neural Network.

Net structure is unknown (LSTM?) but they're providing the net with a queue of the 15 most recent hidden states. Seems quite a bit easier to train than normal RNNs which need to learn to encode historical data and can have issues like vanishing gradients for longer time windows.

The velocity and acceleration predictions is new, by giving the last 15 frames (~1s) of data I'd expect you can train a highly accurate net to predict velocity + acceleration based off of the learned time series.

They've already been using these queue based RNNs with the normal position nets for a few months presumably to improve stability of the predictions.

This matches with the recent public statements from Tesla about new models training on video instead of static images.

To evaluate the performance compared to radar, I bet Tesla has run some feature importance techniques on the models and radar importance has probably dropped quite a bit with the new nets. See tools like https://captum.ai for more info.

I still think that radar is going to stick around for quite a while for highway usage since the current camera performance in rain and snow isn't great.

NoA often disables in mild rain. City streets might behave better since the relative rain speed is lower.

One other nifty trick they've recently added is a task to rectify the images before feeding them into the neural nets.

This is a common in classical CV applications so surprised it only popped up in the last couple of months.

This makes a lot of sense since it means that the nets don't need to learn the lens distortion. It also likely makes it a lot easier for the nets to correlate objects across multiple cameras since the movement is now much more linear.

For more background on LSTMs (Long Short-Term Memory) see https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21

They're tricky to train because they need to encode history which is fed into future runs. The more times you pass the state, the more the earlier frames is diluted hence "vanishing gradients".

Tesla’s FSD beta v9 will be a big improvement forward from what FSD beta users have been using where the system was still relying on radar. And it’ll be an even bigger leap from what non-beta testers currently have access to. We can’t wait. Now where’s that button?

Almost ready with FSD Beta V9.0. Step change improvement is massive, especially for weird corner cases & bad weather. Pure vision, no radar.

— Elon Musk (@elonmusk) April 9, 2021

Follow

Follow

Follow

Tesla Expected to Fix Cybertruck Charging and Regen Issue in Update 2025.2

BrianASilver3 / Tesla

Cybertruck owners in northern areas - especially those in Canada, have noticed slower charging rates on some vehicles. These owners are experiencing low charging amperage even when using a 48-amp Wall Connectors.

The 4680 cell - the Cybercell, seems to be sensitive to temperature changes and, as such, loses the ability to charge or discharge rapidly. Tesla accommodates this by warming the battery in advance of driving or while driving. However, there seems to be a software issue preventing the battery pack from warming up properly.

In addition, other users have noted that they don’t have regen braking due to the battery not warming up properly, even after driving. If you drive other Tesla vehicles while they’re cold-soaked, they will eventually warm up to offer you full regen and full performance. The Cybertruck, interestingly, will offer you full performance at cold-soaked temperatures as low as -20C (-4F), but will have regen unavailable in temperatures as warm as -5C (41F). 

Software Fix Confirmed

Brian Silver over on X reached out to Tesla Service with his findings, who confirmed that this is a known software issue that has now been validated. Tesla Service responded by saying that they’re hopeful a fix can be built into the 2025.2 software update. This is the first time we’re hearing about Tesla’s next major software update, which is expected to be released around the second or third week of 2025.

Tesla has also confirmed that this is not a physical issue with the heating system, so no repair is required for this one - it’ll be another OTA fix, just like many others.

We’re glad to see this one getting fixed so quickly. The issue appears to have started with the 2024 Holiday Update, or it could just be a coincidence since Cybertrucks begin to settle into Canada this year, where winters are much harsher and colder on average.

Workaround

We’re hoping to see this fix go out soon, as the current procedure to get around this is annoying. It requires you to set the navigation to a nearby Supercharger so that your vehicle will precondition the battery. You can stop the navigation to the Supercharger after 5-10 minutes, at which point the battery pack will be more than warm enough to charge at the full amperage of your home charger.

This workaround also resolves the issue of limited regenerative braking by warming up the battery pack sufficiently.

Follow

Follow

Follow

Tesla Introduces Multi-Vehicle Coordinated Light Shows [VIDEO]

Not a Tesla App

As part of Tesla’s Holiday Update, Tesla has introduced an awesome new feature - multi-vehicle synchronized Light Shows.

Now, you’re probably going to say: Wait! We’ve seen those kinds of light shows already.

And you’re not wrong - thanks to the hard work of people using the Light Show tool, you could synchronize light shows and run different shows on different vehicles. But this was a lot of manual, time-consuming work. Tesla now officially supports creating a light show involving up to eight vehicles.

Coordinated Shows

These new coordinated shows can be set up in the new xLights interface, making it far easier for Light Show designers to assemble everything. They’ll be able to edit and view multiple vehicles at the same time, synchronize them all together - and build variants with up to eight vehicles.

The output is still eight different Light Shows - which will still need to be put on USB flash drives and distributed to the correct vehicles in the correct order, but designers can now build a unified and coordinated light show. It could be possible to coordinate more than eight vehicles in a single Light Show - and we’re sure we’ll find out in the coming weeks what is possible with the updated xLights software.

Tesla showed off a great example below, putting together a jaw-dropping 8-vehicle Light Show using the new “Ready for Assault” show that arrived with the 2024 Holiday Update. Tesla has also gone ahead and contributed example Light Shows on Github to get Light Show designers familiar with the new process.

Stand in the middle to enter boss fight pic.twitter.com/IHvuzsQgG9

— Cybertruck (@cybertruck) December 23, 2024

Everything New in Light Shows

Tesla’s light show feature received a massive update with this year’s holiday update, but we’re only finding out about some of the features now. Here’s a quick rundown of everything new in Tesla’s Light Show feature:

• Support the Off-road Lightbar for the Cybertruck

• Start, Stop, or Schedule a Light Show from the Tesla app

• Create longer Light Shows

• Two new songs, including Cyber Symphony and Ready for Assault

• Ability to control Ambient Lighting in 2024 Model 3 or Cybertruck

• Display color effects on the vehicle’s center screen

• Create Light Shows that involve up to eight vehicles

Cybertruck Lightbar Support

Another interesting addition is the recent support for the Cybertruck’s off-road lightbar. The off-road lightbar can be addressed in three zones - the forward main bar and left and right ditch beams. With the holiday update, Tesla also added the ability to program the vehicle’s ambient lighting, display color effects, and create longer light shows than ever before.

Tying everything together in this update is the ability to start, schedule, or stop a light show directly from the Tesla app.

Follow

Follow

Follow