Tesla uses a combination of map data and vision data for FSD visualizations
@ArthurFromX
Tesla’s FSD visualizations are admittedly one of the coolest parts of owning a Tesla. Seeing vehicles all around you on the screen, and watching the car make decisions can be mesmerizing.
However, have you ever wondered exactly how the car generates the visualizations? Is it real-time, or does it rely on map data that is downloaded separately from the vehicle’s firmware?
Map Data
Tesla distributes map data to vehicles worldwide separately from its vehicle firmware versions. This means that the map data updates can have a different tempo than the firmware updates. For instance, some cars on 2024.8.9 (an FSD V11 version from ~3mo ago), as well as cars on 2024.14.7 (an FSD V12 version from just a few days ago) just started receiving the NA-2024.8-14924 map update last week. These updates ensure that vehicles can get the most current information about road layouts, traffic patterns, and other critical driving data.
While FSD can usually work with different map versions, how much does FSD and the in-car visualizations rely on the predefined map data? It obviously uses it for some things such as speed limit, but does it use it more than that? Thanks to user, ArthurFromX on X, we now have a better idea of how much relies on map data.
FSD Visualizations
Tesla’s FSD visualizations rely on a combination of pre-mapped data and real-time information gathered by the cameras. This approach provides both background context (pre-mapped data), as well as real-time context via computer vision.
It all comes together to provide an accurate representation of what the car sees around it, taking into account the width, height, and length of nearby vehicles, curbs, and other objects, such as garbage bins or traffic cones.
A post on X highlighted that FSD visualizations might be more impacted by background data than initially thought. The post mentioned that construction to a local roundabout rendered the visualizations inaccurate, although the car was correctly navigating the lane and making its exit.
This suggests that while real-time data is crucial, the accuracy of pre-mapped data still plays a significant role in the overall effectiveness of FSD visualizations, and perhaps a lesser role in the actual decision-making process of FSD.
FSD Navigation
Of course, these visualizations impact more than just what you see on screen. The mapping data and real-time data provide information on how the vehicle plans its path, and how it makes driving decisions, such as moving into right, or left-turn lanes as required.
Mapping data most likely provides background information, enabling the onboard hardware to process and work on driving the vehicle with an initial idea of where it is and what the road looks like.
In short, map data plays a backseat role to the actual driving of FSD, but does play a bigger role in route decision, as well as providing context and predictions to what the vehicle sees. It seems at the moment that visualizations are based on a combination of map data and camera data the vehicle gathers.
Tesla’s been on a roll with Supercharger improvements lately, from the 325kW charging update for the Cybertruck, to 500kW with V4 Superchargers coming next year. While those improvements have been limited to the Cybertruck, Tesla didn’t put all their focus on their new flagship vehicle, but looked at their more affordable vehicles as well.
LFP Battery Heating
Tesla’s Superchargers can now heat LFP Batteries - those that are in the Model 3 and Model Y Rear Wheel Drive variants. This applies to Long Range and Standard Range models, which saw a limited run. This is another update included as part of the 2024 Tesla Holiday Update - which really arrived with a lot of unannounced new features and capabilities.
The change is pretty interesting - Superchargers of the V3 and V4 variety can now pre-heat batteries for Model 3 and Model Y vehicles equipped with LFP battery packs. That means those vehicles are able to get back on the road faster when it's extremely cold. Of course, Tesla still advises you to precondition before you arrive, saving drivers time and money.
Max de Zegher, Tesla’s Director of Charging, also commented on the new feature. Essentially, Tesla is inducing an AC (alternating) ripple current through the battery to warm it up. Keep in mind that Superchargers are DC charging. That means it is possible to get a cold-soaked LFP vehicle on the road 4x faster than before, assuming that it didn’t precondition at all and that it is in the worst-case scenario (below 0ºF).
In essence, Tesla is using some engineering magic to turn the circuits inside the LFP battery into an electric heater - and powering that heater through the Supercharger. An AC ripple current is a small oscillation in the DC charging current that generates heat through electrical resistance, warming up the battery. Those ripples are a byproduct of converting AC to DC and back - so Tesla is using the onboard charger to induce those ripples to warm up the battery. Definitely an innovative technique that’s really only possible with the versatility of the NACS connector.
We’re hoping Tesla can implement this across their full lineup of vehicles, but we’ll have to wait and see how it is trialed across LFP vehicles first and if it is even possible on vehicles with 2170 or 4680 battery packs.
Tesla launched two FSD updates simultaneously on Saturday night, and what’s most interesting is that they arrived on the same software version. We’ll dig into that a little later, but for now, there’s good news for everyone. For Hardware 3 owners, FSD V12.6.1 is launching to all vehicles, including the Model 3 and Model Y. For AI4 owners, FSD V13.2.4 is launching, starting with the Cybertruck.
FSD V13.2.4
A new V13 build is now rolling out to the Cybertruck and is expected to arrive for the rest of the AI4 fleet soon. However, this build seems to be focused on bug fixes. There are no changes to the release notes for the Cybertruck with this release, and it’s unlikely to feature any changes when it arrives on other vehicles.
FSD V12.6.1 builds upon V12.6, which is the latest FSD version for HW3 vehicles. While FSD V12.6 was only released for the redesigned Model S and Model X with HW3, FSD V12.6.1 is adding support for the Model 3 and Model Y.
While this is only a bug-fix release for users coming from FSD V12.6, it includes massive improvements for anyone coming from an older FSD version. Two of the biggest changes are the new end-to-end highway stack that now utilizes FSD V12 for highway driving and a redesigned controller that allows FSD to drive “V13” smooth.
It also adds speed profiles, earlier lane changes, and more. You can read our in-depth look at all the changes in FSD V12.6.
Same Update, Multiple FSD Builds
What’s interesting about this software version is that it “includes" two FSD updates, V12.6.1 for HW3 and V13.2.4 for HW4 vehicles. While this is interesting, it’s less special when you understand what’s happening under the hood.
The vehicle’s firmware and Autopilot firmware are actually completely separate. While a vehicle downloading a firmware update may look like a singular process, it’s actually performing several functions during this period. First, it downloads the vehicle’s firmware. Upon unpacking the update, it’s instructed which Autopilot/FSD firmware should be downloaded.
While the FSD firmware is separate, the vehicle can’t download any FSD update. The FSD version is hard-coded in the vehicle’s firmware that was just downloaded. This helps Tesla keep the infotainment and Autopilot firmware tightly coupled, leading to fewer issues.
What we’re seeing here is that HW3 vehicles are being told to download one FSD version, while HW4 vehicles are being told to download a different version.
While this is the first time Tesla has had two FSD versions tied to the same vehicle software version, the process hasn’t actually changed, and what we’re seeing won’t lead to faster FSD updates or the ability to download FSD separately. What we’re seeing is the direct result of the divergence of HW3 and HW4.
While HW3/4 remained basically on the same FSD version until recently, it is now necessary to deploy different versions for the two platforms. We expect this to be the norm going forward, where HW3 will be on a much different version of FSD than HW4. While each update may not include two different FSD versions going forward, we may see it occasionally, depending on which features Autopilot is dependent on.
Thanks to Greentheonly for helping us understand what happened with this release and for the insight into Tesla’s processes.
_300w.png)
