Thanks to a Tesla patent published last year, we have a great look into how FSD operates and the various systems it uses. SETI Park, who examines and writes about patents, also highlighted this one on X.
This patent breaks down the core technology used in Tesla’s FSD and gives us a great understanding of how FSD processes and analyzes data.
To make this easily understandable, we’ll divide it up into sections and break down how each section impacts FSD.
Vision-Based
First, this patent describes a vision-only system—just like Tesla’s goal—to enable vehicles to see, understand, and interact with the world around them. The system describes multiple cameras, some with overlapping coverage, that capture a 360-degree view around the vehicle, mimicking but bettering the human equivalent.
What’s most interesting is that the system quickly and rapidly adapts to the various focal lengths and perspectives of the different cameras around the vehicle. It then combines all this to build a cohesive picture—but we’ll get to that part shortly.
Branching
The system is divided into two parts - one for Vulnerable Road Users, or VRUs, and the other for everything else that doesn’t fall into that category. That’s a pretty simple divide - VRUs are defined as pedestrians, cyclists, baby carriages, skateboarders, animals, essentially anything that can get hurt. The non-VRU branch focuses on everything else, so cars, emergency vehicles, traffic cones, debris, etc.
Splitting it into two branches enables FSD to look for, analyze, and then prioritize certain things. Essentially, VRUs are prioritized over other objects throughout the Virtual Camera system.
The many data streams and how they're processed.
Not a Tesla App
Virtual Camera
Tesla processes all of that raw imagery, feeds it into the VRU and non-VRU branches, and picks out only the key and essential information, which is used for object detection and classification.
The system then draws these objects on a 3D plane and creates “virtual cameras” at varying heights. Think of a virtual camera as a real camera you’d use to shoot a movie. It allows you to see the scene from a certain perspective.
The VRU branch uses its virtual camera at human height, which enables a better understanding of VRU behavior. This is probably due to the fact that there’s a lot more data at human height than from above or any other angle. Meanwhile, the non-VRU branch raises it above that height, enabling it to see over and around obstacles, thereby allowing for a wider view of traffic.
This effectively provides two forms of input for FSD to analyze—one at the pedestrian level and one from a wider view of the road around it.
3D Mapping
Now, all this data has to be combined. These two virtual cameras are synced - and all their information and understanding are fed back into the system to keep an accurate 3D map of what’s happening around the vehicle.
And it's not just the cameras. The Virtual Camera system and 3D mapping work together with the car’s other sensors to incorporate movement data—speed and acceleration—into the analysis and production of the 3D map.
This system is best understood by the FSD visualization displayed on the screen. It picks up and tracks many moving cars and pedestrians at once, but what we see is only a fraction of all the information it’s tracking. Think of each object as having a list of properties that isn’t displayed on the screen. For example, a pedestrian may have properties that can be accessed by the system that state how far away it is, which direction it’s moving, and how fast it’s going.
Other moving objects, such as vehicles, may have additional properties, such as their width, height, speed, direction, planned path, and more. Even non-VRU objects will contain properties, such as the road, which would have its width, speed limit, and more determined based on AI and map data.
The vehicle itself has its own set of properties, such as speed, width, length, planned path, etc. When you combine everything, you end up with a great understanding of the surrounding environment and how best to navigate it.
The Virtual Mapping of the VRU branch.
Not a Tesla App
Temporal Indexing
Tesla calls this feature Temporal Indexing. In layman’s terms, this is how the vision system analyzes images over time and then keeps track of them. This means that things aren’t a single temporal snapshot but a series of them that allow FSD to understand how objects are moving. This enables object path prediction and also allows FSD to understand where vehicles or objects might be, even if it doesn’t have a direct vision of them.
This temporal indexing is done through “Video Modules”, which are the actual “brains” that analyze the sequences of images, tracking them over time and estimating their velocities and future paths.
Once again, heavy traffic and the FSD visualization, which keeps track of many vehicles in lanes around you—even those not in your direct line of sight—are excellent examples.
End-to-End
Finally, the patent also mentions that the entire system, from front to back, can be - and is - trained together. This training approach, which now includes end-to-end AI, optimizes overall system performance by letting each individual component learn how to interact with other components in the system.
How everything comes together.
Not a Tesla App
Summary
Essentially, Tesla sees FSD as a brain, and the cameras are its eyes. It has a memory, and that memory enables it to categorize and analyze what it sees. It can keep track of a wide array of objects and properties to predict their movements and determine a path around them. This is a lot like how humans operate, except FSD can track unlimited objects and determine their properties like speed and size much more accurately. On top of that, it can do it faster than a human and in all directions at once.
FSD and its vision-based camera system essentially create a 3D live map of the road that is constantly and consistently updated and used to make decisions.
Subscribe
Subscribe to our newsletter to stay up to date on the latest Tesla news, upcoming features and software updates.
Tesla has just announced the contents and features of its 2025 Spring Update. There’s a lot of new content that we expected, as well as some stuff we didn’t see coming that will be arriving in Tesla’s next major release. Awesome new features, such as Adaptive Matrix High Beams, will finally become available in North America, while others like Grok’s voice assistant aren’t quite ready yet.
So, without further ado, let’s get cracking and take a look at everything in this awesome update.
Adaptive High Beams
The headliner feature of this update is the much-awaited Adaptive High Beams for North America - specifically the United States and Canada. We’ve been waiting a little over a year since it was launched in Europe last year. Tesla faced some regulatory delays in getting this approved, but it’s finally arriving for vehicles with newer headlights.
Adaptive High Beams reduce glare for traffic ahead of you by individually dimming specific pixels on the LED matrix. The feature shipped with the refreshed Model Y first and is now arriving for all other vehicles with matrix headlights. This includes newer Model S, Model 3, Model X, and Model Y vehicles - but not the Cybertruck.
The adaptive headlights in action.
Not a Tesla App
The Cybertruck’s signature headlights are too small to fit the LED matrix, and as such, this feature won’t be supported on the Cybertruck for the time being. Hopefully, Tesla will figure something out, but given that this is a hardware limitation, we don’t expect to see much here.
You can check out our guide on how to determine whether your vehicle is equipped with Matrix Headlights. If your vehicle has the hardware, you will see an Adaptive Headlights option under Controls > Lights > Adaptive Headlights after receiving the Spring Update. This feature will be enabled by default.
Improved Blind Spot Camera for Model S / X
The new blindspot camera in the driver's instrument cluster.
Not a Tesla App
In a surprise addition, Tesla is improving support for the Blind Spot Camera on the 2021+ refreshes of their flagship vehicles. Previously, the blind spot camera on these vehicles would only appear on the primary infotainment screen, not the driver’s instrument panel, which was essentially copied over from the Model 3/Y.
Now, drivers will have the option to choose which display the blind spot camera appears on. A setting under Controls > Display > Automatic Blind Spot Camera will allow drivers to choose “Driver Screen”, so that the blind spot camera appears to the left or right side of the instrument cluster, depending on which turn signal you activate. For these vehicles with an instrument cluster directly in front of the driver, this is a much better implementation of the feature than how it was originally designed.
Dashcam Update - B Pillar Cameras
As part of a much-requested update, given the increased and misguided vandalism against Tesla vehicles, Tesla’s team has finally updated their software to record the B-pillar (upper side) cameras as part of both Dashcam and Sentry Mode.
While this means that Dashcam and Sentry Mode footage will now likely take up more room on your USB drive due to recording two additional cameras, it also means that your vehicle is much better protected. Dashcam and Sentry Mode now record from every camera except for the additional front-facing cameras and the interior camera.
Note: It looks like this feature will be limited to newer vehicles, likely those with AI4.
Improved Dashcam Viewer
The updated dashcam viewer.
Not a Tesla App
The Dashcam Viewer in the vehicle is also being improved with this update. Taking a page from the Tesla app, the app in the vehicle will now display multiple camera feeds at the same time, with users having the option to focus on an individual feed if desired.
Due to the additional cameras being recorded, Tesla is now laying out all the camera feeds along the bottom, instead of at each corner of the screen.
The new UI also reveals that there will be buttons to jump back or forward in 15-second increments, while at the top right, you’ll have a link to the next video, instead of having to go back to the list of videos.
Requirements for Dashcam and Sentry Mode Updates
Unfortunately, there is some bad news regarding compatibility with the B-pillar camera recording and this improved Dashcam Viewer. Tesla says the Dashcam updates will only apply to newer “S3XY” vehicles, but they don’t specify the exact requirement.
Based on previous Tesla posts, where they usually list if a feature requires the AMD Ryzen infotainment processor, this requirement doesn’t sound like an Intel vs AMD issue, but instead one that relies on AI4 hardware, which is responsible for processing the video feeds.
Tesla’s “S3XY” requirement also leaves out the Cybertruck, but this seems like an oversight. Given some previously leaked footage of this feature, we expect the Cybertruck to also receive this feature with the Spring Update.
We recently covered routing options on the site, and we believe a lot of people will be pleased with these additions, so if you’ve been craving improved routing options, keep reading.
There are three new routing options to check out. Users will now be able to pick from three types of routing options when choosing a destination. We originally saw these as part of the navigation source code discovered in December 2024.
Fastest: This offers the quickest path to the destination, ignoring any attempts at efficiency or stopping more often to do short charges.
Best Amenities & Fewer Stops:This routing mode minimizes your charge stops in exchange for making them longer, but also allows you to stop near highly rated restaurants, shops, and restrooms for a more relaxing trip.
Avoid Highways: This much-requested feature will enable you to keep your navigation routing away from highways unless they are absolutely required to reach your destination. Hurray for the country roads and relaxed driving.
Requirements: While we’re not sure yet which vehicles will receive these options, we expect it to arrive on all vehicles except for potentially legacy Model S/X.
Trunk Height Based on Location
Another neat and useful little feature: you will now be able to save your trunk opening height based on location rather than applying a general maximum trunk height. If you didn’t already know, you could set the maximum height your automated trunk opens, which can help prevent it from hitting a lower garage ceiling.
This feature is already available on the refreshed Model Y but is now coming to all Model Ys, all Model 3s with automated trunks, and the 2021+ Model S and Model X.
In order to set your height, manually adjust the liftgate to your preferred opening height, and then press and hold the trunk button until you hear a chime in the vehicle, indicating that the height for this location has been set.
Save Frunk Height - Cybertruck
Tesla didn’t forget about the Cybertruck either - you can now do the same with the opening height for the Cybertruck as well. You’ll have to press the exterior (below the bottom center) frunk button and hold it until you hear a chime for the Cybertruck. Pressing the in-frunk button will simply close the frunk.
Accessory Power Option Enables 12V Sockets
Tesla is finally re-enabling 12V accessory power sockets throughout its cars with a new “Accessory Power” option, enabling anyone to use the 12V power sockets in Tesla’s vehicle lineup when they’re away from their vehicle, without needing Camp Mode. This also applies to the USB ports and wireless phone chargers throughout the vehicle.
The Model Y and Model X include a 12V socket in the rear left pillar of the vehicle, alongside a 12V socket in the front of the vehicle. The Model 3 and Model S only have a 12V socket in the front of the vehicle.
You can turn this feature on by going to Controls > Charging > Keep Accessory Power On. This feature is disabled by default and is turned off once the vehicle battery drops to 20% or below. Tesla warns that this feature will use additional power, so it’s best to only use it when needed.
Comfort Drive Mode on the Cybertruck
Following the recent addition of the Comfort Mode option in the Model 3, Tesla is adding the feature to the Cybertruck as well. This feature will automatically switch the vehicle dynamics to “Comfort”, which includes a higher ride height, softer suspension and steering response, and reduction in acceleration profile to Chill Mode while FSD or TACC are active.
You can enable or disable this feature from Controls > Autopilot > Use Comfort Mode in Autopilot. This feature will be enabled by default.
Lane Departure Avoidance on the Cybertruck
Interestingly, the Cybertruck launched without several Autopilot safety and assistance features - namely, because Basic Autopilot itself is missing from the Cybertruck - only FSD and TACC are available. As part of an improvement to safety, Lane Departure Avoidance has now arrived on the Cybertruck with the Spring Update.
This will show a blue indicator on the screen if you begin or are about to begin crossing a lane marking. You will have three options, just like with other Tesla vehicles, including None, Warning, and Assistance. Assistance will provide active feedback and move the vehicle back into the lane lines, while the warning will sound an audio tone and provide visual and physical feedback (vibration) to the steering wheel.
This feature will be enabled by default with Assistance selected and can be changed from Controls > Autopilot > Lane Departure Avoidance.
Minor Updates
Tesla also lists some other smaller details that will be included as part of the 2025 Spring Update, which include these features below:
Keyboard Languages
Go to Controls > Display > Keyboards to switch languages on the touchscreen keyboard.
Media search results are filtered by sources, which provides faster access to your content.
You can now shuffle an entire Apple Music playlist that contains more than 100 songs!
You can scroll through SiriusXM favorites by tapping the left steering wheel button left or right, similar to other services.
You can now sign in to Amazon Music with an Amazon Music Free account. You still require Premium Connectivity or WiFi to stream music.
YouTube Music now shows what song will play next in the Up Next view of the media player.
If you normally connect your vehicle to your phone’s hotspot, this feature will now be enabled every time you drive instead of having you manually connect it each time.
Features We’re Hoping Come Soon
This was an awesome update, but there are always more features we’d love to see come next. Here’s our short list of features we’re still waiting and hoping for.
Everyone’s favorite question is always, When will it be released? Well, it looks like soon. We haven’t seen any vehicles, including employees, receive the Spring Update just yet. However, given that Tesla has officially announced the update, we expect it to go out to employees as soon as this weekend.
Update: Tesla has now started rolling out this update to employees. As expected, it’ll be software update 2025.14.
If no major issues are found, we could see it start rolling out to the lucky first customers in about a week, but be prepared for a slightly longer wait if Tesla needs to reduce multiple revisions of the update before rolling it out publicly.
Tesla’s factories are more than just gigantic locations to produce cars at an insanely fast rate - they’re also locations that live and thrive alongside the environment around them. At Gigafactory Berlin, Tesla planted over 1 million trees in 2024 to help offset the footprint of Giga Berlin. Tesla originally cut down less than half that amount of trees.
One of Tesla’s major goals is to electrify the planet and reduce carbon emissions - and what better way to do that than to create more green space. Tesla has some absolutely fantastic green-centric initiatives, and Giga Berlin really showed that off when they planted double the acreage of forest that they cut down to build the facility.
A rendering of the expected finished locations.
Not a Tesla App
Ecological Paradise
Giga Berlin isn’t the only place that Tesla is working to bring some green to - Giga Texas is also on the list. Tesla has shown off their official renderings and plans for the upcoming Ecological Paradise that will be build around Giga Texas.
Tesla has some absolutely staggering plans for the location, and it's far more than a simple park. With nearly 70,000 residents within 15 minutes of Giga Texas and 15,000 employees commuting into work, there is a considerable number of people to appeal to. That also comes with nearly 4 miles of riverfront - so Tesla will be making the best use of the space to benefit the local community and its employees.
A map of the planned paths, routes, and areas.
Not a Tesla App
The plans for the ecological paradise include a 2-acre riverfront rain garden, as well as 6 rainwater buffer and treatment ponds - all intended to enhance, protect, and expand the 53 acres of local wetland - for a total of 150 million gallons of reclaimed rainwater that will be recycled for landscape irrigation.
Just like green spaces, Tesla places considerable emphasis on water reuse and aims to make its factories as water-neutral as possible. This Ecological Paradise will help to offset the water usage from Giga Texas while also greening up the entire community around it - because all four of the new trailheads, as well as the riverbank, will be publicly accessible.
You can check out the entire filing for the Ecological Paradise, which includes a benefit report to the local community, at this link here. It looks like quite a bit of green space, which will go right alongside the 32,400kW of solar panels installed at Giga Texas - which is the largest single installation of panels in the world.
