Tesla now dynamically resizes vehicle models in latest FSD Beta
Tesla's early FSD Betas included driving visualizations that used simple wireframe boxes to represent vehicles and lane markings were made up of individual dots.
The visualizations were a great look into some of the information that is provided to Autopilot, but even then only a fraction of the information Autopilot uses was actually displayed onscreen.
In reality, Autopilot is creating a 3D representation of every object it tracks. Each object detected then has various attributes. For example, a detected vehicle will have attributes for how fast it's going, how far away it is, the type of vehicle, its predicted path, and more.
The car visualizations are an important part of FSD because they help us better understand what the car is capable of seeing and reacting to. However, the information and visualization Autopilot needs is drastically different than what humans need.
In order for Tesla to achieve FSD, they essentially need to be able to build a highly accurate video game that represents the real world, in real-time.
The car wants access to as much information about each object as possible. Meanwhile, humans want a visualization that closely resembles the real world.
With the introduction of FSD Betas 9.x, Tesla released a more human consumable visualization. One that included proper 3D models of general vehicle types, road pylons, and solid lane markers.
The road edges and lane markings are more distinguished lines, 3D models have working brake lights, and other objects such as speed bumps, bike lanes, and crosswalks are depicted using visualizations that match the real world.
In order for Tesla to achieve FSD, they essentially need to be able to build a highly accurate video game that represents the real world, in real-time.
However, something that has been missing is visualizations is dynamic vehicle sizing. The 3D vehicle models that Tesla has been using have a static size. When the vehicle sees a bus, it calculates its length, width, and height in addition to a bunch of other metrics. However, the 3D model that is shown onscreen is a predefined size, meaning that it does not actually match what the vehicle saw.
This is why you may have seen a tractor-trailer shift forward and backward or you may have seen two vehicles on top of each other. One is signifying the start of the vehicle and since the vehicle is so much longer than the model, it's adding another vehicle to the end to signify the end of the vehicle.
Scalable Vehicle Models
However, in the latest 10.10.2 FSD update, we are now seeing Tesla scale individual vehicle models so that they represent the calculated size of surrounding vehicles. Contextually this could be helpful in better understanding our car’s situation in the world.
In 10.10.2, the car shrinks or stretches the 3D vehicle models in each dimension so that the 3D model matches the calculated dimensions for each vehicle. This is especially apparent in longer vehicles such as buses, trucks, and tractor-trailers, where the vehicle lengths are more likely to vary, but you can also see it scale other vehicle models such as very small cars.
In this example below, you'll see that Tesla is now able to accurately represent buses of different sizes. Tesla only has a model for a full length bus, but in this case, Tesla detected that the length of one of the buses is considerably shorter than the vehicle model so it chose to reduce the length of the bus to the length Autopilot had calculated. In the image below you can see how the same bus model is shown in two different sizes.
It's important to realize the difference between the visualizations and what Autopilot uses. The visualizations are there merely to help us better understand what Autopilot can see. The FSD computer itself has always been taking note of the size of surrounding objects and various other data points. Trajectory, approach velocity, proximity, and so forth have also been a part of this, but this update helps Tesla achieve visualizations that provide a more accurate representation of reality.
It's not only buses and trucks that are scaled up or down. Tesla resized a bobcat down to a vehicle that is about half the length of its normal sedan model.
Models are adjusted in all three dimensions. We witnessed some truck models that were stretched to become taller while also having their length reduced. It's not perfect because you're scaling all components of the truck at the same rate, but it produces a much more accurate representation of the vehicle and the amount of space it takes up.
Tesla has come a long way in a short period with how many objects they're able to detect, but obviously, when you compare the environment the car sees today, there is still a lot missing.
In the short term, we'd like to see more objects visualized. Objects that are commonly encountered while driving, such as trailers and gates.
We'd also like to see other common objects added, such as additional traffic light configurations, crosswalks, mailboxes, and maybe even a generic object that lets us know the vehicle sees something it needs to maneuver around, but it may not know exactly what it is.
In the future, I think we'll see Tesla display a rich, fuller 3D environment that will display static and moving objects that are important for the vehicle to avoid, objects such as barriers, buildings, trees, sidewalks, and more. Today Tesla is one step closer to achieving this goal.
Be sure to check out our full list of every visualization in Tesla's latest FSD Beta.
Ordering a New Tesla?
Consider using our referral code (nuno84363) to get up to $2,000 off your new Tesla and get 3 Months of FSD for free.
