During another one of his gaming live streams, Elon Musk confirmed that Grok in Teslas is coming soon. According to Elon, you’ll be able to talk to your Tesla and ask for anything.
This seems in line with what we recently predicted - the arrival of a Smart Assistant from xAI, integrated right into your Tesla. We’ve seen the initial indications of this Smart Assistant beginning to form.
We’ve been super excited to see the Smart Assistant, but this is the first confirmation that it will be Grok.
Real-time Information
Grok is xAI’s advanced large language model (LLM) AI, designed to deliver highly capable and dynamic responses. Unlike many other LLMs, Grok can access real-time information and adapt its answers based on the latest news and discussions happening on X’s platform, making it uniquely responsive to breaking news and evolving events.
However, Grok still doesn’t have voice support like other models - including OpenAI’s ChatGPT and Google’s Gemini. It can generate images, documents, tables, and reason very effectively, but it has yet to deliver that information audibly.
Adjust Vehicle Features
Grok’s real-time nature is a capability that could be super useful in a vehicle. When you want to stop on a road trip for food, you can ask for the best local restaurant that is the least busy. Grok will potentially be able to gather all this information, and then enter the navigation entry as your next stop - and then Tesla’s FSD will take you there.
Grok will also be able to process in-vehicle commands intelligently - commands like “Open Charge Port” or “My Hands are Cold” will work through Grok, and it’ll make adjustments to your vehicle as required. Interfacing closely with the vehicle will be a capability that most other smart assistants won’t have - and none will be able to set a navigation destination, choose charging stops, or ask questions that depend on your vehicle’s location, such as send me the to closest 250kW Supercharger.
While voice commands can already perform some of these features, the voice system is expected to be greatly improved with Grok’s integration, letting drivers speak more naturally instead of remembering the syntax that the voice command system understands.
All Teslas will be able to use Grok - Elon Musk
General Knowledge
Tesla owners will finally have a Google Assistant-like smart assistant where you’ll be able to get quick answers to everyday questions, like what’s the weather for tomorrow or what’s Tesla’s stock price.
Supported Vehicles
Grok will continue to analyze and deliver responses on Tesla’s or xAI’s servers rather than inside your vehicle. Processing language and then running it through the AI network isn’t a small task, and given FSD taking priority on-vehicle, Grok will be dependent on an internet connection. The good news here is that since the heavy lifting is all down server-side, Grok will be available for all vehicles, according to Musk. Musk said during his live stream that “all Teslas will be able to use Grok.”
This will certainly mean that vehicles with Intel and Ryzen processors will have the new voice assistant available, while vehicles with MCU 1 are unknown at this time. Since this is mostly a server-side change, we also expect legacy Model S and X vehicles to receive the feature.
Premium Connectivity?
While voice commands are free to all Tesla owners, Tesla has been developing a lot of features lately that require its Premium Connectivity package for about $10/month. Just in the last few months, Tesla has added YouTube Music, Amazon Music, SiriusXM, the ability to stream dashcam footage on your phone, weather radar overlays, and more.
There are several paths Tesla could take. Tesla could bundle Grok into Premium Connectivity, it could require a Premium subscription on X, or it could be a completely free feature.
Release Date
Seeing Grok finally on the horizon is super exciting after we’ve been pining for a Smart Voice Assistant in Teslas for years. Musk said that it’s “coming soon,” during his stream, but we still don’t know exactly when. However, Tesla and xAI have been making several changes over the past few months that hint at this upcoming change. We’d expect this feature to launch in the first half of 2025 and potentially even much sooner.
The National Highway Traffic Safety Administration (NHTSA) in the US has said that it is opening an initial probe into Tesla’s new Actually Smart Summon feature. This preliminary investigation into Actually Smart Summon follows the report of four (4) crashes allegedly involving the use of Actually Smart Summon.
NHTSA has said that the reports say that users “had too little reaction time to avoid a crash, either with the available line of sight or releasing the phone app button, which stops the vehicle’s movement.”
Mainstream headlines have included that NHTSA has opened a probe into 2.6 million vehicles, but they’ve really opened a probe into four incidents, which seems a lot more reasonable.
Initial Probe
This is just an initial investigation to determine the circumstances of the four crashes and not actually any indication of whether the NHTSA will perform a full investigation or request a recall/change of the feature from Tesla.
Interestingly, the NHTSA has said it will assess Actually Smart Summon’s use on public roads, maximum speed, and line of sight requirements. Currently, Actually Smart Summon cannot be used on public roads, has an extremely low overall speed, and can only be used within a relatively small area.
The summary of the report mentions that the incidents involve Actually Smart Summon failing to detect posts or parked vehicles, resulting in a crash. Additionally, the summary also mentions that some of the incidents reported are actually under the older Smart Summon feature - which has been known to have more issues, as it relied solely upon ultrasonic sensors (USS) rather than the newer Tesla Vision system.
We’re interested to see what the NHTSA finds and whether this means that Tesla will have to roll back functionality of Actually Smart Summon, which has only recently launched. We’re hoping that the NHTSA finds results in favor of Tesla - who has put safety first with many of its features.
Transport Canada
Transport Canada, the NHTSA’s Canadian equivalent, has made no motions to introduce a similar probe, and the feature is available in Canada. We reached out to Transport Canada, and we received a statement that Transport Canada has not received any reports related to Actually Smart Summon as of today.
Crashing into a pole with Actually Smart Summon is a difficult feat - as it will stop for pedestrians crossing the road 15 feet away or decide on its own that it is not safe to continue and that it must stop. We expect these incidents involving poles likely occurred because users parked their vehicles right against a parking bollard or similar object, and the car moved slightly forward or backward to check what was around it, a noted issue in Actually Smart Summon.
For now, we’ll watch this one closely, but we may have to cross our fingers that Actually Smart Summon’s range or other capabilities are reduced in the future.
As a continuation of our series on Tesla’s patents, we’re taking a look at how Tesla optimizes the performance of AI - FSD, in this case - in autonomous vehicles and robots. Patent WO2024073115A1 goes over efficiently running complex AI models on specialized hardware.
Before we dive into this article, we recommend reading our article on How FSD Works and our other article on Tesla’s Universal Translator for streamlining FSD deployments. While they’re not necessary, the background knowledge will help you appreciate all the details behind how Tesla does their optimization.
Just like before, we’ll be breaking this down into sections and making them as easily understandable as possible.
AI Subnetworks
FSD isn’t a monolithic entity - it is composed of smaller, specialized sub-networks, each dedicated to a specific aspect or function of autonomous operation. This modular design means that Tesla can work on improving one or all sections through training. When one section is improved, the end-to-end nature of the AI also means that the other sections will learn to adapt to the improvements and, therefore, perform better. It also allows for more efficient processing and adaptability during deployment and initial platform training.
These sub-networks might be responsible for tasks such as:
Recognizing and interpreting traffic signals
Detecting and tracking moving objects including vehicles, pedestrians, cyclists, and more
Maintaining lane position and navigating roads
Generating 3D maps of the surrounding environment
Planning paths and making real-time driving decisions
This division of labor allows FSD to handle the complexities of autonomous driving with greater efficiency and precision
Tailored Compilers
Different hardware components are good at different things - and they also require different types of instructions. CPUs, GPUs, and specialized AI accelerators (NPUs) all have unique architecture and capabilities.
Tesla uses a compiler toolchain to translate FSD into machine code that is specifically tailored to each hardware component. This ensures that instructions are executed optimally on each processor, maximizing performance and efficiency.
Strategic Assignment
To further optimize performance, Tesla employs a system that intelligently assigns each FSD sub-network to the most suitable hardware component. This ensures that computationally demanding tasks are handled by the most powerful processors while simpler tasks are delegated to more efficient units.
This strategic assignment of tasks maximizes the overall efficiency of the system, ensuring that each component operates within its optimal performance range.
Optimized Scheduling
The order in which the hardware executes instructions also plays a crucial role in performance. Tesla's system includes an "execution scheduler" that determines the most efficient sequence of operations, minimizing delays and maximizing real-time responsiveness.
This optimized scheduling ensures that the FSD can react quickly and make informed decisions in dynamic driving situations - or quick-response situations with Optimus - like catching a ball.
While the demo here has been confirmed to be teleoperated, Tesla has said they’re working to let Optimus do this autonomously in the future.
To reduce the computational burden and power consumption of FSD, Tesla employs a technique called "quantization-aware training." This involves training FSD to work with lower-precision numbers, which require less processing power and memory. Essentially - rounding.
This approach allows the AI to operate efficiently without significantly compromising accuracy, striking a balance between performance and resource utilization.
Clock Synchronization
In hardware systems with multiple chips, maintaining precise timing is crucial for accurate and synchronized operation. Tesla's system incorporates mechanisms to synchronize the clocks of all processing units, preventing timing errors and ensuring seamless coordination between different components.
This precise clock synchronization is essential for FSD to make accurate real-time calculations and respond effectively to changing conditions.
Redundancy and Failover
To ensure reliability and safety, Tesla's system supports redundant hardware configurations. This means that if a critical component fails, a backup component can seamlessly take over, preventing disruptions in operation.
This redundancy and failover capability is crucial for maintaining the safety and integrity of autonomous systems, especially when driving. Tesla has built-in both physical and software redundancy to FSD, ensuring that it maintains a minimum standard of safety when operating autonomously.
In Simpler Terms…
Imagine a large company (FSD) with different departments (sub-networks) responsible for specific tasks. Each department has its own specialized tools and equipment (hardware components). Tesla's system acts like an efficient management structure, assigning the right tasks to the right departments, providing them with the appropriate tools, and coordinating their efforts for optimal productivity and performance.
![Tesla's Hidden 'Elon Mode' Unveiled: A Sneak Peek into the Future of FSD [Video]](https://www.notateslaapp.com/images/news/2022/fsd-beta-inside_300w.jpg)
![Tesla Adds HEPA Filter to New Model 3, Will it Get Bioweapon Defense Mode? [Update: Tesla Edits Post]](https://www.notateslaapp.com/images/news/2023/new-model-3-hepa-filter_300w.jpg)
