Elon Musk doesn't want Tesla to just be a car company — he wants it to be an AI company, the company that figures out how to make cars drive themselves.
Critical to this mission is Dojo, Tesla's custom-built supercomputer designed to train Tesla's Full Self-Driving (FSD) neural network. FSD is not actually full self-driving; it can perform some self-driving tasks but still requires an attentive human behind the wheel. But with more data, more computing power, and more training, Tesla believes it can cross the boundary from near self-driving to fully self-driving.
This is where Dojo comes in.
Musk has hinted at Dojo before, but he has ramped up discussions about the supercomputer throughout 2024. Dojo's importance to Tesla may be existential, as EV sales have slumped and investors want assurances that Tesla can achieve autonomous driving. Below is a timeline of Dojo mentions and promises:
2019
First mention of the dojo
April 22 – At Tesla's Autonomy Day, the automaker brought its AI team on stage to talk about Autopilot and Full Self-Driving, and the AI that powers both. The company shared information about Tesla's custom chips designed specifically for neural networks and self-driving cars.
During the event, Musk briefly talked about Dojo, revealing it to be a supercomputer for training AI, and also said that all Tesla cars produced at the time had all the hardware needed for full self-driving, and only needed a software update.
2020
Musk kicks off the Dojo Roadshow
February 2 – Musk says Tesla will soon sell more than 1 million connected cars worldwide that have the sensors and computing needed for fully self-driving cars, touting Dojo's capabilities.
“Our training supercomputer, Dojo, will be able to process massive amounts of video training data and efficiently run hyperspace arrays with a huge number of parameters, ample memory, and ultra-high bandwidth between cores — more on this later.”
August 14 – Musk reiterates Tesla's plans to develop “Dojo,” a neural network training computer that will “process really huge amounts of video data,” calling the computer “amazing.” He also says the first version of Dojo is “about a year away,” with a release date around August 2021.
December 31 — Elon says that Dojo isn't necessary, but self-driving will improve: “It's not enough to be safer than a human driver. Ultimately, autopilot needs to be at least 10 times safer than a human driver.”
2021
Tesla Makes Dojo Official
August 19 – The automaker officially launched Dojo at Tesla's first-ever AI Day, an event aimed at attracting engineers to Tesla's AI team. Tesla also unveiled the D1 chip, which Tesla says will be used to power the Dojo supercomputer along with Nvidia's GPUs. Tesla says the AI cluster will have 3,000 D1 chips.
October 12 – Tesla released a Dojo Technology white paper, “Tesla's Guide to Configurable Floating-Point Formats and Arithmetic.” The white paper outlines a technical standard for a new type of binary floating-point arithmetic used in deep learning neural networks, which can be implemented “fully in software, fully in hardware, or in any combination of software and hardware.”
2022
Tesla Reveals Dojo Progress
August 12 – Musk says Tesla will “introduce Dojo gradually. We won't need to buy that many additional GPUs next year.”
September 30th – At Tesla's second AI Day, the company revealed that it had installed its first Dojo cabinet and performed a 2.2 megawatt load test. Tesla said that one tile (comprised of 25 D1 chips) was being built per day. Tesla demoed the Dojo on stage, running a stable diffusion model to create an AI-generated “Cybertruck on Mars” image.
Importantly, the company says it has set a target date of completing the full Exapod cluster by the first quarter of 2023, and that it plans to build a total of seven Exapods in Palo Alto.
2023
“Longshot Gamble”
April 19 – During Tesla's first-quarter earnings call, Musk tells investors that Dojo “has the potential to reduce training costs by orders of magnitude” and “could potentially become a sellable service that we can offer to other companies, much like Amazon Web Services offers web services.”
Musk also said that he “views Dojo as kind of a long-term gamble,” but that it's “a gamble worth making.”
June 21 – The Tesla AI X account posted that the company's neural network is already in customer vehicles. The thread includes a graph showing a timeline of Tesla's current and projected computing power, with Dojo listed as starting production in July 2023, though it is unclear if this refers to the D1 chip or the supercomputer itself. Musk said the same day that Dojo is already online and running tasks in Tesla's data centers.
The company also predicts that Tesla's computing power will be in the top five in the world by around February 2024 (though there is no indication that this has been achieved), and that it will reach 100 exaflops by October 2024.
July 19 – Tesla announces during its second quarter earnings call that Dojo has begun production. Musk also says that Tesla plans to spend more than $1 billion on Dojo by 2024.
September 6 – Musk posts on X that Tesla is limited in the compute power required for AI training, but that Nvidia and Dojo will help solve that. Musk says managing the data from the roughly 160 billion frames of video Tesla captures from its cars per day is extremely difficult.
2024
Expansion Plans
January 24 – During Tesla's fourth quarter and full year earnings call, Musk reaffirms that Dojo is a high-risk, high-reward project. He also says that Tesla is pursuing a “dual Nvidia and Dojo path,” that “Dojo is working,” and that they are “doing training work.” He also says that Tesla is expanding Dojo and has “plans for Dojo 1.5, Dojo 2, Dojo 3, etc.”
January 26th – Tesla announces plans to spend $500 million to build its Dojo supercomputer in Buffalo. Musk then plays down the investment a bit, posting to X that while $500 million is a big amount, it's “only the equivalent of a 10k H100 system from Nvidia. Tesla will spend more than that on Nvidia hardware this year. The minimum to be competitive in AI right now is at least several billion dollars per year.”
On April 30th – at TSMC's North American Technology Symposium – the company announced that the next generation of Dojo training tiles, D2, is already in production. Rather than connecting 25 chips together to make one tile, D2 places the entire Dojo tile on a single silicon wafer, according to IEEE Spectrum.
May 20 – Mask Notes The rear of the GigaTexas factory expansion will include the construction of an “ultra-high density, water-cooled supercomputer cluster.”
June 4 – CNBC reports that Musk has diverted thousands of Nvidia chips reserved for Tesla to work on X and xAI. CNBC initially denied the report, but then Musk posted on X Tesla said that with construction ongoing on the south side of Giga Texas, there was no place to send the NVIDIA chips to power them up and they “would have just been left in a warehouse,” he said, noting that the expansion “will house 50,000 H100s for FSD training.”
He also Posts:
“Of the roughly $10 billion in AI spending that I said Tesla would do this year, about half is internal, primarily for the Tesla-designed AI inference computer and sensors in all of our vehicles, and for Dojo. Building out the AI training supercluster, NVidia hardware accounts for about two-thirds of the cost. We now expect Tesla to spend $3 billion to $4 billion on NVIDIA this year.”
July 1 – Musk reveals at X that current Tesla cars may not have the right hardware for the company's next-generation AI models. He says the roughly five-fold increase in the number of parameters that next-generation AI will require will be “very hard to achieve without upgrading the vehicle inference computer.”
Nvidia's supply challenges
July 23 – During Tesla's second-quarter earnings call, Musk said that demand for Nvidia hardware is “extremely high and GPUs are often hard to come by.”
“So I think we need to put a lot more effort into Dojo to make sure we get the training capabilities we need,” Musk said, “and I see a path with Dojo that allows us to compete with Nvidia.”
A graph in Tesla's investor deck projects that Tesla's AI training capacity will grow from about 40,000 in June to about 90,000 H100 equivalent GPUs by the end of 2024. Later that day on X, Musk posted that Dojo 1 would have “about 8,000 H100 equivalent training online by the end of the year.” He also posted a photo of the supercomputer, which appears to use the same refrigerator-like stainless steel exterior as Tesla's Cybertruck.
XXX
July 30 – Musk says the AI5 is 18 months away from starting mass production. reply A post from someone claiming to be starting a club for “Tesla HW4/AI4 owners who are angry about being left out when the AI5 launches.”
August 3 – Musk posts on X that he has given a walkthrough of “the Tesla supercomputing cluster in Giga Texas (aka Cortex).” He says the cluster is made up of roughly 100,000 H100/H200 Nvidia GPUs and has “massive storage for FSD and Optimus video training.”