I rode along in Nvidia’s AV pilot. It made better decisions than me

SAN FRANCISCO — It’s a gorgeous California day in the Bay Area, one where the city is doing its most flattering impression of itself — cold sun, hard shadows, and that particular Friday energy where everyone’s late and somehow still stopping for a matcha. I’m in the passenger seat of a brand new Mercedes-Benz CLA sedan zipping around the crowded, two-lane streets of the city when an everyday urban traffic trap unfolds: A delivery van in front of me has stopped to unload, there’s a bus coming quickly in the other direction, and pedestrians are darting across the street as if crosswalks are just a suggestion. The driver, Lucas, doesn’t have his hands on the wheel as the car glides forward. And I have exactly zero worries about how things will play out — because I’m inside Nvidia $NVDA’s autonomous vehicle pilot, and this car makes better decisions than I can. 

It has to. It’s trained to.

The car starts by using its 10 cameras, five radar sensors, and 12 ultrasonic sensors to read the obvious tell: hazard lights. It doesn’t immediately commit to swinging around the van, though. It verifies that the vehicle is truly stopped, gauges the bus’ speed and distance, waits out the pedestrian flow, and then nudges left with the smoothness of someone who has done this a thousand times before — but without the energy of someone who has done this a thousand times before. (The best self-driving flex is “no drama.”)

“That was pretty well-handled,” Ali Kani, Nvidia’s VP of the automotive team, tells me from the backseat. 

He’s not wrong.

For almost an hour, the car — using Nvidia’s driver-assist system — confidently handles everything the road throws at it under driver supervision: from the Ferry Building, down the Embarcadero, up the hills of Fillmore Street, and onto Union Street, where the shops are adorable and the streets are packed. We see Waymo robotaxis. We see Teslas in Full-Self Driving mode. We see drivers who probably shouldn’t be behind the wheel of anything more complicated than a shopping cart. We change lanes, make an unprotected left-turn, zip through intersections, cede plenty of space to pedestrians and bikers, glide to easy stops, and easily navigate everything else the city — and its drivers — want to throw at us on a Friday afternoon.

I ask how the car is at parallel parking — “great,” Kani says, “we’re very proud of it” — which is excellent news to anyone (me) who grew up driving across the Bay Area and who still wakes up in a cold sweat thinking about cramming into a too-tight spot on a too-steep hill in a too-crowded city. 

Lucas keeps one hand hovering above the wheel as the car “drives.” The car’s sensors are constantly gathering data; for every input, the system runs 10 quick “Huh, what happens next?” simulations. When eight of them agree on the same safe move, the car commits and does it. Still, Nvidia’s system has tunable driving personality knobs — acceleration, deceleration, lane-change timing, how hard it commits — plus “cooperative steering,” where the driver can nudge decisions without detonating the whole thing. The city had already tried to kill me three times this week — once with a driver who mounted a sidewalk — so my standards for “trustworthy decision-making” coming into the demo weren’t exactly high, but the tech never left me feeling wary. Some drivers could learn from it.

More than once, the car “talks” to us because the moment anyone in the cabin says “Mercedes,” it trips the vehicle assistant, which responds like an over-eager coworker who heard their name in a meeting and now wants to contribute. “It’s always gonna be the M-word,” Kani jokes when I ask if you can rename the tech to a less common name, as Amazon $AMZN did with its Echo and “Alexa.” But that communication is a key selling point. Nvidia’s next-era pitch is that autonomy doesn’t just need to see, it needs to understand — language, intent, context, and everything else — so that the software behaves less like a feature and more like a driver. There should be a conversation between car and passenger, Nvidia says, a “make the car think” phase where you’ll be able to tell it what you want — go faster, go around the car in front of you, pull over — which Nvidia sees as another (faster) path for AV progression.

The streets of San Francisco force issues repeatedly. But the Mercedes CLA keeps making the right kind of choices: verify, predict, commit — then move. I get comfortable in the seat, entirely forgetting I’m being shuttled around by a computer and a series of probabilities and edge cases. It was smooth sailing — or, rather, smooth driving.

Photo courtesy of Nvidia $NVDA

That smoothness is the point — and also the pitch. Right now, Nvidia isn’t trying to out-Waymo Waymo by running a robotaxi empire, or out-Tesla $TSLA Tesla by turning every customer into a rolling beta tester. It’s trying to do something more Nvidia than either: turn autonomy into infrastructure — a stack that automakers can plug into, tune, and ship, without reinventing the whole brainstem from scratch. 

The version of that stack I rode in was built with Mercedes, and Nvidia’s team is clear about what that means behind the scenes: the driving, parking, and active safety software was built by Nvidia with Mercedes’ help, trained on Nvidia infrastructure, and then tested in the cloud and on-road through both companies’ processes — with the premise that nothing goes anywhere near a customer until both sides sign off that it’s safe. That’s supplier talk — the same tone Nvidia uses when it shows up at CES sounding like the Department of Compute.

Nvidia talks about its demo car as an enhanced “Level 2” (L2++) — a point-to-point system designed to handle highway and city streets supervised, driver-hands-hovering, one human still legally on the hook — even as Level 4 is the endgame. Level 4 is entirely human free. But even at L2, the data flywheel spins fast, and Nvidia needs to get the system strong enough to handle the real world under supervision, collect the weird edge cases, feed the model, generate more of the hard stuff synthetically, and come back sharper. Kani is careful about the premise. This is a Level 2 car, he says, and it’s “telling you it is a Level 2 car,” and that “you cannot trust me.” A Waymo, by contrast, is a Level 4 ride — and “it’s telling you that you can trust me.” The point of Nvidia’s approach seems to be to earn the right to move up levels, not declare victory and hope the public agrees.

For Nvidia to get to L4, Kani says, the car needs more compute than the single Orin chip the car currently uses. He points to Nvidia’s higher-end compute and sensor redundancy — including lidar — as table stakes for a system that doesn’t demand a hovering human hand. Then, on Tuesday, lidar-maker Hesai said it was selected as a lidar partner for Nvidia’s “reference compute and sensor architecture” aimed at making vehicles L4-ready. The setup is the expensive, belt-and-suspenders hardware Kani is talking about — a full “sensor ecosystem” (cameras, radar, lidar, ultrasonic) fused by two Thor systems, with Nvidia pitching Hyperion as “the backbone” for the autonomy transition.

In the pre-drive presentation, Nvidia’s VP of automotive, Xinzhou Wu, made the roadmap feel less like aspiration and more like a schedule. In his framing, 2026 is about expanding the L2 envelope (highway and urban), adding more automation around parking, and pushing toward broader U.S. coverage — while also starting a “small-scale trial” of an L4 “alpha” track in 2026 to expand the L2++ envelope across highway and city driving.  “Partner deployment” of an L4 “alpha robotaxi” is scheduled for 2027, with a longer arc toward L3 highway and L3/L4 consumer vehicles in 2028.

The reason Nvidia thinks it can play here is the part that sounds the most like Nvidia: practice at scale. I asked Wu how Nvidia can close the gap with the obvious incumbents when Waymos are everywhere on the streets of San Francisco. Simulation, he said. Wu said Nvidia simulation is running at industrial volume — “a million tests per day,” “100,000 unique scenarios,” and “2,000 GPUs” dedicated to it, with plans to grow those numbers by orders of magnitude on the road to L4. 

Kani translates the same idea into a before-and-after story: He says they ran a version of this route weeks earlier and “it was really bad.” They broke down the hardest scenarios, fed more data, generated synthetic versions of the long-tail edge cases, and came back to find it “so much better in six weeks.” He also draws a line around how Nvidia intends to scale: It’s not about “mapping a city” the old-school way, because these cars have to work “all over the world.” The miles Nvidia lacks on public streets, it seems to intend to manufacture in silicon.

“We’re close,” Kani says — as he admits the project “was a little bit of a science project” for a long time. The difference now, he says, is that “there are the core pieces of technology”: “a Vision Language Action model with reasoning that really can handle long tails” plus a synthetic-data engine “quite good at creating these long tails,” accurate enough to train from. 

But the path from L2 to L4 has a habit of eating calendars — see: Elon Musk’s grand vision and grander promises. Nvidia claims that it only took a year to move from highway driving to city driving — while Tesla took eight years. Tesla began rolling out FSD Beta in October 2020 yet still describes its current FSD as supervised, explicitly not autonomous. Waymo opened its fully driverless service to the public in metro Phoenix in October 2020 and has spent years turning “driverless” from a demo adjective into an operations discipline. Nvidia acknowledges that it has different constraints: Robotaxis can afford sensor-and-compute setups that are wildly more expensive than what belongs in a consumer passenger car, Kani noted — which makes “solve it in a passenger car” a harder, more commercially portable problem.

But Nvidia is trying to solve that problem with the confidence of a company that can afford to be patient, vertically helpful — and stubborn. It has been working on autonomy “for more than 10 years,” Kani says, even if it hasn’t been doing it with Waymo’s visibility or Tesla’s main-character energy. Nvidia’s empire was built by selling the picks and shovels for other people’s AI ambitions. Automotive, by comparison, is still small — Nvidia reported $592 million in quarterly automotive revenue (automotive and robotics) in the most recent quarter. Now, with Nvidia sitting at the very top of the market-cap food chain, the pitch reads less like a side quest and more like a corporate mandate — a company that has already eaten AI deciding it would like to eat the physical world, too.

Nvidia wants to sell autonomy, not rides

The AV market landscape was sitting at stoplights all over this city. Over the two days I spent with Nvidia — and the 17 miles I walked up and down San Francisco’s hills — I counted 96 Waymos. I rode in one for the first time, too, to compare it with the Nvidia Mercedes. 

The first problem was dumb and immediate: which seat to sit in. I always sit in the back of an Uber $UBER or Lyft $LYFT, because I’m not a monster and because I don’t always want to make small talk with a stranger who knows my home address. But when there’s no driver, the social logic collapses. I hovered, then slid into the front passenger seat so I could watch the wheel move on its own — which is both deeply cool and, on an animal-brain level, a little unsettling. Then my mind did what it always does: leapt straight past the engineering to the logistics. What happens if someone throws up in one? Not me, with my perfect rideshare rating and my pathological fear of being a problem — but, statistically, someone. Who cleans it? Does the car pull over and quietly text its manager?

Here’s the thing about Waymo: The tech is impressive, but the business is the flex. It has a real consumer product, in real cities, doing real rides. Nvidia, by contrast, is selling the idea that the hard part of autonomy can be packaged — not as a fleet but as a brain that other companies can buy.

Nvidia’s strategy reads as both ambitious and cautious. It’s ambitious because the incumbents already have what Nvidia doesn’t: public road miles, public trust, and the brutal learning that comes from putting the product in front of actual paying customers. It’s cautious because Nvidia isn’t trying to own the whole mess. It wants to be the platform underneath the mess.

Waymo is a robotaxi company. Tesla is a consumer car company that markets itself like a software company. Nvidia is a supply-chain company wearing a leather jacket. It wants to sell a solution to what Wu described as the industry’s core “three-computer” problem — the computer in the car, the computer that trains the model, and the computer that simulates the world. Nvidia wants to provide one integrated stack that any automaker can plug into, whether they’re trying to build a supervised driver-assist feature or a full driverless fleet. Nvidia can’t pretend it has Waymo’s on-street footprint, and it can’t pretend it has Tesla’s distribution machine. But it can claim something both of them need: the compute DNA, the tooling, the simulation, the partner network, the ability to treat autonomy as a repeatable industrial product rather than a single company’s monastic quest.

Both Waymo and Tesla should be nervous about Nvidia’s road to AVs. Wu claimed that Nvidia is “very close” to Tesla’s supervised system, saying the internal takeover rates in head-to-head tests land in the same neighborhood. Nvidia doesn’t need to win the robotaxi race to win the autonomous era. Waymo and Tesla do. So do BYD, Zoox, Cruise, and whoever else. Otherwise, they could wake up one day to find that the rest of the industry has even a “good enough,” faster, cheaper path to autonomy that doesn’t require building a car from scratch. And Nvidia already has a seat in the car — its chips sit inside millions of vehicles on the road today, including Teslas, quietly doing the work long before Nvidia ever asked it to do the driving.

Wu described his mandate from CEO Jensen Huang in a way that’s almost comically on-brand for a company that thinks in platforms: Huang told Wu that his job is “to make everything that moves autonomous.” Cars are the most regulated, most litigated version of “things that move.” Robots are the adjacent act — and Nvidia has been selling “physical AI” as the next frontier of the stack. The Nvidia autonomy push is bigger than sedans; the sedan is just where the stakes are easiest to understand.

In a demo car, it’s easy to fall in love with the smoothness. In the real world, the smoothness has to survive inclement weather, bad signs, weird human decisions, and the kind of unlucky day that turns into an insurance case. But by the end of my ride, I saw what Nvidia’s execs keep circling: This didn’t feel like a sci-fi future; it felt like tomorrow. 

I kept thinking about that first Waymo problem — where to sit — because it’s the same problem the industry has with autonomy: When the old roles vanish, everyone has to renegotiate the rules in real time. Nvidia’s pitch is that the renegotiation doesn’t have to be existential. Autonomy can be purchased, integrated, tested, signed off, and shipped like any other critical system in a modern car. That’s a very Silicon Valley fantasy — and perhaps the most plausible way this stuff actually scales.