Waymo AI Driverless Car Goes Rogue: Blocks Traffic, Evades Capture | JJRicks Rides With Waymo #54
JJRicks0 HN recommenders · 2021–2024
554K views · demo · dated · Uploaded 2021-05-12
JJRicks rides a driverless Waymo in Phoenix when it gets confused by traffic cones, blocks traffic, and evades the support vehicles sent to retrieve it. The episode is one of a long series of rider-view Waymo videos.
Cited in self-driving threads as a documented case of a robotaxi failing at traffic cones, with some comments also noting the support team response.
54:37How Self-Driving Cars will Destroy Cities (and what to do about it)
Not Just Bikes
52:22Google Self-Driving Car Project | SXSW Interactive 2016
SXSW
30:09Andrej Karpathy - AI for Full-Self Driving at Tesla
Matroid
1:59:37George Hotz: Comma.ai, OpenPilot, and Autonomous Vehicles | Lex Fridman Podcast #31
Lex Fridman
4:30Traffic Has a Perfect Solution. Humans Are the Problem.
CGP Grey
3:08:46George Hotz: Hacking the Simulation & Learning to Drive with Neural Nets | Lex Fridman Podcast #132
Lex Fridman
40:37ai.bythebay.io: George Hotz, Self-Driving Lessons from Comma AI
FunctionalTV
3:28:48Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI | Lex Fridman Podcast #333
Lex Fridman
13 comments from Hacker News
At 30:18 there's a very interesting tidbit. [VIDEO] They way I understand that is the route planner, which I assume is ran completely separately to the self-driving car, removes un-navigable spaces from the route planning. So in this situation, the roadside assistance driver is surprised that the entire lane has not been removed from the route planner. That's pretty damning and it makes me wonder if there is a huge gap in autonomy here that is being papered over by clever route planning (ie. avoid that lane altogether and the AI won't have to worry about the cones). The AI should be able to figure out on its own that the right lane was closed, it shouldn't need route planning to do that for it.
[VIDEO] Here's a video of Waymo car in Phoenix. Around min. 17 it gets confused by a cone and stops in the middle of two lane road, hugging both lanes. Then google sends support but just when they arrive, the car drives away from them. They were lucky it didn't lead to an accident and as long as they keep the fleet to 600 cars then yeah, the accident rate will be much lower than Tesla's AutoPilot, shipped in more than a million cars. My point is not to rag on Waymo just to inject some reality. We don't have a choice between "safe and unsafe" way of developing self-driving software. We have a choice between "test software we know can't handle all situation on real roads and make it better based on that testing" or "we'll never have self-driving software". Except the other option will rather be: "U.S. doesn't allow testing of self-driving software on real roads and a Chinese company will develop it and will capture a trillion-dollar market in U.S."
It took a while, here's the video I watched that made me not want ANY Waymos near where I live: [VIDEO] I think the thing that made me so furiously mad is that there's all these situations in the video that people might laugh at, but there are situations where I've been stuck in traffic in an emergency and if I found out it's because Google was trying to make it's next pentillionth dollar I would blow my top. Can you imagine one of these during the Campfire evacuation?
I had no idea it was available anywhere to just anyone, and without a backup driver in the driver seat. It still isn't to me in the UK, but really cool to see, it's a lot further along (especially in terms of regulation/being allowed) than I realised. Couple of articles deep from the submission, blogger taking one for a ride (alone, in the back seat) in Arizona: [VIDEO]
Yeah Cruise announced this Walmart delivery with self-driving cars in Nov 2020 to launch in 2021 [1] and it is now available at the first location in Arizona. Cruise has been testing here but probably more on proving grounds like Apple has. Apple just bought a proving ground this year that they have been leasing since 2017 [2]. Testing picking up as well as competition. I am sure Cruise will expand it in Arizona. Most of the self-driving operations are in the East Valley (Chandler, Gilbert, Tempe) as the roads are pretty simple and square. The Scottsdale area this is in also pretty simple and by the reservation. Self-driving cars are everywhere with Waymo in Arizona as well, many entirely unattended. If you drive anywhere you see them. For Waymo samples, this JJRicks Studios channel [3] did a series on it for a while to show rides with no driver. Waymo does have a team that follows cars in trouble, on the infamous "cone" one you can see it [4]. For the most part the operation is solid, very few incidents like the "cone" incident where the cone was just in between LiDAR and CV detection and probably caused a covering of the line that confused the decision process. These types of issues are best to resolve in a baseline test with little change before the more challenging ones. [1] [LINK] [2] [LINK] [3] [LINK] [4] [VIDEO]
IIRC correctly in their Waymo videos you can see inside the cabin. Each passenger gets a video feed of that video, so I think it is real-time. Let me find a link. EDIT: This was posted above for different reasons, but you can see the render is real-time. [VIDEO]
I must say, the linked video in the article to the rogue Waymo car is like a hilarious Black Mirror episode: [VIDEO]
You can easily find videos of waymo cars doing crazy stuff that stretches the definition of "working" pretty far. E.g., [VIDEO] Nobody has time for shenanigans like that.
I don't understand why authorities allow self-driving cars to beta test out in public. I thought waymo was the best with something like millions of hours on the road but when they started accepting passengers, it failed to proceed past a traffic cone: [VIDEO] I wouldn't trust any of the others like Tesla and Uber that think self-driving is easy. And Tesla with their cost-saving no LIDAR nonsense. These companies should be fined hefty amounts and barred from testing out in public.
I've said it before when a self-driving Uber killed a woman, and I'll say it again: I don't understand why authorities allow self-driving cars to beta test out in public. I thought Waymo was the best with something like millions of hours on the road but when they started accepting passengers, it failed to proceed past a traffic cone: [VIDEO] I wouldn't trust any of the others like Tesla and Uber that think self-driving is as easy putting it out on the road. And Tesla with their cost-saving no LIDAR nonsense. These companies should be fined hefty amounts and barred from testing out in public.
I've said this previously when a self-driving Uber killed a woman, and I'll say it again: I don't understand why authorities allow self-driving cars to beta test out in public. I thought Waymo was the best of the lot with something like millions of hours on the road and no incidents, but when they started accepting passengers, it failed to proceed past a traffic cone on a sunny day: [VIDEO] The government should initiate a comprehensive test on a self-driving car's capabilities. In their current form, I wouldn't trust any of the others like Tesla and Uber that think self-driving is as easy putting it out on the road. And that too with Tesla and their cost-saving no-LIDAR nonsense. These companies should be fined hefty amounts and barred from beta testing out in public.
> I also don't recall traffic cones being a very difficult problem beyond Cruise + cones on windshield in SF. I have no love for Cruise. But its straightforward to pause if there's a large object on the windshield. Waymo had a big mistake with cones where a lane was blocked off for construction and they thought it was the inverse and started driving down the blocked off side: [LINK] full video: [VIDEO] In the statement before that timestamp they say some of it was caused by remote operator error. But it seems to have enough problems with cones to need an operator in the first place, and you can see the planner is wrong. I think Cruise would be using a human or at least summoning one to oversee in any situation with unexpected cones, based on what they have said about how often they use remote assistance too.
Google's AI is just as bad as the rest. I remember when it couldn't get past a traffic cone after not being in an accident for "countless hours": [VIDEO] These companies should be fined hefty amounts and barred from beta testing out in public and putting people in danger. Governments should carry out comprehensive tests on a self-driving car's capabilities, same as cars without proven passenger safety (Euro NCAP) aren't allowed to be on roads carrying passengers. And now this great AI company is also providing AI services to pariah states like Apartheid Israel to be used on people. That's scary.