The self-driving car UI problem: what should

There’s a screen in the back of every Waymo that shows what the car “sees.” Other vehicles are blue boxes. Pedestrians are stick figures. Cyclists are outlined shapes. The road is a simplified map with the car’s planned path drawn as a line.

I find it mesmerizing. I watch it the entire ride. Every pedestrian detected, every car tracked, every lane change planned. It’s like watching the car’s brain think in real time.

My mom took her first Waymo ride last month. She watched the screen for about 30 seconds, then asked me to turn it off. “I don’t want to see how many things it’s watching. It makes me think about all the things it might miss.”

Same screen. Same data. Completely opposite reactions.

The trust spectrum

When you ride in a car with a human driver, you don’t see what they see. You don’t have a display showing every object in their visual field, every decision point, every potential collision they’re tracking. You trust them based on the smoothness of the ride, the confidence of the driving, and the absence of near-misses.

Waymo gives you more information than you’d get from a human driver. And more information isn’t always more comfort.

For me, seeing the LIDAR point cloud and the object detection is reassuring. It proves the car is paying attention. The comprehensiveness of its attention makes me trust it.

For my mom, the same comprehensiveness highlights the complexity. All those objects to track. All those decisions to make. All those things that could go wrong. The information made her anxious.

The design question

What should the screen show? Tesla shows a simplified view. Zoox shows a stylized cityscape. Waymo shows the raw detection output.

I think the answer is different for different people, which makes it a hard design problem. Some passengers want transparency. Some want simplicity. Some want nothing at all. Just a smooth ride and a destination.

The best answer might be adaptive. A “zen mode” that shows nothing. A “curious mode” that shows what the car sees. A “nervous mode” that shows only reassuring information: “all clear ahead,” “smooth stop in 3 seconds,” “arriving in 4 minutes.”

The technology of self-driving is a robotics problem. The experience of self-driving is a design problem. And I think the design problem is harder, because it involves understanding human psychology, which is messier than LIDAR.

My mom will take another Waymo. She told me the ride was smooth, the car was safe, and she never wants to see that screen again.

Fair enough.

Related thinking: