You're responsible now if you wreck your car and are found to be at fault. It's the same thing to me. Having the car drive itself is an aid or a tool to me, not something to completely take over for common sense.
Let's look at this from another angle that's been debated recently. If the car works as it's supposed to and it can read situations...Let's say you're in your driverless car and it's taking you to work, you get in the situation where either the car can hit another car or a group of people in a crosswalk or it can crash you into a guardrail. In this scenario, someone has to die. It's either you, or multiple other people. It's been tossed back and forth whether the car should choose to kill you (1 life) or multiple other people if there is absolutely no other option.
So, let's say these cars are programmed to kill the least amount of people in a scenario like that. Let's also say that we all know this when buying one of these cars. We also all know that there's an ability to override.
Now, let's say you find yourself in a situation like I've described above, but instead of letting the car make the decision, you decide and act. You override the car's programming and kill 4 people to save your own life.
In that situation, would you blame the driver/passenger of the car for overriding the car's programming and killing those people?
Keep in mind, for this scenario, there was no other option. Whatever happened, you found yourself suddenly in this situation where it was your life or these other people's lives and there wasn't anything else that could happen. Maybe you came around the corner and the car was going the proper speed, but there's this crosswalk and these people decided to go even though the sign said not to, or they're jaywalking. Either way, are you to blame for killing those people, knowing that everyone knows the car should've killed you based on it's programming?
Because I don't know how you couldn't hold that driver responsible for that scenario. And I can't see the difference between choosing to act and override the car's programming, resulting in death for other people and choosing NOT to act, letting the car have complete control, and getting in some other accident where your car kills other people.
Maybe it's just me.