Admittedly, my view is heavily biased by what I know about flying airplanes. But as autopilot systems in airplanes are the closest thing in technology and practice to driverless car systems, it seems a good comparison.
On liability, I'm just working backwards to consider the limiting factors. All industries are limited by regulation/license on the governmental side and insurance on the private side. While safety is an operator/best practices concern, safety is regulated by these two forces. In the US, a personal driver's license is easy to obtain and renew and allows the holder to operate most vehicles except those of a certain size/weight/wheel structure. Adding complex technology to the use of a car is not considered by States administering PDLs. One option on the State side is to create a licensing requirement; maybe something as simple as requiring a commercial driver's license to operate a driverless car.
On the insurance side, individual underwriters can create their own requirements. I'm rated by the FAA to operate most single or multi-engine non-turbojet aircraft with a gross weight less than 12,500 lbs (i.e., by license I can fly a
King Air). But I could never get insurance to fly one with initial and recurrent training specific to this aircraft that is acceptable to the insurer.
On the operations side, if I take controls of an aircraft of any level of complexity, I am the responsible entity (pilot in command in FAA terms, since most aircraft have duplicate controls unlike cars). No matter what systems I employ to aid my travel, I am ultimately responsible for myself, the aircraft, passengers and cargo, and any action I take with such. In less complex aircraft, the autopilot system does little more than hold heading and altitude. In more complex aircraft, all that's left to do between takeoff and touchdown is talk on the radio. Never is the responsibility of safe travel removed from the pilot in command, however, because of any technological complexity.
I see the same in cars (i.e., no drinking and driving or any other action, including the various sorts of breastfeeding, that would otherwise incapacitate or prevent the designated driver from performing his/her role). Liability in an airplane is pretty straightforward: the commercial operator and pilot in command bear chief responsibility. Manufacturers seldom bear primary responsibility in airplane crashes.
The manufacturer defect problem may be rather difficult; how do we determine whether the car veered because of automated or manual input? Other liabilities are rather straightforward--a person who is not capable of being responsible for such a vehicle should not be in charge; any person deemed responsible for such a vehicle is therefore responsible and liable. On the driverless Johnny Cab, the safety/liability issue is determining who is in charge/responsible. When I get a cab at O'Hare and sit in the back seat, responsibility is mutually agreed to fall on the driver. Similarly, when I ride a subway/train, responsibility is both on the commercial operator (transit authority that maintains the trains/tracks) and vehicle operator. I'm only responsible for not falling onto the tracks when I get in and out.
I don't disagree that there are too many deaths on the roads due to unsafe driving and that we should move forward to "driverless" cars; I think that the method for moving forward has been largely discovered by other industries and should be applied similarly in the auto industry.