That's the big thing. Forget antivirus. People really need to pay attention to those privileges. I found myself kinda ignoring them, but I pay more attention now after the recent scares.
It's hard to get people to pay attention, but I think that the privileges screen should at least be a little scarier. If the app asks for access to sd card, contacts, and internet I think there should be a clearer, sterner warning.
In the end, though, I don't care much if an app steals my contacts or sd card data. I don't have anything to hide. The one thing I really watch out for is what apps I put passwords into. The big one is the browser. I love Dolphin HD, but if I ever do anything sensitive (banking), I'm going with the stock browser. I use K9 for email but I trust those guys...I think.