A lot of it really does boil down to what you're used to. I jumped on the android train when the first one came out, even though the iPhone was available already (not on my carrier of choice, however).
I'm an IT guy and the guy who everyone I know calls for support with a large variety of devices, but I do my best to stay away from iDevices. When I do end up having to help someone do something on an ipad or iphone, I feel the pain you mention, but I'm convinced that that's only because things aren't where I feel they should be and/or don't work like I feel they should. But, if I had chosen the other path, I think I could see the same issue with android.
I do, however, believe that my own personal tastes would've led me to android eventually anyway, but I guess I'll never really know.