No, it's not. GPS is only running when you see the crosshairs icon in the notification bar. On Gingerbread, it was on the right side of the bar with other system indicators. On ICS, it flashes on the left side of the bar.
GPS (along with WiFi, Bluetooth, sync, and screen brightness) can be toggled via the built-in Power control widget or with a plethora of third-party widgets available in the Play store. The option to turn on and off the various aspects of GPS can also be found in the settings under Location services (which is the same place they were located on Gingerbread).
GPS is turned on by default after updating to ICS. That said, turning it off really won't save battery. It's a myth that just having GPS switched on will drain your battery. GPS will only use battery when it's active, and it's only active when the crosshairs are flashing in the notification bar. That will only happen when you have an app running that uses GPS. So, as long as you're not running something that's using GPS, leaving GPS switched on will not drain any more battery than leaving it switched off.
