They actually are really well camouflaged (Google made it a little too hard to find them, imo). In GMail app, Menu --> More --> Settings --> select account --> Labels to notify. Select your label (should be Inbox and Priority Inbox) and set the ringtone to silent (make sure you do it for all labels available). I have mine set to Silent, Never, Notify once for new email (means the ringtone is Silent, and vibrate is set to Never). This will make it so GMail emails show up in your notification bar (and trigger your notification LED), but they never make a noise or vibrate the phone.
Thanks Zandar. That was very helpful.