Having the same issues with JellyBelly as with the Liquid Jelly Bean ROM. Back to ICS again...... BAH!
You're doing something wrong then. Do you have clockwork touch? Have you tried using superwipe? Your issues are most likely caused by lingering remnants of ICS.