Ok so I've done a FDR and did not let anything restore... Re-installed all of my apps manually and reconfigured my settings.
I must say that the FDR without restoring any apps automatically tremendously helped my battery life. I'm at 5 hours now and at 70% battery. That's with about 30 minutes of music playing, 15 minute of phone calls, about 5 minutes of texting, and maybe 10 minutes of searching (with 4G). I'm assuming that's pretty good.
I doubt that I'll ever get 2 days and stuff like most of you, but whatever -- at least it has improved. Also, I did this FDR yesterday so, like others have stated, it may just need a few days of battery training. Other than what I've already done, I'm not about to invest anymore time in trying to optimize the battery -- I bought the phone to actually use it instead ^_^.