On Google Maps if you have your "Home" address filled in it will use that also. ( I just hate to have my HOME address listed in case someone gets my phone)
If you use a screen lock and trusted places\devices, no one can get into your phone to see where you live, brother.