You're misreading it, that's just the Location Icon that's on not Location Services. You need to turn Location back on, switch off the icon and the turn Location off again.
The toggle for the icon is still on though and short of turning off ALL the radios, Cellular and WiFi and turning off the GPS, it will keep trying to get a location fix. If that icon is there permanently then you obviously have a background task that needs your location.
As it states in the description all the toggle does is turn off location services for apps and services that require it, it doesn't stop the phone from finding your location.