Funny thing is: I was "sharing my Internet", but I did turn it off after I was done. Also, I never noticed that this icon was there when I tried Internet sharing... I never "pulled down" the notification area. The icon did not go away though after I was done. That's why I was curious... I did check if Internet sharing was on/off. I restarted my phone, and now it's gone.
Funny, I see no mention of this icon in the user guide.
It won't switch off as long as the device using it is still connected, your iPad, laptop or whatever it is. Best to manually switch it off so you don't run up a bill.