Go to settings>speech if you're using WP8.
There, you'll find some settings, but...
No way to turn it off altogether.
Yep, you heard that right.
I checked, and I couldn't find anything.
Guys, the question is obviously about the speech recognition that happens when you hold down the Windows button. He said "is there a way to get rid of that pop up screen every time it activates by accident". There's no other speech-related functionality that shows a pop up screen and activates by accident.
The answer is no, there's no way to disable this in the speech or ease of access settings. It would be nice if there was. The closest you can do is to disable the setting "Use Speech when the phone is locked" in the speech settings.