I've had this issue as well. Even threw out a perfectly fine pair of headphones, because I thought they were broken. - I don't know what causes this, but in my case it's a software problem, because a phone restart always solves the problem. (Lumia 920)