Why is my plastic dust plug charm putting my phone into headphone mode?

[BudgiePhone228]

I got this kiwi plastic dust plug charm for my windows phone but it's putting my phone into headphone mode. My ringtone still works when somebody calls me but when I go to answer the call I can't hear anybody until I take it out. Is there anyway I can fix this?

 

[Guytronic]

The dust plug is probably a bit too long and opening the phone jack contacts.

 

[BudgiePhone228]

Hmm it's 3.5mm jack and i have a Nokia lumina 521 which has a 3.5 mm audio connector so I'm not sure that's the problem.