After I check the tags (ID3V1 and ID3V2), I can't find anything wrong. I don't see any capital letter at the end of any field. To verify the tags, I use the ID3-TagIT V3.3.0. Here are some pics.

T Moore, could you tell me what software are you using to edit the tags?
BTW, the last Xbox Music update don't fix this issue.
EDIT: I just noticed another thing: Once I copied the folder with the songs in to the music folder, when the artist index is updating, all the index becomes a mess. Some letters disappear, some artist are merged in different letter and some artist just disappeared