Yea, the Xbox Music+Video app is severely lacking and quirky. But visually stunning when working properly.
I've just spent the last 3 days experimenting with how to get music on the phone with a SD card cleanly without duplicates, etc. Basically you want to remove all your music from the phone first and start over. Trying to clean it up any other way just doesn't work. You also want to make sure that there isn't any music on the Phone side either.
Plug the phone into a computer and look at it via Windows Explorer. You should see the phone under the computer, and you should see "Phone" and "SD Card", both will have a folder named "Music", make sure that there isn't any music in those folders. Then after that, make sure there isn't any music in your collection in the music app.
Now the fun part, getting your music on the phone. The BEST way I found was to take the SD Card out of the phone, and put it in my computer. Then literally "drag-n-drop" my music collection into the "Music" folder on the SD Card. Then take the card back to the phone, and let it sort through all the songs and rebuild it's internal database of your music.
If you don't have the ability to connect your SD Card directly to your computer, the next best way is leave the SD Card in the phone, connect it to your computer via USB and again "drag-n-drop" your music via Windows Explorer into the Music folder of the SD Card. I found this be slower than above, mainly because the phone checks each file for some reason.
Again, you will have to wait awhile for the phone to index all the files into it's database.
Using good old Window Media Player, or the Windows Phone Program (beta), is a bit of a crap shoot on how it turns out on the phone. I haven't tried the Windows 8 program, since I still running Windows 7.
Now I did run into a situation where no matter what I tried, the app on the phone didn't see any music what so ever (this was after several experiments with messing around with the contents of hidden folders on the phone). Rebooting didn't work. The only thing I could do was to reset the phone back to factory, and starting over.