I wouldn't say I'm an audiophile or anything, but can hear the differences between good quality music/playback and bad.
The Lumia is perfectly fine in this respect in my opinion. Certainly nothing glaringly wrong with it, no hissing, crackling or popping and both the Zune and Nokia Music provide a nice environment to launch music from.
If you're very concerned about it though it would be worth your while trying to find a display model so you can test it for yourself.