If you open the music app and go to setting in the hidden menu bar at the bottom there is a setting "Xbox music cloud collection", turn this off to remove Xbox music you haven't downloaded from showing up in your collection.
If you are taking about the "cloud collection" - you cannot modify that on windows phone. The only thing that can modify the cloud collection now is music app on windows 8.
On the other hand, if the song is downloaded to the phone, you can delete it for sure. The problem is you need to delete the songs one by one. You cannot delete the whole album