I had the same opinions on my N7, I have a 16 GB wifi model. On the first long flight I took with it, I spent 3-4 hrs watching video (sitcom episodes in MP4 format in an AVI container, playing via VLC Beta) and maybe 2 hrs reading using the Kindle app. I found that I had ~35% battery left at the end of it. That led me to believe that it was the radio taking up most power, and now I leave the tablet in airplane mode most of the time. My usage is mostly watching local video and Kindle books, so it may not fit your usage patterns. I haven't faced this Media Server issue, but I have no music on the tablet.
Re my N4, after 14 hours outside the house with light usage: 15-20 mins GPS navigation, maybe 30 mins of voice calls, lots of time checking Facebook, reading email and occasional light gaming for maybe 60 mins, I still have 40% battery left. It's quite satisfactory, have never had a reason to complain,