This is such a subjective subject, and everyone will have their own opinion. There is, of course, no right answer other than what is right for you.
I have tried a number of sizes in hand to test them out, and have found that with my wide hands and short, thick fingers, that a 4" is the perfect size for me. Any larger and I'm constantly dropping it unless I use 2 hands while using it.
It doesn't however, appear to be big enough to do any serious browsing. However, a few quick peeks while I'm out and about is okay, but if I want to do anything serious, even a 6" screen isn't big enough for me. I prefer a larger platform for things most people do on a smart phone. 15.6" laptop minimum. I could see using an 11" tablet for casual browsing, or reading an e-book, but not for anything that requires me to enter text. For that, I want my full laptop. SP3 is even too small for me for that.