Wow, this is a tough one. Comes down to personal choice. What you enjoy another will find annoying.
Size wise.............I still own the Sony Xperia Z Ultra. NOTHING can compare to that for size, I think its not even a phablet but tablet! :grin:
The question of screen size is always going to be a hot potato. I don't find the 640XL big, I find the 640 small? The positive of the Sony mentioned is its beautiful large screen, its biggest negative its the size of that beautiful large screen! Its just not pocket friendly. I find the 640xl as big as I care to go. Decent size screen while being pocket friendly. I also own the samsung Galaxy S6 Edge+ and that is also a large mobile but again pocketable.
What I suggest is look at the ratio of screen size to body and ensure the maximum possible screen size is packed into the smallest possible body like the SG S6Edge+, vis: Sony ZUltra 68.3%. SG S6Edge+ 75.6% and the 640XL 69.1% screen to body ratio. The higher the ratio the better as this signifies more screen real estate in a smaller body.
It really is choice. If you have the opportunity to get to a phone shop with a large display that you can handle and do side by side comparisons this will so often make your mind up for you. One more thing, its not only the size that counts but how the mobile sits in the hand, comfort is key to the user experience. I could NOT use my SG S6Edge+ without a cover as the sides are so thin they are most uncomfortable to hold. I would 'think' you would find the 640XL a delight to hold? Again it realy is your choice that counts.
Very best wishes, Ryland