I totally understand wanting the best specs. Lots of people have stated that Windows Phone doesn't really need a crazy processor, and I agree. I've had a Lumia 1520 for a month, and in day to day operations, I notice no performance improvement over my ATIV S with the S4. Now, on paper the S800 is a huge leap over the S4, but it doesn't make much of a difference (although I'm sure that it would really help the Lumia 1020 process photos faster).
Apps do seem to install faster on the 1520, and I've heard that web pages load faster in the broswer (I never really use the broswer, so haven't noticed). Like I said, day to day use such as email, facebook, wpcentral - the S4 matches the S800.
Whether that shows if WP8 truely is a lean, mean, optimized machine, or that WP8.1 simply isn't optimized to take advantage of the power of S800. Who knows. All I know is that whether your device is a WP8 launch device (Lumia 920, ATIV S, HTC 8x using S4), a Lumia 1520 / 930 with S800, or one of the many S400 devices - you are going to have a great experience with your device.
The only difference I've ever noticed is with the Lumia 520. I bought one for my Mom last year to replace her Nokia flip phone, and the 520 takes longer to boot up than the higher end Windows Phones, and I've noticed it take a little longer to perform a few tasks in the OS, but still a good experience.