I owned a 1520, I currently own an 830, and I had a 640XL in my midst for an entire week, as I ordered one for my father (check my posts in the 640XL forum).
The answer regarding which of these to get depends on your specific use case. One of the reasons I sold my 1520 was the phantom touch issue as well. Mine wasn't even as bad as some others, but it was just annoying. Plus I just fell in love with the 830's aesthetics.
Now, to the 830 vs the 640 XL...
I got a chance to play with a glossy white 640 XL for a week, as mentioned above. I was impressed with it. Found no problems. No software issues (like random reboots), camera is very solid (especially outdoors), and things like Miracast (Project My Screen) work well. Performance is pretty much on par with the 830.
However, I had no desire to get one of my own to replace my 830 for a few reasons (in no particular order)...
- Design and build quality. The 640 XL's build seems well engineered and sturdy enough. It's actually quite light, which I'm not sure is always a good thing. The 920 for instance was also all polycarbonate but seemed a lot more solid. That thing can take anything you throw at it. I wonder if the difference here is that the 640XL was built under the auspices of Microsoft and not Nokia. As for overall design... the 640 XL looks a lot more like the average smartphone. The 830 is unique, with a beautiful aluminum frame and curved glass.
- Size. While I think the sweetspot is 5.2"-5.3" for a display size, I think 5.7" is a little too big. I actually have proportionate hand size for someone 6'1" and I didn't mind handling it, but the 1520 for instance was simply a behemoth and made pocketability (especially sitting down) almost impossible. The 640 XL is only marginally better in that regard. In fact the difference is negligible.
- OIS and low-light photography. This is the biggest weakness of the 640XL. While the camera is a lot sharper than its predecessor (Lumia 1320), low-light photography is just bad right now. Might not be a hardware thing... I'm thinking an update with some tuning to the automatic settings could fix this, but most 640XL owners seem to be complaining about it and it's the flaw that stood out most to me in testing it out for a week. I've come to appreciate OIS a ton since owning a 920, and not having it has become close to a deal-breaker for me.
- Wireless charging. It's just really convenient to be able to drop the phone on my wireless charging stand without fiddling with wires, and have it double as my nightstand clock with Glance screen.