So I am currently using a Lumia 630 for over a year and looking at all of these newer phones was quite hard for me without having one of those.
I have been reading other threads where 640 was compared to 650, but I am more into 640 XL vs 650 as I really want to buy a phone that will last a bit longer with it's specs (my 630 only has 512 MB of RAM). As far as I understood, the main difference between 640 and 640 XL is the screen size, rear camera and the battery life.
I have also seen many complaints about the Lumia 650, mostly about the processor and as far as I have seen, people who were complaning were gamers. Personally, I do not really game a lot on my phone, so that's not the deal braker for me, but I would be very thankful if the ones who used both 640 XL and 650 or have any kind of other expirience with these two phones could state their opinions here.
P.S.: I am not living in US and Lumia 640 XL here would cost about 300 € and Lumia 650 about 250€.