Of the Lumia/Microsoft phones you can still buy, the 640 is an 8.1 phone so you would need to upgrade to W10, the 650 is a good all rounder but doesn't have Continuum, the 950 and 950 XL are top end spec phones with Continuum, as mentioned there are now other makers like the very powerful but very expensive HP Elite.
If you aren't a massive app user (And you will be surprised how good the MS supplied apps are) then apps really don't cause an issue, I have a banking app can control my lights, check the stars, there are plenty more but you don't have the choice you get on some other platforms.
And as mentioned, there is no need to go on insiders, I used to be, but I am now back to production build, yes it means I am a couple of months behind some users on some parts of the OS/apps, but I don't have the issues they have when bugs get introduced.
I really am liking where MS are going with the OS now, and the integration of my phone with my PC is fantastic and getting better, the latest Skype preview (Fast ring insiders) allows Skype to become your fully integrated message centre, so SMS and MMS will show up and be send-able from your PC using your phones allowances.