The UI is just one reason, but obviously an important one. Interacting with a UI made for mouse and keyboard with your finger is a major pain in the rear, even on a large screen (a finger can't hover over an element like a mouse, there is no right mouse button, etc) . On a small phone sized screen it's entirely unusable. It's only viable when hooked up to a larger screen with the required peripherals. Period.
There is also the issue of expectations. Mobile OSes are basically maintenance free. Anybody can use them without requiring any formal understanding of IT whatsoever. That is simply not true of desktop OSes. People dislike the maintenance-heavy and complicated desktop OSes for a reason. The simplicity of mobile OSes is why consumer's have come to prefer mobile computing devices. Putting a desktop OS on a phone is guaranteed to annoy the majority of consumer's for those reasons. The second the average consumer is told to update a driver, use a registry cleaner or reinstall Windows on their phone sized device (even if MS sells it as something different), is the second most consumers will throw W10-on-a-phone (at least mentally) in the trash bin.
Security is another issue. A security breach on a Windows PC is already bad enough. That problem is compounded on a device that is loaded with sensors, used for payments and has precise tracking capabilities. That provides a gateway to acquire far more personal information than what can typically be cleaned from a compromised PC. W10 is a very insecure compared to W10S or W10M, so as an OS, it's simply not a good match for such a personal device.
There are more reasons, but I consider those the most important.
Technically, putting W10 on a phone or phone-like device poses no problem whatsoever. The problem is that it's just fundamentally a bad idea. On a side note, that should also make us a bit skeptical of the potential behind the upcoming W10 on ARM, since it's exactly that.