Follow these steps to delete the Windows To Go registry value:
Open regedit.exe
Browse to: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control
Delete this value: PortableOperatingSystem
You will most probably notice that the issue reappears when you connect a particular USB drive to the PC. Subsequent removal of that drive will cause the PC to crash. This is happening because the unique IDs of the USB disk and the internal HDD are the same. The easiest way to fix this issue is to "clean" the disk. A clean will destroy all data on the disk, so back it up first.
Open diskpart.exe
Type "list disk" to determine the disk number of the USB disk.
Type "select disk #", where # is the disk number you determined.
Type "clean".
Type "exit".
Restart your PC. You can use diskmgmt.msc or diskpart.exe to repartition your USB drive to resume using it normally.