I have the same problem in years. I have waited to see if there is any application improvement but apparently there is no more work to fix or update OneNote on Windows Phone, even a single iota! I bet Microsoft hired very low cost engineer to make this done, so until today we can see the worst mobile application design by the-world-most-hated engineer. And other Microsoft staffs never noticed the problem because they are using OneDrive with very high availability of internet connection. For short: they forget us!
You can remove notebooks, renaming means adding new one and syncing data to it. Pages/folders you can rename, delete it whatever. Are you using windows 10 version?