re:
By far the easiest way to manage your Calendar is on the web via a PC browser. Log into your outlook.com, select Calendar from your list of services...
Whether it is on the phone or via browser, selecting "More Details" when creating or editing an entry gets you to the Reminder field.
If that is set up and you are not getting notifications, check Settings > ringtones + sounds and make sure something is set up in the Reminders field.