Add people to appointment without inviting

[MBY]

I frequently save info in People for offices I contact frequently (like my doctor's office). When I add an appointment, I'd like to be able to add their info to the appointment, maybe even have the location fill in from their address. But I don't want to email them an invite. In fact, for these offices I usually don't even have an email address saved, so they don't come up on the "Add Someone" list.

Are there any work-arounds for this? Is there actually any benefit to adding someone to your appointment if you aren't sending invites? For example, does the appointment show in the history for that contact?