Don't want to have to carry around tweezers? How often are you needing to change the SIM? Usually you will only change a SIM when you change carriers, not on a whim.
Personally, I remove SIM cards with a thumb tack. The point is great at grabbing them, and only a very small amount of pressure is needed. You don't want to apply enough pressure to puncture the SIM, but it gets great traction.
I don't, however, carry one with me. Pointy blade of a pocket knife will do the same thing. Those are easier to carry with you, and many people carry them anyway, unless they are going to the airport.
I'm not sure how the 640XL SIM card slots are, but on the 635, you can actually push it back a bit from the opposite end to push it out some before removing. That usually gives you enough room to pinch it between your fingernails to remove.