You don't necessarily need to connect your device to your computer. You can use any of the file managers from the Windows Phone Store. Whatsapp media (images & videos) is stored in /pictures/Whatsapp. In WM10 that folder contains another subfolder which is holding all voice media.
I'm not entirely sure how it's managed in WP8.1. At least I don't remember the exact location of the folder. Could be just like in WM10 a subfolder of pictures, or directly in the root area of the phone's memory or SD card (depending on where you installed the app).