I have a server box with Windows Server 2008 R2 and no audio hardware. I have installed the 'Desktop Experience' feature, and if I plug in a USB headset audio works on the console.
However when I use a TS session to the machine from Windows 7 there is no audio and the icon on the notification area shows 'no audio output devices installed'.
Terminal Server Device Redirector driver is a windows driver. Common questions for Terminal Server Device Redirector driver Q: Where can I download the Terminal Server Device Redirector driver's driver? HDX Plug and Play USB device redirection. HDX Plug and Play USB device redirection enables dynamic redirection of media devices, including cameras, scanners, media players, and point of sale (POS) devices to the server. You or the user can restrict redirection of all or some of the devices.
How can I make audio work on TS sessions?
John
3 Answers
Terminal Server Setup
In RDP settings, under Local Resources tab, click 'Settings...' button in Remote audio section and check that 'Play on this computer' is selected.
Simple solution we used at work for this issue: cheap USB audio dongles from eBay, less than £2.50 apiece. Stick one in the back of the server and it now has an audio device that it can redirect.
Terminal Server Device Redirector Driver

It sounds like the USB headset is the audio adapter, as well as being a headphone/mic.
If you're unplugging the headset from the server before you RDP into it, then there's no audio device to redirect.
Terminal Server Install 2012
If you are leaving them plugged in, and it doesn't redirect it then it's not an audio device that terminal server recognizes as compatible with redirection.
If you are determined to have audio available for redirection perhaps look into a more 'permanent' audio adapter for the server. Basic PCI(e) sound cards can be had for $20-30 pretty easily these days.
Hope that helps.