basically it depends ... on MANY things..
a) how much you care about your voice quality
b) how many concurrent calls
c) do you modify the RTP stream or just pass it through.
basically if your real picky or have more than a handful of concurrent calls.. then dont do it.
if your using conferences etc... dont do it.
that being said, you can get away with a lot if you dont mind the occasional blip and chirp in your media..
Thing to not though... above is for virtualization ... ( ESX, Xen, etc )
Jails like OpenVZ, BSD Jails, etc work well and provide many of the benefits found in virtualization.
Jay