Hi Daniel,
I'm running FreeSWITCH on KVM (Debian 7 Guest / Proxmox hosts and RHEL 6 hosts) without problems, even conferences and softfax (stuff where I assume timing is more critical) works very well. I exercised some caution when configuring the VM though, I don't know if all those things are necessary, best practice or even useful, but they result from some experience with strange timing in VMs:
I have set up mod_timerfd, although I don't know if it's any different from using enable-softtimer-timerfd in core:
freeswitch@default> show timer
type,name,ikey
timer,soft,CORE_SOFTTIMER_MODULE
timer,timerfd,mod_timerfd
My guest kernel supports linking the guest clock source to the host, so I don't have any issues with clock drifting away and don't need NTP in the guest. Maybe this also affects FreeSWITCH's ability to keep time?
root@freeswitch:~# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
kvm-clock
On systems that don't need the CPU power, I make sure only one virtual CPU exists and/or pin the guest CPUs to host CPUs.
Maybe the most useful thing to check is to switch the virtual NIC in KVM, I usually try to use virtio where possible, but hat strange issues before (like a DHCP Server VM not seeing DISCOVERs with virtio NIC while the same VM with e1000 worked fine). So, maybe try to switch NIC type and see if it helps. Currently I'm using FreeSWITCH on both e1000 and virtio without issues.
Regards, Markus