i have this problem all the time (the error mentioned in this issue was only related with this).
It has to do with windows handle sleep() method, you has say sleep(1) for 1ms but on my
machine it waits 15ms (it depends on your hardware, other PCs behave different!). So i tested around with high performance counters.
For now i ignore that problem an set jitterbuffers on other device big enough.
Thomas