[Freeswitch-users] Conference delay increasing over time

I have a very well specified Windows Server connected to sound card input via mod_portaudio. I want multiple callers to be able to listen (only) to the incoming feed, so I set up dynamic conferences, with the dialplan flags setting mute. An autocall bridges the portaudio. My assumption is that this is the best way. I am happy to be wrong if there's a better plan. The problem is that delay builds up, seconds every minute on the conference. I have RTP-Timer-Name=none and enabling both RTP AutoFlushes. Any other suggestions I could try? Would CentOS have better timing? I don't need callers to hear each other, but I'm sure that I'm missing something. Any thoughts? Erik