Somehow this all reminds me about the importance of using the right
software configuration (and hardware) for handling real-time audio.

See for example:
http://manual.ardour.org/setting-up-your-system/the-right-computer-system-for-digital-audio/

And since you're using ubuntu:
https://help.ubuntu.com/community/UbuntuStudio/RealTimeKernel

As the ubuntu page describes, the normal ubuntu stock kernel is
different compared to the studio version.

More real-time related information, links, articles and such:
http://wiki.linuxaudio.org/wiki/real_time_info

I hope this information may assist you into solving your issue.
--
EL

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com

Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org