Re: [Freeswitch-dev] Questions regarding the architecture
Anthony, Great answers indeed. Thanks a lot! Is there a documentation system that I can save it to ... like a wiki? These information are too valuable to be buried in the mail archive. It took me a while but I think I have come to agreement for pretty much every point you made, except one. The only thing I felt suspicious is the busy waiting in which the originator polls check_channel_status() a million times after the originatee starts ringing but before it answers. Can't we wait for a mutex or condition waken up by CALL_ANSWERED? Imagine if you have a thousand incoming calls, there will be 1000 while loops spinning the processor. Another issue I noticed is that hold/resume is not working properly. There was no 200OK for the re-INVITE. I tried to debug further but somehow couldn't get into mod_exosip.c using gdb. (DDD and Eclipse behaved the same. Does anybody have the same issue?) I kind of guess the issue is what kind of SDP is the SIP phone sending for hold. The old way is to set an all zero O field, as opposed to SENDONLY or INACTIVE. Anyways, if mod_exopsip.c is going way, we should probably just wait for mod_sofia. Two more basic questions: 1. Licensing. A main problem I saw with Asterisk is it's GPL/Digium dual licensing scheme. I don't see any serious business development could be possible under this license. I think Mark and his company have made a big mistake on this subject. (And their business model as well.) What's the main rationale for you/other founders to choose MPL? Do you favour/oppose to commercial development using freeswitch in their proprietary systems? 2. Memory usage. I am seeing freeswitch using 25M bytes of memory with just one basic SIP call. It's a little surprising. Should I turn on the -Os to see if we can save some space? PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25295 root 0 -20 27040 25m 3696 S 0 2.5 0:00.12 lt-freeswitch Thanks, Alex ________________________________ From: Anthony Minessale [mailto:anthmct@yahoo.com] Sent: Wed 9/6/2006 2:01 PM To: freeswitch-dev@lists.freeswitch.org Subject: Re: [Freeswitch-dev] Questions regarding the architecture 1. Thread architecture. For a basic call, why do we need two threads for the sessions (one for each endpoint)? While the thread in charge of originator is in the switch_core_standard_on_execute(), the other thread is in the switch_core_session_run() while loop. What's advantage of doing this compared to Asterisk's one thread approach? Why do we need the thread for the originatee? A) Every session has a thread which is managed by the state machine in the core which sees it from creation to destruction. In the case of a bridged call the existing thread of both sessions are used to move inbound audio across to the opposite session at the same time. A one threaded approach such as asterisk will leave you doing poll's on 2 sockets at once and blocking in one direction will impeed the other as well as forcing you to expose something that can be polled, (not all channels have sockets) The states of the orignatee channel are overridden from the default behaviour to perform the bridge in this case: CS_HOLD pauses the thread. CS_RING moves the state to CS_HOLD where it will wait until it's moved to CS_LOOPBACK or CS_HANGUP by the originate function. CS_LOOPBACK runs the audio bridge code sending the audio back to the other channel. The originator is in CS_EXECUTE meaning it's running instructions installed by a dialplan module. At this point both channels are in the same code reading A and writing B as well as moving messages and dtmf across one another. If either is interrupted the functions exits in both sessions and the originatee state override table is cleared and the session is moved to the new state where the execution continues. if this state is hangup the call ends. 2. What does CS_LOOPBACK really mean? When A calls B, B is in the LOOPBACK state after the call is established. Yet, the switch_core_on_loopback() is not really called. A) The default for loopback is to echo the audio back to the channel, in the case of a bridge it's overridden to send it audio to the other session instead of itself. 3. When A calls B and before B answers, the caller session thread does busy waiting by constantly calling check_channel_status(). Is there a better approach than this? And with this architecture how are we going to incorporate with the voice mail and other timer-based applications? A) Yes this is the best approach, the originator has created the originatee and is waiting for the endpoint module to place it into the CS_RING state when this happens because we overrode it we move it to CS_HOLD and wait for it to be answered or for a timeout to occur. The point is we have waited for it to be safe for us to control. 4. When the orginatee hangs up, the exosip_kill_channel() was called 3 times just for the orginatee channel. The same function is called twice for the originator channel. Is this by design? A) The should be as few calls to this as possible once it's ok to have mutiple calls to it it just calls for any blocking to end on a read or write operation such as the socket api call "shutdown()" does We are not using exosip anymore, we have started coding mod_sofia which is our preferred sip code. There are several changes comming up this week you may want to study. Thanks, Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com JABBER:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@66.250.68.194 iax:guest@66.250.68.194/888 googletalk:freeswitch@gmail.com pstn:712-432-7800 ----- Original Message ---- From: Alex Guan <alex.guan@prodeasystems.com> To: freeswitch-dev@lists.freeswitch.org Sent: Wednesday, September 6, 2006 10:13:43 AM Subject: [Freeswitch-dev] Questions regarding the architecture Gang, I am new to freeswitch and have been studying it for the last week. A few questions regarding the architecture. Any input will be greatly appreciated. 1. Thread architecture. For a basic call, why do we need two threads for the sessions (one for each endpoint)? While the thread in charge of originator is in the switch_core_standard_on_execute(), the other thread is in the switch_core_session_run() while loop. What's advantage of doing this compared to Asterisk's one thread approach? Why do we need the thread for the originatee? 2. What does CS_LOOPBACK really mean? When A calls B, B is in the LOOPBACK state after the call is established. Yet, the switch_core_on_loopback() is not really called. 3. When A calls B and before B answers, the caller session thread does busy waiting by constantly calling check_channel_status(). Is there a better approach than this? And with this architecture how are we going to incorporate with the voice mail and other timer-based applications? 4. When the orginatee hangs up, the exosip_kill_channel() was called 3 times just for the orginatee channel. The same function is called twice for the originator channel. Is this by design? Thanks, Alex _______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev <http://console.mxlogic.com/redir/?5dOWbXb8USC-yejhjhdw0PVkDjDo_Z34FS9Y01MjlS67OFek7qUX7_EoBeNcKxVATvzhOYMOMyOYeodECTo_Z34FS9BQfcBO5mUm-wafBitetz_QciDoDM04Sy-yMCrhhv7fK6zBxUQsCTNNI5-Aq83iSbN_W69jIjbEupdbFEw0tLWjBm53h17OeNgB0yq89A_d46NIjBfXjrzVKvxYYmfSk3q9JASC-yejKYUr3wsm> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev <http://console.mxlogic.com/redir/?5dOWbXb8USC-yejhjhdw0PVkDjDo_Z34FS9Y01MjlS6lmDaIaoX7_EoBeNcKxVATvzhOYMOMyOYeodECTo_Z34FS9BQfcBO5mUm-wafBitetz_QciDoDM04Sy-yMCrhhv7fK6zBxUQsCTNNI5-Aq83iSbN_W69jIjbEupdbFEw0tLWjBm53h17OeNgB0yq89A_d46NIjBfXjrzVKvxYYmfSk3qpJASC-yejKYUr3wsm> http://www.freeswitch.org <http://console.mxlogic.com/redir/?5dOWbXb8USC-yejhjhdw0Do_Z34FS9Y01MTvzhOYMOMyOYeodECTo_Z34FS9BQfcBO5mUm-wafBitetz_QciDoDM04Sy-yMCrhhv7fK6zBxUQsCTNNI5-Aq83iSbN_W69jIjbEupdbFEw0tLWjBm53h17OeNgB0yq89A_d46NIjBfXjrzVKvxYYmfSk3r9JASC-yejKYUr3wsm>
On Sep 7, 2006, at 2:26 PM, Alex Guan wrote:
Anthony,
Great answers indeed. Thanks a lot! Is there a documentation system that I can save it to ... like a wiki? These information are too valuable to be buried in the mail archive.
We have trac.freeswitch.org or voip-info.org (I think we prefer it being on trac)
It took me a while but I think I have come to agreement for pretty much every point you made, except one. The only thing I felt suspicious is the busy waiting in which the originator polls check_channel_status() a million times after the originatee starts ringing but before it answers. Can't we wait for a mutex or condition waken up by CALL_ANSWERED? Imagine if you have a thousand incoming calls, there will be 1000 while loops spinning the processor.
Anthony will need to answer this one. ;)
Another issue I noticed is that hold/resume is not working properly. There was no 200OK for the re-INVITE. I tried to debug further but somehow couldn't get into mod_exosip.c using gdb. (DDD and Eclipse behaved the same. Does anybody have the same issue?) I kind of guess the issue is what kind of SDP is the SIP phone sending for hold. The old way is to set an all zero O field, as opposed to SENDONLY or INACTIVE. Anyways, if mod_exopsip.c is going way, we should probably just wait for mod_sofia.
We still have work to do on sofia-sip to bring it up to speed with all the features we want.
Two more basic questions: 1. Licensing. A main problem I saw with Asterisk is it's GPL/ Digium dual licensing scheme. I don't see any serious business development could be possible under this license. I think Mark and his company have made a big mistake on this subject. (And their business model as well.) What's the main rationale for you/other founders to choose MPL? Do you favour/oppose to commercial development using freeswitch in their proprietary systems?
The MPL allows you to commercialize the code. But Anthony can elaborate on this subject more than I can. I don't want to speak on his behalf.
2. Memory usage. I am seeing freeswitch using 25M bytes of memory with just one basic SIP call. It's a little surprising. Should I turn on the -Os to see if we can save some space?
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25295 root 0 -20 27040 25m 3696 S 0 2.5 0:00.12 lt- freeswitch
You can try it out and see. Also look at what modules you load. When I start freeswitch up on my mac I use 6 or 7 megs of ram.
Thanks, Alex
_______________
Brian West wrote:
On Sep 7, 2006, at 2:26 PM, Alex Guan wrote:
Two more basic questions:
1. Licensing. A main problem I saw with Asterisk is it's GPL/Digium dual licensing scheme. I don't see any serious business development could be possible under this license. I think Mark and his company have made a big mistake on this subject. (And their business model as well.) What's the main rationale for you/other founders to choose MPL? Do you favour/oppose to commercial development using freeswitch in their proprietary systems?
The MPL allows you to commercialize the code. But Anthony can elaborate on this subject more than I can. I don't want to speak on his behalf.
The Open Source Initiative (OSI) is actively working to reduce the number of licenses. For instance we have deprecated the Jabber Open Source License (which was quite similar to MPL, but more friendly to companies since it reduced some risk regarding warranties etc.): http://www.opensource.org/licenses/jabberpl.php But perhaps a BSD-style license is a little too loose for FreeSWITCH? Peter -- Peter Saint-Andre Jabber Software Foundation http://www.jabber.org/people/stpeter.shtml
The reason that I started putting stuff on voip-info and will continue to put it there is that you have to login to get to trac. I do think it's too much to ask to have people register to get documentation on a project that they are trying to figure out if they care about being part of. And no-one will even know there is any documentation unless they are registered, nullifying most of the usefulness of the content. On 9/7/06, Brian West <brian.west@mac.com> wrote:
On Sep 7, 2006, at 2:26 PM, Alex Guan wrote:
Anthony,
Great answers indeed. Thanks a lot! Is there a documentation system that I can save it to ... like a wiki? These information are too valuable to be buried in the mail archive.
We have trac.freeswitch.org or voip-info.org (I think we prefer it being on trac)
It took me a while but I think I have come to agreement for pretty much every point you made, except one. The only thing I felt suspicious is the busy waiting in which the originator polls check_channel_status() a million times after the originatee starts ringing but before it answers. Can't we wait for a mutex or condition waken up by CALL_ANSWERED? Imagine if you have a thousand incoming calls, there will be 1000 while loops spinning the processor.
Anthony will need to answer this one. ;)
Another issue I noticed is that hold/resume is not working properly. There was no 200OK for the re-INVITE. I tried to debug further but somehow couldn't get into mod_exosip.c using gdb. (DDD and Eclipse behaved the same. Does anybody have the same issue?) I kind of guess the issue is what kind of SDP is the SIP phone sending for hold. The old way is to set an all zero O field, as opposed to SENDONLY or INACTIVE. Anyways, if mod_exopsip.c is going way, we should probably just wait for mod_sofia.
We still have work to do on sofia-sip to bring it up to speed with all the features we want.
Two more basic questions:
1. Licensing. A main problem I saw with Asterisk is it's GPL/Digium dual licensing scheme. I don't see any serious business development could be possible under this license. I think Mark and his company have made a big mistake on this subject. (And their business model as well.) What's the main rationale for you/other founders to choose MPL? Do you favour/oppose to commercial development using freeswitch in their proprietary systems?
The MPL allows you to commercialize the code. But Anthony can elaborate on this subject more than I can. I don't want to speak on his behalf.
2. Memory usage. I am seeing freeswitch using 25M bytes of memory with just one basic SIP call. It's a little surprising. Should I turn on the -Os to see if we can save some space?
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25295 root 0 -20 27040 25m 3696 S 0 2.5 0:00.12 lt-freeswitch
You can try it out and see. Also look at what modules you load. When I start freeswitch up on my mac I use 6 or 7 megs of ram.
Thanks,
Alex
_______________
_______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com JABBER:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@66.250.68.194 iax:guest@66.250.68.194/888 googletalk:freeswitch@gmail.com pstn:712-432-7800 ----- Original Message ---- From: Alex Guan <alex.guan@prodeasystems.com> To: freeswitch-dev@lists.freeswitch.org Sent: Thursday, September 7, 2006 2:26:41 PM Subject: Re: [Freeswitch-dev] Questions regarding the architecture First of all, thanks for paying attention. I have been waiting for someone to finally raise any questions at all about the code so congrats. I am open to suggestions in that matter. You may want to look at the newest revision of the code as we discuss this because it has become more complicated from the one you are describing. The functionality to wait for and outbound call has been moved to the switch_ivr_originate function which is a standard function now used anywhere in the code where an outbound call must be made. The function takes a destination string which represents a channel endpoint url and may contain one or more of these strings separated by an & character. when there are multiple channels being called at once the complexity goes up and the while loop in question now polls the status of all the channels that are candidates for the outgoing connection which makes the conditional mutex idea harder to properly implement. The good news is we do a switch_yield(1000) [sleep 1 ms and yield the cpu] and this loop tends to never last very long in a traditional call. I am not against optimizing as long as it doesn't compromize stability. There is a bigger problem I see in a similar situation in the RTP. The defualt behaviour in the config for things that use RTP is to set the read socket in async mode meaning the reads will timeout exactly in tune with the desired sample rate and packet interval regardless of reading a valid packet or not. Artificial packets are generated and flagged as CNG so they are not interpreted by the codec engine and are simply passed through to keep the timing right. We currently are using switch_time_now() which on unix translates to gettimeofday() using a method where we take an initial timestamp then increment that number by the desired ms interval and do several gettime calls until we land on or past the desired ms mark at which point we calculate the next mark based on the last (not based on the current time as that source cannot be trusted) so if we start at 1000 ms epoch we go up to 1020 1040 1060 etc. This works but is very taxing on the box. I am considering comming up with a way to start a single thread in the core for every unique sample rate and interval combo desired by anything using the code and having interested threads register to be woken up at this interval somehow so the inner read loop in rtp can try to read on a nonblocking socket then hit a mutex that is reliably woken up at the exact desired interval. This is of course somewhat tricky so I need to think about it more but possibly using some other form of timer would also help this matter like POSIX or something (anyone have any examples?) I am fairly sure that this timing thing leaves room to double how many calls we can get at once on a hefty box that can already do upwards of 1000 channels even with this bottleneck in place... So let me know what you think.... Anthony, Great answers indeed. Thanks a lot! Is there a documentation system that I can save it to ... like a wiki? These information are too valuable to be buried in the mail archive. It took me a while but I think I have come to agreement for pretty much every point you made, except one. The only thing I felt suspicious is the busy waiting in which the originator polls check_channel_status() a million times after the originatee starts ringing but before it answers. Can't we wait for a mutex or condition waken up by CALL_ANSWERED? Imagine if you have a thousand incoming calls, there will be 1000 while loops spinning the processor. Another issue I noticed is that hold/resume is not working properly. There was no 200OK for the re-INVITE. I tried to debug further but somehow couldn't get into mod_exosip.c using gdb. (DDD and Eclipse behaved the same. Does anybody have the same issue?) I kind of guess the issue is what kind of SDP is the SIP phone sending for hold. The old way is to set an all zero O field, as opposed to SENDONLY or INACTIVE. Anyways, if mod_exopsip.c is going way, we should probably just wait for mod_sofia. Two more basic questions: 1. Licensing. A main problem I saw with Asterisk is it's GPL/Digium dual licensing scheme. I don't see any serious business development could be possible under this license. I think Mark and his company have made a big mistake on this subject. (And their business model as well.) What's the main rationale for you/other founders to choose MPL? Do you favour/oppose to commercial development using freeswitch in their proprietary systems? 2. Memory usage. I am seeing freeswitch using 25M bytes of memory with just one basic SIP call. It's a little surprising. Should I turn on the -Os to see if we can save some space? PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25295 root 0 -20 27040 25m 3696 S 0 2.5 0:00.12 lt-freeswitch Thanks, Alex ________________________________ From: Anthony Minessale [mailto:anthmct@yahoo.com] Sent: Wed 9/6/2006 2:01 PM To: freeswitch-dev@lists.freeswitch.org Subject: Re: [Freeswitch-dev] Questions regarding the architecture 1. Thread architecture. For a basic call, why do we need two threads for the sessions (one for each endpoint)? While the thread in charge of originator is in the switch_core_standard_on_execute(), the other thread is in the switch_core_session_run() while loop. What's advantage of doing this compared to Asterisk's one thread approach? Why do we need the thread for the originatee? A) Every session has a thread which is managed by the state machine in the core which sees it from creation to destruction. In the case of a bridged call the existing thread of both sessions are used to move inbound audio across to the opposite session at the same time. A one threaded approach such as asterisk will leave you doing poll's on 2 sockets at once and blocking in one direction will impeed the other as well as forcing you to expose something that can be polled, (not all channels have sockets) The states of the orignatee channel are overridden from the default behaviour to perform the bridge in this case: CS_HOLD pauses the thread. CS_RING moves the state to CS_HOLD where it will wait until it's moved to CS_LOOPBACK or CS_HANGUP by the originate function. CS_LOOPBACK runs the audio bridge code sending the audio back to the other channel. The originator is in CS_EXECUTE meaning it's running instructions installed by a dialplan module. At this point both channels are in the same code reading A and writing B as well as moving messages and dtmf across one another. If either is interrupted the functions exits in both sessions and the originatee state override table is cleared and the session is moved to the new state where the execution continues. if this state is hangup the call ends. 2. What does CS_LOOPBACK really mean? When A calls B, B is in the LOOPBACK state after the call is established. Yet, the switch_core_on_loopback() is not really called. A) The default for loopback is to echo the audio back to the channel, in the case of a bridge it's overridden to send it audio to the other session instead of itself. 3. When A calls B and before B answers, the caller session thread does busy waiting by constantly calling check_channel_status(). Is there a better approach than this? And with this architecture how are we going to incorporate with the voice mail and other timer-based applications? A) Yes this is the best approach, the originator has created the originatee and is waiting for the endpoint module to place it into the CS_RING state when this happens because we overrode it we move it to CS_HOLD and wait for it to be answered or for a timeout to occur. The point is we have waited for it to be safe for us to control. 4. When the orginatee hangs up, the exosip_kill_channel() was called 3 times just for the orginatee channel. The same function is called twice for the originator channel. Is this by design? A) The should be as few calls to this as possible once it's ok to have mutiple calls to it it just calls for any blocking to end on a read or write operation such as the socket api call "shutdown()" does We are not using exosip anymore, we have started coding mod_sofia which is our preferred sip code. There are several changes comming up this week you may want to study. Thanks, Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com JABBER:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@66.250.68.194 iax:guest@66.250.68.194/888 googletalk:freeswitch@gmail.com pstn:712-432-7800 ----- Original Message ---- From: Alex Guan <alex.guan@prodeasystems.com> To: freeswitch-dev@lists.freeswitch.org Sent: Wednesday, September 6, 2006 10:13:43 AM Subject: [Freeswitch-dev] Questions regarding the architecture Gang, I am new to freeswitch and have been studying it for the last week. A few questions regarding the architecture. Any input will be greatly appreciated. 1. Thread architecture. For a basic call, why do we need two threads for the sessions (one for each endpoint)? While the thread in charge of originator is in the switch_core_standard_on_execute(), the other thread is in the switch_core_session_run() while loop. What's advantage of doing this compared to Asterisk's one thread approach? Why do we need the thread for the originatee? 2. What does CS_LOOPBACK really mean? When A calls B, B is in the LOOPBACK state after the call is established. Yet, the switch_core_on_loopback() is not really called. 3. When A calls B and before B answers, the caller session thread does busy waiting by constantly calling check_channel_status(). Is there a better approach than this? And with this architecture how are we going to incorporate with the voice mail and other timer-based applications? 4. When the orginatee hangs up, the exosip_kill_channel() was called 3 times just for the orginatee channel. The same function is called twice for the originator channel. Is this by design? Thanks, Alex _______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev <http://console.mxlogic.com/redir/?5dOWbXb8USC-yejhjhdw0PVkDjDo_Z34FS9Y01MjlS67OFek7qUX7_EoBeNcKxVATvzhOYMOMyOYeodECTo_Z34FS9BQfcBO5mUm-wafBitetz_QciDoDM04Sy-yMCrhhv7fK6zBxUQsCTNNI5-Aq83iSbN_W69jIjbEupdbFEw0tLWjBm53h17OeNgB0yq89A_d46NIjBfXjrzVKvxYYmfSk3q9JASC-yejKYUr3wsm> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev <http://console.mxlogic.com/redir/?5dOWbXb8USC-yejhjhdw0PVkDjDo_Z34FS9Y01MjlS6lmDaIaoX7_EoBeNcKxVATvzhOYMOMyOYeodECTo_Z34FS9BQfcBO5mUm-wafBitetz_QciDoDM04Sy-yMCrhhv7fK6zBxUQsCTNNI5-Aq83iSbN_W69jIjbEupdbFEw0tLWjBm53h17OeNgB0yq89A_d46NIjBfXjrzVKvxYYmfSk3qpJASC-yejKYUr3wsm> http://www.freeswitch.org <http://console.mxlogic.com/redir/?5dOWbXb8USC-yejhjhdw0Do_Z34FS9Y01MTvzhOYMOMyOYeodECTo_Z34FS9BQfcBO5mUm-wafBitetz_QciDoDM04Sy-yMCrhhv7fK6zBxUQsCTNNI5-Aq83iSbN_W69jIjbEupdbFEw0tLWjBm53h17OeNgB0yq89A_d46NIjBfXjrzVKvxYYmfSk3r9JASC-yejKYUr3wsm> _______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
participants (5)
-
Alex Guan
-
Anthony Minessale
-
Brian West
-
Paul Tinsley
-
Peter Saint-Andre