[Freeswitch-users] Getting FreeSWITCH to use RTP port + 1 for RTCP
James Mortensen
james.mortensen at synclio.com
Wed Dec 4 22:18:50 MSK 2013
Yes, we're on 1.5.8 from the latest master. I believe nonmux is what
Asterisk was using, where the ports were incremented by 1. I'm not sure if
that is related or not, it was just something we noticed that was different
between the two SDP's.
Thanks,
James
On Wed, Dec 4, 2013 at 10:58 AM, Anthony Minessale <
anthony.minessale at gmail.com> wrote:
> Typically we try to use mux rtcp and fall back to nonmux if the other end
> does not support it.
> This is necessary for mozilla vs chrome.
>
> Are you running latest master of FS? Because the media code changes quite
> often at this juncture.
>
>
>
>
>
>
> On Tue, Dec 3, 2013 at 9:43 PM, James Mortensen <
> james.mortensen at synclio.com> wrote:
>
>> Ah yes! That's my fault. Let me clarify. We're using WebRTC to implement
>> the media plane using libjingle, but the signaling portion is all SIP over
>> UDP (or TCP if needed). This is why you see ICE candidates and a SAVPF
>> profile, despite the SIP messaging being sent over UDP.
>>
>> The AppRTCDemo (1) didn't use SIP, and (2) sent the SDP over the Channel
>> API. This means it's near impossible to interoperate with legacy systems
>> such as the PSTN, which is still a necessity. Hence, this is why we're
>> doing things a little different.
>>
>> The fact that this came off as confusing suggests that this is sort of an
>> edge case and not as common as I'd mistakenly assumed. What's more, the
>> errors we're seeing appear to be in the libjingle library we're using,
>> *not* in FreeSWITCH. However, audio flows both ways when using our setup
>> with Asterisk, and this is why I thought there might be a configuration
>> setting we could change in the FreeSWITCH SDP, in order to get libjingle to
>> play nice during the call setup.
>>
>> At this time, it seems we may have to just dig into this more ourselves,
>> which we're more than happy to do. With FreeSWITCH connected to our Chrome
>> App (which *does* use WebRTC with SIP over WebSockets) we get consistent
>> two way audio, whereas with Asterisk we experience one way audio on about
>> 20% of calls. In short, FreeSWITCH provides a more reliable media plane
>> than Asterisk; otherwise, this wouldn't be an issue as we would simply use
>> what works.
>>
>> In summary, if you or the other experts here can think of more questions
>> or things for us to try in FreeSWITCH, we're happy to share the results
>> here if that provides value. Otherwise, if you're sure it's not a
>> FreeSWITCH configuration problem, we'll continue troubleshooting from other
>> angles. Hope this helps clarify, and thanks again for all your help.
>>
>>
>>
>> James
>>
>>
>>
>>
>> On Tue, Dec 3, 2013 at 5:41 PM, Anthony Minessale <
>> anthony.minessale at gmail.com> wrote:
>>
>>> Maybe you should elaborate more because I think everyone assumes you are
>>> doing webrtc since you showed tiny snippet of ICE cands.
>>>
>>>
>>>
>>>
>>> On Tue, Dec 3, 2013 at 4:48 PM, James Mortensen <
>>> james.mortensen at synclio.com> wrote:
>>>
>>>> > It has nothing to do with the SIP TLS settings.
>>>>
>>>> I don't get what does it have to do with WSS settings? This is an
>>>> Android client connecting through UDP/TCP ports 5060/5070. It doesn't
>>>> connect SIP over WebSockets, hence my confusion.
>>>>
>>>> Thanks,
>>>> James
>>>>
>>>>
>>>>
>>>> On Tue, Dec 3, 2013 at 1:59 PM, Anthony Minessale <
>>>> anthony.minessale at gmail.com> wrote:
>>>>
>>>>> It has nothing to do with the SIP TLS settings.
>>>>> I believe I already listed what you need to make it work.
>>>>> You may want to start over with the default configs, place the files
>>>>> in the correct place and add the one binding for wss.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Dec 3, 2013 at 11:58 AM, James Mortensen <
>>>>> james.mortensen at synclio.com> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> We followed the steps in this document to set up TLS:
>>>>>> http://wiki.freeswitch.org/wiki/SIP_TLS#If_your_SIP_profile_fails_to_start_upand followed these troubleshooting instructions. We have the certs in
>>>>>> ${prefix}/conf/ssl
>>>>>>
>>>>>> oot at ip-10-188-136-13:/opt/freeswitch-dev/conf/ssl# l
>>>>>> total 12
>>>>>> -rw-r----- 1 root root 579 Dec 3 10:27 cafile.pem
>>>>>> drwxr-x--- 2 root root 4096 Dec 3 10:28 CA
>>>>>> -rw-r----- 1 root root 999 Dec 3 10:28 agent.pem
>>>>>>
>>>>>> Also, I've verfied that libssl-dev is indeed installed, version 1.0.1.
>>>>>>
>>>>>>
>>>>>> When we try to load (or reload) the internal profile with the
>>>>>> wss-binding parameter enabled for port 7066, we see the following output:
>>>>>>
>>>>>> nta.c:2389 agent_init_via() nta: agent_init_via: SIP/2.0/udp
>>>>>> 50.18.X.X:5080 (sip)
>>>>>> nta.c:2389 agent_init_via() nta: agent_init_via: SIP/2.0/tcp
>>>>>> 50.18.X.X:5080 (sip)
>>>>>> nta.c:2389 agent_init_via() nta: agent_init_via: SIP/2.0/ws
>>>>>> 10.188.136.13:5066 (sip)
>>>>>> nta.c:2255 nta_agent_add_tport() nta: Via fields initialized
>>>>>> nta.c:2263 nta_agent_add_tport() nta: Contact header created
>>>>>> tport.c:1620 tport_bind_server() tport_bind_server(0x7fbf9c003ca0) to
>>>>>> wss/10.188.136.13:7443/sips
>>>>>> tport.c:1690 tport_bind_server() tport_bind_server(0x7fbf9c003ca0):
>>>>>> calling tport_listen for wss
>>>>>> nua.c:575 nua_set_params() nua: nua_set_params: entering
>>>>>> tport.c:628 tport_alloc_primary()
>>>>>> tport_alloc_primary(0x7fbf9c003ca0): new primary tport 0x7fbf9c005e80
>>>>>> nua_stack.c:529 nua_signal() nua((nil)): sent signal r_set_params
>>>>>> nua_params.c:480 nua_stack_set_params() nua: nua_stack_set_params:
>>>>>> entering
>>>>>> soa.c:405 soa_set_params() soa_set_params(static::0x7fbfa8001ca0,
>>>>>> ...) called
>>>>>> tport.c:734 tport_listen() tport_listen(0x7fbf9c003ca0): unknown(pf=2
>>>>>> wss/[10.188.136.13]:7443): No such file or directory
>>>>>> nta.c:2237 nta_agent_add_tport() nta: bind(10.188.136.13:7443;transport=wss):
>>>>>> No such file or directory
>>>>>> nua_stack.c:195 nua_stack_init() nua: initializing SIP stack failed
>>>>>> nua_stack.c:212 nua_stack_deinit() nua: nua_stack_deinit: entering
>>>>>> sresolv.c:190 sres_sofia_update() sres_sofia_update(0x7fbf9c0038a0,
>>>>>> -1, -1)
>>>>>> sresolv.c:190 sres_sofia_update() sres_sofia_update((nil), -1, -1)
>>>>>> tport.c:562 tport_destroy() tport_destroy(0x7fbf9c003ca0)
>>>>>> su_epoll_port.c:166 su_epoll_port_deinit()
>>>>>> su_epoll_port_deinit(0x7fbf9c0008c0) called
>>>>>> 2013-12-03 17:47:36.815063 [ERR] sofia.c:2545 Error Creating SIP UA
>>>>>> for profile: internal (sip:mod_sofia at 50.18.X.X
>>>>>> :5080;maddr=10.188.136.13;transport=udp,tcp)
>>>>>> The likely causes for this are:
>>>>>> 1) Another application is already listening on the specified address.
>>>>>> 2) The IP the profile is attempting to bind to is not local to this
>>>>>> system.
>>>>>>
>>>>>>
>>>>>> And when trying to reload the profile, this is what we're seeing:
>>>>>>
>>>>>> freeswitch at ip-10-188-136-13> sofia profile internal restart
>>>>>>
>>>>>> Invalid Profile [internal]
>>>>>> freeswitch at ip-10-188-136-13>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I'm a little unclear as to whether or not any certs actually need to
>>>>>> be installed on Android. The webrtc.freeswitch.org demo seems to
>>>>>> work without needing to do anything on the client side. What's more, that
>>>>>> shouldn't have any effect on the server's ability to start the profile.
>>>>>> What are we missing here? Thank you!
>>>>>>
>>>>>>
>>>>>>
>>>>>> James
>>>>>>
>>>>>>
>>>>>> On Wed, Nov 27, 2013 at 10:14 PM, Anthony Minessale <
>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>
>>>>>>> *HINT* you need to use a valid cert for the dtls and the ssl
>>>>>>> websocket
>>>>>>> Android is 100 times more paranoid about certs and doesn't bother to
>>>>>>> explain.
>>>>>>> A common theme in google code for webrtc to not take the time and
>>>>>>> effort to explain errors.
>>>>>>>
>>>>>>> go to /usr/local/freeswitch/certs
>>>>>>> put both the key and the cert in wss.pem
>>>>>>> put the same key in dtls-srtp.key and the same cert in dtls-srtp.crt
>>>>>>> put a valid chain cert in ca-bundle.crt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Nov 27, 2013 at 7:09 PM, James Mortensen <
>>>>>>> james.mortensen at synclio.com> wrote:
>>>>>>>
>>>>>>>> I have limited information at this time, but we're basically seeing
>>>>>>>> RTP packets flowing from Chrome to FreeSWITCH and from FreeSWITCH to
>>>>>>>> Chrome, but very minimal activity between Android and FreeSWITCH.
>>>>>>>>
>>>>>>>> Before my colleague hit the road for the day, we did notice there
>>>>>>>> were two separate INVITES being sent to Android, and that's where we
>>>>>>>> stopped. It is possible there's a port mismatch somewhere, assuming we
>>>>>>>> really did see two INVITES in the FreeSWITCH SIP trace. To be sure, we
>>>>>>>> registered Chrome and Android on different networks so there was absolutely
>>>>>>>> no confusion.
>>>>>>>>
>>>>>>>> I've attached a SIP trace from several of these call attempts.
>>>>>>>> Just look from the top where you see a JsSIP client send an INVITE from
>>>>>>>> 50.43.x.x and then scroll down to see FreeSWITCH send two INVITES to two
>>>>>>>> separate ports on the 115.11.x.x network. Moreover, the candidates in both
>>>>>>>> INVITES contain different ports. Here are those two INVITES inline, where
>>>>>>>> user 1000 is the Android client on 115.11.x.x and 1100 is the Chrome JsSIP
>>>>>>>> client at 50.43.x.x, JsSIP initiates the call:
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>> send 2031 bytes to udp/[115.11.X.X]:9322 at 23:25:08.408616:
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>> INVITE sip:1000 at 115.11.X.X:9322;transport=udp SIP/2.0
>>>>>>>> Via: SIP/2.0/UDP 50.18.X.X:5070;rport;branch=z9hG4bKrv59pQQvDaagc
>>>>>>>> Route: <sip:1000 at 115.11.X.X:9322;transpo;lr;ovid=c5ab5428>
>>>>>>>> Max-Forwards: 68
>>>>>>>> From: "James" <sip:971200xxxx at 10.188.136.13>;tag=UX0748rrX2Bgm
>>>>>>>> To: <sip:1000 at 115.11.X.X:9322;transport=udp>
>>>>>>>> Call-ID: fed8b6a7-d25d-1231-2cad-22000abc880d
>>>>>>>> CSeq: 52461290 INVITE
>>>>>>>> Contact: <sip:mod_sofia at 50.18.X.X:5070>
>>>>>>>> User-Agent:
>>>>>>>> FreeSWITCH-mod_sofia/1.5.8b+git~20131127T162035Z~bbe1fe1a31~64bit
>>>>>>>> Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE,
>>>>>>>> REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE
>>>>>>>> Supported: precondition, path, replaces
>>>>>>>> Allow-Events: talk, hold, conference, presence,
>>>>>>>> as-feature-event, dialog, line-seize, call-info, sla,
>>>>>>>> include-session-description, presence.winfo, message-summary, refer
>>>>>>>> Content-Type: application/sdp
>>>>>>>> Content-Disposition: session
>>>>>>>> Content-Length: 975
>>>>>>>> X-FS-Support: update_display,send_info
>>>>>>>> Remote-Party-ID: "James" <sip:971200xxxx at 10.188.136.13
>>>>>>>> >;party=calling;screen=yes;privacy=off
>>>>>>>>
>>>>>>>> v=0
>>>>>>>> o=FreeSWITCH 1385562204 1385562205 IN IP4 50.18.X.X
>>>>>>>> s=FreeSWITCH
>>>>>>>> c=IN IP4 50.18.X.X
>>>>>>>> t=0 0
>>>>>>>> a=sendrecv
>>>>>>>> a=msid-semantic: WMS oXSGoBbKflmyggzbJEZP120R0qAOBfsQ
>>>>>>>> m=audio 32504 RTP/SAVPF 0 8 101 13
>>>>>>>> a=rtpmap:0 PCMU/8000
>>>>>>>> a=rtpmap:8 PCMA/8000
>>>>>>>> a=rtpmap:101 telephone-event/8000
>>>>>>>> a=fingerprint:sha-256
>>>>>>>> 91:9E:50:FA:2E:A5:8E:02:41:1E:FC:A5:8A:B0:0C:1F:98:1F:1B:49:BB:D6:44:A8:E1:01:EA:82:7C:B6:59:1D
>>>>>>>> a=rtcp-mux
>>>>>>>> a=rtcp:32504 IN IP4 50.18.X.X
>>>>>>>> a=ssrc:177794596 cname:2bmhDoRIz36GawIJ
>>>>>>>> a=ssrc:177794596 msid:oXSGoBbKflmyggzbJEZP120R0qAOBfsQ a0
>>>>>>>> a=ssrc:177794596 mslabel:oXSGoBbKflmyggzbJEZP120R0qAOBfsQ
>>>>>>>> a=ssrc:177794596 label:oXSGoBbKflmyggzbJEZP120R0qAOBfsQa0
>>>>>>>> a=ice-ufrag:1xoTpfAgv0VX5dDX
>>>>>>>> a=ice-pwd:eQ5I4MguGd1GAzfr
>>>>>>>> a=candidate:5290007956 1 udp 659136 50.18.X.X 32504 typ host
>>>>>>>> generation 0
>>>>>>>> a=candidate:5290007956 2 udp 659136 50.18.X.X 32504 typ host
>>>>>>>> generation 0
>>>>>>>> a=crypto:1 AES_CM_128_HMAC_SHA1_80
>>>>>>>> inline:TTl8L9nzxRRf4R3jWWukpQaOSmkB5tncyLIuwvaA
>>>>>>>> a=ptime:20
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>> send 2034 bytes to udp/[115.11.X.X]:62163 at 23:25:08.409283:
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------
>>>>>>>> INVITE sip:1000 at 115.11.X.X:62163;transport=udp SIP/2.0
>>>>>>>> Via: SIP/2.0/UDP 50.18.X.X:5070;rport;branch=z9hG4bKS5y2rj8ZaK02Q
>>>>>>>> Route: <sip:1000 at 115.11.X.X:62163;transpo;lr;ovid=c5ab5428>
>>>>>>>> Max-Forwards: 68
>>>>>>>> From: "James" <sip:971200xxxx at 10.188.136.13>;tag=v6S0639UtB22F
>>>>>>>> To: <sip:1000 at 115.11.X.X:62163;transport=udp>
>>>>>>>> Call-ID: fed8d029-d25d-1231-2cad-22000abc880d
>>>>>>>> CSeq: 52461290 INVITE
>>>>>>>> Contact: <sip:mod_sofia at 50.18.X.X:5070>
>>>>>>>> User-Agent:
>>>>>>>> FreeSWITCH-mod_sofia/1.5.8b+git~20131127T162035Z~bbe1fe1a31~64bit
>>>>>>>> Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE,
>>>>>>>> REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE
>>>>>>>> Supported: precondition, path, replaces
>>>>>>>> Allow-Events: talk, hold, conference, presence,
>>>>>>>> as-feature-event, dialog, line-seize, call-info, sla,
>>>>>>>> include-session-description, presence.winfo, message-summary, refer
>>>>>>>> Content-Type: application/sdp
>>>>>>>> Content-Disposition: session
>>>>>>>> Content-Length: 975
>>>>>>>> X-FS-Support: update_display,send_info
>>>>>>>> Remote-Party-ID: "James" <sip:971200xxxx at 10.188.136.13
>>>>>>>> >;party=calling;screen=yes;privacy=off
>>>>>>>>
>>>>>>>> v=0
>>>>>>>> o=FreeSWITCH 1385569930 1385569931 IN IP4 50.18.X.X
>>>>>>>> s=FreeSWITCH
>>>>>>>> c=IN IP4 50.18.X.X
>>>>>>>> t=0 0
>>>>>>>> a=sendrecv
>>>>>>>> a=msid-semantic: WMS YGqq3KBdA3doGvggsuWTkskoGwvg99oY
>>>>>>>> m=audio 24778 RTP/SAVPF 0 8 101 13
>>>>>>>> a=rtpmap:0 PCMU/8000
>>>>>>>> a=rtpmap:8 PCMA/8000
>>>>>>>> a=rtpmap:101 telephone-event/8000
>>>>>>>> a=fingerprint:sha-256
>>>>>>>> 91:9E:50:FA:2E:A5:8E:02:41:1E:FC:A5:8A:B0:0C:1F:98:1F:1B:49:BB:D6:44:A8:E1:01:EA:82:7C:B6:59:1D
>>>>>>>> a=rtcp-mux
>>>>>>>> a=rtcp:24778 IN IP4 50.18.X.X
>>>>>>>> a=ssrc:177896132 cname:FFeyfGCFzFU610ck
>>>>>>>> a=ssrc:177896132 msid:YGqq3KBdA3doGvggsuWTkskoGwvg99oY a0
>>>>>>>> a=ssrc:177896132 mslabel:YGqq3KBdA3doGvggsuWTkskoGwvg99oY
>>>>>>>> a=ssrc:177896132 label:YGqq3KBdA3doGvggsuWTkskoGwvg99oYa0
>>>>>>>> a=ice-ufrag:lZ3vhdKOA6UA59oA
>>>>>>>> a=ice-pwd:EtZJZsepYZmPPpZ1
>>>>>>>> a=candidate:6658001960 1 udp 659136 50.18.X.X 24778 typ host
>>>>>>>> generation 0
>>>>>>>> a=candidate:6658001960 2 udp 659136 50.18.X.X 24778 typ host
>>>>>>>> generation 0
>>>>>>>> a=crypto:1 AES_CM_128_HMAC_SHA1_80
>>>>>>>> inline:CKFCzTuq04VQ1E0QJMmo0WjXDUtp+GTuK4CLgAS7
>>>>>>>> a=ptime:20
>>>>>>>>
>>>>>>>>
>>>>>>>> It seems like something is amiss here. This does seem odd. Hope
>>>>>>>> this helps!
>>>>>>>>
>>>>>>>>
>>>>>>>> James
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Nov 27, 2013 at 4:44 PM, Michael Jerris <mike at jerris.com>wrote:
>>>>>>>>
>>>>>>>>> Try looking at a pcap and comparing to the android logs. Are they
>>>>>>>>> sending the stun requests to the right place and are they getting to
>>>>>>>>> freeswitch?
>>>>>>>>>
>>>>>>>>> On Nov 27, 2013, at 7:38 PM, James Mortensen <
>>>>>>>>> james.mortensen at synclio.com> wrote:
>>>>>>>>>
>>>>>>>>> The issue is that Android is showing STUN ping timeouts in the
>>>>>>>>> libjingle logs. This doesn't happen with Asterisk, so we just asked
>>>>>>>>> ourselves, "What is different in the SDP's" and we noticed it's the way
>>>>>>>>> RTCP is handled....
>>>>>>>>>
>>>>>>>>> Now, I'm not suggesting using the port +1 method would solve the
>>>>>>>>> problem; it was mostly just something we thought to try to see if we'd get
>>>>>>>>> different results. If it's not worth trying, we'll try something else. :)
>>>>>>>>>
>>>>>>>>> I don't have the Android debug logs handy at this time, but we'll
>>>>>>>>> work on putting them together.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> James
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Nov 27, 2013 at 4:14 PM, Michael Jerris <mike at jerris.com>wrote:
>>>>>>>>>
>>>>>>>>>> We are using rtcp mux where it uses the same port. This is the
>>>>>>>>>> default for webrtc and I don't think we have a way to disable it for
>>>>>>>>>> outbound invites for webrtc media, but there shouldn't ever be a reason to
>>>>>>>>>> disable it as all the browsers support it and its required by the spec. As
>>>>>>>>>> for why its not working with chrome for android, I'm not sure, do you have
>>>>>>>>>> a debug log of the call?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Nov 27, 2013, at 5:56 PM, James Mortensen <
>>>>>>>>>> james.mortensen at synclio.com> wrote:
>>>>>>>>>>
>>>>>>>>>> > Hello,
>>>>>>>>>> >
>>>>>>>>>> > I've searched the list and documentation, and it's not yet
>>>>>>>>>> clear if FreeSWITCH is able to send candidates in the same manner as we
>>>>>>>>>> were doing with Asterisk 1.5 where it sends the RTCP to the RTP port +1.
>>>>>>>>>> In other words, I'd see candidates that looked like this:
>>>>>>>>>> >
>>>>>>>>>> > a=candidate:3441596188 1 udp 659136 10.3.1.1 20622 typ host
>>>>>>>>>> generation 0
>>>>>>>>>> > a=candidate:3441596188 1 udp 659136 50.18.X.X 20622 typ srflx
>>>>>>>>>> generation 0
>>>>>>>>>> > a=candidate:3441596188 2 udp 659136 10.3.1.1 20623 typ host
>>>>>>>>>> generation 0
>>>>>>>>>> > a=candidate:3441596188 2 udp 659136 50.18.X.X 20623 typ srflx
>>>>>>>>>> generation 0
>>>>>>>>>> >
>>>>>>>>>> > For instance, if the RTP port was 20622, the RTCP port would be
>>>>>>>>>> 20623. We're getting no audio when trying to make Chrome to Android and
>>>>>>>>>> Android to Android WebRTC calls, and we thought we'd try a different RTCP
>>>>>>>>>> method as we did have it working with Asterisk but want to use FreeSWITCH
>>>>>>>>>> instead.
>>>>>>>>>> >
>>>>>>>>>> > I've checked out the latest FreeSWITCH code from the master
>>>>>>>>>> branch.
>>>>>>>>>> >
>>>>>>>>>> > Not sure if FreeSWITCH can do this yet, but if anyone knows
>>>>>>>>>> that would be helpful.
>>>>>>>>>> >
>>>>>>>>>> > Thank you!
>>>>>>>>>> > James
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>> consulting at freeswitch.org
>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>> http://www.cluecon.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://wiki.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>>
>>>>>>> _________________________________________________________________________
>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>> consulting at freeswitch.org
>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Official FreeSWITCH Sites
>>>>>>> http://www.freeswitch.org
>>>>>>> http://wiki.freeswitch.org
>>>>>>> http://www.cluecon.com
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:
>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://wiki.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>> ClueCon http://www.cluecon.com/
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>> MSN:anthony_minessale at hotmail.com
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>> sip:888 at conference.freeswitch.org
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>> pstn:+19193869900
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>>
>>>>
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Anthony Minessale II
>>>
>>> FreeSWITCH http://www.freeswitch.org/
>>> ClueCon http://www.cluecon.com/
>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>
>>> AIM: anthm
>>> MSN:anthony_minessale at hotmail.com
>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>> IRC: irc.freenode.net #freeswitch
>>>
>>> FreeSWITCH Developer Conference
>>> sip:888 at conference.freeswitch.org
>>> googletalk:conf+888 at conference.freeswitch.org
>>> pstn:+19193869900
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>>
>>>
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>>
>>
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
>
>
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20131204/56a6e11a/attachment-0001.html
Join us at ClueCon 2013 Aug 6-8, 2013
More information about the FreeSWITCH-users
mailing list