[Freeswitch-users] FreeSWITCH Message Bus / Shared Key Value Store

Sun Feb 17 21:10:24 MSK 2013

Regarding #3, it seems they got to it for zeromq v3.
>
> From ØMQ 3.x, filtering happens at the publisher side, when using a
> connected protocol (tcp: *or ipc:*). Using the epgm:// protocol,
> filtering happens at the subscriber side.

In ØMQ/2.x, all filtering happened at the subscriber side.

-Avi

On Sun, Feb 17, 2013 at 7:51 PM, Anthony Minessale <
anthony.minessale at gmail.com> wrote:

> It has a strict philosophy on not using locks or mutexes and only doing
> com over socket.  When you create an instance of it, it uses its own
> threads internally and that model gets messed up severely if you call fork
> from the process (even when you are doing it for something else like system
> command).  Its basically an engine waiting for a host program rather than
> something you can integrated into a more complex application.  Also the
> multicast has issues, there is no way to specificy host local scope for
> hops.  Finally, their philosophy is to ignore the dilemma of sub/pub.
>  There are no subscriptions to events, there is only send all events to
> everyone and let them filter locally.  Its easy to believe this is adequate
> when doing pre-determined test scenarios but we know in reality that in a
> busy FS box we, cant send *all* the events to *everyone* and let them
> filter locally.
>
>
>
> On Sun, Feb 17, 2013 at 11:32 AM, Avi Marcus <avi at avimarcus.net> wrote:
>
>> Please excuse my ignorance, but can you explain again why zmq won't work?
>>
>> Regarding boasting about messaging capability.. it was made to be quite
>> lightweight - supposedly it can easily saturate a 1gigabit link with
>> message size over 150 bytes each:
>> http://www.zeromq.org/results:10gbe-tests
>>
>> If you want to use it for a queue, then it's lack of persistence and
>> assurance that messages get through might not be helpful, but for a
>> broadcast to whoever wants to listen it sounds great for heartbeats,
>> events, etc.
>>
>> -Avi Marcus
>>
>> 1-718-989-9485 (USA)
>> 1-866-202-5850 (USA & Canada Toll Free)
>> 02-372-1570 (Israel)
>> 020-3298-2875 (UK)
>>
>>
>> On Sun, Feb 17, 2013 at 5:50 PM, Eliot Gable <egable+freeswitch at gmail.com
>> > wrote:
>>
>>> ZMQ also does not work with a fork, which is needed in order to execute
>>> any system commands (like iptables or anything else which has no
>>> programming API). That pretty much eliminates ZMQ as a possibility.
>>>
>>>  I did some research this weekend, and of all the possibilities I could
>>> find, the one that caught my attention the most was Spread:
>>> http://www.spread.org/
>>>
>>> There are some drawbacks to it, namely:
>>>
>>> 1) It requires any marketing material mentioning FreeSWITCH or any
>>> project / solution utilizing FreeSWITCH to also include a prepared
>>> statement about the use of the Spread toolkit. This is a fairly major
>>> licensing issue, as all commercial solutions utilizing FreeSWITCH as a core
>>> component would then also need to mention the use of the Spread toolkit
>>> within FreeSWITCH. It is essentially a "viral clause," which would be very
>>> tedious for people to make sure they honor correctly.
>>>
>>> 2) The toolkit uses a stand-alone daemon which would then need to be
>>> monitored separately from FreeSWITCH and would be another point of failure,
>>> adding complexity to the system. Basically, if that daemon were to crash,
>>> FreeSWITCH would need to know about it and would need to either respawn it
>>> or shut down. This would bring the need for Pacemaker or something similar
>>> back into the picture for any viable HA solution. Alternatively, we could
>>> write some code similar to daemontools into FreeSWTICH which respawns the
>>> daemon if it dies, but we would have to test the impact of this respawning
>>> on the overall cluster to determine if it impacts visibility of the node at
>>> any point in time. Another alternative would be to wrap the daemon into a
>>> thread inside FreeSWITCH such that if the daemon caused a segfault or
>>> something, it would force FreeSWITCH to terminate, as well. I have not done
>>> a code review of the daemon yet to determine if this is viable alternative,
>>> but assuming they have not coded it on LSD or something, it is more than
>>> likely possible.
>>>
>>> 3) They boast about how it can handle up to 8,000 1KB messages per
>>> second. I don't consider that boast-worthy. When I worked at Broadvox a few
>>> years ago, I had a FS pair which ran around 380 calls per second (760
>>> sessions per second). Each call generates dozens of events. That hardware
>>> was getting dated when I left Broadvox, and today's hardware along with the
>>> performance improvements done to FS since then means we could conceivably
>>> have a single node which runs over 1k calls per second firing dozens of
>>> events per call. That means a single box could completely consume the
>>> message bandwidth of the entire Spread network. Imagine trying to have 64
>>> such boxes running. We are really in need of a solution which boasts
>>> hundreds of thousands of messages per second. Spread seems like it might be
>>> off by an order of magnitude and then some.
>>>
>>> Despite these issues, Spread still seems to come closer to our needs
>>> than any other solution I found. FYI, I also looked at the Corosync IPC
>>> system, and was not at all impressed. On paper, Spread exceeds Corosync's
>>> capabilities by a fair margin.
>>>
>>> There are some strategies for mitigating issue #3 with Spread, as well.
>>> For example, we could limit messages across the Spread network to things
>>> like heartbeats and / or other HA and synchronization related messages.
>>> Basically, think of it like a D-channel on a PRI. For sending high packet
>>> per second streams of messages, we could do standard unicast connections or
>>> even try straight up mutlicasting to all nodes on the LAN. Sending
>>> heartbeats every 10ms across the Spread network would put a 64-node cluster
>>> at 6,400 messages per second just with heartbeats. That would still leave a
>>> decent amount of message bandwidth available for other types of negotiation
>>> messages and should still allow for sub-second fail-over detection and
>>> reaction. Of course, this is all assuming we can actually get 8,000 1KB
>>> messages per second out of a 64-node cluster.
>>>
>>> There are likely lots of things that impact how many messages per second
>>> Spread can handle. A lot of it has to do with network latency and CPU
>>> power. Spread uses acknowledgements and message reordering to ensure
>>> delivery in a way that accounts for things like node membership changes
>>> during the time the message is in transit and whether the message has been
>>> received by all nodes in the cluster. Network latency is probably one of
>>> the biggest factors in how many messages per second it can handle. On a
>>> faster network link, the messages per second would be higher and on a
>>> slower network, it would be lower. Obviously, CPU processing time and
>>> scheduling is important, as well. If one system is extremely overloaded and
>>> the Spread daemon is being starved for CPU resources, that will add extra
>>> latency in processing and also reduce message throughput. Obviously, this
>>> could also impact whether the node is seen as visible. So, this is one more
>>> argument for why we would need to try to run the daemon as a thread under
>>> FreeSWITCH. Then it has the same scheduling priority as FreeSWITCH and
>>> cannot be starved for resources by FreeSWITCH itself. It also would exhibit
>>> the same amount of resource starvation FreeSWITCH experiences on the node,
>>> so would more accurately reflect the state of FreeSWITCH on the node.
>>>
>>> If anyone has any other suggestions than Spread, I would like to hear
>>> it. Also, some feedback on item #1 would be great, as I cannot really judge
>>> for everyone else how willing they are to accept such a licensing clause.
>>>
>>>
>>>
>>> On Tue, Feb 12, 2013 at 8:48 PM, João Mesquita <jmesquita at freeswitch.org
>>> > wrote:
>>>
>>>> I have used ZeroMQ in the past for this sorts of things but it really
>>>> won't be able to detect failures really fast. It is not made for this.
>>>> Maybe we can gather the requirements for such message bus? Zmq for example
>>>> provides you with this cool interface to build messaging protocols on top
>>>> of it but it does not provide reliability when it comes to endpoint to
>>>> endpoint connection without a heartbeat implemented on the user end. Can
>>>> this be used for FS as well? Anyhow, just throwing some ideas...
>>>>
>>>> João Mesquita
>>>> FreeSWITCH™ Solutions
>>>>
>>>>
>>>> On Tue, Feb 12, 2013 at 5:42 PM, Dave R. Kompel <drk at drkngs.net> wrote:
>>>>
>>>>> **
>>>>> I've done a few experments with using both Redis, and the evil
>>>>> "Microsoft Azure Service bus" (the server on prem based version) to extend
>>>>> the eventing system to have global PUB/SUB. This way things like
>>>>> registrations, and Limit stuff could be made global.
>>>>>
>>>>> I'm looking for a way, in my carrier switch implmentation, to implment
>>>>> both HA Failover and Scaleout clustering.
>>>>>
>>>>> --Dave
>>>>>
>>>>>  ------------------------------
>>>>> *From:* Eliot Gable [mailto:egable+freeswitch at gmail.com]
>>>>> *To:* FreeSWITCH Users Help [mailto:
>>>>> freeswitch-users at lists.freeswitch.org]
>>>>> *Sent:* Tue, 12 Feb 2013 05:49:13 -0800
>>>>> *Subject:* [Freeswitch-users] FreeSWITCH Message Bus / Shared Key
>>>>> Value Store
>>>>>
>>>>>
>>>>> Tony and Mike and I had a discussion last night about FreeSWITCH with
>>>>> regards to implementing some form of core message bus or shared key-value
>>>>> store. We discussed a few different options, but did not really settle on
>>>>> anything. If you are writing modules or using FreeSWITCH in a multi-node
>>>>> setting, please share what features / functionality you would like to see
>>>>> implemented in this regard, how you would use it, and why you want to see
>>>>> the specific mechanism of your choice rather than some alternative. Also,
>>>>> please consider and mention whether "cluster awareness" is something that
>>>>> factors into your use case. By this, I mean having each FS node have some
>>>>> idea about the state / status of each other node in terms of taking calls
>>>>> vs acting as a standby or slave node, etc.
>>>>>
>>>>> --
>>>>> Eliot Gable
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Eliot Gable
>>>
>>> "We do not inherit the Earth from our ancestors: we borrow it from our
>>> children." ~David Brower
>>>
>>> "I decided the words were too conservative for me. We're not borrowing
>>> from our children, we're stealing from them--and it's not even considered
>>> to be a crime." ~David Brower
>>>
>>> "Esse oportet ut vivas, non vivere ut edas." (Thou shouldst eat to live;
>>> not live to eat.) ~Marcus Tullius Cicero
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130217/b51ada3a/attachment-0001.html 