Hello List, I have read the current thread about scalability and I would need some advice about a callcenter setup: First where I come from: I have lot's of problems with an asterisk solution. I have regular crash's and lock-ups, with downgrading and other stuff i got it somewhat stable, but have nevertheless regular hickups. I am desperate and want to get rid of asterisk and I hope that freeSwitch will provide me with a more stable solution. Our Setup (really nothing special): * 1 Asterisk box, New IBM Hardware (3 month old), 2 HE rack server, 3 GIG of RAM, Xircom analog switch connected to mobile stations for 4 different providers, Digium 4port cards TP400<something> * 8 queues * ~60 agents (which logon, logoff, pause, unpause), not more than 40 concurrently online * ~ 7K - 9K calls (well, CDR entries) a day (not that much for a bpx) * Music on Hold in the call-queues * No special announcement * Transfers between calls in queues and different agents as well as non agents (i mention this because we have transfer related chrashes in asterisk) The current Problems: * Lockups with different causes (ranging from calls not terminated to heavy thread locking through the AMI interface) * Crashes and library aborts (pthread, libc, crashes related to music on hold, app_queue, transfers) We used Asterisk: 1.4.23, 1.4.24, 1.4.26rc3, 1.4.26rc5, 1.4.26 and are now back to 1.4.21.2 (stock debian) as anything beyond that is for whatever reason highly unstable for our szenario. Maybe we should have been segmenting the box into one asterisk dedicted to talking to the hardware, one especial for queue/sip handling, i do not know. (all issues are well documented in issues.asterisk.org, but it seems to be very, very difficult to get to the bottom of them as they exist since 1.4.23 as it seems and are open until know and not fixable since month.) Now, I really would appreciate some success-stories on how you guys managed to get a stable pbx system with freeSWITCH in regard of HA and scalability: * How to segment freeSWITCH? Or is it stable enough to handle all in one for such a szenario as outlined above? * What would be the best strategie for High Availability / Failover? -> I read in the WIKI (featurelist) that Livemigration of calls from one box to another should be possible? -> I was thinking about using memcached for storing all state information so another freeswitch box can takeover calls from the first box if it dies, is this possible? If so, how? -> Is there anotherway to somehow configure freeSWITCH that in the event of a crash i do not loose the current established calls? Basically I just want a stable PBX where I do not have to fear every day it will core-dump or abort or Lock up. Is freeSWITCH mature enough so i can sleep at night for at least 3 month without a crash? Thank you for your Time and help in advance, and I am more than willing to take all the information gathered here and create a wiki page to help other people with the same questions/problems. best Ray -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
Usually you don't need to worry about stability issues with FS. For scalability, peoples tend to use openser or some other sip loadbalancer in fron of fs, but you probably would not need that. Live migration of calls is not yet possible, tough. -giovanni On Fri, Aug 28, 2009 at 8:52 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
Hello List,
I have read the current thread about scalability and I would need some advice about a callcenter setup:
First where I come from: I have lot's of problems with an asterisk solution. I have regular crash's and lock-ups, with downgrading and other stuff i got it somewhat stable, but have nevertheless regular hickups. I am desperate and want to get rid of asterisk and I hope that freeSwitch will provide me with a more stable solution.
Our Setup (really nothing special): * 1 Asterisk box, New IBM Hardware (3 month old), 2 HE rack server, 3 GIG of RAM, Xircom analog switch connected to mobile stations for 4 different providers, Digium 4port cards TP400<something> * 8 queues * ~60 agents (which logon, logoff, pause, unpause), not more than 40 concurrently online * ~ 7K - 9K calls (well, CDR entries) a day (not that much for a bpx) * Music on Hold in the call-queues * No special announcement * Transfers between calls in queues and different agents as well as non agents (i mention this because we have transfer related chrashes in asterisk)
The current Problems: * Lockups with different causes (ranging from calls not terminated to heavy thread locking through the AMI interface) * Crashes and library aborts (pthread, libc, crashes related to music on hold, app_queue, transfers)
We used Asterisk: 1.4.23, 1.4.24, 1.4.26rc3, 1.4.26rc5, 1.4.26 and are now back to 1.4.21.2 (stock debian) as anything beyond that is for whatever reason highly unstable for our szenario. Maybe we should have been segmenting the box into one asterisk dedicted to talking to the hardware, one especial for queue/sip handling, i do not know. (all issues are well documented in issues.asterisk.org, but it seems to be very, very difficult to get to the bottom of them as they exist since 1.4.23 as it seems and are open until know and not fixable since month.)
Now, I really would appreciate some success-stories on how you guys managed to get a stable pbx system with freeSWITCH in regard of HA and scalability:
* How to segment freeSWITCH? Or is it stable enough to handle all in one for such a szenario as outlined above? * What would be the best strategie for High Availability / Failover? -> I read in the WIKI (featurelist) that Livemigration of calls from one box to another should be possible? -> I was thinking about using memcached for storing all state information so another freeswitch box can takeover calls from the first box if it dies, is this possible? If so, how? -> Is there anotherway to somehow configure freeSWITCH that in the event of a crash i do not loose the current established calls?
Basically I just want a stable PBX where I do not have to fear every day it will core-dump or abort or Lock up. Is freeSWITCH mature enough so i can sleep at night for at least 3 month without a crash?
Thank you for your Time and help in advance, and I am more than willing to take all the information gathered here and create a wiki page to help other people with the same questions/problems.
best Ray
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
On Fri, Aug 28, 2009 at 12:11 PM, Giovanni Maruzzelli <gmaruzz@celliax.org>wrote:
Usually you don't need to worry about stability issues with FS.
For scalability, peoples tend to use openser or some other sip loadbalancer in fron of fs, but you probably would not need that.
Live migration of calls is not yet possible, tough.
Brian West has done some testing with live migrations but I don't know where he left off. Brian, were you using OpenVZ? I forget... In any case, FS allows you to try to do this with the hope that it will actually work in a production environment. As for the other things - yes, FS can work with the TDM card and the queues, etc. If you are in a position to install FS on a sandbox machine for testing then that would be your best bet. I recommend diving in, which is probably what you did when you first started learning Asterisk... Have fun! -MC
Hi Raimund, One FreeSWITCH box will be quite enough to handle the call volumes that you're talking about, and it ought to be much more stable than the Asterisk solution which you've outlined below. It's probably best to forget about live failover without calls dropping - this isn't something that's supported, and there'd be a lot of work to do to develop code to keep two boxed in sync. Once you get used to a stable solution - i.e. something which doesn't crash - then live failover, HA, etc., will seem somewhat irrelevant. I recently did some work on an FS-based call center solution - drop me a note if I could be of any help with yours. Cheers -- Dave
Hello List,
I have read the current thread about scalability and I would need some advice about a callcenter setup:
First where I come from: I have lot's of problems with an asterisk solution. I have regular crash's and lock-ups, with downgrading and other stuff i got it somewhat stable, but have nevertheless regular hickups. I am desperate and want to get rid of asterisk and I hope that freeSwitch will provide me with a more stable solution.
Our Setup (really nothing special): * 1 Asterisk box, New IBM Hardware (3 month old), 2 HE rack server, 3 GIG of RAM, Xircom analog switch connected to mobile stations for 4 different providers, Digium 4port cards TP400<something> * 8 queues * ~60 agents (which logon, logoff, pause, unpause), not more than 40 concurrently online * ~ 7K - 9K calls (well, CDR entries) a day (not that much for a bpx) * Music on Hold in the call-queues * No special announcement * Transfers between calls in queues and different agents as well as non agents (i mention this because we have transfer related chrashes in asterisk)
The current Problems: * Lockups with different causes (ranging from calls not terminated to heavy thread locking through the AMI interface) * Crashes and library aborts (pthread, libc, crashes related to music on hold, app_queue, transfers)
We used Asterisk: 1.4.23, 1.4.24, 1.4.26rc3, 1.4.26rc5, 1.4.26 and are now back to 1.4.21.2 (stock debian) as anything beyond that is for whatever reason highly unstable for our szenario. Maybe we should have been segmenting the box into one asterisk dedicted to talking to the hardware, one especial for queue/sip handling, i do not know. (all issues are well documented in issues.asterisk.org, but it seems to be very, very difficult to get to the bottom of them as they exist since 1.4.23 as it seems and are open until know and not fixable since month.)
Now, I really would appreciate some success-stories on how you guys managed to get a stable pbx system with freeSWITCH in regard of HA and scalability:
* How to segment freeSWITCH? Or is it stable enough to handle all in one for such a szenario as outlined above? * What would be the best strategie for High Availability / Failover? -> I read in the WIKI (featurelist) that Livemigration of calls from one box to another should be possible? -> I was thinking about using memcached for storing all state information so another freeswitch box can takeover calls from the first box if it dies, is this possible? If so, how? -> Is there anotherway to somehow configure freeSWITCH that in the event of a crash i do not loose the current established calls?
Basically I just want a stable PBX where I do not have to fear every day it will core-dump or abort or Lock up. Is freeSWITCH mature enough so i can sleep at night for at least 3 month without a crash?
Thank you for your Time and help in advance, and I am more than willing to take all the information gathered here and create a wiki page to help other people with the same questions/problems.
best Ray
-- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
Hi David, What have you used on FS for call center, mod_fifo? Can you describe your experience with that, I'm currently interested in call center + FS scenario. Diego On Fri, Aug 28, 2009 at 9:01 PM, David Knell <dave@3c.co.uk> wrote:
Hi Raimund,
One FreeSWITCH box will be quite enough to handle the call volumes that you're talking about, and it ought to be much more stable than the Asterisk solution which you've outlined below.
It's probably best to forget about live failover without calls dropping - this isn't something that's supported, and there'd be a lot of work to do to develop code to keep two boxed in sync. Once you get used to a stable solution - i.e. something which doesn't crash - then live failover, HA, etc., will seem somewhat irrelevant.
I recently did some work on an FS-based call center solution - drop me a note if I could be of any help with yours.
Cheers --
Dave
Hello List,
I have read the current thread about scalability and I would need some advice about a callcenter setup:
First where I come from: I have lot's of problems with an asterisk solution. I have regular crash's and lock-ups, with downgrading and other stuff i got it somewhat stable, but have nevertheless regular hickups. I am desperate and want to get rid of asterisk and I hope that freeSwitch will provide me with a more stable solution.
Our Setup (really nothing special): * 1 Asterisk box, New IBM Hardware (3 month old), 2 HE rack server, 3 GIG of RAM, Xircom analog switch connected to mobile stations for 4 different providers, Digium 4port cards TP400<something> * 8 queues * ~60 agents (which logon, logoff, pause, unpause), not more than 40 concurrently online * ~ 7K - 9K calls (well, CDR entries) a day (not that much for a bpx) * Music on Hold in the call-queues * No special announcement * Transfers between calls in queues and different agents as well as non agents (i mention this because we have transfer related chrashes in asterisk)
The current Problems: * Lockups with different causes (ranging from calls not terminated to heavy thread locking through the AMI interface) * Crashes and library aborts (pthread, libc, crashes related to music on hold, app_queue, transfers)
We used Asterisk: 1.4.23, 1.4.24, 1.4.26rc3, 1.4.26rc5, 1.4.26 and are now back to 1.4.21.2 (stock debian) as anything beyond that is for whatever reason highly unstable for our szenario. Maybe we should have been segmenting the box into one asterisk dedicted to talking to the hardware, one especial for queue/sip handling, i do not know. (all issues are well documented in issues.asterisk.org, but it seems to be very, very difficult to get to the bottom of them as they exist since 1.4.23 as it seems and are open until know and not fixable since month.)
Now, I really would appreciate some success-stories on how you guys managed to get a stable pbx system with freeSWITCH in regard of HA and scalability:
* How to segment freeSWITCH? Or is it stable enough to handle all in one for such a szenario as outlined above? * What would be the best strategie for High Availability / Failover? -> I read in the WIKI (featurelist) that Livemigration of calls from one box to another should be possible? -> I was thinking about using memcached for storing all state information so another freeswitch box can takeover calls from the first box if it dies, is this possible? If so, how? -> Is there anotherway to somehow configure freeSWITCH that in the event of a crash i do not loose the current established calls?
Basically I just want a stable PBX where I do not have to fear every day it will core-dump or abort or Lock up. Is freeSWITCH mature enough so i can sleep at night for at least 3 month without a crash?
Thank you for your Time and help in advance, and I am more than willing to take all the information gathered here and create a wiki page to help other people with the same questions/problems.
best Ray
-- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hi Diego, We didn't use mod_fifo; we built our own queues using an application hanging off the event socket. This was partly down to typical programmer hubris, and partly to allow us to do things that mod_fifo might not without either (a) trawling through mod_fifo, or (b) pestering Anthony. As a brief outline: - operators each live in their own conference - supervisors can add themselves to these conferences for monitoring or coaching - callers sit in queues for the various skills that the operators have. When an operator is free, look at each queue for the skills that the operator has, and choose the caller who's been waiting longest. I used a one-thread-per-call model, so there's a little bit of thought needed to get the access to shared structures right but, essentially, given the low frequency of access to them and the fact that nothing was time-critical at the sub-second level, I just wrapped all of the accesses in a single global mutex, which was easy and is pretty foolproof. The latter is important, given that it was me writing the stuff ;-) Cheers -- Dave
Hi David,
What have you used on FS for call center, mod_fifo?
Can you describe your experience with that, I'm currently interested in call center + FS scenario.
Diego
On Fri, Aug 28, 2009 at 9:01 PM, David Knell <dave@3c.co.uk> wrote: Hi Raimund,
One FreeSWITCH box will be quite enough to handle the call volumes that you're talking about, and it ought to be much more stable than the Asterisk solution which you've outlined below.
It's probably best to forget about live failover without calls dropping - this isn't something that's supported, and there'd be a lot of work to do to develop code to keep two boxed in sync. Once you get used to a stable solution - i.e. something which doesn't crash - then live failover, HA, etc., will seem somewhat irrelevant.
I recently did some work on an FS-based call center solution - drop me a note if I could be of any help with yours.
Cheers --
Dave
> Hello List, > > I have read the current thread about scalability and I would need some > advice about a callcenter setup: > > First where I come from: > I have lot's of problems with an asterisk solution. I have regular > crash's and lock-ups, with downgrading and other stuff i got it > somewhat stable, but have nevertheless regular hickups. I am desperate > and want to get rid of asterisk and I hope that freeSwitch will > provide me with a more stable solution. > > > Our Setup (really nothing special): > * 1 Asterisk box, New IBM Hardware (3 month old), 2 HE rack server, 3 > GIG of RAM, Xircom analog switch connected to mobile stations for 4 > different providers, Digium 4port cards TP400<something> > * 8 queues > * ~60 agents (which logon, logoff, pause, unpause), not more than 40 > concurrently online > * ~ 7K - 9K calls (well, CDR entries) a day (not that much for a bpx) > * Music on Hold in the call-queues > * No special announcement > * Transfers between calls in queues and different agents as well as > non agents (i mention this because we have transfer related chrashes > in asterisk) > > The current Problems: > * Lockups with different causes (ranging from calls not terminated to > heavy thread locking through the AMI interface) > * Crashes and library aborts (pthread, libc, crashes related to music > on hold, app_queue, transfers) > > We used Asterisk: 1.4.23, 1.4.24, 1.4.26rc3, 1.4.26rc5, 1.4.26 and are > now back to 1.4.21.2 (stock debian) as anything beyond that is for > whatever reason highly unstable for our szenario. Maybe we should have > been segmenting the box into one asterisk dedicted to talking to the > hardware, one especial for queue/sip handling, i do not know. (all > issues are well documented in issues.asterisk.org, but it seems to be > very, very difficult to get to the bottom of them as they exist since > 1.4.23 as it seems and are open until know and not fixable since month.) > > > Now, I really would appreciate some success-stories on how you guys > managed to get a stable pbx system with freeSWITCH in regard of HA and > scalability: > > * How to segment freeSWITCH? Or is it stable enough to handle all in > one for such a szenario as outlined above? > * What would be the best strategie for High Availability / Failover? > -> I read in the WIKI (featurelist) that Livemigration of calls from > one box to another should be possible? > -> I was thinking about using memcached for storing all state > information so another freeswitch box can takeover calls from the > first box if it dies, is this possible? If so, how? > -> Is there anotherway to somehow configure freeSWITCH that in the > event of a crash i do not loose the current established calls? > > Basically I just want a stable PBX where I do not have to fear every > day it will core-dump or abort or Lock up. Is freeSWITCH mature enough > so i can sleep at night for at least 3 month without a crash? > > > > Thank you for your Time and help in advance, and I am more than > willing to take all the information gathered here and create a wiki > page to help other people with the same questions/problems. > > best > Ray > > --
David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
Yes, FreeSWITCH is a system that you can trust 100%. I have switched my Asterisk servers to FreeSWITCH and have peace now. If I were you I would get rid of Asterisk and use FreeSWITCH, FS will handle all what you want very well. And I agree with David, fail-over is kinda irrelevant since the FS doesn't crash like Asterisk does. Diego On Fri, Aug 28, 2009 at 6:52 PM, Raimund Sacherer <rs@runsolutions.com>wrote:
Hello List,
I have read the current thread about scalability and I would need some advice about a callcenter setup:
First where I come from: I have lot's of problems with an asterisk solution. I have regular crash's and lock-ups, with downgrading and other stuff i got it somewhat stable, but have nevertheless regular hickups. I am desperate and want to get rid of asterisk and I hope that freeSwitch will provide me with a more stable solution.
Our Setup (really nothing special): * 1 Asterisk box, New IBM Hardware (3 month old), 2 HE rack server, 3 GIG of RAM, Xircom analog switch connected to mobile stations for 4 different providers, Digium 4port cards TP400<something> * 8 queues * ~60 agents (which logon, logoff, pause, unpause), not more than 40 concurrently online * ~ 7K - 9K calls (well, CDR entries) a day (not that much for a bpx) * Music on Hold in the call-queues * No special announcement * Transfers between calls in queues and different agents as well as non agents (i mention this because we have transfer related chrashes in asterisk)
The current Problems: * Lockups with different causes (ranging from calls not terminated to heavy thread locking through the AMI interface) * Crashes and library aborts (pthread, libc, crashes related to music on hold, app_queue, transfers)
We used Asterisk: 1.4.23, 1.4.24, 1.4.26rc3, 1.4.26rc5, 1.4.26 and are now back to 1.4.21.2 (stock debian) as anything beyond that is for whatever reason highly unstable for our szenario. Maybe we should have been segmenting the box into one asterisk dedicted to talking to the hardware, one especial for queue/sip handling, i do not know. (all issues are well documented in issues.asterisk.org, but it seems to be very, very difficult to get to the bottom of them as they exist since 1.4.23 as it seems and are open until know and not fixable since month.)
Now, I really would appreciate some success-stories on how you guys managed to get a stable pbx system with freeSWITCH in regard of HA and scalability:
* How to segment freeSWITCH? Or is it stable enough to handle all in one for such a szenario as outlined above? * What would be the best strategie for High Availability / Failover? -> I read in the WIKI (featurelist) that Livemigration of calls from one box to another should be possible? -> I was thinking about using memcached for storing all state information so another freeswitch box can takeover calls from the first box if it dies, is this possible? If so, how? -> Is there anotherway to somehow configure freeSWITCH that in the event of a crash i do not loose the current established calls?
Basically I just want a stable PBX where I do not have to fear every day it will core-dump or abort or Lock up. Is freeSWITCH mature enough so i can sleep at night for at least 3 month without a crash?
Thank you for your Time and help in advance, and I am more than willing to take all the information gathered here and create a wiki page to help other people with the same questions/problems.
best Ray
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
On Sat, Aug 29, 2009 at 2:34 PM, Diego Viola <diego.viola@gmail.com> wrote:
Yes, FreeSWITCH is a system that you can trust 100%. I have switched my Asterisk servers to FreeSWITCH and have peace now.
If I were you I would get rid of Asterisk and use FreeSWITCH, FS will handle all what you want very well.
And I agree with David, fail-over is kinda irrelevant since the FS doesn't crash like Asterisk does.
You still have hardware failures and fail-over is also useful for hit-less maintenance on boxes. I'd be interested to know how Brian West was approaching his live migration work. Steve
Oh yeah, that would be so helpfull for my situation, as my client *demands* now a solution where he can press a big red button and all fails over to another box. Hi es totally scared because of the Lockups in Asterisk which under specific situations including AMI, Automated Call Setup, and murphy led to a lockup of the entire machine, no console was working anymore, only cold-reset could do it. So, IF there is the possibility for life-takeover, / failover etc. I would love to here how has been done. I am very experienced with openvz and use for about two years now only openvz virtualization servers for anything because of live-migration etc. But as I am new in this company we could not adopt this until now. So Please Ken, if you can, describe what need's to be done to get a failover / takeover working (an outline would be enough) Thanks in Advance -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 11:58 AM, Steve Kurzeja wrote:
On Sat, Aug 29, 2009 at 2:34 PM, Diego Viola <diego.viola@gmail.com> wrote: Yes, FreeSWITCH is a system that you can trust 100%. I have switched my Asterisk servers to FreeSWITCH and have peace now.
If I were you I would get rid of Asterisk and use FreeSWITCH, FS will handle all what you want very well.
And I agree with David, fail-over is kinda irrelevant since the FS doesn't crash like Asterisk does.
You still have hardware failures and fail-over is also useful for hit-less maintenance on boxes.
I'd be interested to know how Brian West was approaching his live migration work.
Steve _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
Thinking about it, maybe we can create a solution, if some of us work together: My strength are in virtualization, linux, development, databases, integration, etc. What I do not now much about is how SIP (and everything else for that matter in the Voice world) works under the hood, and how it's implemented in FS. I know that the state information for a call has to be stored and retrieved somewhere and somehow, only I do not know that part. What I know is that it hast to be do-able to store all the stream information (ip's, port's, current state's, etc.) in a very fast database (e.g. my idea would be memcached) so another FS could just take this information and take over the call, maybe you loose a second of voice, maybe you loose the recorded call file or a part of it, but that should be it. (SipFoundry has a boxed opensource PBX, which, of course is not flexible like FreeSWITCH or Asterisk, but has Call Live Migration and Call Live Failover integrated!). What I want is for my company to be able to sell a 99.99 uptime PBX (we do mostly call-center related stuff), which can scale well, and can grow with the company without lot's of hassles, my Dream would be: To begin with: One Hardware Node with the essential hardware (digium cards for example). On this node are OpenVZ virtualized containers: [VirtCnt1: FS which only talks to the Hardware and forwards everything] = Could be replaced with hardware media gateway, etc. [VirtCnt2: FS which handles the PBX] \___ Loadbalanced, with odbc or xml, Failover, Livetakeover [VirtCnt3: FS which handles the PBX] / [VirtCnt4: Database for state information] (maybe something as resource-friendly as memcached? ressource heavvy database?) With this we can achieve all this: Problem with VirtCnt2 (e.g. crash, lock, ...) * VirtCnt3 can take over. -> You are free without stress to investigate the problem, you can debug and analyze whyle the machine is still running -> you can also create a machine-state-dump of the virtual container, dump the container as well, copy the data to your lab and restore the machine up the state which it was running with the problem, so you can liveinvestigate it in the lab (some prerequirements given, but easy doable) -> just think about the possibility of better bugreports because someone can take the time to read out all the data with GDB to investigate the proper cause of a machine Lock! You want to upgrade to a new FreeSWITCH version? * Take VirtCnt2 out of the LoadBalancing Scheme, * Stop it, Clone it, * Upgrade FreeSWITCH in the cloned Container * Start the cloned container * if there's something wrong, stop it and restart the original VirtCnt2 -> No problem at all, you can Test on the Live Hardware, with part of the Live users (maybe a low-volume queue) to be sure everything works out fine before you activate the full loadbalance Server on it's own can't handle the load * Buy new machine * Setup Hardware Node * Livemigrate VirtCnt3 (no downtime) Now the first Server with the VrtCnt1 and VirtCnt2 as well has to much load * Buy new machine * Setup Hardware Node * Livemigrate VirtCnt2 (no downtime) -> Now you have a 3 server solution (1 mediaprox, 2 loadbalanced / failover PBXes) out of the first box you bought, without headaches, because the system was built for it from the beginning! The Database drains to much? * Buy new machine * Setup Hardware Node * Livemigrate database VirtCnt4 (no downtime) You want to upgrade Hardware/Kernel in Hardware node 1? * Livemigrate VirtCnt2 to a hotstandby machine, or to the other PBX machine, upgrade the hardware, Re-Livemigrate the containers. (no downtime) * OR just break the loadbalancing, wait until all current calls are teared down correctly, upgrade machine, reenable the loadbalancer You want an exact copy of the first server for Hardware HA? * Buy new machine * Setup Hardware node * Buy hardware PRI switchover box * Clone VirtCnt1 - VirtCnt4 to the new machine * Make basic failover configuration -> the sky's the limit, as the saying goes ... So, I can do all the openvz stuff and the integration with database / memcached / heartbeat / whatever is needed here, someone there to be willing to work with me on this on the FreeSWITCH side? or at least provide me with the necessary information about what's needed / how to talk / what states from FreeSWITCH? I know this seems very ambitious but if this could be made in a rather relativly easy to setup package, with good documentation, it would be a boost for FreeSWITCH, i am sure, because after all this is what everyone is grown accustomed to from good old phone companys and the good old pbx's: carrier grade uptimes ... Thanks for everyone reading up until here, all the best, Ray -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 3:17 PM, Raimund Sacherer wrote:
Oh yeah, that would be so helpfull for my situation, as my client *demands* now a solution where he can press a big red button and all fails over to another box. Hi es totally scared because of the Lockups in Asterisk which under specific situations including AMI, Automated Call Setup, and murphy led to a lockup of the entire machine, no console was working anymore, only cold-reset could do it.
So, IF there is the possibility for life-takeover, / failover etc. I would love to here how has been done.
I am very experienced with openvz and use for about two years now only openvz virtualization servers for anything because of live- migration etc. But as I am new in this company we could not adopt this until now.
So Please Ken, if you can, describe what need's to be done to get a failover / takeover working (an outline would be enough)
Thanks in Advance
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
On Aug 29, 2009, at 11:58 AM, Steve Kurzeja wrote:
On Sat, Aug 29, 2009 at 2:34 PM, Diego Viola <diego.viola@gmail.com> wrote: Yes, FreeSWITCH is a system that you can trust 100%. I have switched my Asterisk servers to FreeSWITCH and have peace now.
If I were you I would get rid of Asterisk and use FreeSWITCH, FS will handle all what you want very well.
And I agree with David, fail-over is kinda irrelevant since the FS doesn't crash like Asterisk does.
You still have hardware failures and fail-over is also useful for hit-less maintenance on boxes.
I'd be interested to know how Brian West was approaching his live migration work.
Steve _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
Hi Raimund, The difficult bit in all of this is having calls seamlessly transferred from one box to another when the first box dies. There's a lot of state associated with a call, not all of which is easy to replicate across. 99.99% uptime implies an average of no more than 15 seconds downtime during a 40 hour business week (or about one unscheduled reboot every couple of months), which is easily achievable using FreeSWITCH (and, in my experience, Asterisk) on standard hardware. People agonize about their four or five nines too much, in my opinion. Folk are used to their phones crashing, needing rebooting and dropping calls - we've cellphones to thank for that - and a half-decent VoIP solution will knock the spots off your favourite mobile carrier for reliability. Plus there's all the external factors - 99.999% uptime on your PRI means that someone can only drive a digger through the cable once every 273 years, assuming that it takes a day to fix, and no telco's going to give you that in an SLA. And your power can't go off. And so on. Lastly, I'm afraid that virtualization and VoIP don't play well together, at least not if you want to achieve a sensible density. The large number of small packets being moved around the network interfaces - both physical and virtual - will quickly chew up your CPU. Cheers -- Dave
Thinking about it, maybe we can create a solution, if some of us work together:
My strength are in virtualization, linux, development, databases, integration, etc. What I do not now much about is how SIP (and everything else for that matter in the Voice world) works under the hood, and how it's implemented in FS.
I know that the state information for a call has to be stored and retrieved somewhere and somehow, only I do not know that part. What I know is that it hast to be do-able to store all the stream information (ip's, port's, current state's, etc.) in a very fast database (e.g. my idea would be memcached) so another FS could just take this information and take over the call, maybe you loose a second of voice, maybe you loose the recorded call file or a part of it, but that should be it. (SipFoundry has a boxed opensource PBX, which, of course is not flexible like FreeSWITCH or Asterisk, but has Call Live Migration and Call Live Failover integrated!).
What I want is for my company to be able to sell a 99.99 uptime PBX (we do mostly call-center related stuff), which can scale well, and can grow with the company without lot's of hassles, my Dream would be:
To begin with:
One Hardware Node with the essential hardware (digium cards for example). On this node are OpenVZ virtualized containers: [VirtCnt1: FS which only talks to the Hardware and forwards everything] = Could be replaced with hardware media gateway, etc. [VirtCnt2: FS which handles the PBX] \___ Loadbalanced, with odbc or xml, Failover, Livetakeover [VirtCnt3: FS which handles the PBX] / [VirtCnt4: Database for state information] (maybe something as resource-friendly as memcached? ressource heavvy database?)
With this we can achieve all this:
Problem with VirtCnt2 (e.g. crash, lock, ...) * VirtCnt3 can take over. -> You are free without stress to investigate the problem, you can debug and analyze whyle the machine is still running -> you can also create a machine-state-dump of the virtual container, dump the container as well, copy the data to your lab and restore the machine up the state which it was running with the problem, so you can liveinvestigate it in the lab (some prerequirements given, but easy doable) -> just think about the possibility of better bugreports because someone can take the time to read out all the data with GDB to investigate the proper cause of a machine Lock!
You want to upgrade to a new FreeSWITCH version? * Take VirtCnt2 out of the LoadBalancing Scheme, * Stop it, Clone it, * Upgrade FreeSWITCH in the cloned Container * Start the cloned container * if there's something wrong, stop it and restart the original VirtCnt2 -> No problem at all, you can Test on the Live Hardware, with part of the Live users (maybe a low-volume queue) to be sure everything works out fine before you activate the full loadbalance
Server on it's own can't handle the load * Buy new machine * Setup Hardware Node * Livemigrate VirtCnt3 (no downtime)
Now the first Server with the VrtCnt1 and VirtCnt2 as well has to much load * Buy new machine * Setup Hardware Node * Livemigrate VirtCnt2 (no downtime) -> Now you have a 3 server solution (1 mediaprox, 2 loadbalanced / failover PBXes) out of the first box you bought, without headaches, because the system was built for it from the beginning!
The Database drains to much? * Buy new machine * Setup Hardware Node * Livemigrate database VirtCnt4 (no downtime)
You want to upgrade Hardware/Kernel in Hardware node 1? * Livemigrate VirtCnt2 to a hotstandby machine, or to the other PBX machine, upgrade the hardware, Re-Livemigrate the containers. (no downtime) * OR just break the loadbalancing, wait until all current calls are teared down correctly, upgrade machine, reenable the loadbalancer
You want an exact copy of the first server for Hardware HA? * Buy new machine * Setup Hardware node * Buy hardware PRI switchover box * Clone VirtCnt1 - VirtCnt4 to the new machine * Make basic failover configuration
-> the sky's the limit, as the saying goes ...
So, I can do all the openvz stuff and the integration with database / memcached / heartbeat / whatever is needed here, someone there to be willing to work with me on this on the FreeSWITCH side? or at least provide me with the necessary information about what's needed / how to talk / what states from FreeSWITCH?
I know this seems very ambitious but if this could be made in a rather relativly easy to setup package, with good documentation, it would be a boost for FreeSWITCH, i am sure, because after all this is what everyone is grown accustomed to from good old phone companys and the good old pbx's: carrier grade uptimes ...
Thanks for everyone reading up until here, all the best,
Ray
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
On Aug 29, 2009, at 3:17 PM, Raimund Sacherer wrote:
Oh yeah, that would be so helpfull for my situation, as my client *demands* now a solution where he can press a big red button and all fails over to another box. Hi es totally scared because of the Lockups in Asterisk which under specific situations including AMI, Automated Call Setup, and murphy led to a lockup of the entire machine, no console was working anymore, only cold-reset could do it.
So, IF there is the possibility for life-takeover, / failover etc. I would love to here how has been done.
I am very experienced with openvz and use for about two years now only openvz virtualization servers for anything because of live-migration etc. But as I am new in this company we could not adopt this until now.
So Please Ken, if you can, describe what need's to be done to get a failover / takeover working (an outline would be enough)
Thanks in Advance
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
On Aug 29, 2009, at 11:58 AM, Steve Kurzeja wrote:
On Sat, Aug 29, 2009 at 2:34 PM, Diego Viola <diego.viola@gmail.com> wrote: Yes, FreeSWITCH is a system that you can trust 100%. I have switched my Asterisk servers to FreeSWITCH and have peace now.
If I were you I would get rid of Asterisk and use FreeSWITCH, FS will handle all what you want very well.
And I agree with David, fail-over is kinda irrelevant since the FS doesn't crash like Asterisk does.
You still have hardware failures and fail-over is also useful for hit-less maintenance on boxes.
I'd be interested to know how Brian West was approaching his live migration work.
Steve _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
On PRI circuits failover with ha is possible using a red tone appliance, but I m sure it's going to be almost impossible to transfer the running calls during the event of fs box going down. Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com On 29-Aug-2009, at 11:35 PM, David Knell <dave@3c.co.uk> wrote:
Hi Raimund,
The difficult bit in all of this is having calls seamlessly transferred from one box to another when the first box dies. There's a lot of state associated with a call, not all of which is easy to replicate across.
99.99% uptime implies an average of no more than 15 seconds downtime during a 40 hour business week (or about one unscheduled reboot every couple of months), which is easily achievable using FreeSWITCH (and, in my experience, Asterisk) on standard hardware.
People agonize about their four or five nines too much, in my opinion. Folk are used to their phones crashing, needing rebooting and dropping calls - we've cellphones to thank for that - and a half-decent VoIP solution will knock the spots off your favourite mobile carrier for reliability. Plus there's all the external factors - 99.999% uptime on your PRI means that someone can only drive a digger through the cable once every 273 years, assuming that it takes a day to fix, and no telco's going to give you that in an SLA. And your power can't go off. And so on.
Lastly, I'm afraid that virtualization and VoIP don't play well together, at least not if you want to achieve a sensible density. The large number of small packets being moved around the network interfaces - both physical and virtual - will quickly chew up your CPU.
Cheers --
Dave
Thinking about it, maybe we can create a solution, if some of us work together:
My strength are in virtualization, linux, development, databases, integration, etc. What I do not now much about is how SIP (and everything else for that matter in the Voice world) works under the hood, and how it's implemented in FS.
I know that the state information for a call has to be stored and retrieved somewhere and somehow, only I do not know that part. What I know is that it hast to be do-able to store all the stream information (ip's, port's, current state's, etc.) in a very fast database (e.g. my idea would be memcached) so another FS could just take this information and take over the call, maybe you loose a second of voice, maybe you loose the recorded call file or a part of it, but that should be it. (SipFoundry has a boxed opensource PBX, which, of course is not flexible like FreeSWITCH or Asterisk, but has Call Live Migration and Call Live Failover integrated!).
What I want is for my company to be able to sell a 99.99 uptime PBX (we do mostly call-center related stuff), which can scale well, and can grow with the company without lot's of hassles, my Dream would be:
To begin with:
One Hardware Node with the essential hardware (digium cards for example). On this node are OpenVZ virtualized containers: [VirtCnt1: FS which only talks to the Hardware and forwards everything] = Could be replaced with hardware media gateway, etc. [VirtCnt2: FS which handles the PBX] \___ Loadbalanced, with odbc or xml, Failover, Livetakeover [VirtCnt3: FS which handles the PBX] / [VirtCnt4: Database for state information] (maybe something as resource-friendly as memcached? ressource heavvy database?)
With this we can achieve all this:
Problem with VirtCnt2 (e.g. crash, lock, ...) * VirtCnt3 can take over. -> You are free without stress to investigate the problem, you can debug and analyze whyle the machine is still running -> you can also create a machine-state-dump of the virtual container, dump the container as well, copy the data to your lab and restore the machine up the state which it was running with the problem, so you can liveinvestigate it in the lab (some prerequirements given, but easy doable) -> just think about the possibility of better bugreports because someone can take the time to read out all the data with GDB to investigate the proper cause of a machine Lock!
You want to upgrade to a new FreeSWITCH version? * Take VirtCnt2 out of the LoadBalancing Scheme, * Stop it, Clone it, * Upgrade FreeSWITCH in the cloned Container * Start the cloned container * if there's something wrong, stop it and restart the original VirtCnt2 -> No problem at all, you can Test on the Live Hardware, with part of the Live users (maybe a low-volume queue) to be sure everything works out fine before you activate the full loadbalance
Server on it's own can't handle the load * Buy new machine * Setup Hardware Node * Livemigrate VirtCnt3 (no downtime)
Now the first Server with the VrtCnt1 and VirtCnt2 as well has to much load * Buy new machine * Setup Hardware Node * Livemigrate VirtCnt2 (no downtime) -> Now you have a 3 server solution (1 mediaprox, 2 loadbalanced / failover PBXes) out of the first box you bought, without headaches, because the system was built for it from the beginning!
The Database drains to much? * Buy new machine * Setup Hardware Node * Livemigrate database VirtCnt4 (no downtime)
You want to upgrade Hardware/Kernel in Hardware node 1? * Livemigrate VirtCnt2 to a hotstandby machine, or to the other PBX machine, upgrade the hardware, Re-Livemigrate the containers. (no downtime) * OR just break the loadbalancing, wait until all current calls are teared down correctly, upgrade machine, reenable the loadbalancer
You want an exact copy of the first server for Hardware HA? * Buy new machine * Setup Hardware node * Buy hardware PRI switchover box * Clone VirtCnt1 - VirtCnt4 to the new machine * Make basic failover configuration
-> the sky's the limit, as the saying goes ...
So, I can do all the openvz stuff and the integration with database / memcached / heartbeat / whatever is needed here, someone there to be willing to work with me on this on the FreeSWITCH side? or at least provide me with the necessary information about what's needed / how to talk / what states from FreeSWITCH?
I know this seems very ambitious but if this could be made in a rather relativly easy to setup package, with good documentation, it would be a boost for FreeSWITCH, i am sure, because after all this is what everyone is grown accustomed to from good old phone companys and the good old pbx's: carrier grade uptimes ...
Thanks for everyone reading up until here, all the best,
Ray
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
On Aug 29, 2009, at 3:17 PM, Raimund Sacherer wrote:
Oh yeah, that would be so helpfull for my situation, as my client *demands* now a solution where he can press a big red button and all fails over to another box. Hi es totally scared because of the Lockups in Asterisk which under specific situations including AMI, Automated Call Setup, and murphy led to a lockup of the entire machine, no console was working anymore, only cold-reset could do it.
So, IF there is the possibility for life-takeover, / failover etc. I would love to here how has been done.
I am very experienced with openvz and use for about two years now only openvz virtualization servers for anything because of live-migration etc. But as I am new in this company we could not adopt this until now.
So Please Ken, if you can, describe what need's to be done to get a failover / takeover working (an outline would be enough)
Thanks in Advance
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
On Aug 29, 2009, at 11:58 AM, Steve Kurzeja wrote:
On Sat, Aug 29, 2009 at 2:34 PM, Diego Viola <diego.viola@gmail.com> wrote: Yes, FreeSWITCH is a system that you can trust 100%. I have switched my Asterisk servers to FreeSWITCH and have peace now.
If I were you I would get rid of Asterisk and use FreeSWITCH, FS will handle all what you want very well.
And I agree with David, fail-over is kinda irrelevant since the FS doesn't crash like Asterisk does.
You still have hardware failures and fail-over is also useful for hit-less maintenance on boxes.
I'd be interested to know how Brian West was approaching his live migration work.
Steve _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/ freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org -- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
I was able to do this using OpenVZ, You can get away with it on smaller instances... like if you're doing one instance per company but don't expect live migration to work as well on large instances with thousands of calls up at once. You need a fast network, fast disks and to follow the howto on the wiki. /b On Aug 29, 2009, at 4:58 AM, Steve Kurzeja wrote:
You still have hardware failures and fail-over is also useful for hit-less maintenance on boxes.
I'd be interested to know how Brian West was approaching his live migration work.
Steve
We have previously estimated the development of live fail over (after a box dies where live migration is no longer possible) to exceed 100k in development costs. It requires several additions to the sofia sip library, freeswitch and a dependancy on some other code we would have to implement to manage it. It may or may not be worth it to raise that kind of funding just to avoid an occasional disaster. Then there is a matter of securing the time of the developers necessary to carry out the implementation. On Aug 29, 2009 10:19 AM, "Brian West" <brian@freeswitch.org> wrote: I was able to do this using OpenVZ, You can get away with it on smaller instances... like if you're doing one instance per company but don't expect live migration to work as well on large instances with thousands of calls up at once. You need a fast network, fast disks and to follow the howto on the wiki. /b On Aug 29, 2009, at 4:58 AM, Steve Kurzeja wrote: > You still have hardware failures and fail-over... _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists...
Hmm, so basically 100 interested companys which each chip in 1000 bucks :-) sounds like lots of manpower, but on the other hand, I do not know the issues regarding the SIP protocoll, but, basically, isn't it *just* to tell another FS box to listen on port x for voicetraffic, forward it to ip on port y? ok, i understand there's a lot going on under the hood, i guess it would mean to setup a call, but take care to not really set up the call, just the internal state ... hmm, could it theoretically be done with the event system? ok, i guess I have to dive further into the internals to fully understand the scope. But a live migration, where the box is available, be possible right now? Would be a step i would like to implement just to be able to do work on a hardware node if necesary without interrupting the service ... -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 6:01 PM, Anthony Minessale wrote:
We have previously estimated the development of live fail over (after a box dies where live migration is no longer possible) to exceed 100k in development costs.
It requires several additions to the sofia sip library, freeswitch and a dependancy on some other code we would have to implement to manage it.
It may or may not be worth it to raise that kind of funding just to avoid an occasional disaster.
Then there is a matter of securing the time of the developers necessary to carry out the implementation.
On Aug 29, 2009 10:19 AM, "Brian West" <brian@freeswitch.org> wrote:
I was able to do this using OpenVZ, You can get away with it on smaller instances... like if you're doing one instance per company but don't expect live migration to work as well on large instances with thousands of calls up at once. You need a fast network, fast disks and to follow the howto on the wiki.
/b On Aug 29, 2009, at 4:58 AM, Steve Kurzeja wrote: > You still have hardware failures and fail-over...
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists...
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
It is not possible to do a live migration at all right now... There is no way to move a call and have all the states required for that call to magically re-appear on a different instance. This will require a fair bit of work to get there. It is possible to configure fs via some back end DB magic to share configurations ie: n+1 or ³warm standby² style fault tolerance but you are going to loose the calls that are up when the failure occurs. From: Raimund Sacherer <rs@runsolutions.com> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Sat, 29 Aug 2009 20:33:21 +0200 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing Hmm, so basically 100 interested companys which each chip in 1000 bucks :-) sounds like lots of manpower, but on the other hand, I do not know the issues regarding the SIP protocoll, but, basically, isn't it *just* to tell another FS box to listen on port x for voicetraffic, forward it to ip on port y? ok, i understand there's a lot going on under the hood, i guess it would mean to setup a call, but take care to not really set up the call, just the internal state ... hmm, could it theoretically be done with the event system? ok, i guess I have to dive further into the internals to fully understand the scope. But a live migration, where the box is available, be possible right now? Would be a step i would like to implement just to be able to do work on a hardware node if necesary without interrupting the service ... -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 6:01 PM, Anthony Minessale wrote:
We have previously estimated the development of live fail over (after a box dies where live migration is no longer possible) to exceed 100k in development costs.
It requires several additions to the sofia sip library, freeswitch and a dependancy on some other code we would have to implement to manage it.
It may or may not be worth it to raise that kind of funding just to avoid an occasional disaster.
Then there is a matter of securing the time of the developers necessary to carry out the implementation.
On Aug 29, 2009 10:19 AM, "Brian West" <brian@freeswitch.org> wrote:
I was able to do this using OpenVZ, You can get away with it on smaller instances... like if you're doing one instance per company but don't expect live migration to work as well on large instances with thousands of calls up at once. You need a fast network, fast disks and to follow the howto on the wiki.
/b
On Aug 29, 2009, at 4:58 AM, Steve Kurzeja wrote: > You still have hardware failures and fail-over...
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists...
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar. Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com On 30-Aug-2009, at 12:14 AM, Ken Rice <krice@freeswitch.org> wrote:
It is not possible to do a live migration at all right now... There is no way to move a call and have all the states required for that call to magically re-appear on a different instance. This will require a fair bit of work to get there.
It is possible to configure fs via some back end DB magic to share configurations ie: n+1 or “warm standby” style fault tolerance but you are going to loose the calls that are up when the failure oc curs.
From: Raimund Sacherer <rs@runsolutions.com> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Sat, 29 Aug 2009 20:33:21 +0200 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Hmm, so basically 100 interested companys which each chip in 1000 bucks :-)
sounds like lots of manpower, but on the other hand, I do not know the issues regarding the SIP protocoll, but, basically, isn't it *just* to tell another FS box to listen on port x for voicetraffic, forward it to ip on port y?
ok, i understand there's a lot going on under the hood, i guess it would mean to setup a call, but take care to not really set up the call, just the internal state ...
hmm, could it theoretically be done with the event system? ok, i guess I have to dive further into the internals to fully understand the scope.
But a live migration, where the box is available, be possible right now? Would be a step i would like to implement just to be able to do work on a hardware node if necesary without interrupting the service ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares
On Aug 29, 2009, at 6:01 PM, Anthony Minessale wrote:
We have previously estimated the development of live fail over (after a box dies where live migration is no longer possible) to exceed 100k in development costs.
It requires several additions to the sofia sip library, freeswitch and a dependancy on some other code we would have to implement to manage it.
It may or may not be worth it to raise that kind of funding just to avoid an occasional disaster.
Then there is a matter of securing the time of the developers necessary to carry out the implementation.
On Aug 29, 2009 10:19 AM, "Brian West" <brian@freeswitch.org> wrote:
I was able to do this using OpenVZ, You can get away with it on smaller instances... like if you're doing one instance per company but don't expect live migration to work as well on large instances with thousands of calls up at once. You need a fast network, fast disks and to follow the howto on the wiki.
/b
On Aug 29, 2009, at 4:58 AM, Steve Kurzeja wrote: > You still have hardware failures and fail-over...
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists...
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication.
This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed.
Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ... -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task.
/b
On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication.
This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed.
Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
High Availability is always an interesting problem. PSTN equipment generally overcomes it through duplication of messaging and hardware. Obviously at an E1 (T1 for the US folks) or Subscribers Line Level this is not possible. This duplication comes at a cost and lowers the calls per minute a softswitch can handle for the amount of hardware purchased. Like everything it is a trade-off between cost and reliability. Per Dave's comment, duplication on a network level can be achieved easily through duplicating network switches and using OSPF, VRRP or network bonding to ensure continuity if a path is compromised. From experience this is exactly what the vendors do and it does work. During failover the calls experience a minut breakup of voice quality but they still hold up ok. Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue). Regards, On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly... Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hi Ken, I don't disagree, the scenario you mentioned is just the tip of the iceberg when it comes to getting something like this working smoothly. However to try and cover them all in a single post (assuming that I could) would have resulted in something akin to War and Peace and probably bored the pants off of most readers :) For any HA solution to move from a concept into reality, it would require some serious brainstorming to identify all scenarios and how to handle them. Given that many posts in this thread mentioned transfer of call state at time of failure or some form of virtualisation, I wanted to provide another solution that would provide real time call state updates between FS boxes utilising mechanisms that are largely already in-place and thus (hopefully) reducing software development time if someone was willing to cough up the $$$$$. Regards, On Mon, Aug 31, 2009 at 1:23 PM, Ken Rice<krice@freeswitch.org> wrote:
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly...
Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
I was thinking more like something the size of war and peace but written with the style of an RFC (and about as interesting and an RFC) I know there has already been some discussion on several fronts of atleast getting the core and several other pieces to where they need to be for stateful failover and I'm not sure if its been mentioned here, but sofia is going to be a bit of work and estimates run in the 100K USD range. Now if we could get a get a couple of corporate sponsors to help here it would be great. K
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 14:32:22 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Hi Ken,
I don't disagree, the scenario you mentioned is just the tip of the iceberg when it comes to getting something like this working smoothly. However to try and cover them all in a single post (assuming that I could) would have resulted in something akin to War and Peace and probably bored the pants off of most readers :) For any HA solution to move from a concept into reality, it would require some serious brainstorming to identify all scenarios and how to handle them.
Given that many posts in this thread mentioned transfer of call state at time of failure or some form of virtualisation, I wanted to provide another solution that would provide real time call state updates between FS boxes utilising mechanisms that are largely already in-place and thus (hopefully) reducing software development time if someone was willing to cough up the $$$$$.
Regards,
On Mon, Aug 31, 2009 at 1:23 PM, Ken Rice<krice@freeswitch.org> wrote:
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly...
Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Ken Rice <krice@freeswitch.org> wrote:
I know there has already been some discussion on several fronts of atleast getting the core and several other pieces to where they need to be for stateful failover and I'm not sure if its been mentioned here, but sofia is going to be a bit of work and estimates run in the 100K USD range. Now if we could get a get a couple of corporate sponsors to help here it would be great.
I agree. Perhaps some of those corporations which are saving money over proprietary solutions by using FreeSWITCH, and who also want the high-availability functionality, would be ideal sponsors for the work.
Shouldn't be so hard, right? But there's human nature, even if the corporations get to save, like, 100k worth of licenses and hardware and other costs it is very hard to get them to spend a little back to the community which helped to achive that goal. I think my first, very ambitous goal to make the sofia rewrite a personal long-term hobby is not going to be very realistic, because of my daytime job and my private life. But I will try to persuade my employer to make at least a little contribution, as I really want this in FS, maybe we should * make it a bounty project * try to bring in some big sponsors (who are the biggest companys using FS anyway?) * make some awareness in the voip scene?? seriously what else can be done to get the estimated 100k to become reality? I really want this feature in FS! :-) best, -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 31, 2009, at 7:57 AM, Jason White wrote:
Ken Rice <krice@freeswitch.org> wrote:
I know there has already been some discussion on several fronts of atleast getting the core and several other pieces to where they need to be for stateful failover and I'm not sure if its been mentioned here, but sofia is going to be a bit of work and estimates run in the 100K USD range. Now if we could get a get a couple of corporate sponsors to help here it would be great.
I agree. Perhaps some of those corporations which are saving money over proprietary solutions by using FreeSWITCH, and who also want the high-availability functionality, would be ideal sponsors for the work.
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
A random thought. It strikes me that adding support for live call failover to FreeSWITCH might well be the wrong thing to do - it's something which is probably not too difficult to achieve if starting from the ground up, but which is going to be a right PITA to do retrospectively. And, once you've done switching, there's conferencing, IVR stuff... Here's an alternative approach. Have two FS boxes (which I'll call FS1 and FS2 - you can't fault *my* imagination) and a front-end B2BUA/RTP proxy with a bit of a difference. The B2BUA sends incoming stuff to both FS1 and FS2 as two separate call legs, and the RTP proxy forwards inbound RTP to both FS1 and FS2. Assuming that FS1 is the current "master", it forwards RTP and call state transitions from FS1 back to the caller. If it detects that FS1 has gone away, then - as FS2 should be in roughly the same state as FS1 - it flips to using FS2 as its source. The internal state of the B2BUA/RTP proxy should be relatively simple, and therefore should be able to be duplicated to a hot standby. I can see a collection of gotchas, such as registration (the B2BUA needs access to the user database), and I haven't even begun to try to think of how to deal with calls *from* the FS boxes. Actually, that's not true: I have, and they clearly need to go through the same sort of mechanism in reverse, which is fine until I start trying to work out how to pair up calls from FS1 and FS2 reliably, and then my head hurts. But I think that this idea has merit. Firstly, it abstracts the failover mechanism away into one place; secondly, it is general - FS1 and FS2 could just as easily be Asterisk1 and Asterisk2. And I'll do it for just $98,650.. --Dave
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly...
Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
Hi David, On Aug 31, 2009, at 2:22 PM, David Knell wrote:
A random thought. It strikes me that adding support for live call failover to FreeSWITCH might well be the wrong thing to do - it's something which is probably not too difficult to achieve if starting from the ground up, but which is going to be a right PITA to do retrospectively. And, once you've done switching, there's conferencing, IVR stuff...
Here's an alternative approach. Have two FS boxes (which I'll call FS1 and FS2 - you can't fault *my* imagination) and a front-end B2BUA/RTP proxy with a bit of a difference. The B2BUA sends incoming stuff to both FS1 and FS2 as two separate call legs, and the RTP proxy forwards inbound RTP to both FS1 and FS2. Assuming that FS1 is the current "master", it forwards RTP and call state transitions from FS1 back to the caller. If it detects that FS1 has gone away, then - as FS2 should be in roughly the same state as FS1 - it flips to using FS2 as its source.
The internal state of the B2BUA/RTP proxy should be relatively simple, and therefore should be able to be duplicated to a hot standby.
I can see a collection of gotchas, such as registration (the B2BUA needs access to the user database), and I haven't even begun to try to think of how to deal with calls *from* the FS boxes. Actually, that's not true: I have, and they clearly need to go through the same sort of mechanism in reverse, which is fine until I start trying to work out how to pair up calls from FS1 and FS2 reliably, and then my head hurts. Yeah, sounds like a real challenge
But I think that this idea has merit. Firstly, it abstracts the failover mechanism away into one place; secondly, it is general - FS1 and FS2 could just as easily be Asterisk1 and Asterisk2.
this, in theory, sound quite interesting ... would be nice to have a proof of concept setup thought. -> Sound's not quite THAT nice than a full-built-in solution to FS ... but if it works ...
And I'll do it for just $98,650..
Haha, that's really funny :-)
--Dave
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly...
Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com
wrote: So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/ freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/ freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
-- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
And I'll do it for just $98,650..
Haha, that's really funny :-) Dave is talking New Zealand Dollars ;) On Mon, Aug 31, 2009 at 10:36 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
Hi David,
On Aug 31, 2009, at 2:22 PM, David Knell wrote:
A random thought. It strikes me that adding support for live call failover to FreeSWITCH might well be the wrong thing to do - it's something which is probably not too difficult to achieve if starting from the ground up, but which is going to be a right PITA to do retrospectively. And, once you've done switching, there's conferencing, IVR stuff...
Here's an alternative approach. Have two FS boxes (which I'll call FS1 and FS2 - you can't fault *my* imagination) and a front-end B2BUA/RTP proxy with a bit of a difference. The B2BUA sends incoming stuff to both FS1 and FS2 as two separate call legs, and the RTP proxy forwards inbound RTP to both FS1 and FS2. Assuming that FS1 is the current "master", it forwards RTP and call state transitions from FS1 back to the caller. If it detects that FS1 has gone away, then - as FS2 should be in roughly the same state as FS1 - it flips to using FS2 as its source.
The internal state of the B2BUA/RTP proxy should be relatively simple, and therefore should be able to be duplicated to a hot standby.
I can see a collection of gotchas, such as registration (the B2BUA needs access to the user database), and I haven't even begun to try to think of how to deal with calls *from* the FS boxes. Actually, that's not true: I have, and they clearly need to go through the same sort of mechanism in reverse, which is fine until I start trying to work out how to pair up calls from FS1 and FS2 reliably, and then my head hurts. Yeah, sounds like a real challenge
But I think that this idea has merit. Firstly, it abstracts the failover mechanism away into one place; secondly, it is general - FS1 and FS2 could just as easily be Asterisk1 and Asterisk2.
this, in theory, sound quite interesting ... would be nice to have a proof of concept setup thought.
-> Sound's not quite THAT nice than a full-built-in solution to FS ... but if it works ...
And I'll do it for just $98,650..
Haha, that's really funny :-)
--Dave
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly...
Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com
wrote: So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/ freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/ freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
-- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
Actually Dave, our idea's are very similar. The difference being is that you fork the calls into 2 separate legs at a B2BUA and sending each to an FS box, while I would duplicate messages to an FS box acting in Standby mode using extra SIP headers to let the standby machine know the RTP port number and the UUID. These 2 FS boxes would essentially become the core switching units in Active/Standby configuration and be used to handle switching logic and billing. Thus if FS2 became the Active box, it would have exactly the same call information as FS1 had. Theoretically there would nothing stopping FS1 and FS2 being master and standby for each other at the same time and both handling traffic, although this has the potential to cause over load if something did fail. i.e FS1 receives INVITE, handles call and forwards INVITE to Gateway and then sends a second INVITE (Duplication of Original message, utilising extra SIP headers to pass on required information) to FS2. FS1 fails FS2 takes over and handles call. Then you would look at way's to distribute functionality across other boxes to allow scalability. An example would be to use dedicated FS boxes for TDM cards and Opensips to handle registrations, SIP proxy or SBC controllers. Of course this assumes you are looking to build a softswitch that can handle large call numbers and have the $$$ to do so. Sadly, I have niether the money or the requirement. On Mon, Aug 31, 2009 at 10:22 PM, David Knell<dave@3c.co.uk> wrote:
A random thought. It strikes me that adding support for live call failover to FreeSWITCH might well be the wrong thing to do - it's something which is probably not too difficult to achieve if starting from the ground up, but which is going to be a right PITA to do retrospectively. And, once you've done switching, there's conferencing, IVR stuff...
Here's an alternative approach. Have two FS boxes (which I'll call FS1 and FS2 - you can't fault *my* imagination) and a front-end B2BUA/RTP proxy with a bit of a difference. The B2BUA sends incoming stuff to both FS1 and FS2 as two separate call legs, and the RTP proxy forwards inbound RTP to both FS1 and FS2. Assuming that FS1 is the current "master", it forwards RTP and call state transitions from FS1 back to the caller. If it detects that FS1 has gone away, then - as FS2 should be in roughly the same state as FS1 - it flips to using FS2 as its source.
The internal state of the B2BUA/RTP proxy should be relatively simple, and therefore should be able to be duplicated to a hot standby.
I can see a collection of gotchas, such as registration (the B2BUA needs access to the user database), and I haven't even begun to try to think of how to deal with calls *from* the FS boxes. Actually, that's not true: I have, and they clearly need to go through the same sort of mechanism in reverse, which is fine until I start trying to work out how to pair up calls from FS1 and FS2 reliably, and then my head hurts.
But I think that this idea has merit. Firstly, it abstracts the failover mechanism away into one place; secondly, it is general - FS1 and FS2 could just as easily be Asterisk1 and Asterisk2.
And I'll do it for just $98,650..
--Dave
Your forgetting that with RTP theres the duplication of RTP that may be required along with a mechanism that keeps media timers working properly...
Where many things don't support RTP timers they sure come in handy on a regular basis especially dealing with several of the other software based platforms out there that just love to have calls end but never want to send a BYE
From: Jim Burke <jim@evolutiontel.net> Reply-To: <freeswitch-users@lists.freeswitch.org> Date: Mon, 31 Aug 2009 12:54:50 +1000 To: <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing
Conceptually in a pure SIP installation of Freeswitch, hot standby or active standby could be achieved through duplication of messages (INVITE, 200OK and BYE for calls) between 2 boxes and then using VRRP to change the IP address when the box falls over. Going this way allows call state to be as up to date as possible. Obviously this would require logic added to sofia to transfer the RTP port information and UUID information between the FS instances (Easily achieved using additional SIP headers). It would also require that sofia does not forward duplicated messages to gateways or user agents. CDR information would also need to be generated correctly (My experience of Radius with Opensips is that it generates start and stop messages on INVITE/200OK then BYE respectively so this could be an easy solution to the CDR issue).
Regards,
On Sun, Aug 30, 2009 at 11:19 PM, Raimund Sacherer<rs@runsolutions.com> wrote:
So, the first way would be to have a look on the sip stack, which is, in fact, sofia ..., well, sound's like a nice, fun, long-going hobby project to me :-) I will definitly at least look at the sip code to check if it is something i would myself willingly give over to ... But I *really* do want a setup where it does not matter if one specific FS instance is UP or NOT ...
-- Raimund Sacherer - RunSolutions Open Source It Consulting -
Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 10:04 PM, Brian West wrote:
The sip stack needs to be modified to spin that data up into the state machine so that it can take over calls once the fail over takes place... its not an easy task. /b On Aug 29, 2009, at 2:56 PM, Mitul Limbani wrote:
Is itnpossible to have a db cluster know the state of each and every call and then use Heartbeat on this db + fs cluster so that clients see only one ip where as internally all fs boxes refer db for call states, db again is under replication. This in the thioery can be written, but I am sure if we think bit more on this direction the problem seem to be getting addressed. Other guys also chip in their 2 cents, we just need 50 of em to make a full dollar.
Thanks & Regards, Mitul Limbani, Founder & CEO, Enterux Solutions Pvt. Ltd., The Enterprise Linux Company (r), http://www.enterux.com http://www.entVoice.com
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- David Knell, Director, 3C Limited T: +44 20 3298 2000 E: dave@3c.co.uk W: http://www.3c.co.uk
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Jim Burke Director Evolutiontel. http://www.evolutiontel.net
Also please remember this means every _application_ also must be "failover" aware. So, if you're using a little calling card or billing app to control your routing, that app will have to explicitly enroll its state as well as restore itself. Live migration is possible via OpenVZ. OVZ does not have the same overhead of other virtualization techniques such as Hyper-V, ESX, Xen, etc. So you can do media without much worry. OVZ can work with other virtualization, too; you can spin it up inside of a VMware machine. But this is *live* migration of the container. It's "just" pausing your container, copying the whole memory, and resuming it on another node. This is not "dead" migration or failover after calls break. However, you could try running a very frequent sync in OVZ, so that if there was a node failure, you could resume from a previously saved state. But, I highly doubt performance would be good enough for this to work successfully on any serious deployment. -Michael From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Raimund Sacherer Sent: Saturday, August 29, 2009 12:33 PM To: freeswitch-users@lists.freeswitch.org Subject: Re: [Freeswitch-users] FreeSWITCH HA + Loadbalancing Hmm, so basically 100 interested companys which each chip in 1000 bucks :-) sounds like lots of manpower, but on the other hand, I do not know the issues regarding the SIP protocoll, but, basically, isn't it *just* to tell another FS box to listen on port x for voicetraffic, forward it to ip on port y? ok, i understand there's a lot going on under the hood, i guess it would mean to setup a call, but take care to not really set up the call, just the internal state ... hmm, could it theoretically be done with the event system? ok, i guess I have to dive further into the internals to fully understand the scope. But a live migration, where the box is available, be possible right now? Would be a step i would like to implement just to be able to do work on a hardware node if necesary without interrupting the service ... -- Raimund Sacherer - RunSolutions Open Source It Consulting - Parc Bit - Centro Empresarial Son Espanyol Edificio Estel - Local 3D 07121 - Palma de Mallorca Baleares On Aug 29, 2009, at 6:01 PM, Anthony Minessale wrote: We have previously estimated the development of live fail over (after a box dies where live migration is no longer possible) to exceed 100k in development costs. It requires several additions to the sofia sip library, freeswitch and a dependancy on some other code we would have to implement to manage it. It may or may not be worth it to raise that kind of funding just to avoid an occasional disaster. Then there is a matter of securing the time of the developers necessary to carry out the implementation. On Aug 29, 2009 10:19 AM, "Brian West" <brian@freeswitch.org<mailto:brian@freeswitch.org>> wrote: I was able to do this using OpenVZ, You can get away with it on smaller instances... like if you're doing one instance per company but don't expect live migration to work as well on large instances with thousands of calls up at once. You need a fast network, fast disks and to follow the howto on the wiki. /b On Aug 29, 2009, at 4:58 AM, Steve Kurzeja wrote: > You still have hardware failures and fail-over... _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists... _______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (13)
-
Anthony Minessale
-
Brian West
-
David Knell
-
Diego Viola
-
Giovanni Maruzzelli
-
Jason White
-
Jim Burke
-
Ken Rice
-
Michael Collins
-
Michael Giagnocavo
-
Mitul Limbani
-
Raimund Sacherer
-
Steve Kurzeja