How to update a live system with very little downtime (How does 'make current' work?)
Sorry for such a basic question: 1. If I do a 'make current' while a system is running it will not have any effect until I restart FS, correct? 2. What if 'make current' fails? What if it fails and the server gets restarted, will FS work? 3. Is it more advisable to stop FS, do the 'make current', then start FS even though it will have a longer down time? I know it's best to keep FS up-to-date - but it is difficult when you only have 1 system available and it is expected to have 100% uptime.. Thanks, Jason
Jason, This may not be the best way but is the way I do it on my Ubuntu 10.04-based system. I found that make current does the build and install without pausing. On my li'l Atom-based system, the build can take a while, meaning that the install could be at an unpredictable/inconvenient time. Also, I have a Sangoma ISDN card, so the wanpipe kernel drivers need rebuilding on every kernel update; I take this opportunity to do everything else, like this: 0. Backup all your config files, recordings, logs, etc. 1. Download any updated Sangoma drivers (libsngisdn and wanpipe). Install the libsngisdn drivers. On my system, they also need to be copied from /lib64 to /usr/lib64 and the various symlinks remade before rerunning ldconfig. None of this affects the running system as the libsngisdn library version is hard coded into the executables at FreeTDM build time. Decompress the wanpipe drivers but don't build them (this should be done after the kernel upgrade). 2. In /usr/local/src/freeswitch (YMMV) do a git pull. I keep an eye on the mailing list so know whether I need to do a ./bootstrap.sh or not. If in doubt, do. Then `./configure` and `make` as normal. (Don't do `make install` yet.) You then have 2 options: either do the kernel upgrade and rebuild the wanpipe drivers with the current FreeSWITCH, then upgrade FreeSWITCH, or do FreeSWITCH first then the kernel upgrade. Doing both at once is mad as if something goes wrong you then have a problem identifying what caused it. Assuming the first approach (my preferred as it is easier (for me) to roll the kernel back). 3. Do the kernel upgrade (in my case, `apt-get dist-upgrade`, `reboot`). mod-freetdm will fail to load as wanpipe will be to the wrong kernel version; this is expected. This will cause the first outage. 4. Upgrade the wanpipe drivers. If the wanpipe drivers have been upgraded themselves, then you need to `./Setup install` otherwise just `./Setup drivers`. Use the same options as when first building them. `wanrouter start` to get the wanpipe devices live again. 5. At the fs_cli prompt, `load mod_freetdm` should now succeed. You may wish to reboot the system again to check that all comes up from a cold start as it should do. Here ends the first outage (usually about 10 minutes), but you could run it straight into the second if you wish, or pause here if you need to. 6. Now comes the time to upgrade FreeSWITCH. This will cause an outage. Stop FreeSWITCH then `make install`. Start FreeSWITCH; all should be well. This normally only takes about 5-10mins on my system. This doesn't answer your Y question but might answer your X question. [http://www.perlmonks.org/?node_id=542341] John On 16/04/12 20:01, Jason Moran wrote:
Sorry for such a basic question:
1. If I do a 'make current' while a system is running it will not have any effect until I restart FS, correct?
2. What if 'make current' fails? What if it fails and the server gets restarted, will FS work?
3. Is it more advisable to stop FS, do the 'make current', then start FS even though it will have a longer down time?
I know it's best to keep FS up-to-date -- but it is difficult when you only have 1 system available and it is expected to have 100% uptime..
Thanks, Jason
On Mon, Apr 16, 2012 at 12:01 PM, Jason Moran <jmoran@secureachsystems.com>wrote:
Sorry for such a basic question:****
**1. **If I do a ‘make current’ while a system is running it will not have any effect until I restart FS, correct?****
**2. **What if ‘make current’ fails? What if it fails and the server gets restarted, will FS work?****
**3. **Is it more advisable to stop FS, do the ‘make current’, then start FS even though it will have a longer down time?****
** **
I know it’s best to keep FS up-to-date – but it is difficult when you only have 1 system available and it is expected to have 100% uptime..****
Thanks, Jason
Since you don't have a spare system (which is totally naughty, btw, especially if your corporate overlords expect 100% uptime) then my next question would be this: how are you set on CPU power, RAM, and disk space? The reason I ask is that it is entirely possible to create a secondary git clone of the FS repo and use a completely different install path when running the configure script. You could "make current" in that secondary repo and get a sort of preview of what would happen w/o touching your production build env or install dirs. If the update goes well in your test scenario then you could do the update in your production environment. It's kinda hackish but if you have spare disk space then it's a really cheap option. It isn't perfect, but it does give you at least some semblance of "try before you buy." -MC
Or you could always build a debian-live image that includes freeswitch-current and all the wanpipe driver etc, and swap them out at will at 5 mins downtime maximum, store all you data on HDD of /etc and /freeswitch/conf and you are good to go.... On Tue, Apr 17, 2012 at 12:02 AM, Michael Collins <msc@freeswitch.org>wrote:
On Mon, Apr 16, 2012 at 12:01 PM, Jason Moran <jmoran@secureachsystems.com
wrote:
Sorry for such a basic question:****
**1. **If I do a ‘make current’ while a system is running it will not have any effect until I restart FS, correct?****
**2. **What if ‘make current’ fails? What if it fails and the server gets restarted, will FS work?****
**3. **Is it more advisable to stop FS, do the ‘make current’, then start FS even though it will have a longer down time?****
** **
I know it’s best to keep FS up-to-date – but it is difficult when you only have 1 system available and it is expected to have 100% uptime..****
Thanks, Jason
Since you don't have a spare system (which is totally naughty, btw, especially if your corporate overlords expect 100% uptime) then my next question would be this: how are you set on CPU power, RAM, and disk space? The reason I ask is that it is entirely possible to create a secondary git clone of the FS repo and use a completely different install path when running the configure script. You could "make current" in that secondary repo and get a sort of preview of what would happen w/o touching your production build env or install dirs. If the update goes well in your test scenario then you could do the update in your production environment.
It's kinda hackish but if you have spare disk space then it's a really cheap option. It isn't perfect, but it does give you at least some semblance of "try before you buy."
-MC
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Kerem Erciyes - Sistem Danismani http://keremerciyes.com
For minimum downtime just leave FS running, go to where you have the FS source and do: git pull make /etc/init.d/freeswitch stop make install /etc/init.d/freeswitch start That's assuming you have a freeswitch init script. That'll leave you with only the downtime that it takes to restart FS, and the length of the make install(usually very short). William King Senior Engineer Quentus Technologies, INC 1037 NE 65th St Suite 273 Seattle, WA 98115 Main: (877) 211-9337 Office: (206) 388-4772 Cell: (253) 686-5518 william.king@quentustech.com On 04/30/2012 11:52 AM, Kerem Erciyes wrote:
Or you could always build a debian-live image that includes freeswitch-current and all the wanpipe driver etc, and swap them out at will at 5 mins downtime maximum, store all you data on HDD of /etc and /freeswitch/conf and you are good to go....
On Tue, Apr 17, 2012 at 12:02 AM, Michael Collins <msc@freeswitch.org <mailto:msc@freeswitch.org>> wrote:
On Mon, Apr 16, 2012 at 12:01 PM, Jason Moran <jmoran@secureachsystems.com <mailto:jmoran@secureachsystems.com>> wrote:
Sorry for such a basic question:
1. If I do a 'make current' while a system is running it will not have any effect until I restart FS, correct?
2. What if 'make current' fails? What if it fails and the server gets restarted, will FS work?
3. Is it more advisable to stop FS, do the 'make current', then start FS even though it will have a longer down time?
I know it's best to keep FS up-to-date -- but it is difficult when you only have 1 system available and it is expected to have 100% uptime..
Thanks, Jason
Since you don't have a spare system (which is totally naughty, btw, especially if your corporate overlords expect 100% uptime) then my next question would be this: how are you set on CPU power, RAM, and disk space? The reason I ask is that it is entirely possible to create a secondary git clone of the FS repo and use a completely different install path when running the configure script. You could "make current" in that secondary repo and get a sort of preview of what would happen w/o touching your production build env or install dirs. If the update goes well in your test scenario then you could do the update in your production environment.
It's kinda hackish but if you have spare disk space then it's a really cheap option. It isn't perfect, but it does give you at least some semblance of "try before you buy."
-MC
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Kerem Erciyes - Sistem Danismani http://keremerciyes.com
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Personally I think all of the in place suggestions are suicide if you are looking for minimum downtime. Collins' suggestion is best given the lack of a second server. Install FS to /usr/local/freeswitch-dev/ and test there. You should be able to simply change your FS ports and test against the dev version to make sure everything is working. FreeSWITCH config changes and sometimes upgrading your config to support new settings can be a pain. Inplace requires downtime vs a second dev copy on the same server should be able to nearly eliminate it. Testing allows your normal FS to go fine, change configs and once dev is working just install to your primary install folder and copy the config over. I have been running FS for along time and there have been a few times the upgrade has not quite just instantly worked, having a nice test side to work on things on is a lot easier than having to figure out what broke while people are waiting. ~mitch
participants (6)
-
Jason Moran
-
John
-
Kerem Erciyes
-
Michael Collins
-
Mitch Capper
-
William King