Frequent sqlite database corruption
Hi all, Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers. I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff? As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem. Cheers, Jeremy
As a confirmation, the profile which corrupted was exclusively WSS connections.
On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org> wrote:
Hi all,
Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers.
I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff?
As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem.
Cheers,
Jeremy
I do not believe that is what is going on. https://freeswitch.org/jira/browse/FS-9371 https://freeswitch.org/jira/browse/FS-10721 Are you using freeswitch.Dbh from LUA? I think its probably closing the file out from under you and corrupting the db in the process. Maybe these JIRAs will provide a way to replicate the issue on demand. /b On Sun, Jan 7, 2018 at 5:51 PM, Geoff Mina <gmina@connectfirst.com> wrote:
As a confirmation, the profile which corrupted was exclusively WSS connections.
On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org> wrote:
Hi all,
Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers.
I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff?
As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem.
Cheers,
Jeremy
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Brian West | General Operations Director FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045 <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g> Email: brian@freeswitch.com Mobile: 918-424-9378 Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/> [image: color-facebook-96.png] <https://www.facebook.com/freeswitch/>[image: color-twitter-96.png] <https://twitter.com/freeswitch?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor>
+1 I have experienced sqlite corruption on physical servers, but it was very seldom... but on EC2s it happens rather frequently. I'm not using freeswitch.Dbh from lua anywhere... ᐧ Regards, David Villasmil email: david.villasmil.work@gmail.com phone: +34669448337 On Tue, Jan 9, 2018 at 1:12 AM, Brian West <brian@freeswitch.com> wrote:
I do not believe that is what is going on.
https://freeswitch.org/jira/browse/FS-9371 https://freeswitch.org/jira/browse/FS-10721
Are you using freeswitch.Dbh from LUA? I think its probably closing the file out from under you and corrupting the db in the process.
Maybe these JIRAs will provide a way to replicate the issue on demand.
/b
On Sun, Jan 7, 2018 at 5:51 PM, Geoff Mina <gmina@connectfirst.com> wrote:
As a confirmation, the profile which corrupted was exclusively WSS connections.
On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org> wrote:
Hi all,
Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers.
I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff?
As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem.
Cheers,
Jeremy
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
--
Brian West | General Operations Director
FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045 <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
Email: brian@freeswitch.com
Mobile: 918-424-9378 <(918)%20424-9378>
Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
[image: color-facebook-96.png] <https://www.facebook.com/freeswitch/>[image: color-twitter-96.png] <https://twitter.com/freeswitch?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor>
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Not using LUA or really anything special. Just proxying UDP to WSS. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager.
On Jan 8, 2018, at 6:49 PM, David Villasmil <david.villasmil.work@gmail.com> wrote:
+1
I have experienced sqlite corruption on physical servers, but it was very seldom... but on EC2s it happens rather frequently. I'm not using freeswitch.Dbh from lua anywhere... ᐧ
Regards,
David Villasmil email: david.villasmil.work@gmail.com phone: +34669448337
On Tue, Jan 9, 2018 at 1:12 AM, Brian West <brian@freeswitch.com> wrote: I do not believe that is what is going on.
https://freeswitch.org/jira/browse/FS-9371 https://freeswitch.org/jira/browse/FS-10721
Are you using freeswitch.Dbh from LUA? I think its probably closing the file out from under you and corrupting the db in the process.
Maybe these JIRAs will provide a way to replicate the issue on demand.
/b
On Sun, Jan 7, 2018 at 5:51 PM, Geoff Mina <gmina@connectfirst.com> wrote: As a confirmation, the profile which corrupted was exclusively WSS connections.
On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org> wrote:
Hi all,
Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers.
I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff?
As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem.
Cheers,
Jeremy
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
--
Brian West | General Operations Director FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045 Email: brian@freeswitch.com Mobile: 918-424-9378 Website: https://www.FreeSWITCH.com
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
We run on EC2 and do not have this problem. We did have the problem in the past now and then. But I think that was with an older version of FS. But we are only using FS with Verto. Ideally, we would be using postgresql, but we have not got there yet. Robert From: FreeSWITCH-users [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of David Villasmil Sent: Monday, January 8, 2018 8:50 PM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Frequent sqlite database corruption +1 I have experienced sqlite corruption on physical servers, but it was very seldom... but on EC2s it happens rather frequently. I'm not using freeswitch.Dbh from lua anywhere... [https://mailfoogae.appspot.com/t?sender=aZGF2aWQudmlsbGFzbWlsLndvcmtAZ21haWwuY29t&type=zerocontent&guid=5e77cac0-3905-4589-94f9-cad7c1f039c8]ᐧ Regards, David Villasmil email: david.villasmil.work@gmail.com<mailto:david.villasmil.work@gmail.com> phone: +34669448337 On Tue, Jan 9, 2018 at 1:12 AM, Brian West <brian@freeswitch.com<mailto:brian@freeswitch.com>> wrote: I do not believe that is what is going on. https://freeswitch.org/jira/browse/FS-9371<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffreeswitch.org%2Fjira%2Fbrowse%2FFS-9371&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=KlDgUzhj5w9WW%2FmOlUI3ipwfRotnr6dKJHwTxxkw%2Fig%3D&reserved=0> https://freeswitch.org/jira/browse/FS-10721<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffreeswitch.org%2Fjira%2Fbrowse%2FFS-10721&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=ymepJHptZZduq%2FhCF98BGenK5jQB5%2BMb8w1Gk2J4PX8%3D&reserved=0> Are you using freeswitch.Dbh from LUA? I think its probably closing the file out from under you and corrupting the db in the process. Maybe these JIRAs will provide a way to replicate the issue on demand. /b On Sun, Jan 7, 2018 at 5:51 PM, Geoff Mina <gmina@connectfirst.com<mailto:gmina@connectfirst.com>> wrote: As a confirmation, the profile which corrupted was exclusively WSS connections.
On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org<mailto:jeremy.laine@m4x.org>> wrote:
Hi all,
Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers.
I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff?
As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem.
Cheers,
Jeremy
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitchsolutions.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xoLWALmaKeynInjiyXI6v6NhaRDZrtdwgmeizshtUnA%3D&reserved=0> Official FreeSWITCH Sites http://www.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0> http://confluence.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fconfluence.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=hR3mLNDpJ%2Bf5%2BTwTRltofWjUAf6eDPtdh7Rq%2BgCEX2I%3D&reserved=0> http://www.cluecon.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cluecon.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=tLf4PeEQRBPcBWhtpn62j7D%2FuaeLH%2FwiByELM1V%2FvUg%3D&reserved=0> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Flistinfo%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xIqSe7q%2BUOpxONbFIxJDWHxSpaNs%2Fz0n8aEX%2BkTWKK4%3D&reserved=0> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Foptions%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=LQgJpY%2FGt43O%2FIuSBrAA31i4s6IHn2OE4K9Bse136C8%3D&reserved=0> http://www.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0> -- [http://24.112.99.44/fss.png] Brian West | General Operations Director FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaps.google.com%2F%3Fq%3D17345%2BCivic%2BDrive%2B%25232531%2BBrookfield%2C%2BWI%2B53045%26entry%3Dgmail%26source%3Dg&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xaILh9iibnJefh0jT2zji8HImz2YNO7L75NOGaAa2LQ%3D&reserved=0> Email: brian@freeswitch.com<mailto:brian@freeswitch.com> Mobile: 918-424-9378<tel:(918)%20424-9378> Website: https://www.FreeSWITCH.com<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.freeswitch.com%2F&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=Hyzwpj0gMbeqokbh7PHrG4PNnJiCLntw4vL81lTaP4s%3D&reserved=0> [color-facebook-96.png]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2Ffreeswitch%2F&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=sq9ooktk3ZS34ulN%2B5RTgBw%2FVJINM6u1xJnri0htiyw%3D&reserved=0>[color-twitter-96.png]<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Ffreeswitch%3Fref_src%3Dtwsrc%255Egoogle%257Ctwcamp%255Eserp%257Ctwgr%255Eauthor&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=rBnKff6g%2FA5yWxy%2FyO4w8nQgyhuLsBftFTPEga0hKFU%3D&reserved=0> _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitchsolutions.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xoLWALmaKeynInjiyXI6v6NhaRDZrtdwgmeizshtUnA%3D&reserved=0> Official FreeSWITCH Sites http://www.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0> http://confluence.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fconfluence.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=hR3mLNDpJ%2Bf5%2BTwTRltofWjUAf6eDPtdh7Rq%2BgCEX2I%3D&reserved=0> http://www.cluecon.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cluecon.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=tLf4PeEQRBPcBWhtpn62j7D%2FuaeLH%2FwiByELM1V%2FvUg%3D&reserved=0> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Flistinfo%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xIqSe7q%2BUOpxONbFIxJDWHxSpaNs%2Fz0n8aEX%2BkTWKK4%3D&reserved=0> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Foptions%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=LQgJpY%2FGt43O%2FIuSBrAA31i4s6IHn2OE4K9Bse136C8%3D&reserved=0> http://www.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0> ________________________________ This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ________________________________
The only scenario where i have this problem is when using WSS. On 01/09/2018 03:37 PM, Mundkowsky, Robert wrote:
We run on EC2 and do not have this problem. We did have the problem in the past now and then. But I think that was with an older version of FS. But we are only using FS with Verto. Ideally, we would be using postgresql, but we have not got there yet.
Robert
*From:*FreeSWITCH-users [mailto:freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *David Villasmil *Sent:* Monday, January 8, 2018 8:50 PM *To:* FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> *Subject:* Re: [Freeswitch-users] Frequent sqlite database corruption
+1
I have experienced sqlite corruption on physical servers, but it was very seldom... but on EC2s it happens rather frequently. I'm not using freeswitch.Dbh from lua anywhere...
ᐧ
Regards,
David Villasmil
email: david.villasmil.work@gmail.com <mailto:david.villasmil.work@gmail.com>
phone: +34669448337
On Tue, Jan 9, 2018 at 1:12 AM, Brian West <brian@freeswitch.com <mailto:brian@freeswitch.com>> wrote:
I do not believe that is what is going on.
https://freeswitch.org/jira/browse/FS-9371 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffreeswitch.org%2Fjira%2Fbrowse%2FFS-9371&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=KlDgUzhj5w9WW%2FmOlUI3ipwfRotnr6dKJHwTxxkw%2Fig%3D&reserved=0>
https://freeswitch.org/jira/browse/FS-10721 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffreeswitch.org%2Fjira%2Fbrowse%2FFS-10721&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=ymepJHptZZduq%2FhCF98BGenK5jQB5%2BMb8w1Gk2J4PX8%3D&reserved=0>
Are you using freeswitch.Dbh from LUA? I think its probably closing the file out from under you and corrupting the db in the process.
Maybe these JIRAs will provide a way to replicate the issue on demand.
/b
On Sun, Jan 7, 2018 at 5:51 PM, Geoff Mina <gmina@connectfirst.com <mailto:gmina@connectfirst.com>> wrote:
As a confirmation, the profile which corrupted was exclusively WSS connections.
> On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org <mailto:jeremy.laine@m4x.org>> wrote: > > Hi all, > > Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers. > > I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff? > > As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem. > > Cheers, > > Jeremy >
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitchsolutions.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xoLWALmaKeynInjiyXI6v6NhaRDZrtdwgmeizshtUnA%3D&reserved=0>
Official FreeSWITCH Sites http://www.freeswitch.org <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0> http://confluence.freeswitch.org <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fconfluence.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=hR3mLNDpJ%2Bf5%2BTwTRltofWjUAf6eDPtdh7Rq%2BgCEX2I%3D&reserved=0> http://www.cluecon.com <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cluecon.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=tLf4PeEQRBPcBWhtpn62j7D%2FuaeLH%2FwiByELM1V%2FvUg%3D&reserved=0>
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Flistinfo%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xIqSe7q%2BUOpxONbFIxJDWHxSpaNs%2Fz0n8aEX%2BkTWKK4%3D&reserved=0> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Foptions%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=LQgJpY%2FGt43O%2FIuSBrAA31i4s6IHn2OE4K9Bse136C8%3D&reserved=0> http://www.freeswitch.org <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0>
--
Brian West | General Operations Director
FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045 <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaps.google.com%2F%3Fq%3D17345%2BCivic%2BDrive%2B%25232531%2BBrookfield%2C%2BWI%2B53045%26entry%3Dgmail%26source%3Dg&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xaILh9iibnJefh0jT2zji8HImz2YNO7L75NOGaAa2LQ%3D&reserved=0>
Email: brian@freeswitch.com <mailto:brian@freeswitch.com>
Mobile: 918-424-9378 <tel:%28918%29%20424-9378>
Website: https://www.FreeSWITCH.com <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.freeswitch.com%2F&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=Hyzwpj0gMbeqokbh7PHrG4PNnJiCLntw4vL81lTaP4s%3D&reserved=0>
color-facebook-96.png <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2Ffreeswitch%2F&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=sq9ooktk3ZS34ulN%2B5RTgBw%2FVJINM6u1xJnri0htiyw%3D&reserved=0>color-twitter-96.png <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Ffreeswitch%3Fref_src%3Dtwsrc%255Egoogle%257Ctwcamp%255Eserp%257Ctwgr%255Eauthor&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=rBnKff6g%2FA5yWxy%2FyO4w8nQgyhuLsBftFTPEga0hKFU%3D&reserved=0>
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitchsolutions.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xoLWALmaKeynInjiyXI6v6NhaRDZrtdwgmeizshtUnA%3D&reserved=0>
Official FreeSWITCH Sites http://www.freeswitch.org <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0> http://confluence.freeswitch.org <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fconfluence.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=hR3mLNDpJ%2Bf5%2BTwTRltofWjUAf6eDPtdh7Rq%2BgCEX2I%3D&reserved=0> http://www.cluecon.com <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cluecon.com&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=tLf4PeEQRBPcBWhtpn62j7D%2FuaeLH%2FwiByELM1V%2FvUg%3D&reserved=0>
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Flistinfo%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=xIqSe7q%2BUOpxONbFIxJDWHxSpaNs%2Fz0n8aEX%2BkTWKK4%3D&reserved=0> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Foptions%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=LQgJpY%2FGt43O%2FIuSBrAA31i4s6IHn2OE4K9Bse136C8%3D&reserved=0> http://www.freeswitch.org <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7Ce8751ef78ab94a2b8fba08d55703b119%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636510595763566288&sdata=yhpc8x9Uft03onJRGvYx5PF8TSUMypMsUXhvPylVqQw%3D&reserved=0>
------------------------------------------------------------------------
This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.
Thank you for your compliance.
------------------------------------------------------------------------
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Saludos / Regards / Cumprimentos António Silva
I don't use lua at all, just receiving calls over WSS / DTLS-SRTP and forwarding them to a SIP trunk. Jeremy
Hi Brian, Is not related to sqlite dbs only, it affects also the access to postgresql, in the last case the connection to database is lock, some jiras i could get, different errors but same setup, and i think that is the same root cause: *mod_callcenter: callcenter_api locked https://freeswitch.org/jira/browse/FS-10638 *error message: fail to send query to pgsql: https://freeswitch.org/jira/browse/FS-10474 *sofia stop response to requests: https://freeswitch.org/jira/browse/FS-10676 I all i solve it by restart only postgresql, with forces FS to clean up all dbhandles open. Jeremy, for what i could compile from the different times it happens is the presence of wss endpoints, i've other similar setups without wss that i don't face this issues. I couldn't figure the what is the root cause, it could be some bug in openssl lib... for sure is some corruption in open db handles by fs. I try proxy wss connections with nginx, but same issue. Now i was working on trying to connect to FS without ssl using the proxy this way: cliente <- [WSS] -> nginx <-[WS]-> fs But of course i need to implement in nginx a "WSS helper" to change all tls contact to non tls. Right now i've a server in "quarantine" where i put the my clients that are using wss, so i can restart FS without affecting many calls. Saludos / Regards / Cumprimentos, António silva On 01/09/2018 01:12 AM, Brian West wrote:
I do not believe that is what is going on.
https://freeswitch.org/jira/browse/FS-9371 https://freeswitch.org/jira/browse/FS-10721
Are you using freeswitch.Dbh from LUA? I think its probably closing the file out from under you and corrupting the db in the process. Maybe these JIRAs will provide a way to replicate the issue on demand. /b
On Sun, Jan 7, 2018 at 5:51 PM, Geoff Mina <gmina@connectfirst.com <mailto:gmina@connectfirst.com>> wrote:
As a confirmation, the profile which corrupted was exclusively WSS connections.
> On Jan 7, 2018, at 4:24 PM, Jeremy Lainé <jeremy.laine@m4x.org <mailto:jeremy.laine@m4x.org>> wrote: > > Hi all, > > Just as Geoff Mina recently reported, I too have encountered numerous instances of sqlite database corruptions, without any power cuts or disk failures. I have experienced this both on EC2 instances and on physical servers. > > I saw António Silva asked whether Geoff Mina was using wss endpoints, could you elaborate on why this might be the root cause? In the setups where I encountered the corruption, I am indeed using wss endpoints. How about you Geoff? > > As a workaround I am considering migrating to postgresql (core + sofia) but it's frustating not to understand the root cause of the problem. > > Cheers, > > Jeremy >
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com>
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org <http://confluence.freeswitch.org> http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org
--
Brian West | General Operations Director
FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045 <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
Email: brian@freeswitch.com <mailto:brian@freeswitch.com>
Mobile: 918-424-9378
Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
color-facebook-96.png <https://www.facebook.com/freeswitch/>color-twitter-96.png <https://twitter.com/freeswitch?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor>
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (7)
-
Antonio Silva
-
António Silva
-
Brian West
-
David Villasmil
-
Geoff Mina
-
Jeremy Lainé
-
Mundkowsky, Robert