Hi, I am also looking for similar kind of solution. I have a web application for which I am using backend as Cassandra. My web application also controls Freeswitch (as in it contains the UI for setting up bridging for calls). So, right now, for that I wrote a sync service which fetches data from Cassandra and puts it in MySQL. On Thu, Aug 1, 2013 at 9:53 PM, Lloyd Aloysius <lloyd.aloysius@sunteltech.ca

wrote:

if you use MySQL replication is painful. It is not faster. Binary logs are killing the cpu and memory. Lots of interest in this direction. I saw OpenSIP using Casendra Driver for distribution.

LLoyd * <http://www.sunteltech.ca/blog/>*

On Thu, Aug 1, 2013 at 12:04 PM, Jeff Leung <jleung@v10networks.ca> wrote:

...

I’m not sure of Cassandra database will work as a database for the FreeSWICH core since the syntax looks SQL but isn’t.****

** **

If It provides ODBC connectivity, you may as well as give it a shot.****

** **

*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Lloyd Aloysius *Sent:* Thursday, August 1, 2013 8:14 AM *To:* FreeSWITCH Users Help *Subject:* [Freeswitch-users] FreeSWITCH Core - Cassandra Database****

** **

Hi All****

** **

Does any one have experience with FreeSWITCH Core and Cassandra Database. Sounds like Cassandra (http://cassandra.apache.org/) have a better replication system. This will help to distribute freeswitch across multiple data centers and provide a better fail over system.****

** **

Any feedback ? Thank you in advance.****

** **

Thanks****

Lloyd****

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Thanks and Regards, Ashwin Jain Director (Engineering) MetroGuild

Ken, Thank you for the information. Can you please tell me how can we scale with postgres with multiple Data centers. I think postgres - replication is not possible? Thanks Lloyd * * * <http://www.sunteltech.ca/blog/>* On Thu, Aug 1, 2013 at 12:52 PM, Ken Rice <krice@freeswitch.org> wrote:

the core of freeswtich requires a sql database. changing it to use a nosql database is not a trivial task.

Postgres is probably the most recommended database to use and is used in many high volume production environments. (its that the developers of freeswitch primarily use) Ken Sent from my iPad

On Aug 1, 2013, at 11:40, Ashwin Jain <ashwinrkjain@gmail.com> wrote:

Hi,

I am also looking for similar kind of solution.

I have a web application for which I am using backend as Cassandra. My web application also controls Freeswitch (as in it contains the UI for setting up bridging for calls). So, right now, for that I wrote a sync service which fetches data from Cassandra and puts it in MySQL.

On Thu, Aug 1, 2013 at 9:53 PM, Lloyd Aloysius < lloyd.aloysius@sunteltech.ca> wrote:

...

if you use MySQL replication is painful. It is not faster. Binary logs are killing the cpu and memory. Lots of interest in this direction. I saw OpenSIP using Casendra Driver for distribution.

LLoyd * <http://www.sunteltech.ca/blog/>*

On Thu, Aug 1, 2013 at 12:04 PM, Jeff Leung <jleung@v10networks.ca>wrote:

...

I’m not sure of Cassandra database will work as a database for the FreeSWICH core since the syntax looks SQL but isn’t.****

** **

If It provides ODBC connectivity, you may as well as give it a shot.****

** **

*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Lloyd Aloysius *Sent:* Thursday, August 1, 2013 8:14 AM *To:* FreeSWITCH Users Help *Subject:* [Freeswitch-users] FreeSWITCH Core - Cassandra Database****

** **

Hi All****

** **

Does any one have experience with FreeSWITCH Core and Cassandra Database. Sounds like Cassandra (http://cassandra.apache.org/) have a better replication system. This will help to distribute freeswitch across multiple data centers and provide a better fail over system.****

** **

Any feedback ? Thank you in advance.****

** **

Thanks****

Lloyd****

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Thanks and Regards, Ashwin Jain Director (Engineering) MetroGuild

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

Hello, If you are looking for a master-master replication I can say that Percona Xtradb cluster works very well. There are a few extra settings that need to be in my.cnf pertaining to wsrep but not too bad to set up, certainly not painful. I am currently running a 3 master cluster for an eCommerce site. I can not speak to cross data center replication though as all three are racked above one another. MMySQL is pretty quick but can it not compare to MongoDB or CouchBase. I have used many SQL db and a few NoSQL and the NoSQL are much faster, easier to scale, and just plain more fun. My opinion and of course YMMV JohnM On 8/1/2013 1:38 PM, Ken Rice wrote:

Re: [Freeswitch-users] FreeSWITCH Core - Cassandra Database Postgresql offers streaming replication in PG9 versions... Its a master/(warm)standby system but it works quite well (I use the native pgsql replication for high volume calling). PostgreSQL is BSD licensed so this has also lead to several addons that use various methods for replication... Some are opensourced others are not...

On 8/1/13 12:10 PM, "Lloyd Aloysius" <lloyd.aloysius@sunteltech.ca> wrote:

Ken,

Thank you for the information. Can you please tell me how can we scale with postgres with multiple Data centers. I think postgres - replication is not possible?

Thanks Lloyd * <http://www.sunteltech.ca/blog/> *

On Thu, Aug 1, 2013 at 12:52 PM, Ken Rice <krice@freeswitch.org> wrote:

the core of freeswtich requires a sql database. changing it to use a nosql database is not a trivial task.

Postgres is probably the most recommended database to use and is used in many high volume production environments. (its that the developers of freeswitch primarily use) Ken Sent from my iPad

On Aug 1, 2013, at 11:40, Ashwin Jain <ashwinrkjain@gmail.com> wrote:

Hi,

I am also looking for similar kind of solution.

I have a web application for which I am using backend as Cassandra. My web application also controls Freeswitch (as in it contains the UI for setting up bridging for calls). So, right now, for that I wrote a sync service which fetches data from Cassandra and puts it in MySQL.

On Thu, Aug 1, 2013 at 9:53 PM, Lloyd Aloysius <lloyd.aloysius@sunteltech.ca> wrote:

if you use MySQL replication is painful. It is not faster. Binary logs are killing the cpu and memory. Lots of interest in this direction. I saw OpenSIP using Casendra Driver for distribution.

LLoyd * <http://www.sunteltech.ca/blog/> *

On Thu, Aug 1, 2013 at 12:04 PM, Jeff Leung <jleung@v10networks.ca> wrote:

I'm not sure of Cassandra database will work as a database for the FreeSWICH core since the syntax looks SQL but isn't.

If It provides ODBC connectivity, you may as well as give it a shot.

*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Lloyd Aloysius *Sent:* Thursday, August 1, 2013 8:14 AM *To:* FreeSWITCH Users Help *Subject:* [Freeswitch-users] FreeSWITCH Core - Cassandra Database

Hi All

Does any one have experience with FreeSWITCH Core and Cassandra Database. Sounds like Cassandra (http://cassandra.apache.org/) have a better replication system. This will help to distribute freeswitch across multiple data centers and provide a better fail over system.

Any feedback ? Thank you in advance.

Thanks

Lloyd

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Ken _http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org _irc.freenode.net #freeswitch

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

You can modify this behavior in MongoDB by changing the "write concern" [1] to ensure that the MongoDB client only returns to the caller when at least one of the replicas has acknowledged the data. The problem is that if you do that the performance drops dramatically. In my opinion, MongoDB is only suitable for storing non-critical data. I would never use it for a failover solution with FreeSWITCH. BTW, moving from a relational database to a NoSQL one is not trivial at all, especially if your application requires any kind of transactionality from the DB. [1] http://docs.mongodb.org/manual/core/write-concern/ On Thu, Aug 1, 2013 at 5:19 PM, Patrick Lists < freeswitch-list@puzzled.xs4all.nl> wrote:

On 08/01/2013 09:45 PM, john@millican.us wrote: [snip]

...

I have used many SQL db and a few NoSQL and the NoSQL are much faster, easier to scale, and just plain more fun. My opinion and of course YMMV

Doesn't MongoDB acknowledge a write when it still has the data in mem as opposed to written on disk? So when the power fails and both FreeSWITCH server 1 and MongoDB server 1 are down there was data in memory that has not been replicated to MongoDB server 2. How is that going to result in proper failover? How can you fire up FreeSWITCH server 2 with all call data when MongoDB server 2 does not have all the call data? I'm no NoSQL expert so would love to hear how a NoSQL based solution could do the job reliably.

Regards, Patrick

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

That's true, but what's unique about MongoDB, AFAIK, is that by default the official client would inform the caller that the write had been successful before the data had even reached the server. This has been changed recently, though, but the default durability settings are still too weak. On Thu, Aug 1, 2013 at 7:32 PM, Avi Marcus <avi@avimarcus.net> wrote:

Writing to disk isn't a uniquely mongodb issue - even many SQL dbs have configurable options for write durability. But for failover replication, the writing to disk is less of the issue than the latency of the replication... -Avi

On Fri, Aug 2, 2013 at 1:13 AM, Juan Jose Comellas <juanjo@comellas.org>wrote:

...

You can modify this behavior in MongoDB by changing the "write concern" [1] to ensure that the MongoDB client only returns to the caller when at least one of the replicas has acknowledged the data. The problem is that if you do that the performance drops dramatically. In my opinion, MongoDB is only suitable for storing non-critical data. I would never use it for a failover solution with FreeSWITCH.

BTW, moving from a relational database to a NoSQL one is not trivial at all, especially if your application requires any kind of transactionality from the DB.

[1] http://docs.mongodb.org/manual/core/write-concern/

On Thu, Aug 1, 2013 at 5:19 PM, Patrick Lists < freeswitch-list@puzzled.xs4all.nl> wrote:

...

On 08/01/2013 09:45 PM, john@millican.us wrote: [snip]

...

I have used many SQL db and a few NoSQL and the NoSQL are much faster, easier to scale, and just plain more fun. My opinion and of course YMMV

Doesn't MongoDB acknowledge a write when it still has the data in mem as opposed to written on disk? So when the power fails and both FreeSWITCH server 1 and MongoDB server 1 are down there was data in memory that has not been replicated to MongoDB server 2. How is that going to result in proper failover? How can you fire up FreeSWITCH server 2 with all call data when MongoDB server 2 does not have all the call data? I'm no NoSQL expert so would love to hear how a NoSQL based solution could do the job reliably.

Regards, Patrick

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

On 08/02/2013 05:12 AM, Nathan Neulinger wrote:

I've had good luck with a small deployment on percona as well, though there are a few minor issues (maybe stuff that can be corrected):

1. When FS tries to create indexes on various tables that already exist, it winds up causing a bunch of complaints/failures in the percona logs and creating various GAL_ dump files.

2. The external management of percona can be a mess. I tried running with Pacemaker/Corosync and really didn't have good luck with it failing over the floating IP properly, so I switched things back over to Keepalived, which I'm using for the freeswitch instances.

For the floating IP to follow to the new box, on a *2* node setup, you need to disable quorum and add a constraint in your pacemaker/corosync config so the resource (PerconaDB) follows the floating IP. Something like: $ pcs property set no-quorum-policy=ignore $ pcs constraint colocation add PerconaDB ClusterIP INFINITY $ pcs constraint order ClusterIP then PerconaDB And this example helps to prevent resources to move around when the failed node comes back online. Make sure the 'resource-stickiness' value is higher than the 'prefers' one: $ pcs constraint location PerconaDB prefers node-1=50 $ pcs resource rsc defaults resource-stickiness=100 I'm curious, did you use a shared filesystem (something like DRBD with GFS2/OCFS2) and regular Percona XtraDB on both nodes or did you use Percona XtraDB Cluster? Regards, Patrick

On 08/02/2013 04:10 PM, Lloyd Aloysius wrote:

Core Dev'e preferred database is PostgreSQL. I will choose PostgreSQL.

There are so many issues with depends on one data centers. The more critical now a days is weather cause lots of issues.

I want to distribute freeswitch, minimum two data centers and two different regions.

Does anyone have a PostgreSQL master-master replication working reliably? Anything I need to consider using PostgreSQL master-master.

Make sure that any autoincrement value is unique across all nodes. So it should not be possible that both node1 and node2 can generate the same autoincrement value. Triggers may casue trouble and row based locking is a good idea. How much of that applies to PostgreSQL you should be able to find by RTFM and searching on Google. Regards, Patrick

For the floating IP to follow to the new box, on a *2* node setup, you need to disable quorum and add a constraint in your pacemaker/corosync config so the resource (PerconaDB) follows the floating IP. Something like:

$ pcs property set no-quorum-policy=ignore $ pcs constraint colocation add PerconaDB ClusterIP INFINITY $ pcs constraint order ClusterIP then PerconaDB

I had something along those lines w/ a three-node cluster. I think this was close to the last config I used: node $id="1945540483" freesw-db1.srv.mst.edu node $id="1962317699" freesw-db2.srv.mst.edu node $id="1979094915" freesw-db3.srv.mst.edu \ attributes standby="off" primitive p_ip_mysql ocf:heartbeat:IPaddr2 \ params nic="eth0:0" iflabel="mysql" ip="131.151.246.119" primitive p_mysql ocf:heartbeat:mysql \ params config="/local/mysql/mysql.conf" \ pid="/local/mysql/data/mysqld.pid" \ socket="/var/lib/mysql/mysql.sock" \ binary="/local/mysql/server/bin/mysqld" \ datadir="/local/mysql/data" \ log="/local/mysql/data/log" \ additional_parameters="--basedir=/local/mysql/server --log-error=/local/mysql/data/log" \ op monitor interval="30s" timeout="30s" \ op start interval="0" timeout="120s" \ op stop interval="0" timeout="120s" clone cl_mysql p_mysql \ meta interleave="true" colocation c_ip_mysql inf: p_ip_mysql cl_mysql property $id="cib-bootstrap-options" \ dc-version="1.1.7-2.fc17-ee0730e13d124c3d58f00016c3376a1de5323cff" \ cluster-infrastructure="corosync" \ stonith-enabled="false" rsc_defaults $id="rsc-options" \ resource-stickiness="100"

I'm curious, did you use a shared filesystem (something like DRBD with GFS2/OCFS2) and regular Percona XtraDB on both nodes or did you use Percona XtraDB Cluster?

Percona XtraDB Cluster. The symptom/problem I ran into - aside from Corosync/PM being excessively complex for the situation - was that a DB instance failed, and despite the others being online and the cluster itself being up and working - it didn't fail the IP over. It may have been a glitch - didn't see it during testing - but just more trouble than it was worth for the limited use case. -- Nathan ------------------------------------------------------------ Nathan Neulinger nneul@mst.edu Missouri S&T Information Technology (573) 612-1412 System Administrator - Architect

I'm using PostgreSQL 9.1 with both log shipping and sync streaming replication. Couple that with pacemaker/corosync and I have full master/slave hot-standby (slave is read-only). Literally only takes about 5 minutes to set up replication. Failover is a different matter. On 7 September 2013 22:22, Gabriel Gunderson <gabe@gundy.org> wrote:

On Thu, Aug 1, 2013 at 10:52 AM, Ken Rice <krice@freeswitch.org> wrote:

...

Postgres is probably the most recommended database to use and is used in many high volume production environments. (its that the developers of freeswitch primarily use)

We use PostgreSQL with a master and several slaves; it scales wonderfully (doesn't really work if you're trying to keep your CDRs on it --too many writes).

I know it's unlikely to happen anytime soon, but Cassandra support in the core would be amazing. It would make it nearly trivial to scale FreeSWITCH to hundreds of servers. I'm smiling just thinking about it :)

Best, Gabe

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com

Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- *Andrew Cassidy BSc (Hons) MBCS SSCA* Managing Director *T <info@cassidywebservices.co.uk> *03300 100 960 *F<info@cassidywebservices.co.uk> *03300 100 961 *E <info@cassidywebservices.co.uk> *andrew@cassidywebservices.co.uk *W <info@cassidywebservices.co.uk> *www.cassidywebservices.co.uk