This sounds like so many "redundancy" projects that will probably offer nothing in the real world. On 08/30/2009 05:52 AM, Pete Mueller wrote:

I guess I should also mention that Xen is a side-project.

When considering this issue for an existing production systems, we chose to put as much HA into hardware as we can. We are not concerned with FS crashing, as so far we've never seen that happen (except when our module caused it :) So for each of our systems: - We have dual NIC cards (onboad NIC + PCI card) both bridged together in case one fails NICs hardly ever fail. Its the wiring which is the vulnerable area. How independent can you make the two wiring paths, when they come from the same box? - We have redundant power supplies. Even with a good UPS, power fails more often than a high quality power supply. Just how independent are the two power sources feeding your two power supplies? Do you have two completely independent UPS sets? Do you have spacially diverse wiring from them? - We use Mirrored Solid State Disks for local storage (far better MTBF than HDD, a lot faster too) My experience so far is that SSD reliability is very poor, with entire drives disappearing, rather than just getting the odd bad sector. I guess to balance this, hard disk drive reliability seems to have plummeted in the last year or so, after several good years. - All but OS and speed-critical data is stored on a NAS device NAS == more wiring. More wiring == more vulnerabilities. Are you sure your setup is a win? NAS tends to help keep the data secure, but it isn't good for reliable access to that data. - We have redundant DBs with Memcache infront for speed

At the same time we chose to use COTS hardware (SuperMicro chassis/MoBo) rather than the big-boys like IBM or Dell. This kept the overall cost per machine low. Initially some were concerned that not having a name like IBM on our servers would be concerning to some potential clients. The solution was to pay a company to deisgn and build a custom face plate for the SuperMicro boxes. Which oddly looks more impressive to clients that a rack full of IBM faceplates. It was suprisingly low cost for the faceplates too. Some years ago we made an entire custom chassis for off the shelf boards. The quotes for fabricating that in small numbers were all over the place, but we ended with a good quality chassis at low cost. Most off the shelf rack mount enclosures are really pricy, so it isn't that hard to match their price with a custom build. We ended up with a better design (at least for our purposes) that cost us no more. It can really make your stuff stand out.

A simple respray of the front panel can achieve a distinctive look at low cost too. :-)

For scalability, OpenSIPS was our choice. There's a very nice tutorial on their website on how to configure Load Balancing.

Regards, Steve