Understand....and do know that when services/sites are not engineered correctly this will happen.
Here is how Flickr have architected their service.

Platform


  • PHP
  • MySQL
  • Shards
  • Memcached for a caching layer.
  • Squid in reverse-proxy for html and images.
  • Linux (RedHat)
  • Smarty for templating
  • Perl
  • PEAR for XML and Email parsing
  • ImageMagick, for image processing
  • Java, for the node service
  • Apache
  • SystemImager for deployment
  • Ganglia for distributed system monitoring
  • Subcon stores essential system configuration files in a subversion repository for easy deployment to machines in a cluster.
  • Cvsup for distributing and updating collections of files across a network.

    The Stats

  • More than 4 billion queries per day.
  • ~35M photos in squid cache (total)
  • ~2M photos in squid's RAM
  • ~470M photos, 4 or 5 sizes of each
  • 38k req/sec to memcached (12M objects)
  • 2 PB raw storage (consumed about ~1.5TB on Sunday
  • Over 400,000 photos being added every day
  • -E



    On Tue, Jun 3, 2008 at 2:19 PM, Albano Daniele Salvatore - Personale <d.albano@gmail.com> wrote:
    A couple of suggestions:
    - take a look to nginx, it's really fast and for a fast switch there is
    need for a fast data backend (it serves data)
    - dosen't use php (i love php, but it is slow ... a lot slow even using
    apc, eaccelerator or similar!)

    Memcached (or similar like sharedance) would be really useful to serve
    pre generated XML.

    It would be really nice if the WUI (Web UI) would be written using Mono
    or Python

    these are just my two cents :)