[aprssig] distributed findu possible ?
scott at opentrac.org
Sun Aug 10 15:42:24 EDT 2008
> The hard stuff, as I've said, is keeping two servers with 200+GB
> databases serving millions of dynamic web pages a day operating
> reliably. Companies hire full time IT staffs to handle lower loads.
Well, what's the hardest part there? Database maintenance? Log
management? OS maintenance? Yes, some companies might have a full-time
staff for that, but some get by on less.
I don't have enough experience with large MySQL databases to know how
much of a pain that is. Oracle certainly scales just fine at those
sizes... there's a steep learning curve (and it's not cheap) but with a
properly structured database and the right hardware you can do a lot.
Back when I did big Oracle databases (mostly on VMS clusters), we relied
a lot on the features of the database rather than the hardware. No
RAID, except for mirroring on the system drives. Oracle could handle
redundancy more efficiently by itself, mirroring tablespaces to multiple
drives (and distributing read operations). With partitioning (something
I'm not sure is fully supported yet in MySQL) you could even further
distribute the I/O load. Major indexes each had their own drives, and
the load was further distributed between multiple SCSI (or DSSI) buses.
Anyone know how much of that has filtered down to MySQL? Does anyone
even bother, or do they just throw it on a big RAID 5 and buy faster
drives and more RAM? I know that's what happened with the systems I
used to run - the hardware improved faster than the system grew, and
eventually all of that careful planning and design lost out to
convenience and simplicity.
More information about the aprssig