[aprssig] distributed findu possible ?
steve at dimse.com
Fri Aug 8 09:10:31 EDT 2008
On Aug 8, 2008, at 3:30 AM, Michael Conrad wrote:
> When a critical mass of peers in a distributed findu network will
> you do not have such hard requirements for the availability of a
> single peer.
> Some peers can fail, other peers are coming back and the whole
> system is
> available all the time. In addition each peer only handles a fraction
> of all requests, needs only a fraction of the overall bandwidth....
> A dream would be, that each peer operator only invests one hour per
> to install new updates. If you have 100 active peers, this would be
> time than Steve has to invest each month, but it is better shared
Or you could get 1000 people willing to donate 6 minutes of time a
Unfortunately computer science does not scale that way, a fact which
has been known for four decades. Probably out of print, but 30 (gulp)
years ago for my systems design course we studied a book called "The
Mythical Man-month" by an IBM manager named Brooks, which debunked the
idea that the way to speed up a CS project was to throw more people at
it. Indeed, beyond a certain critical number, which depends on
complexity of the project, more people result in a worse product which
takes more calendar time to deliver.
Distributed computing is a great way to obtain more resources. CPU
time, disk storage, and network bandwidth are all great reasons for
using this paradigm. Program design and system administration, not so
much. Even with your idea, you still need one person or a small group
to design the system, write and maintain the code, recruit the
members, and answer the hundreds of questions. This person or persons
will spend a lot more than 20 hours a week initially. Maybe (big
maybe) long term, once the system is stable, this person or group will
drop below the 20 hour line.
Administratively, you have a whole new set of problems that could
actually raise the amount of time needed. Say the programmer finds a
bug or adds a new feature. He offers up an update. How are you going
to be sure all 100 or 1000 users have updated their machine? If they
don't, users will get a different experience based on who's server
they randomly get. This will not be accepted by users. So maybe you
write an automated system to push code. You need to be sure all
updates get installed. And you need to write the push code. You need
someone with the keys to the kingdom to decide what gets pushed. This
is just one of the many hurdles your system would face.
It cam be done. Google is a massive distributed system of cheap, small
servers. They also have hundreds, if not thousands of really bright
people working to administer the system.
Again, I'd love to see myself proved wrong on this, so go for it!
More information about the aprssig