[aprssig] distributed findu possible ?

[Steve Dimse]

On Aug 8, 2008, at 3:30 AM, Michael Conrad wrote:

> When a critical mass of peers in a distributed findu network will  
> exists,
> you do not have such hard requirements for the availability of a  
> single peer.
> Some peers can fail, other peers are coming back and the whole  
> system is
> available all the time. In addition each peer only handles a fraction
> of all requests, needs only a fraction of the overall bandwidth....
>
> A dream would be, that each peer operator only invests one hour per  
> month
> to install new updates. If you have 100 active peers, this would be  
> more
> time than Steve has to invest each month, but it is better shared  
> across
> users.
>
Or you could get 1000 people willing to donate 6 minutes of time a  
month.

Unfortunately computer science does not scale that way, a fact which  
has been known for four decades. Probably out of print, but 30 (gulp)  
years ago for my systems design course we studied a book called "The  
Mythical Man-month" by an IBM manager named Brooks, which debunked the  
idea that the way to speed up a CS project was to throw more people at  
it. Indeed, beyond a certain critical number, which depends on  
complexity of the project, more people result in a worse product which  
takes more calendar time to deliver.

Distributed computing is a great way to obtain more resources. CPU  
time, disk storage, and network bandwidth are all great reasons for  
using this paradigm. Program design and system administration, not so  
much. Even with your idea, you still need one person or a small group  
to design the system, write and maintain the code, recruit the  
members, and answer the hundreds of questions. This person or persons  
will spend a lot more than 20 hours a week initially. Maybe (big  
maybe) long term, once the system is stable, this person or group will  
drop below the 20 hour line.

Administratively, you have a whole new set of problems that could  
actually raise the amount of time needed. Say the programmer finds a  
bug or adds a new feature. He offers up an update. How are you going  
to be sure all 100 or 1000 users have updated their machine? If they  
don't, users will get a different experience based on who's server  
they randomly get. This will not be accepted by users. So maybe you  
write an automated system to push code. You need to be sure all  
updates get installed. And you need to write the push code. You need  
someone with the keys to the kingdom to decide what gets pushed. This  
is just one of the many hurdles your system would face.

It cam be done. Google is a massive distributed system of cheap, small  
servers. They also have hundreds, if not thousands of really bright  
people working to administer the system.

Again, I'd love to see myself proved wrong on this, so go for it!


Steve K4HG