[aprssig] Re: FindU Maps Quirky

[James Jefferson Jarvis]

On Monday 07 February 2005 14:11, Ray McKnight wrote:
> Jim, I'm not criticizing any of the incredible volunteers who
> make this service possible.  

No worries, I didn't perceive it as criticism either. I was trying to point 
out that the particular problem you are experiencing is not related to the 
TIGER data.

> Now I do recall that Steve provided you
> the old FindU machine when the current server was purchased.
> Isn't it less than a 1GHz box with something like 128mb?

It is a dual Pentium III 550 MHz with 386 megabytes of memory and 5400 RPM IDE 
disks.

> Memory isn't all that expensive and might be the easiest way
> for an immediate improvement to be realized, so what type
> and how much do you need?

It is not really memory bound. It's actually a CPU bound problem. Certainally 
memory would help, but it isn't the root problem. 

Steve, do you have the specs on the machine or remember what type of memory it 
uses? If not, I can go visit the machine and check it out later this week.

The root problem is slow CPU AND slow disks AND too little RAM. I suspect the 
following hardware would be a good match for generating maps for findu:

Dual processor > 2 GHz machine
2 gigabytes of RAM or more
multiple 7200 or 10000 RPM disk drives (IDE or SCSI)

Machine needs to run Linux and have software RAID setup.

I don't have the money to buy such a machine. I also don't have the time to go 
and fund raise the money needed. But if somebody or a group were to come up 
with a machine and give it to me or put it on a fast 'net connection, then I 
would be more than happy to setup it up and maintain it. 

> Steve has suggested a load-share effort to split the processing
> between two or more servers.  

It's an easy parallel problem, so round robin DNS would be a reasonable 
solution to the problem. In fact, my mapserver for aprsworld can be put in 
round-robin with geo.aprsworld.net. It works fine and I do that when I'm 
maintaining or updating geo.aprsworld.net. However, I don't leave it in round 
robin because I don't want to slow down maps for aprsworld users to give 
findu faster maps.

> Inlet.  If I can get this sorted out and manage to get a cable
> modem, I have several big Compaq RAID servers just sitting around
>  and could easily put together a 4-processor system with about
> 24GB RAID-5 and 2GB RAM.  If more storage is needed I have a few
> external RAID cabinets populated with 24GB.  

Last I looked geo.aprsworld.net was pushing approximately 1 gigabyte of data 
per day out its pipe. Storage:

operating system: < 2 gigabytes
base map data: 15 gigabytes
rasterized map cache: the bigger the better

> I'm not sure how quickly 
> I can resolve the cable issue, and to be honest I'm not sure the
> upload speed of cable modems, if this would even be a viable
> location to host the effort, so maybe the efforts should go toward
> upgrading your hardware instead?

My thoughts in summary: 

I'm not going to lead up the effort to get new hardware to make findu deal 
with its load. I have my own site, aprsworld.net, that I would rather focus 
my efforts on. However, I will be happy to accept hardware for a new 
mapserver for findu to physically host at my site -- or setup my software on 
anybody elses computer(s) who have a reasonable chance of handling findu's 
volume. So there is the challenge to aprssig - get the new hardware and I'll 
put it online. 

Note that all of my mapping code is in cvs on sourceforge under the aprsworld 
project. TIGER data is available easily and the rest of the map layers I use 
can probably be found at the same locations where I got them - but I can't 
distribute the map data myself due to a quagmire of licenses.  

-Jim KB0THN