[aprssig] Re: More on the findU server issue...

[Steve Dimse]

On Sep 25, 2005, at 6:01 PM, <scott at opentrac.org>  
<scott at opentrac.org> wrote:

>> Each packet results in multiple writes...for example, a
>> position report writes the raw packet, the position into the
>
> That's why I gave the factor of 100 increase.  Again, I know very  
> little
> about MySQL architecture so I'm not sure if it's feasible to split  
> up table
> and index access between multiple disks.  I don't even know if it's  
> got a
> redo log concept like Oracle.
>
> I do know that I had an Oracle 9i database on an old PII-400  
> machine with
> old 8 GB SCSI drives (non-RAID - had to shut the Gladiator array  
> off due to
> the power bill!) that was keeping up with all of the standard APRS  
> position
> packets (no MIC-E), updating regular and spatial indexes, and  
> handling a few
> queries without breaking a sweat.

When you have a parser that handles more than just positions, like  
findU's weather, status, ariss, pcsat, message, and so on, it is very  
likely that each write must include a seek, whereas if you are  
writing one table of position, the situation is much less likely to  
require a seek on each write.

Anyway, as I say, even the backup server has no problem either with a  
situation involving a few queries. The problem is when you are trying  
to handle half the load of findU, which is around 20 http requests a  
second. From the sound of it, you are were not parsing all of the  
APRS formats, which significantly increases the load, and it sounds  
like this was something in the past, with a lower APRS IS stream  
rate, and even just filtering out the Mic-E takes away something like  
a third to a half of the positions. Putting all this apple-to-orange  
stuff, your parser may have been writing a tenth or a twentieth the  
number of database records the findU parser does.

> I'm surprised that MySQL has such a hard
> time.

Just to be completely clear, MySQL does not have the problem. Running  
on the primary server, it works fine. Even under the Slashdot load it  
was able to almost keep up, even if not snappily.
>
> Anyway, none of that helps FindU.  An Oracle license would cost  
> more than
> the server hardware even if you could convert it.  I'm just curious  
> for my
> own use... I wonder if any of the other free RDBMS's are any better.

Just as I wrote about the hardware vendor issue, it is quite possible  
that another software solution may be somewhat more efficient, but I  
have no interest in investing the considerable time and effort such  
experimentation would take. The fact is I have a solution that works,  
and I want to spend my time improving the user experience, not geeked  
out wringing the last bit of life out of old hardware.

Steve K4HG