[aprssig] High speed datbase inserts (CGI)

[scott at opentrac.org]

I never looked at the JavAPRSSrv database interface before - has anyone
tried to use with Oracle?  In particular, Oracle with the Spatial
extensions?  I wrote my own database interface service in Windows a while
back, but the underlying APRS parsing library got abandoned before it was
completed.

Scott
N1VG

-----Original Message-----
From: aprssig-bounces at lists.tapr.org
[mailto:aprssig-bounces at lists.tapr.org]On Behalf Of AE5PL Lists
Sent: Tuesday, March 01, 2005 4:27 AM
To: TAPR APRS Mailing List
Subject: RE: [aprssig] High speed datbase inserts (CGI)


I agree, Ken.  Even on an "underpowered" system (I am populating a MS
SQL database on a mirrored drive configuration with a 2.4 GHz CPU which
is, IMO, underpowered for this application), insertion directly into the
database is probably the best route.

A couple of caveats:

Use minimal indexes.  Indexes are more performance problematic than most
other factors for databases which are mostly added to instead of
searched.

Bulk backups for large databases can lock out the updating program for
extended periods of time.  This could be an issue, depending on buffer
depths in the socket connection and the database driver.  Also
timestamps can be off unless you timestamp the data upon receipt instead
of at time of row insertion.

We have people using javAPRSSrvr's database interface to populate mySQL,
PostgreSQL, and MS SQL Server databases from the APRS-IS feed with
relatively good performance.  This interface uses the Java database
drivers (JDBC) and appears to work fairly well under load.  It is my
understanding that there are similar drivers for Perl.

73,

Pete Loveall AE5PL
mailto:pete at ae5pl.net 

> -----Original Message-----
> From: Ken Linder
> Posted At: Tuesday, March 01, 2005 1:21 AM
> Subject: Re: [aprssig] High speed datbase inserts (CGI)
> 
> I might not be a findu expert but I manage my share of data 
> (> 3 TB) at work.  I would say that it would probably be 
> easier to just shove the data into mySQL.
> 
> UNLESS...  Your mySQL server is quite underpowered.  Then, it 
> might be better to use the temporary files and let the mySQL 
> engine 'breate'.

_______________________________________________
aprssig mailing list
aprssig at lists.tapr.org
https://lists.tapr.org/cgi-bin/mailman/listinfo/aprssig
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 3080 bytes
Desc: not available
URL: <http://lists.tapr.org/pipermail/aprssig_lists.tapr.org/attachments/20050301/333ee06c/attachment.dat>