[aprssig] Preliminary delayed packet analysis

[mconner at aer.com]

Last week I received about one week's worth of databased APRS data on which I could perform my analysis, which was about 5.8 million packets logged from APRS-IS.  The database contained the time the packet was databased, the originating station's call, TOCALL, digi path info (including callsign of the I-gate), and the payload.  From this, I culled about 7400 packets from US callsigns containing time information in HHMMSS format sent using the /HHMMSSh format (all were TinyTraks or OpenTrackers).  

I then looked for packets that were reported twice on the APRS-IS, but the database time of the first report was close (less than 10 seconds) to the time contained in the APRS packet.  This confirmed that the APRS packet's time hack was good (if this time interval was larger, then the time hack would be suspect - most all of the "first" packets were databased within 10 seconds of the time hack in the packet).  The interval for the second report was always > 30 seconds, which means the 30-second APRS-IS dupe filtering was working properly.  I also removed instances where the TinyTrak/OpenTracker appeared to be repeatedly transmitting the same data at intervals.

This resulted in about 400 instances of delayed packets where sufficient time information was available to show that the packet was a true duplicate (two instances on the APRS-IS feed for one RX transmission).  Most of the delays were in the 1-to-3 minute range, with an extreme of 22 minutes.  There were 45 IGates involved in forwarding this data to the APRS-IS.

[Note that the overall the duplicate problem is not just 400 out of 5.8 million, but that only 400 dupes had enough info to help track down the problem.  There are many other dupes in the APRS-IS but cannot be shown with any confidence to be two APRS-IS reports for one RX transmission.]

Of the 45, 26 were UI-View clients (APU25N), 5 were jAPRSIGates (APJIxx), 3 were APRS+SA (APSxxx), 2 were Xastir (APXxxx), and the rest were others or indeterminate.  This would seem to be enough of a spread among IGate software to be unable to prove UI-View is the sole source of the problem as postulated before.

I filtered further to look for packets that had the same digipeat path or were direct reception by both the "first" and "second" IGates to help take the RF path out of the equation.  However, this sample size was too small to be really useful (about 15 packets) but did show UI-View was not the sole source for this case either.

Sooooo....the bottom line is that I don't think UI-View is the sole source of the duplicate problem.  I had hoped to be able to offer some refinement on where to look, but at this point I can't say "focus on X" - the evidence isn't there.  There may be some other commonality like a TNC, USB-to-serial converter, serial port behavior, or TCP/IP issue introducing a delay, and this will be difficult to diagnose given the obscure and relatively transient nature of the problem.  It would be nice if more dupe checking could be performed on the APRS-IS side, but that could introduce more complexity and other side effects into the system.

73 de Mark N9XTN