[aprssig] APRS-IS Duplicate Packets

[Pete Loveall AE5PL Lists]

> -----Original Message-----
> From: Matti Aarnio
> Sent: Tuesday, May 04, 2010 4:07 AM
> 
> The reality is that much used UNMAINTAINED closed source firmwares on
> radio layer do keep adding space characters to packets that they relay.

Matti, when you make such a claim, you should document it.  As for the past 8 years of implementation of javAPRSSrvr as the primary APRS-IS server software, we have not seen this happen from the RF world.  The reason is simple: an AX.25 digipeater does not decode the information field and therefore does not modify the data.  It does modify the header but not the information field.  If there is a non-compliant AX.25 digipeater in use, that device manufacturer should be contacted as it would prevent even the simplest of binary transfers to occur.  In most cases, I would guess that such a digipeater is a software-based digipeater that is not AX.25 compliant.

> Neither APRSIS nor iGates are allowed to MODIFY the packets that they
> pass,
> but when comparing against duplicate one must see if a packet is
> variation
> of some other that has gotten extra spaces at the end.

And we do attempt to compensate for aprsD mangling of packets with non-printable characters, servers and software trimming whitespace, and (most of all) clients using improperly configured TNCs in monitor mode instead of transparent KISS mode.  Again, the assumption must be made on the side of passing the packet most likely correct (containing the unprintable character, with the trailing whitespace, with a properly formed header) and not "guessing" if a packet might be seen again with one of these characters in place.  Your "must" assumption is invalid and, as stated, not proven out in practice.  Your assumption also modifies the packets by the fact that they prevent the originally transmitted packet from passing.  APRS-IS servers must attempt to compensate for APRS-IS generated errors (improper client configurations, improper software techniques such as data modification, etc.) but not for RF generated errors.

> Essentially for duplicate lookup hashing and subsequent comparison one
> really needs to ignore all 0x20 bytes at the end of the packet.
> Call it "duplicateComparisonLength". Most of the time it is equals to
> packet payload length, but not always.

No, this is an invalid assumption and lowers the server software to making the exact same error that certain authors have made by modifying the data field.  This modification occurs because you are preventing a properly formed packet from passing after a prior mangled packet.  This was debated to exhaustion a number of years ago with some authors trying to cover their poor coding techniques by making the same demand you make above.  Having worked with networks and network appliances (both development and implementation) for over 35 years, I refuse to lower my software to incorporating the same mistakes that others have made just because someone doesn't understand the impact of their mistakes.  My software does try to ensure that the original packet is the last packet passed but it will not prevent that packet from being passed based on someone else's mangling of the packet.

javAPRSSrvr will continue to attempt to maintain the highest quality of APRS-IS possible as it has done for the past 8 years.  I will also continue to assist, off-list, anyone who wishes to implement their own APRS-IS functionality so they don't insert their own errant code such as you describe above.

73,

Pete Loveall AE5PL
pete at ae5pl dot net