[aprssig] aprsd drops mic-e characters

[Bill Diaz]

Tapio,
  See below:

>-----Original Message-----
>From: aprssig-bounces at lists.tapr.org 
>[mailto:aprssig-bounces at lists.tapr.org] On Behalf Of Tapio Sokura
>Sent: Saturday, May 07, 2005 08:47
>To: TAPR APRS Mailing List
>Subject: Re: [aprssig] aprsd drops mic-e characters


>Bill Diaz wrote:
>> This aprsd problem has been known for at least 5 years and 
>it was supposedly
>> fixed a long time ago. This type of data corruption / mangling causes
>> excessive copies of the mangled packets and causes stations 
>in some areas to
>> have errorneous positions reported by affected clients.  

>I guess it wasn't fixed then, at least 2.2.5-13 contains it 
>and that is 
>what I have been observing here. There is no difference in the 
>relevant 
>part of the code to 2.2.5-15.

It was fixed at one point, in some distributions.  Some of the older
versions are still in use unfortunately.  It seems that some old (improper
code) was included in some newer releases.  I think there was a fork
sometime back, and there are at least 2 separate aprsd variants.  You need
to check both to see where the fixes were implemented.

>> Please define invalid characters.  I can understand removing 
>the unneeded
>> space at the end of the packet but any other modifications 
>to the payload
>> can change the entire meaning of the packet and can defeat 
>attempts at dupe
>> suppression.  Past practice has been NOT to modify the payload. 
>
>The heart of the problem as I see it, is that the APRS 
>specification is 
>vague on what are and what are not valid characters. It says in many 
>places that "printable ASCII" is good, but what is printable ASCII? 
>Usually it means characters of decimal value between 32-126, possibly 
>including 127. But characters above 127 are not normally considered to 
>be ASCII.

Many european languages use 8 bit characters and consider them to be
printable.  Users may incorporate 8 bit characters for other perfectly valid
reasons.  We discussed this long ago on this SIG and reached a consenus that
applications should not modifiy the payload.  There is simply no rational
reason for an application to intentionally mangle packets.

>> Not true.  Properly written APRS applications can handle 8 
>bit characters.
>> This was also discussed 4 or 5 some time ago on the sig and 
>supposedly aprsd
>> was modified to eliminate this mangling of payloads.
>
>Maybe, I wasn't here then. The fact is that aprsd mangles certain 
>packets, with or without my patch. It is trivial to modify the code or 
>that patch to pass all bytes that are between 28 and 255. It would be 
>trivial to pass all bytes between 0 and 255, but I'm also 
>guessing that 
>many applications won't handle the zero bytes this can insert into the 
>APRS-IS stream. Not to mention the ambiguity of where a packet starts 
>and where it ends.

As I said before, it was fixed at one point.  Obviously, you are modifying a
distribution that did not include the fixes.

There is no ambiguity on where an APRS IS packet begins and ends.  Packets
are terminated with CR or CR/LF.  If the character following a CR or LF is >
chr(32), it is likely the start of a new packet.  The header generally
should not contain 8 bit char, or control char, whereas the payload can
contain both.

You should not, modify any char in the payload.  Please think about the
effect on the APRS-IS and client applications worldwide before you
unilaterily decide to "improve" APRS.  

>> Please, please do NOT modify the contents of any payload. In 
>the past aprsd
>> was responsible for the majority of packet mangling on the 
>APRS-IS.  Let's
>> not degrade the APRS-IS again.
>
>Ok. Then what do I do to packets that contain cr, lf or any 
>combination 
>of those anywhere in the packet? If I just blindly pass packets 
>containing them into the APRS-IS, they will probably be 
>mishandled. 

You need to spend some time looking at the APRS-IS data.  All packets on the
APRS IS are terminated with a CR or CR and LF.  Extraneous CR would be
considered to be nul packets.  The extra CR may cause a problem on RF, but
not on the APRS-IS.

> What 
>I need is a clear and unambiguous specification on what 
>characters/packets to pass and what not to pass. The "pass everything" 
>just doesn't work in APRS-IS without a clearly defined protocol that 
>somehow embeds or escapes the "bad" characters. 

Escaping what you consider to be "bad" characters is a bad practice since
the originator may have had a valid reason for using them.  Simply put, do
not edit the payload of ANY packet you may relay.  

> You can always 
>say that 
>these things appear rarely in practice, but they do occur and 
>that's why 
>you need a clear spec. Now with the ambiguous spec you see 
>every author 
>making up their own rules on what to filter or not to filter.

As stated  above, a packet on the APRS-IS is delimited with a CR or CR and
LF.  Filtering packets is one thing, changing the meaning of a packet is
another matter entirely.

>> If you are actually running an aprsd version that is 
>mangling packets, you
>> should immediately remove it from service.  
>
>No I'm not running aprsd, but there are some in the local network.

Please do not distribute the changes you have made either.  We don't need to
set the clock back 5 years.  The data volume today is about twice what it
was then, and additional mangled packets will only clog an already busy
network with useless and errorneous data.

Bill KC9XG

>   Tapio
>
>PS. This might be more on topic on the aprsspec-list..


>_______________________________________________
>aprssig mailing list
>aprssig at lists.tapr.org
>https://lists.tapr.org/cgi-bin/mailman/listinfo/aprssig
>