[aprssig] APRS protocol replacement ideas: protobuf

[Heikki Hannikainen]

On Sat, 12 Feb 2022, Andrew Pavlin via aprssig wrote:

> If we start using a more verbose protocol, then we have longer packets, 
> and more likelihood of a single bit error that makes the whole thing 
> useless.

I'm not going to advocate for creating yet another encoding of APRS, due 
to the obvious practical issues (backwards compatibility, legacy fleet, 
fragmentation, etc), but for the sake of the argument, I'd like to make a 
fair comparison based on actual data.

A protobuf encoded packet format would be very similar in size than the 
existing APRS encodings, if the same data is transmitted.

It would also be extensible, and have higher resolution for the numeric 
fields, without being much larger. If for some reason a new protocol would 
be needed, for LoRa or some other new development where there wouldn't be 
such backwards compatibility requirements, I think it'd make a great 
contestant!

I spent some 30 minutes doing a little demo which encodes equivalent info 
using protobuf and compares the lengths with Mic-E and uncompressed 
formats. The code is here, the .proto file is the schema:

    https://github.com/hessu/aprs-protobuf-test

$ python3 test.py
Protobuf: b'\n\x0f\rj\xa6pB\x15T\xae\xc1A\x18y \x8e\x02\x12\x02/>\x18\xef\xab\xa0\x90\x06R\x17Hessu, https://aprs.fi/'
Len: 52
Plain: b'@191500h6016.25N/02421.91E$270/052 Hessu, https://aprs.fi/!wPv!'
Len: 63
Mic-E: b'`w(mpSU>/\'"4.}Hessu, https://aprs.fi/!wbC!'
Len: 42
Mic-E with timestamp: b'@191500h`w(mpSU>/\'"4.}Hessu, https://aprs.fi/!wbC!'
Len: 50

Mic-E is shorter than the Protocol Buffers message, protobuf is shorter 
than uncompressed. The protobuf messages are not ASCII, the non-printable 
bytes are shown in hexadecimal (\x18 is one byte of 0x18). The Mic-E 
packets have one escaped single quote too, but none of those escapes are 
included in the length measurements, they're done on raw bytes.

The first Mic-E packet doesn't have a timestamp - if one would be added 
(so that recipients can better filter out-of-order & duplicate packets), 
it would be 8 bytes longer, 50 bytes. The uncompressed and protobuf 
versions have a timestamp. The APRS timestamp only contains hours, minutes 
and seconds. The protobuf version has an unix timestamp, accurate to the 
second, with full date in it too.

On the other hand, the protobuf packet has much more precision and range - 
coordinates are 32-bit floating point, speed is an unscaled integer that 
can go to rocket speeds without loosing precision, and so on. If the !DAO! 
extension would be dropped, Mic-E would be 5 bytes shorter. Byonics sends 
!DAO!, Kenwood probably doesn't, Yaesu doesn't.

I think that 10 bytes could be sacrificed, if for that we get the 
precision, the possibility to extend easily, and most importantly, remove 
the need to implement custom encoders and decoders by hand. 10 bytes is 
some 67 milliseconds at 1200 bit/s.

In any case, with similar data contents, comparable packet length. The 
protobuf packet could be made shorter by reducing data precision and some 
custom encoding, but then the encoding & decoding would require some more 
lines of custom code.

> So, don't look at protobufs as a savior. If anything, it will allow the 
> packets to grow rapidly with extra optional fields, so that the channel 
> is clogged faster with longer transmissions and more collisions.

On the other hand, there is nothing in the current APRS protocol which 
would prevent people from making silly long transmissions by using large 
txdelays, VOX on a Baofeng with a 1-second tail, or just very long comment 
strings and digipeater paths. Nothing prevents developers from adding 
custom extensions in the end of the packet either, or creating 
applications which are spammy, or users from configuring too frequent or 
long transmissions.

For channel clogging, I suspect things could be improved by applications 
which simply would not let users configure too long paths, too long 
comment strings, or too frequent transmissions.

> And protobufs isn't self-describing at the wire level, any more than 
> APRS or OpenTRAC is.

Right; but on the upside, it can be described with a simple schema file 
that is easy to share, and easy to convert to code which does encoding and 
decoding.

If a tight serialization is wanted, the format cannot be self-describing. 
If it needs to be self-describing, it'll be more like JSON or XML, and 
it'll be rather big.

To me, protobuf doesn't seem much fatter than the current encodings. A 
litle fatter than Mic-E or compressed, but that's because it has more 
resolution in the numeric fields.

No, I don't believe this is in any way practical for APRS today. This is 
just an academic discussion, and food for thought if someone is 
considering new a non-APRS protocol. :)

   - Hessu, OH7LZB/AF5QT