[aprssig] Unicode in APRS messages

[Heikki Hannikainen]

Hi,

On Sun, 11 May 2014, SARTrack Admin wrote:

> I have now added UTF8 encoding/decoding to the TCP interface... And this 
> works perfectly when I use my own program, while testing on a 'aprsc' based 
> server. Russian and other languages with 'funny' characters now come out 
> correctly on the other side.
>
> However, when I tested it with the APRS client 'AGWTracker', garbage came 
> out.

I think both APRSISCE/32 (http://aprsisce.wikidot.com/) and YAAC 
(http://www.ka2ddo.org/ka2ddo/YAAC.html) do UTF-8 encoded Unicode 
messaging on the desktop.

APRSdroid for Android (http://aprsdroid.org/) supports it too.

aprs.fi supports UTF-8 just as well.

You can transmit a 'hello' message to a callsign 'UTF-8' on the APRS-IS, 
I've got an autoresponder set up that will reply with a few messages 
containing UTF-8 encoded text, to see if your setup can receive 
international text.


I think AGWTracker is the oddball one, using a fixed 16-bit unicode 
encoding (UCS-16? some Microsoft default internal encoding?), which 
requires the user to tick a checkbox to enable and disable it, depending 
on if they're messaging to another AGWTracker or not. If you send regular 
english text using that encoding to an old ASCII client, it won't work 
(every second byte is a binary 0 / NULL byte for these regular english 
ones). With UTF-8 we don't need to have a setting to enable/disable it, as 
UTF-8 and ASCII are compatible as long as you don't actually transmit 
characters outside the ASCII range. Even then, the old non-UTF8 clients 
will just print 2-3 characters of garbage for the ÅÄÖ and 日本語, and 
print English around them just fine. UTF-8 doesn't use NUL bytes at all, 
either. This is pretty much why UTF-8 has become the standard for most of 
the Internet - it works for latin characters even if you accidentally send 
UTF-8 to an ASCII-only recipient application.

> I would like to know how other APRS clients deal with this issue, and what 
> Encoding (if any) is being used to transfer Unicode APRS packets over TCPIP.
> And how this type of data is dealt with, when it must be transfered over the 
> radio, as this is obviousely ASCII/ANSI based...

UTF-8 encoding is commonly implemented on APRS. It works great over radio, 
and even with old digipeaters and igates, since it's ASCII compatible 
(doesn't use NUL bytes, and 'high' >127 values are only used for the 
characters outside ASCII text range).

   - Hessu, OH7LZB