[aprssig] Project TNMVA (was: Does anybody know what APZMVA is?)

[Frank Knobbe]

Greetings,

The purpose of this email is to present an overview of the TNMVA
project. This is in response to questions that arose in the thread "Does
anybody know what APZMVA is?". Note that it is not an APRS application
designed for use by the general public, but merely a service to our
local APRS members in Tennessee.


Short Intro
===========

First of, thank you for the Welcome and encouragement I have received
from some of you. My name is Frank Knobbe and I'm a relatively new Ham
(2013, still only Tech). The only reason I've gotten into Ham Radio was
to have "backup" or emergency communication in my Jeep when off-roading,
so I'm not what you might call a traditional Ham. But I've been a
computer enthusiast ever since I got a C=64 at age 12 or so. Computers
and specifically computer communication has always been a hobby of mine.
I had developed BBS software for Atari ST in the late eighties and wrote
a gateway between German BBS networks and GEnie (an online service like
CompuServe) in the early nineties. That was before the Internet was
widely available, and it allowed Americans and Germans to exchange
private and forum messages (German Roundtable on GEnie). Since the
mid-nineties (after having moved to the US), I've been engaged in
computer security. I've written private and public OpenSource tools
(like SnortSAM), was engaged early on in IT security communities,
co-founded our MidTN ISSA Chapter in 2001, founded a couple security
related companies, the last of which is still what occupies most of my
time today.

But I've always enjoyed inventing, developing, building and enabling new
things. Being it security related products and services, or enabling
communication. So after having received my Ham Radio license, as a
computer geek I was drawn to the digital modes, and APRS seemed like a
good way to get started.


TNMVA Origins
=============

After I got into APRS, like others I set up an Igate. But I wondered
what else APRS could benefit from or how it might be used. And rather
than doing things on APRS-IS alone, I wanted to come up with something
new and useful, while still staying true to Bob's vision of APRS. So I
wondered what might be useful for APRS users, especially mobile users...
like users in vehicles... wait a second! How about accident reports?
That would be useful to folks on the road with mobile APRS rigs! So in
the Fall of 2013, I set out to build such a system.

When I say "system", I'm referring to the things that make this service
work. It is not a standalone application. Instead, it is an assembly of
a couple Unix shell and PHP scripts. Professionals would refer to it as
a "hack job", since, well, it's just a bunch of scripts cobbled
together. But they seem to be stable and are working great.


Planning
========

I wanted to provide timely notifications of accident reports to mobile
users. After some googleling, I found that the State of Tennessee
Department of Transportation (TDOT) has a web site that lists just such
reports (http://ww2.tdot.state.tn.us/tsw/smartmap.htm). And they also
offer that data as an RSS feed. Perfect! It's easy to poll and filter
relevant data out of that plain-text feed. When new accidents are
reported, they appear in the RSS feed. When they are resolved, they no
longer exist in the next RSS feed poll. So I need to keep track of these
entries in order to detect the removal. The information is read,
processed, stored, and tracked in a local database for comparison to the
previous poll. That way its easy to detected removals.

The data present includes the GPS location information, the timestamp
the accident was reported, and a description, amongst other things. In
order to bring this data to APRS, utilizing an object or item over, say,
a bulletin was the best choice since the accident will visually show up
on any attach GPS device or other display. It would at least provide
bearing and distance to the accident on rigs without a graphical
display.

I chose objects over items after I did some testing with various
application to see how additions and especially removals are handled. I
noticed that APRSISCE handles object kills as expected as did UIView.
However, item kills work in APRSISCE but not in UIView. aprs.fi will not
remove an item or object from the map (a design choice), but
openaprs.net will remove objects, but not items. So I chose to use
objects instead of items, and since I had the timestamp of the initial
accident report available, that was used for the timestamp of the
object.

The next question was what icon to choose. I noticed that someone in the
Eastern US also brought accident reports onto APRS, but he chose Police,
Ambulance, Constructions Site and other icons. I wanted to keep it
simple and consistent, and only use one symbol. Keeping things simple is
often better and less confusing for any sort of human-device interface,
especially one that would be used in a motor vehicle. I eyeballed the
warning triangle but realized that it is reserved for emergencies and
treated differently by a lot of applications. So I settled on the Red
Triangle. I gave it the "I" overlay, not because it stood for Incident
(although that's a good excuse), but simply because if you squint your
eyes, it looks a bit like those road-side warning triangles truckers put
behind their disabled rigs. A simple, consistent, distinctive, and
visually obvious symbol.

The object name of "TNMVAxxxx" was chosen to provide a meaningful, but
consistent name. For example, if operators in other States would set up
a system like that as well, they could name their objects KYMVA, NCMVA,
etc. The last four characters are just an increasing serial number (in
hexadecimal). That way accidents have a unique name, and can even be
archived for offline analysis if desired (at the last the last 65536
accidents).

It was clear that this whole thing would only work if it was easy on RF,
meaning, the transmission was a short as possible. I certainly did not
want to adversely effect the local radio waves. So I chose to compress
the GPS coordinates. Further, the accident description is way longer
than the comment field of objects (43 chars). So the script importing
the data from the RSS feed is shortening the description (via grep and
sed's), only carving out useful info. This is pretty much the roadway
itself and the impact the accident has on that that roadway. (For
example, "I-24-N OVERTURNED SEMI, RDWAY CLOSED", "I-65-S MULT VEHICLES,
LEFT LANES BLOCKED", etc). There is no need to be concerned about
milemarkers or other location tags since the location information is
already present in the form of GPS coordinates. Furthermore,
"congestion" or "slow traffic" warnings are filtered out since I'm only
concerned about accidents that can have large impact on traffic.

So all the information was complete and kept as short as possible
(including compression) while still being most meaningful.


Onto RF
=======

The information is polled and processed on my web server which is hosted
in a data center, away from a VHF radio. So it was clear that the entry
point of those objects into APRS was from the Internet side. These
objects are injected into APRS-IS through a script right from my server.

While transmitting the information onto RF is easy, it should be done
with care so that it does not cause congestion. Getting it onto RF was
simple since I was already operating an Igate. But I wasn't comfortable
having this already long object packet being brought to RF as a
third-party message. It seemed like unnecessary waste of bytes and
transmission time. So in order to keep it as short as possible, I
modified aprx (the Igate software I use) to pick up the TNMVA objects
from APRS-IS and transmit them as direct packets, not third-party. (I've
since changed that where a further modification of aprx allows for input
on TNC-style interfaces, including TCP ports, to be converted into ax.25
packet structures for transmission to KISS interfaces. The TCP port is
fed from a tail of a file through SSH from the server. Basically, the
script on the server simply creates a properly formatted TNC packet,
appends it to a file, which gets "tailed" by my home server, written to
a socket, which is fed into an aprx TNC interface, and then simply
digipeated out the KISS interface to the radio. It avoids the detour
through APRS-IS. This interface is K4FHK-3, while K4FHK-10 is the KISS
interface of the Igate.)

So the method for getting onto RF was done. Next was the question of
path and hops. It was clear that there is no need for mobile Hams in
other States (in my case, Kentucky and Alabama) to receive these
objects, so the path was chosen to be "TN". I experimented with hop
counts and settled on two hops. One hop to get from my station to one of
two digis that can hear me well, and another hop to ensure delivery to
my intended coverage area, Middle Tennessee. It is limited to 80 miles
(130 KM) around Nashville, covering from just east of the Tennessee
River in the West to a bit past Cookeville in the East, and all the way
down to the Alabama border. 

However, since all TDOT reports are brought into APRS-IS, the accidents
for the whole State are covered on IS.

(See father below some notes for others who are interested in covering
other parts of TN).

There you have it. The RSS feed gets polled, processed (coordinates,
description shortened, etc), injected into APRS-IS, picked up by an
Igate and brought onto RF with a path of TN2-2 as a direct message,
keeping on-air time as short as possible.

But how often?


The Timing
==========

The timing of this system is perhaps its most unique aspect. We can't
simply flood APRS-IS with data since someone might gate it back to RF,
which would cause horrible congestion. What I came up with is a
two-tiered, throttled timing. (I will be describing the older timing
here which explains it better, although the timing was recently
changed). It is two-tiered in order to give APRS-IS users immediate and
always current data, while adhering to a certain pattern on RF to avoid
unnecessary transmission.

The main script will start its processing every 3 minutes, so the main
cycle time is 3 minutes. It polls the TDOT RSS feed, converts the new
data and stores it in a local database. Next, that database is checked
for new entries (and for expired entries), and then they are brought
into APRS like so:

A new accident is brought to APRS at the time it is noticed, let's call
it T0. It is brought to both IS and RF. Then at the next 3 min interval
(T1), it is brought to both again, in case the first transmission got
lost in a collision, and once more at the next interval (T2). 

Then for the next four 3 min intervals (T3, T4, T5, T6), the object is
only brought onto APRS-IS, not RF. After the initial three
transmissions, the chances are good that a mobile operator received it. 

Then the next two intervals (T7, T8) are again updated both on APRS-IS
and RF, followed by the next four intervals being updated only on
APRS-IS, not RF.

So for the lifetime of the accident, the object will be "kept alive" on
RF only every 15-18 minutes, while staying current on APRS-IS every 3
minutes.

Once the accident has been resolved and removed from TDOT's RSS feed, it
is brought to APRS and RF twice (with that 3 min interval) as an Object
Kill so that it gets removed from any GPS or other display.

The above pattern is followed for each accident, meaning if the first
accident has been broadcast three times on RF, and is now only being
updated on APRS-IS, and another accident occurs and is listed, that
second accident will start with the 3(RF+IS)/4(IS)/2(RF+IS)/4(IS)
pattern at that point in time. In essence, the pattern is tracked per
accident object.

If multiple accidents are active at the same time, there is a 20 second
delay between each transmission so that the airtime is shared with other
stations and Digis have time to do their stuff.


(Due to some changes on my home station, which apparently made it less
reliable, I had changed the timing recently from the above described
3/4/2 @ 3 min, to 4/3/2 @ 2 min intervals. While a bit more aggressive,
I've been monitoring performance closely and it still performs very
politely on RF. Once I get my attic antenna onto a pole on the roof, and
have a better LoS to my area digis, I will change it back to 3/4/2 or
3/3/2, probably still at 2 min intervals.)


The Callsigns
=============

This dual pattern is achieved by using different callsigns. An object is
injected by K4FHK-15 if it is intended for RF. So any Igates should only
be concerned about that. If an object is intended only for APRS-IS and
not RF (the T3-6 intervals), it is injected by K4FHK-51 (the SSID, being
outside of ax.25's 4 bit range, is only valid on APRS-IS, and thus is
the clue that this object is not to be gated to RF).

At the start of every cycle, the "system" will also inject an object
(Igate) named K4FHK-IS (for Internet Server) together with a status.
This is done such that there is a back reference when users on aprs.fi,
openaprs.net or applications like APRSISCE look at, or hover over, the
accident object. For example, aprs.fi will draw a line back to my Igate,
identifying it as the source. It also lists a contact email address. And
I added the status with a message that attempts to solicit other Hams in
hopes to provide coverage in other areas of TN. A call for help if you
will.

So using two call signs explains why there are two identical objects on
the map (in case you are using software like APRSISCE that creates
unique objects of the same name per callsign). Also, should the RF
transmitted packet be gated back to APRS-IS, there would be a third
one. 

Originally I had transmitted with NOGATE in the path, since the RF
packet originated on IS and doesn't need to be gated back there. But
I've observed that some Igates do not honor NOGATE (that one also didn't
honor RFONLY) and still gate it to APRS-IS. That had the effect that an
RF packet was picked by such Igate, and in an application like APRSISCE,
it showed three identical objects. However, that remote Igate did not
receive the Object Kill packet, so only the two injected objects were
removed, leaving the igated object hanging out there. In order to avoid
situations like that, I decided to remove the NOGATE and simply gate it
back to IS myself so that the Object Kill is effective for all three
callsigns. Since all these duplicates are only on IS, there is no impact
on RF. I just wanted to mention that there is a reason for these
duplicates.


Coverage Area
=============

As mentioned above, these objects are currently on RF only in Middle
Tennessee, about 80 miles around Nashville. It would great if other
areas (West TN/Memphis/Jackson, or East TN/Knoxville/Chattanooga) had
coverage on RF as well.

In order to provide that coverage, an Igate operator would need to be
able to:

** Pick up objects named TNMVA* from APRS-IS that are sourced by
K4FHK-15.

** Limit these objects to a sensible coverage area. (For example, a
Memphis Igate might choose to cover all of West TN up to the TN river).

** Bring these objects to RF with a sensible hop count (depending on
local area connectivity between digis), with the State digipath being
recommended (TN2-2 or TN2-1).

The above can be accomplished with proper filter settings, either in the
APRS-IS server config section, or an per-interface configuration. While
being able to transmit as direct packets would be preferred, to keep
on-air time shorter, transmitting these objects as third party packets
would certainly work as well. I can provide a config example for aprx
that I used, but I'm not able to provide configuration examples for
UIView or others. But it's really just a simple application of filters.

Just do not gate K4FHK-51 sourced objects. These are intended for IS
only.


Performance
===========

Obviously, as a new Ham, I was concerned about the impact this system
would have on the local RF. I certainly did not want to make a bad first
impression. I had studied the APRS101 document many times over and was
very familiar with the protocol, but I needed to gain experience and
tread lightly on RF. This system went live on-air around Thanksgiving
2013, and I have been monitoring the performance closely. Even after
tweaking the intervals and timing, I observed that it has a low impact
on RF. There are weather stations beaconing more aggressively, with
longer packets, and of course "less-optimally" configured mobiles that
beacon more frequently and/or with excessive paths. (I wish there was a
minimum hold-off, the 3rd number, in turn-based Smart Beaconing...). 

So the TNMVA objects appear to be kind to the air waves. The
transmission pattern avoids congestion, while still providing timely
information to mobile rigs. And APRS-IS users have a near real-time view
of events, with only a 2-3 minute delay behind the official TDOT
listing. (Sorry, I can't do anything about accidents being late to be
listed at TDOT, but it seems to be doing great overall.)


Summary
=======

While I have not received a lot of spontaneous feedback from local APRS
users directly, those operators that I did talk to appear to be enjoying
this service. I myself find it very useful when I'm on the road. And
I've tracked a couple Hams on APRS-IS that actually avoided an accident
(Interstate closure) and took an alternate route. If this system was
able to save them from being stuck in traffic for hours, I feel have
accomplished my mission.

I'm glad you were able to read this post until the end. My sincere
apologies, I did not intent for this overview to get this long. There
are certainly sections I didn't fully elaborate on. So if anyone has any
additional questions, comments or concerns, please feel free to contact
me off-list, and I'll be happy to answer any questions you have. If you
do want to reply to the list directly, please be kind enough to trim
your reply the relevant parts.

Thanks again.

Best regards,
Frank / K4FHK