[aprssig] Live Telemetry Data Through a Simple Web Page

[Brad McConahay]

Actually, I'm doing it almost exactly as you describe, in perl.  But -
functions that pass hierarchical looking arguments means never having to
update the script to add new variables.  And the end-user still doesn't have
to know anything about XML to use the replacement script.

For example, the three parts of the main routine look something like this...

-----------

1. $webpage = get_webpage($cgi->param('url'))

(where get_webpage() is a 5 or 6 line function using LWP to grab the remote
web page)


2. $xml = get_xml("http://xml.n8qq.com/...?call=...");

(where get_xml() is a 5 or 6 line function using XML::Simple that grabs the
packet from xml.n8qq.com and automatically puts it into the simple hash
pointer $xml, so you can get things like $xml->{position}->{})


3. $webpage =~ s/aprs\((.*?)\)/filter_aprs($1)/egi;

(only one regex line needed -- filter_aprs() is a 4 or 5 line function that
takes each aprs(foo bar) variable and replaces it with the value in
$xml->{foo}->{bar}.  This makes it totally extensible without ever updating
this script to add new variables.)


print $webpage;

-----------

Using "$speed" type variables in the web page would mean needing to parse
separately for every possible variable, which is a lot more overhead and not
extensible at all.  To avoid that, the variable needs to be something with
at least a distinct beginning and ending token.  The "aprs(foo bar)"
structure just looks standard and clean to me.  And there's actually no
reason for the user not to think of it as a function, even though it's
really just a replacement variable in the end.

Re: XML...
>
> You CAN use it in many situations, but at this
> point in time it doesn't make things easier  
> than direct access of the data. Maybe when better
> tools are available...

I probably would've agreed with you a year ago, but having done it both ways
now, I have to disagree in a big way.  With having the XML interface already
set up, this replacement script basically wrote itself.  No
variable-to-value mapping routines needed.  The single script that maps
mysql fields to XML elements is the only time that should ever happen.  This
script will automatically know whatever is in the APRS XML spec.  And a
quick script to always print a list of every available variable is easy to
create as well. (that's another script that will never need to be updated to
add new variables)

Since you already have to distinguish between many data items in a somewhat
hierarchical manner anyway, why not just make it a ubiquitous standard
hierarchy like XML and bring everything else that offers into the mix at the
same time.  Iow, you can't just say $speed or $direction.  You already have
to say something like $position_speed and $wind_direction.  Why not just
make it all hierarchical, standard, etc.  APRS packets lend themselves too
well to hierarchical organization not to.

I see this is no different a concept than having standard parsing routines
to get data from packets to mysql.  This is just a different layer of
handling the data, and there's no reason it shouldn't be both standard and
centralized.  And by exposing the XML interface to the public, that layer of
the equation is automatically solved for everybody in a standard way, via
http, without exposing any source code, etc, etc, etc...

And as further proof of good existing XML tools... adding the XQL
interpreter took less than an hour last week.  With XML::XQL, the XQL filter
routine took like 10 lines of code.  And I only needed to know enough XQL to
test and make sure it was working okay to do it.  I'd say good tools are
already there now.  I was literally shocked at how easy it was.

I remember you writing early-on that you were planning on doing findu in an
actual XML database, and you had a lot of praise for it.  I understand
completely why you went with mysql, and I agree that it was (and still is) a
good decision.  But that doesn't mean that XML still isn't the best way to
expose the data for public use.  I wonder if that experience soured you
_too_ much on XML.

XML makes it all very centralized, hands-off, and rapidly extensible.
(okay, I'm a little tired of the "extensible" buzzword, but I'm finding that
it's very true... :)

Brad N8QQ


-----Original Message-----
From: Steve Dimse [mailto:steve at dimse.com] 
Sent: Saturday, April 23, 2005 5:47 PM
To: Brad McConahay
Cc: aprssig at lists.tapr.org; ve3pep at rac.ca
Subject: Re: [aprssig] Live Telemetry Data Through a Simple Web Page


On Apr 23, 2005, at 5:13 PM, Brad McConahay wrote:

> On 23 Apr 2005 11:03:19 -0400, Steve Dimse Wrote:
>>
>> This would be a nice project for someone else to do, let the
>> user specify a url in the page request, the server grabs the
>> user's page from his own web space (everyone can get web space
>> somewhere) and replaces any variables using Brad's XML service.
>
> So if I'm understanding you correctly...
>
> Let's say this is my page with variables, as an end-user: (the things  
> that
> look like "aprs()" functions are the variables)
>
> http://home.n8qq.com/aprs
>
> And I then feed that url and my callsign to this script, as you  
> describe:
>
> http://xml.n8qq.com/replace/index.cgi?call=n8qq&url=http:// 
> home.n8qq.com/apr
> s
>
> ...and the variables are replaced.  Is that the sort of thing you had  
> in
> mind?
>
> These variables look more like functions with arguments than variables  
> -
> maybe they should be something embedded in an html comment, like  
> <!--aprs
> position latitude degrees-->.  Not sure what would be best in that  
> regard.

Well, I guess it depends on the implementation. The way I  visualized  
it, the replace script grabs the data first, then goes through and does  
a global replace on the variable names, so there is really no function  
calling going on. Granted, you could also do it so that you parse the  
file and then call a function each time you find a variable, which is  
easier depends on the language you are using to write the replace  
script. For example, in perl you can do this very easily (say that, for  
example speed is represented in the html source as $speed (a perl-like  
syntax, but it really can be any sort of syntax including the comment  
you mention, see next paragraph for more details on this) and you have  
already called the  database code to get the speed of the callsign  
which is now in the perl variable $speed:

$thepage = `wget http://home.n8qq.com/apr`;

$thepage =~ s/\$speed/$speed/g;
(this is followed by a bunch of other regex that replace all the other  
variables defined that may be in a page)

then just

print $thepage;

As to your syntax, I'd say an HTML comment is not the best choice, as  
it would hide mistakes. Say the variable was speed, but the source page  
mistyped the speed as spped:

my speed is $spped

vs

my speed is <!--$spped-->

My syntax outputs

my speed is $spped

the comment method just outputs

my speed is

Which makes it easier to spot the error? Especially important if you  
want to make users write the full XML hierarchy instead of the simple  
variable names I propose. Not my choice since I'm not writing it, but  
adding XML to this seems like a needless complication to me...that is  
sort of the way I feel about XML in general. You CAN use it in many  
situations, but at this point in time it doesn't make things easier  
than direct access of the data. Maybe when better tools are  
available...

Steve K4HG