[TangerineSDR] TangerineSDR Digest, Vol 9, Issue 12

[David Witten]

Rob,

Points well taken.

I didn't mean to suggest that the production project should use Web
Sockets, particularly from Python.  Just that they were useful to me for
testing.

Also I had not considered that anyone was proposing to deploy any of this
facing the open web without serious firewall protection.  Frankly I'm still
curious how proper configuration can ever be achieved for casual users
without institutional resources.  Is it even possible to use something like
Cloudflare for a project like this?  I haven't kept up with service
provisioning at that scale.

Dave Witten, KD0EAG

On Tue, Dec 3, 2019 at 8:50 PM <tangerinesdr-request at lists.tapr.org> wrote:

> Send TangerineSDR mailing list submissions to
>         tangerinesdr at lists.tapr.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
> or, via email, send a message with subject or body 'help' to
>         tangerinesdr-request at lists.tapr.org
>
> You can reach the person managing the list at
>         tangerinesdr-owner at lists.tapr.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of TangerineSDR digest..."
> Today's Topics:
>
>    1. Re: Webserver for local host (Rob Wiesler)
>    2. Re: Web presentation of magnetometer data to multiple clients
>       (David Witten)
>    3. Re: Webserver for local host (Rob Wiesler)
>
>
>
> ---------- Forwarded message ----------
> From: Rob Wiesler <robert.wiesler at case.edu>
> To: TAPR TangerineSDR Modular Software Defined Radio <
> tangerinesdr at lists.tapr.org>
> Cc:
> Bcc:
> Date: Tue, 3 Dec 2019 21:11:23 -0500
> Subject: Re: [TangerineSDR] Webserver for local host
> ACK what's been said about Python, Python 3 vs. 2, and Flask.  I'm not
> so vehemently against Java/JSP/Tomcat, but that's because I wasn't about
> about to form a strong opinion against it if I wasn't the one writing
> the web application.
>
> ... that said, please do not consider writing a web application where
> the backend (running on the SBC) is written in Javascript (e.g. Node).
> That entire software ecosystem is a nightmare, and actively resists
> packaging and normal maintenance.
>
> On Tue, Dec 03, 2019 at 19:34:38 +0000, Engelke, Bill via TangerineSDR
> wrote:
> > It seems that Flask is often used in a container (i.e., venv).  How
> > important is it to do this? Seems like an unnecessary complication for
> > our application, especially if we are going to standardize on Python
> > 3.
>
> Nathan has already responded, but I'd like to follow up with a much
> stronger response in the negative (which seems to be how this thread is
> organized).
>
> We absolutely must *not* use venv in a software package we intend to
> distribute as part of an operating system.  Developers who insist on
> using bleeding-edge versions of everything use venv to isolate
> themselves from a dependency hell of their own creation in a way that
> effectively generates a second, private dependency hell, and in the
> process make it significantly harder to package their software.  We
> avoid that by writing software that uses libraries packaged for a stable
> operating system distribution.
>
> > Documentation indicates (indeed, Flask itself gives a warning) that
> > Flask (by itself) should not be run as a server "in a production
> > environment." The advice is to run it under the control of a
> > production WSGI such as Waitress (in fact, running under a venv).  Do
> > we need to plan to do this?
>
> Flask is a WSGI framework, not a web server.  It comes with a simple
> HTTP server suitable for development only.  A proper WSGI HTTP server
> should exist in front of it to handle requests.  Flask's tutorial
> mentions Waitress (which is probably fine).  Apache with mod_wsgi is
> another option.  For setups more complicated than ours, I like uwsgi.
> Gunicorn is probably fine, too.  Avoid twisted at all costs.  Some WSGI
> servers shouldn't be used to host Internet-facing websites - in this
> case one would typically stick nginx in front of it (but TangerineSDR's
> web interface won't be Internet-facing, so that's largely moot).
>
> So, yes, plan on using Waitress.  Just don't stick it in a venv, and
> don't rely on anything installed with pip, as opposed to apt-get (and
> friends).
>
> For example, my company's system uses uwsgi running in Emperor mode.  We
> cobbled together a simple framework that allows multiple web
> applications to coexist at the same time with different URI prefixes.
> All customers have to do is drop an INI file and a directory containing
> their webapp in a specific subdirectory of /srv/www, and uwsgi
> automatically spots the new webapp, loads it, and serves it.  In order
> to make this work without modifying uwsgi-emperor's global configuration
> in /etc (because that's a bad thing to make a package do), we built a
> new package containing a systemd unit that starts uwsgi in a custom way,
> and gave it a dependency on uwsgi-core (instead of uwsgi-emperor).
>
> For TangerineSDR, I envision a package built out something like this:
> - A webapp written with Flask and Python3 in pretty much the way Flask
>   tells you to do it, minus venv and pip (but including the setup.py
>   file they have you write).  This will live in
>   /usr/lib/python3/dist-packages, and be importable without editing the
>   python path or changing the home directory.
> - Dependencies on at least python3-flask and python3-waitress.
> - A init script that calls waitress-serve (this will also work with
>   systemd).
>
>
>
>
>
> ---------- Forwarded message ----------
> From: David Witten <wittend at wwrinc.com>
> To: "Kim, Hyomin" <hmkim at njit.edu>
> Cc: TAPR TangerineSDR Modular Software Defined Radio <
> tangerinesdr at lists.tapr.org>, "Dr. Nathaniel A. Frissell Ph.D." <
> nathaniel.frissell at scranton.edu>
> Bcc:
> Date: Tue, 3 Dec 2019 20:20:33 -0600
> Subject: Re: [TangerineSDR] Web presentation of magnetometer data to
> multiple clients
> Hyomin,
>
> Thanks for the links.
>
> The screen capture I posted is from earlier code, and I know the values
> were being converted incorrectly at that time.
>
> I was just trying to show folks on the list what was possible with minimal
> code.  I will post more screenshots at a later date, perhaps even a live
> link if I can.
>
> Dve Witten, KD0EAG
>
>
> On Tue, Dec 3, 2019 at 8:12 PM Kim, Hyomin <hmkim at njit.edu> wrote:
>
>> Thank you, David!
>> Please take a look at the attached figure showing magnetic field
>> components near Washington DC based on a magnetic field model. These values
>> are what we can expect at the location. Thus your magnetometer reading
>> should be similar to these. The values in your figure seem too big.
>>
>> The northward (X) and eastward (Y) are in the horizontal field while the
>> downward (Z) points toward the center of the earth. Thus the plane
>> containing X and Y should be leveled. There are some different ways to
>> define where X and Y should point to. Here in the figure, X is pointing to
>> the geographic north and Y to the geographic east. Anyways, a quick and
>> dirty way to make sure your magnetometer reading is correct is to calculate
>> SQRT(X^2+Y^2+Z^2) from your reading to see if this value is close to the
>> total intensity (F) in the figure. Of course, the value varies depending on
>> where you are.
>>
>> Visit the website: http://wdc.kugi.kyoto-u.ac.jp/igrf/point/index.html and
>> enter the coordinates of your location to see expected magnetic fields of
>> your location.
>>
>> Hope this helps.
>>
>> Hyomin KD2MCR
>>
>>
>>
>>
>> [image: NJIT logo] <https://www.njit.edu/> *Hyomin Kim*
>> Assistant Professor
>> Physics
>> Center for Solar-Terrestrial Research
>> Institute for Space Weather Sciences
>> hmkim at njit.edu • (973) 596-5704
>> https://web.njit.edu/~hmkim/
>> 104 Tiernan Hall, 161 Warren Street, Newark, NJ 07102
>>
>>
>> On Tue, Dec 3, 2019 at 4:02 PM Dr. Nathaniel A. Frissell Ph.D. via
>> TangerineSDR <tangerinesdr at lists.tapr.org> wrote:
>>
>>> Thank you, David. This looks good! Do you think you could add in a time
>>> series line plot of the Bx, By, and Bz coordinates? I think this would be
>>> useful to have right away, even for diagnostics.
>>>
>>>
>>>
>>> 73 de Nathaniel W2NAF
>>>
>>>
>>>
>>> *From: *TangerineSDR <tangerinesdr-bounces at lists.tapr.org> on behalf of
>>> TangerineSDR Listserv <tangerinesdr at lists.tapr.org>
>>> *Reply-To: *TangerineSDR Listserv <tangerinesdr at lists.tapr.org>
>>> *Date: *Tuesday, December 3, 2019 at 1:49 PM
>>> *To: *TangerineSDR Listserv <tangerinesdr at lists.tapr.org>
>>> *Cc: *David Witten <wittend at wwrinc.com>
>>> *Subject: *[TangerineSDR] Web presentation of magnetometer data to
>>> multiple clients
>>>
>>>
>>>
>>> All,
>>>
>>>
>>>
>>> This is a work in progress, for my own use right now.
>>>
>>>
>>>
>>> It shows presentation of magnetometer data in a web browser.  Ignore the
>>> actual data shown, it properly signed quantities.
>>>
>>>
>>>
>>> The page is served by HTTP, but the sensor data is sent via Websockets.
>>> This is done using a golang server (minimal) that launches a program
>>> written in C that interrogates the I2C channel. When the golang program
>>> detects a browser connection, it serves the web page.  The web page opens a
>>> Websockets connection and streams the magnetometer data encapsulated in
>>> JSON to the browser client.
>>>
>>>
>>>
>>> No web server, as such, is used.  Only one HTTP/HTTPs request is used.
>>> There are other possible ways to do this, but this one works well for my
>>> purposes.
>>>
>>>
>>>
>>> The intent is to have each local host send its lat/long and ip address
>>> on startup.  The locations would be indicated on the map.  Clicking on the
>>> marker would open a connection to that particular SBC.
>>>
>>>
>>>
>>> Data logging locally and remotely could both be controlled.
>>>
>>>
>>>
>>> Again this is just eye candy, showing what can be done.  No frameworks,
>>> user's language of choice for the service.
>>>
>>>
>>>
>>> Dave Witten
>>> --
>>> TangerineSDR mailing list
>>> TangerineSDR at lists.tapr.org
>>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>>
>>
>
>
> ---------- Forwarded message ----------
> From: Rob Wiesler <robert.wiesler at case.edu>
> To: TAPR TangerineSDR Modular Software Defined Radio <
> tangerinesdr at lists.tapr.org>
> Cc:
> Bcc:
> Date: Tue, 3 Dec 2019 21:49:52 -0500
> Subject: Re: [TangerineSDR] Webserver for local host
> On Tue, Dec 03, 2019 at 12:16:45 -0600, David Witten via TangerineSDR
> wrote:
> > I do not see much need for complication with more than very lightweight
> > 'frameworks'.
>
> I'll agree a little more strongly - we must actively avoid anything
> "full featured".  The reason my company's system has problems with
> Django is not because it serves pages slowly, but because Django takes
> forever to import on a tiny SBC, and makes you import it as part of
> basically all maintenance tasks.
>
> > I would propose an architecture where the localhost runs a simple web
> > service loop.  This loop would be launched at boot and principally serve
> > two pages - one page that just provides status information without
> > authentication.  The other would be a page requiring login by the
> > owner/controller of the localhost that allows control of services and
> > presentation of data to the owner/controller of the device.  All data
> would
> > be served over secure, certificated connections.  All interaction would
> be
> > through small, purpose-built service handlers that autostart but allow
> > management through the owner/controller's page.
>
> Caveat one: "Secure, certified connections" can be approximated, but not
> achieved.  HTTPS really only works (and even then, not well) for
> Internet-facing installations where access to the physical hardware is
> carefully limited.  It is effectively impossible to get properly-signed
> HTTPS certificates for embedded systems not under our physical control
> in the field and behind NAT.  We can (and should) make the TangerineSDR
> generate self-signed certificates, but then we still have a MITM issue,
> plus now our users' web browsers are screaming at them that something is
> wrong.  I imagine that first-time setup via the web interface will
> basically only happen on a private access point or wired connection, and
> most users shouldn't have too much of an issue re-accepting the
> self-signed certificate every year or so when it expires and the web
> server regenerates it, especially if the instructions are well-written,
> but that's not the same thing as "secure".
>
>
> Caveat two: Having the webapp talk to too many different processes is
> probably a bad idea.  I think Bill envisions at least one master control
> process - even if it makes sense to split all the various knobs between
> different processes, the webapp should still probably only talk to the
> master control process (although it might also need a background mule to
> save some dynamic session state between worker threads).
>
>
> > In many cases, such as the transfer of bulk data to the central
> repository,
> > connections would be best handled using Web Sockets or some similar
> > mechanism directly by the service process with only lightweight progress
> > reporting directed through the Web interface.
>
> Caveat three: Web Sockets are a really bad idea, especially in Python.
> That effectively limits us to using Twisted and Autobahn, and either one
> of those is a complete nightmare to be avoided at all costs (I won't get
> into the nonsense we've had to put up with at work to make it halfway
> usable).  TLS over TCP is pretty easy, not to mention sane, and it can
> be worked into basically any event loop.
>
>
> Apart from those small details, I think that would definitely be the
> architecture to shoot for.
>
>
> > These are just my thoughts right now, I'm sure that I can come up with
> more
> > opinions if anyone wants to hear them.
>
> I'm really hoping to free up some time this decade so that I can
> actually get some work done for the project instead of just coming up
> with opinions. :)
>
>
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20191203/97b1bae2/attachment-0001.html>