<div dir="ltr"><div class="gmail_default" style="font-size:x-small">Rob,</div><div class="gmail_default" style="font-size:x-small"><br></div><div class="gmail_default" style="font-size:x-small">Points well taken. </div><div class="gmail_default" style="font-size:x-small"><br></div><div class="gmail_default" style="font-size:x-small">I didn't mean to suggest that the production project should use Web Sockets, particularly from Python. Just that they were useful to me for testing. </div><div class="gmail_default" style="font-size:x-small"><br></div><div class="gmail_default" style="font-size:x-small">Also I had not considered that anyone was proposing to deploy any of this facing the open web without serious firewall protection. Frankly I'm still curious how proper configuration can ever be achieved for casual users without institutional resources. Is it even possible to use something like Cloudflare for a project like this? I haven't kept up with service provisioning at that scale.<br></div><div class="gmail_default" style="font-size:x-small"><br></div><div class="gmail_default" style="font-size:x-small">Dave Witten, KD0EAG</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 3, 2019 at 8:50 PM <<a href="mailto:tangerinesdr-request@lists.tapr.org">tangerinesdr-request@lists.tapr.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send TangerineSDR mailing list submissions to<br>
<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org" rel="noreferrer" target="_blank">http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:tangerinesdr-request@lists.tapr.org" target="_blank">tangerinesdr-request@lists.tapr.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:tangerinesdr-owner@lists.tapr.org" target="_blank">tangerinesdr-owner@lists.tapr.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of TangerineSDR digest..."<br>
Today's Topics:<br>
<br>
1. Re: Webserver for local host (Rob Wiesler)<br>
2. Re: Web presentation of magnetometer data to multiple clients<br>
(David Witten)<br>
3. Re: Webserver for local host (Rob Wiesler)<br>
<br><br><br>---------- Forwarded message ----------<br>From: Rob Wiesler <<a href="mailto:robert.wiesler@case.edu" target="_blank">robert.wiesler@case.edu</a>><br>To: TAPR TangerineSDR Modular Software Defined Radio <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>><br>Cc: <br>Bcc: <br>Date: Tue, 3 Dec 2019 21:11:23 -0500<br>Subject: Re: [TangerineSDR] Webserver for local host<br>ACK what's been said about Python, Python 3 vs. 2, and Flask. I'm not<br>
so vehemently against Java/JSP/Tomcat, but that's because I wasn't about<br>
about to form a strong opinion against it if I wasn't the one writing<br>
the web application.<br>
<br>
... that said, please do not consider writing a web application where<br>
the backend (running on the SBC) is written in Javascript (e.g. Node).<br>
That entire software ecosystem is a nightmare, and actively resists<br>
packaging and normal maintenance.<br>
<br>
On Tue, Dec 03, 2019 at 19:34:38 +0000, Engelke, Bill via TangerineSDR wrote:<br>
> It seems that Flask is often used in a container (i.e., venv). How<br>
> important is it to do this? Seems like an unnecessary complication for<br>
> our application, especially if we are going to standardize on Python<br>
> 3.<br>
<br>
Nathan has already responded, but I'd like to follow up with a much<br>
stronger response in the negative (which seems to be how this thread is<br>
organized).<br>
<br>
We absolutely must *not* use venv in a software package we intend to<br>
distribute as part of an operating system. Developers who insist on<br>
using bleeding-edge versions of everything use venv to isolate<br>
themselves from a dependency hell of their own creation in a way that<br>
effectively generates a second, private dependency hell, and in the<br>
process make it significantly harder to package their software. We<br>
avoid that by writing software that uses libraries packaged for a stable<br>
operating system distribution.<br>
<br>
> Documentation indicates (indeed, Flask itself gives a warning) that<br>
> Flask (by itself) should not be run as a server "in a production<br>
> environment." The advice is to run it under the control of a<br>
> production WSGI such as Waitress (in fact, running under a venv). Do<br>
> we need to plan to do this?<br>
<br>
Flask is a WSGI framework, not a web server. It comes with a simple<br>
HTTP server suitable for development only. A proper WSGI HTTP server<br>
should exist in front of it to handle requests. Flask's tutorial<br>
mentions Waitress (which is probably fine). Apache with mod_wsgi is<br>
another option. For setups more complicated than ours, I like uwsgi.<br>
Gunicorn is probably fine, too. Avoid twisted at all costs. Some WSGI<br>
servers shouldn't be used to host Internet-facing websites - in this<br>
case one would typically stick nginx in front of it (but TangerineSDR's<br>
web interface won't be Internet-facing, so that's largely moot).<br>
<br>
So, yes, plan on using Waitress. Just don't stick it in a venv, and<br>
don't rely on anything installed with pip, as opposed to apt-get (and<br>
friends).<br>
<br>
For example, my company's system uses uwsgi running in Emperor mode. We<br>
cobbled together a simple framework that allows multiple web<br>
applications to coexist at the same time with different URI prefixes.<br>
All customers have to do is drop an INI file and a directory containing<br>
their webapp in a specific subdirectory of /srv/www, and uwsgi<br>
automatically spots the new webapp, loads it, and serves it. In order<br>
to make this work without modifying uwsgi-emperor's global configuration<br>
in /etc (because that's a bad thing to make a package do), we built a<br>
new package containing a systemd unit that starts uwsgi in a custom way,<br>
and gave it a dependency on uwsgi-core (instead of uwsgi-emperor).<br>
<br>
For TangerineSDR, I envision a package built out something like this:<br>
- A webapp written with Flask and Python3 in pretty much the way Flask<br>
tells you to do it, minus venv and pip (but including the setup.py<br>
file they have you write). This will live in<br>
/usr/lib/python3/dist-packages, and be importable without editing the<br>
python path or changing the home directory.<br>
- Dependencies on at least python3-flask and python3-waitress.<br>
- A init script that calls waitress-serve (this will also work with<br>
systemd).<br>
<br>
<br>
<br><br><br>---------- Forwarded message ----------<br>From: David Witten <<a href="mailto:wittend@wwrinc.com" target="_blank">wittend@wwrinc.com</a>><br>To: "Kim, Hyomin" <<a href="mailto:hmkim@njit.edu" target="_blank">hmkim@njit.edu</a>><br>Cc: TAPR TangerineSDR Modular Software Defined Radio <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>>, "Dr. Nathaniel A. Frissell Ph.D." <<a href="mailto:nathaniel.frissell@scranton.edu" target="_blank">nathaniel.frissell@scranton.edu</a>><br>Bcc: <br>Date: Tue, 3 Dec 2019 20:20:33 -0600<br>Subject: Re: [TangerineSDR] Web presentation of magnetometer data to multiple clients<br><div dir="ltr">Hyomin,<br><br>Thanks for the links. <br><br>The screen capture I posted is from earlier code, and I know the values were being converted incorrectly at that time.<br><br>I was just trying to show folks on the list what was possible with minimal code. I will post more screenshots at a later date, perhaps even a live link if I can.<br><br>Dve Witten, KD0EAG<br><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 3, 2019 at 8:12 PM Kim, Hyomin <<a href="mailto:hmkim@njit.edu" target="_blank">hmkim@njit.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thank you, David!</div><div>Please take a look at the attached figure showing magnetic field components near Washington DC based on a magnetic field model. These values are what we can expect at the location. Thus your magnetometer reading should be similar to these. The values in your figure seem too big. </div><div><br></div><div>The northward (X) and eastward (Y) are in the horizontal field while the downward (Z) points toward the center of the earth. Thus the plane containing X and Y should be leveled. There are some different ways to define where X and Y should point to. Here in the figure, X is pointing to the geographic north and Y to the geographic east. Anyways, a quick and dirty way to make sure your magnetometer reading is correct is to calculate SQRT(X^2+Y^2+Z^2) from your reading to see if this value is close to the total intensity (F) in the figure. Of course, the value varies depending on where you are. <br></div><div><br></div><div>Visit the website: <a href="http://wdc.kugi.kyoto-u.ac.jp/igrf/point/index.html" target="_blank">http://wdc.kugi.kyoto-u.ac.jp/igrf/point/index.html</a> and enter the coordinates of your location to see expected magnetic fields of your location.</div><div><br></div><div>Hope this helps. </div><div><br></div><div>Hyomin KD2MCR</div><div><br></div><div><br></div><div><br></div><div><br></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><table style="border-collapse:collapse;width:525px;border-spacing:0px;background-color:transparent;color:rgb(35,38,54);font-family:Poppins,sans-serif;font-size:16px"><tbody style="border:0px;padding:0px;font-size:0.9em"><tr><td style="padding:5px;font-size:0.9em;font-stretch:normal;line-height:1.5;color:rgb(255,0,0);border-width:0px 3px 0px 0px;border-bottom-style:initial;border-bottom-color:initial;border-top-style:initial;border-right-style:solid;border-left-style:initial;border-top-color:initial;border-right-color:rgb(204,0,0);border-left-color:initial;width:180px;vertical-align:middle"><a href="https://www.njit.edu/" style="background-color:transparent;color:rgb(51,122,183)" target="_blank"><img src="https://assets.njit.edu/uicomponents/NJIT-email-logo.png" alt="NJIT logo" style="border: 0px; display: block; vertical-align: middle; width: 150px; height: auto; max-width: 150px;"></a></td><td style="padding:5px 5px 5px 10px;font-size:12px;font-stretch:normal;line-height:16px;border:0px;font-family:Arial,Helvetica,sans-serif"><strong style="color:rgb(111,111,111);font-size:13px">Hyomin Kim</strong><br><font color="#6f6f6f">Assistant Professor</font><br><font color="#6f6f6f">Physics<br></font><font color="#666666">Center for Solar-Terrestrial Research<br>Institute for Space Weather Sciences</font><br><font color="#666666"><a href="mailto:hmkim@njit.edu" target="_blank">hmkim@njit.edu</a> • <span style="white-space:nowrap">(973) 596-5704</span><br><a href="https://web.njit.edu/~hmkim/" target="_blank">https://web.njit.edu/~hmkim/</a><br>104 Tiernan Hall, 161 Warren Street, Newark, NJ 07102<br></font></td></tr></tbody></table></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 3, 2019 at 4:02 PM Dr. Nathaniel A. Frissell Ph.D. via TangerineSDR <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US">
<div>
<p class="MsoNormal">Thank you, David. This looks good! Do you think you could add in a time series line plot of the Bx, By, and Bz coordinates? I think this would be useful to have right away, even for diagnostics.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">73 de Nathaniel W2NAF<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black">TangerineSDR <<a href="mailto:tangerinesdr-bounces@lists.tapr.org" target="_blank">tangerinesdr-bounces@lists.tapr.org</a>> on behalf of TangerineSDR Listserv <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>><br>
<b>Reply-To: </b>TangerineSDR Listserv <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>><br>
<b>Date: </b>Tuesday, December 3, 2019 at 1:49 PM<br>
<b>To: </b>TangerineSDR Listserv <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>><br>
<b>Cc: </b>David Witten <<a href="mailto:wittend@wwrinc.com" target="_blank">wittend@wwrinc.com</a>><br>
<b>Subject: </b>[TangerineSDR] Web presentation of magnetometer data to multiple clients<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">All,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">This is a work in progress, for my own use right now. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">It shows presentation of magnetometer data in a web browser. Ignore the actual data shown, it properly signed quantities.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">The page is served by HTTP, but the sensor data is sent via Websockets. This is done using a golang server (minimal) that launches a program written in C that interrogates the I2C channel. When the golang
program detects a browser connection, it serves the web page. The web page opens a Websockets connection and streams the magnetometer data encapsulated in JSON to the browser client.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">No web server, as such, is used. Only one HTTP/HTTPs request is used. There are other possible ways to do this, but this one works well for my purposes. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">The intent is to have each local host send its lat/long and ip address on startup. The locations would be indicated on the map. Clicking on the marker would open a connection to that particular SBC.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Data logging locally and remotely could both be controlled.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Again this is just eye candy, showing what can be done. No frameworks, user's language of choice for the service.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Dave Witten<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
-- <br>
TangerineSDR mailing list<br>
<a href="mailto:TangerineSDR@lists.tapr.org" target="_blank">TangerineSDR@lists.tapr.org</a><br>
<a href="http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org" rel="noreferrer" target="_blank">http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org</a><br>
</blockquote></div>
</blockquote></div>
<br><br><br>---------- Forwarded message ----------<br>From: Rob Wiesler <<a href="mailto:robert.wiesler@case.edu" target="_blank">robert.wiesler@case.edu</a>><br>To: TAPR TangerineSDR Modular Software Defined Radio <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>><br>Cc: <br>Bcc: <br>Date: Tue, 3 Dec 2019 21:49:52 -0500<br>Subject: Re: [TangerineSDR] Webserver for local host<br>On Tue, Dec 03, 2019 at 12:16:45 -0600, David Witten via TangerineSDR wrote:<br>
> I do not see much need for complication with more than very lightweight<br>
> 'frameworks'.<br>
<br>
I'll agree a little more strongly - we must actively avoid anything<br>
"full featured". The reason my company's system has problems with<br>
Django is not because it serves pages slowly, but because Django takes<br>
forever to import on a tiny SBC, and makes you import it as part of<br>
basically all maintenance tasks.<br>
<br>
> I would propose an architecture where the localhost runs a simple web<br>
> service loop. This loop would be launched at boot and principally serve<br>
> two pages - one page that just provides status information without<br>
> authentication. The other would be a page requiring login by the<br>
> owner/controller of the localhost that allows control of services and<br>
> presentation of data to the owner/controller of the device. All data would<br>
> be served over secure, certificated connections. All interaction would be<br>
> through small, purpose-built service handlers that autostart but allow<br>
> management through the owner/controller's page.<br>
<br>
Caveat one: "Secure, certified connections" can be approximated, but not<br>
achieved. HTTPS really only works (and even then, not well) for<br>
Internet-facing installations where access to the physical hardware is<br>
carefully limited. It is effectively impossible to get properly-signed<br>
HTTPS certificates for embedded systems not under our physical control<br>
in the field and behind NAT. We can (and should) make the TangerineSDR<br>
generate self-signed certificates, but then we still have a MITM issue,<br>
plus now our users' web browsers are screaming at them that something is<br>
wrong. I imagine that first-time setup via the web interface will<br>
basically only happen on a private access point or wired connection, and<br>
most users shouldn't have too much of an issue re-accepting the<br>
self-signed certificate every year or so when it expires and the web<br>
server regenerates it, especially if the instructions are well-written,<br>
but that's not the same thing as "secure".<br>
<br>
<br>
Caveat two: Having the webapp talk to too many different processes is<br>
probably a bad idea. I think Bill envisions at least one master control<br>
process - even if it makes sense to split all the various knobs between<br>
different processes, the webapp should still probably only talk to the<br>
master control process (although it might also need a background mule to<br>
save some dynamic session state between worker threads).<br>
<br>
<br>
> In many cases, such as the transfer of bulk data to the central repository,<br>
> connections would be best handled using Web Sockets or some similar<br>
> mechanism directly by the service process with only lightweight progress<br>
> reporting directed through the Web interface.<br>
<br>
Caveat three: Web Sockets are a really bad idea, especially in Python.<br>
That effectively limits us to using Twisted and Autobahn, and either one<br>
of those is a complete nightmare to be avoided at all costs (I won't get<br>
into the nonsense we've had to put up with at work to make it halfway<br>
usable). TLS over TCP is pretty easy, not to mention sane, and it can<br>
be worked into basically any event loop.<br>
<br>
<br>
Apart from those small details, I think that would definitely be the<br>
architecture to shoot for.<br>
<br>
<br>
> These are just my thoughts right now, I'm sure that I can come up with more<br>
> opinions if anyone wants to hear them.<br>
<br>
I'm really hoping to free up some time this decade so that I can<br>
actually get some work done for the project instead of just coming up<br>
with opinions. :)<br>
<br>
<br>
TangerineSDR mailing list<br>
<a href="mailto:TangerineSDR@lists.tapr.org" target="_blank">TangerineSDR@lists.tapr.org</a><br>
<a href="http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org" rel="noreferrer" target="_blank">http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org</a><br>
</blockquote></div>