[TangerineSDR] Seeking your Review and Comment on Tangerine SBC Functional Specification

[Engelke, Bill]

Hello Rob:

A couple of data points for you -

RAM-based ring buffer: Digital RF includes this as a standard utility, which I have tested and it works great. I have configured it on the Odroid and pushed data to it at a pretty high rate, and it got all the stuff written to separate spinning hard drive (at least right up until the Odroid's CPUs started bumping 100% utilization, which was 12 to 14 channels of I/Q data at 192 ksps). If you want to play with Digital RF, it is available at https://github.com/MITHaystack/digital_rf . Note: you must install HDF5 first.

Regarding handling uploaded data - you raise a lot of good points about data storage and performance implications... I need to study on that a bit more.  At the very least, I envision that the storage method for I & Q data may want to differ from that of pre-processed data, and maybe between the ring buffer and firehose; but we can discuss. I will read up on inotify.

- Updated F-Spec incorporating recent comments to be published soon...  day or two...    -73- Bill



-----Original Message-----
From: Rob Wiesler <robert.wiesler at case.edu> 
Sent: Tuesday, October 8, 2019 10:42 PM
To: Engelke, Bill <bill.engelke at ua.edu>
Cc: TAPR TangerineSDR Modular Software Defined Radio <tangerinesdr at lists.tapr.org>
Subject: Re: Seeking your Review and Comment on Tangerine SBC Functional Specification

Hi, Bill.

On Tue, Oct 8, 2019 at 1:56 PM Engelke, Bill <bill.engelke at ua.edu> wrote:
> Hello Rob - thanks for your most helpful reply - I will be incorporating your thoughts into the next version.  I have a few follow-up remarks:
>
> - I will discuss the Ring Buffer concept with Nathaniel.  It seems like it could be useful, it's just that it would not be practical with today's hardware to run it at the huge data rate originally envisioned.

That's my thought.  I think we might (at least briefly) explore the idea of a RAM-based ringbuffer (I haven't even done the back-of-the-envelope calculation to know if this is even remotely feasible - the N2 has either 2G or 4G RAM).  Again, I don't know which phase this belongs to (I'll let you guys figure it out).

> - Excellent idea to add a statement about making the system able to 
> survive power cycles. That was one of the things that made me abandon 
> Linux for a number of years after it first became available - if the 
> power went off, the system would often get bricked. I have been 
> delighted to find that this is not the case with more modern 
> implementations (i.e., Raspberry, Odroid N2 etc)

The difference is a matter of journaling file systems, better per-program behavior, significantly better sync(1) semantics, and per-package work towards idempotent software updates.  The last bit remaining for TangerineSDR is atomic updates.

> - Discoverability: I am not sure of the value of this either, it is an artifact from how the OpenHPSDR-compatible systems work. With TangerineSDR, we do plan to support the config where there are multiple units on the same network (institutional setting); but the plan is that these would be individually configured using a web running in the SBC.

That plan sounds good.

> - Operating system: you mention Debian several times.  The Odroid N2 uses Ubuntu. If someone wants to go through the effort to gen up Debian to run on it, they are welcome to do so, but I would consider that out of scope for Phase 1.

They likely ship Ubuntu only to reduce the complexity of choice.
Debian runs perfectly well (I've seen a lot of mail to this effect pass through my inbox).  Since we will want to generate our own image and not simply use theirs, and they likely have no significant Ubuntu-related modifications we want to keep, Ubuntu and Debian are more-or-less fungible here.

In any case, this shouldn't be a sticking point.

> - Waiting for files to close: what I mean by this is: if a file is actively being uploaded to the server, the server should not attempt to open and read that file before the transfer is complete; in fact, I would expect the file transfer process to maintain exclusive rights to the file until transfer is done (and file closed).  What I have been imagining for Use Case 3 (data pre-processed by GNURadio in TangerineSDR) is that the FFT snapshots (of small chunks of bandwidth around WWV) would be uploaded in real time and stored as database records (rather than files), which makes them immediately available for real-time analysis on the server.  (This is not to say we can't also support Use Case 1, which involves uploading large files from the ring buffer - in which case we have the condition of waiting for a file transfer to complete before we can access the file on the server).  I'm sure there are other ways to "skin this cat," - if I am missing something, please advise.

Well, the classic way to do this is to process the uploaded stream as it's being uploaded, if real-time processing is desired.  In that case, the part that's doing the processing likely won't be reading the data from the filesystem (or database) - the server that's accepting the data will stream via memory as it arrives.  If chunks are sufficiently small, then we can reduce the complexity of the system (and keep (near-)real-time processing as an option) by using an inotify-based watch loop, which can be set up to only trigger when the file is completely written to the filesystem (I use this kind of thing a lot in my own systems).

If we store the data in a database as the hand-off point between the server and the real-time processing stage, we run into the problem of notifying the processing stage when there's data to be found in the database (sampling for new data is not performant).  There are multiple easy solutions to this problem, but they're all at least as hard as just passing the data to the processing stage via a pipe or some other IPC method.  In contrast, if we write to individual files on a real filesystem, inotify handles the handoff of "ownership".

> - Yes, mender.io is merely an example. I will look into how atomic updates are done with Ubuntu.

I'm not sure too many people have actually solved this problem the way I think we need to, but proof to the contrary is very welcome.
Anything relevant to Ubuntu should be trivially transferable to Debian (and vice-versa), *unless* it involves Canonical (in which case we'd probably have to pay for it anyway).

> QUESTION: I am guessing that some geeks (including self here) will prefer to have control over when updates are applied, and others are happy to let the system handle these automatically; we probably should let people have the choice, shouldn't we? Maybe default to automatic, but let the user opt out of that?  Or, what do you suggest?

My thought is that we should ship with automatic updates enabled, but ask the user what they prefer during setup (when they enter the upload identity token and configure network access).  Then we just make sure that updates don't happen while the system is being configured (easy).
That way we have a system that does the right thing out of the box, but ultimately does whatever the user wants it to do, all without generating some setting the user needs to hunt around for.

> Please keep me informed of your progress on building an example system.  We are being asked (by Nathaniel, et al) to make data collection methods common between TangerineSDR and the CWRU system (does it have a name?) ... I have been playing with a development version of the SatNOGS Network, and plan to use a modified version of it for our Phase 1 Central Control System & database.

We have a meeting tomorrow morning.  I'll keep you posted, and probably introduce you to the guy who seems to be riding herd on the students who are already working on that.

As far as a name goes, we've been calling it (or at least the radio board that John Gibbons (N8OBJ) designed and built) the "Standards Station Receiver".  This isn't exactly an unambiguous name, nor does it properly refer to the entire system as a whole, so let's not engrave that anywhere important just yet :) .

--
73 de AC8YV