[TangerineSDR] Local Host Functional Specification, Version 0.1

Tue May 14 12:52:46 EDT 2019

A few remarks…

My only concern with the term "server" is that we may have several functional pieces that fit that definition, like when the TangerineSDR (including the SBC) acts like a server to users that want to connect to get RX data streams in a non-SWPC use case.  So as long as we can be unambiguous in the use, I am OK with the term.

Maybe I need to quit using the term “server” entirely, because it is ambiguous. I just started using it because Tom reminded me that some big institutions will have powerful “servers” to which the Tangerine DE could upload to directly (without the involvement of the SBC).  I just don’t know what else to call them.

QUESTION FOR YOU:  Will the Tangerine DE have a web service (e.g., Alpine or something) running on it, or will the user configure it using a terminal interface (like Red Pitaya) ? Or??

I will want to define the DE-to-SBC protocol in terms of Server and Client functions, and it will have nothing to do with the Central Server concept.

Understood.

When referring to the DE, can you please define it as the "TangerineSDR Data Engine (DE)" the first time, as a definition?

Will do.

-73- Bill AB4EJ

From: TangerineSDR <tangerinesdr-bounces at lists.tapr.org> On Behalf Of Scotty Cowling via TangerineSDR
Sent: Monday, May 13, 2019 7:06 PM
To: TAPR TangerineSDR Modular Software Defined Radio <tangerinesdr at lists.tapr.org>
Cc: Scotty Cowling <scotty at tonks.com>
Subject: Re: [TangerineSDR] Local Host Functional Specification, Version 0.1

Hi Bill,

My only concern with the term "server" is that we may have several functional pieces that fit that definition, like when the TangerineSDR (including the SBC) acts like a server to users that want to connect to get RX data streams in a non-SWPC use case.  So as long as we can be unambiguous in the use, I am OK with the term.

I will want to define the DE-to-SBC protocol in terms of Server and Client functions, and it will have nothing to do with the Central Server concept.

When referring to the DE, can you please define it as the "TangerineSDR Data Engine (DE)" the first time, as a definition?

73,
Scotty WA2DFI
On 2019-05-13 14:56, Engelke, Bill wrote:
Hello Scotty:

1.      On terminology, how about this…. In the SBC documentation, I need some way refer to the Data Engine.  The first couple of versions of the SBC Functional Spec call it the Tangerine; but now I understand that you mean for the Tangerine to include the SBC.  I can update the SBC spec to refer to the Data Engine (instead of calling it the Tangerine).

2.      I have already started a Parking Lot document on the Google Drive. See https://drive.google.com/drive/folders/1FYlmw3pINacZMEnYCFQg2sNzPeM-NElK We can copy this stuff to github at some point if needed.

3.      On the “server” – I mention the DE-to-server data path primarily because Tom wants to account for organizations like MIT Haystack which will have virtually unlimited bandwidth to an enormous server.  Most users will not have that, and the Central Control system / Database will not be sized to allow uploading at full speed (25 Megabytes/sec). In > 90% of cases, the data will have to go from DE to SBC and then be throttled and or preprocessed/compressed to be uploaded to the Central System.

4.      GNURadio:  I have installed it on an Oxdroid XU4 and it works quite well handling data at 100  ksps (IQ, single slice) from Red Pitaya.  The Odroid N2 seems to be about 15% faster according to benchmarks. I am going to further characterize what it can do. GNURadio has been used successfully in the SatNOGS project. Remember that it does not have to run in real time to pre-process data (for an FFT, for example) for upload.

5.      Compression: Phil Erickson is strongly urging us to consider storing (and maybe also transporting) data in HDF5 format.  I have worked with that in the past  and today ran some tests to see what kind of compression we can get. It compresses wav files down to about 40% of their initial size (I understand we need to verify that this is lossless).

More to come tomorrow.  I will also go more closely thru your comments and make sure to address everything in some way.  Are you going to be at the TAPR/AMSAT dinner at Dayton?   -73- Bill

From: TangerineSDR <tangerinesdr-bounces at lists.tapr.org><mailto:tangerinesdr-bounces at lists.tapr.org> On Behalf Of Scotty Cowling via TangerineSDR
Sent: Monday, May 13, 2019 3:30 PM
To: Tom McDermott <tom.n5eg at gmail.com><mailto:tom.n5eg at gmail.com>; Engelke, Bill <bill.engelke at ua.edu><mailto:bill.engelke at ua.edu>
Cc: Scotty Cowling <scotty at tonks.com><mailto:scotty at tonks.com>; Nathaniel A. Frissell <nathaniel.a.frissell at njit.edu><mailto:nathaniel.a.frissell at njit.edu>; tangerinesdr at lists.tapr.org<mailto:tangerinesdr at lists.tapr.org>
Subject: Re: [TangerineSDR] Local Host Functional Specification, Version 0.1

Hi Bill,

It is good to see specifications take shape!  TAPR has a Github account, will this be sufficient for the "parking lot"?

Comments on the spec.

First, my concept of the TangerineSDR *includes* the SBC. Here is the block diagram from the hardware spec:

[cid:image001.png at 01D50A4B.8B843520]
Without confusing things too much, can we make this distinction? So really, if we take a TangerineSDR and program the SBC with the PSWS Local Host software, we will transform the TangerineSDR into a PSWS. (Likely we would also have to re-program the FPGA with new firmware, and ensure that the TangerineSDR had the correct hardware sub-modules (CKM, RFM, DE, Sensor shield.)

The part that you refer to as the "Tangerine" is really the TangerineSDR DE and associated components (CKM, RFM, Sensor Shield).

You do mention connecting the TangerineSDR DE directly to a server. I want to make sure that our terms are consistently correct (blame my OCD). In the TangerineSDR world, it is a server, a connection between the RF world and our network. The combination of direct UDP data to/from the DE and/or UDP/TCP C&C plus data to/from the SBC makes up the TangerineSDR "Server" function. TangerineSDR talks to "clients", processors, consumers (RX) or producers (TX) of data.

So in the case where the SBC is the consumer and pre-processor of data, the SBC part of my definition of TangerineSDR becomes a client to the DE. It then becomes a "server" to what you refer to as the Server, which I presume is the Central Server of All PSWS Big Data. The Central Server then becomes a "server" to Scientific clients that which to obtain and study the data collected from all the thousands of PSWS.

So what is the best nomenclature to use? Perhaps we should define a PSWS Central Server to be "SWServer" or "SWSS". And then call out the specific pieces of the hardware by their names, since we are really defining them in terms of function in your spec. So please use "TangerineSDR DE" and either "TangerineSDR SBC" or maybe just "SBC", which is clear enough. To me, TangerineSDR by itself means the whole CKM, RFM, DE and SBC.

Again, my goal is for "TangerineSDR" to be configurable as "PSWS", "P4G", "STEM SDR", etc. All of these will include some form of SBC.

The confusion is probably my fault for putting the DE hardware spec cart before the TangerineSDR system spec horse.

On GNURadio, I doubt that you will find that any affordable SBC will provide GNURadio with adequate run-time resources. Have you loaded loaded and run it on a RPi? My opinion is that it is a great R&D tool, but it is nowhere stable enough to include in any release of automated PSWS software. I can't even imagine a thousand copies of GNURadio on PSWS all around the world. It would simply not work. I am willing to be proven wrong, but we will need a lot more testing to prove stability before then.

On system updates, I plan to have the ability for the LCC to supporting updates to the DE firmware from the SBC. In my case, I was thinking manual update by connecting the SBC to a web page and letting the user pick an update from a list. The Remote C&C could implement an external prcedure to virtualize a firmware update to the DE via the LCC interface. It could even be made automatic or a push update from the SWSS, with proper authentication.

73,
Scotty WA2DFI

On 2019-05-12 06:52, Tom McDermott wrote:
Hi Bill - thanks for generating the spec.  It's good to see various pieces of the project starting
to get documentation. It's a nice document.  Can I borrow the nice HAMSCI and TAPR logos?

Here are some comments on the spec - it may be too early to address most of them (perhaps
put them in the parking lot and get back to them later?).

1.User Interface.  Would it be useful to have an optional status / eye-candy display
of space weather, propagation, or measurements?   This might be a selling
point for people to acquire a PSWS. Would it need to download something
from the central server to do this?

2. Does the system need a way to do unattended recovery / restart?  Once the
system has been configured for unattended operation and measurement, should
the system have a GUI settable configuration to enable the ability to auto
discover radios, program them to the current observational needs (frequency, band,
etc.), and establish server reporting?  Essentially, get back to what it was doing
before power failed, or the node was rebooted, etc.

3. One of the key issues will be the time required to upload data to the central
server. For example:  a dual-receiver 8-band 192 ks/s 15-minute observation would
be about 20 GB of data. Assuming a 1 Mbit/s upload speed it would take about 2 days
to upload (assuming near 100% efficiency).  During that upload the 24-hour buffer
would be over-written.

4. Could the data to be uploaded to the server be compressed effectively?  Lossless
compression might not achieve much reduction in data size.  Lossy compression might
obscure science data. Can the SBC compress data while doing other tasks (concern
about CPU performance).

5. Should Gnuradio support be optional?  It is a lot of overhead, there may be
lower-resource approaches to data processing.  Does Gnuradio have reliability
issues for long-running / continuous tasks?

-- Tom, N5EG

On Sat, May 11, 2019 at 12:46 PM Engelke, Bill <bill.engelke at ua.edu<mailto:bill.engelke at ua.edu>> wrote:
See attached, first draft of Functional Spec for the Local Host (SBC).   Hope to discuss at Dayton.

-73- Bill AB4EJ

W. D. Engelke (Bill), Asst. Research Engr.
Center for Advanced Public Safety
Cyber Hall
The University of Alabama
Tuscaloosa, AL 35487
Desk: (205) 348-7244
Mobile: (205) 764-3099

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190514/d1d62d3a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 45516 bytes
Desc: image001.png
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190514/d1d62d3a/attachment-0001.png>