[TangerineSDR] Local Host Functional Specification, Version 0.1

Mon May 13 20:06:08 EDT 2019

Hi Bill,

My only concern with the term "server" is that we may have several 
functional pieces that fit that definition, like when the TangerineSDR 
(including the SBC) acts like a server to users that want to connect to 
get RX data streams in a non-SWPC use case.  So as long as we can be 
unambiguous in the use, I am OK with the term.

I will want to define the DE-to-SBC protocol in terms of Server and 
Client functions, and it will have nothing to do with the Central Server 
concept.

When referring to the DE, can you please define it as the "TangerineSDR 
Data Engine (DE)" the first time, as a definition?

73,
Scotty WA2DFI

On 2019-05-13 14:56, Engelke, Bill wrote:
>
> Hello Scotty:
>
> 1.On terminology, how about this…. In the SBC documentation, I need 
> some way refer to the Data Engine.  The first couple of versions of 
> the SBC Functional Spec call it the Tangerine; but now I understand 
> that you mean for the Tangerine to include the SBC.  I can update the 
> SBC spec to refer to the Data Engine (instead of calling it the 
> Tangerine).
>
> 2.I have already started a Parking Lot document on the Google Drive. 
> See 
> https://drive.google.com/drive/folders/1FYlmw3pINacZMEnYCFQg2sNzPeM-NElK 
> We can copy this stuff to github at some point if needed.
>
> 3.On the “server” – I mention the DE-to-server data path primarily 
> because Tom wants to account for organizations like MIT Haystack which 
> will have virtually unlimited bandwidth to an enormous server.  Most 
> users will not have that, and the Central Control system / Database 
> will not be sized to allow uploading at full speed (25 Megabytes/sec). 
> In > 90% of cases, the data will have to go from DE to SBC and then be 
> throttled and or preprocessed/compressed to be uploaded to the Central 
> System.
>
> 4.GNURadio: I have installed it on an Oxdroid XU4 and it works quite 
> well handling data at 100  ksps (IQ, single slice) from Red Pitaya.  
> The Odroid N2 seems to be about 15% faster according to benchmarks. I 
> am going to further characterize what it can do. GNURadio has been 
> used successfully in the SatNOGS project. Remember that it does not 
> have to run in real time to pre-process data (for an FFT, for example) 
> for upload.
>
> 5.Compression: Phil Erickson is strongly urging us to consider storing 
> (and maybe also transporting) data in HDF5 format.  I have worked with 
> that in the past  and today ran some tests to see what kind of 
> compression we can get. It compresses wav files down to about 40% of 
> their initial size (I understand we need to verify that this is lossless).
>
> More to come tomorrow.  I will also go more closely thru your comments 
> and make sure to address everything in some way. Are you going to be 
> at the TAPR/AMSAT dinner at Dayton? -73- Bill
>
> *From:*TangerineSDR <tangerinesdr-bounces at lists.tapr.org> *On Behalf 
> Of *Scotty Cowling via TangerineSDR
> *Sent:* Monday, May 13, 2019 3:30 PM
> *To:* Tom McDermott <tom.n5eg at gmail.com>; Engelke, Bill 
> <bill.engelke at ua.edu>
> *Cc:* Scotty Cowling <scotty at tonks.com>; Nathaniel A. Frissell 
> <nathaniel.a.frissell at njit.edu>; tangerinesdr at lists.tapr.org
> *Subject:* Re: [TangerineSDR] Local Host Functional Specification, 
> Version 0.1
>
> Hi Bill,
>
> It is good to see specifications take shape!  TAPR has a Github 
> account, will this be sufficient for the "parking lot"?
>
> Comments on the spec.
>
> First, my concept of the TangerineSDR *includes* the SBC. Here is the 
> block diagram from the hardware spec:
>
>
> Without confusing things too much, can we make this distinction? So 
> really, if we take a TangerineSDR and program the SBC with the PSWS 
> Local Host software, we will transform the TangerineSDR into a PSWS. 
> (Likely we would also have to re-program the FPGA with new firmware, 
> and ensure that the TangerineSDR had the correct hardware sub-modules 
> (CKM, RFM, DE, Sensor shield.)
>
> The part that you refer to as the "Tangerine" is really the 
> TangerineSDR DE and associated components (CKM, RFM, Sensor Shield).
>
> You do mention connecting the TangerineSDR DE directly to a server. I 
> want to make sure that our terms are consistently correct (blame my 
> OCD). In the TangerineSDR world, it is a server, a connection between 
> the RF world and our network. The combination of direct UDP data 
> to/from the DE and/or UDP/TCP C&C plus data to/from the SBC makes up 
> the TangerineSDR "Server" function. TangerineSDR talks to "clients", 
> processors, consumers (RX) or producers (TX) of data.
>
> So in the case where the SBC is the consumer and pre-processor of 
> data, the SBC part of my definition of TangerineSDR becomes a client 
> to the DE. It then becomes a "server" to what you refer to as the 
> Server, which I presume is the Central Server of All PSWS Big Data. 
> The Central Server then becomes a "server" to Scientific clients that 
> which to obtain and study the data collected from all the thousands of 
> PSWS.
>
> So what is the best nomenclature to use? Perhaps we should define a 
> PSWS Central Server to be "SWServer" or "SWSS". And then call out the 
> specific pieces of the hardware by their names, since we are really 
> defining them in terms of function in your spec. So please use 
> "TangerineSDR DE" and either "TangerineSDR SBC" or maybe just "SBC", 
> which is clear enough. To me, TangerineSDR by itself means the whole 
> CKM, RFM, DE and SBC.
>
> Again, my goal is for "TangerineSDR" to be configurable as "PSWS", 
> "P4G", "STEM SDR", etc. All of these will include some form of SBC.
>
> The confusion is probably my fault for putting the DE hardware spec 
> cart before the TangerineSDR system spec horse.
>
> On GNURadio, I doubt that you will find that any affordable SBC will 
> provide GNURadio with adequate run-time resources. Have you loaded 
> loaded and run it on a RPi? My opinion is that it is a great R&D tool, 
> but it is nowhere stable enough to include in any release of automated 
> PSWS software. I can't even imagine a thousand copies of GNURadio on 
> PSWS all around the world. It would simply not work. I am willing to 
> be proven wrong, but we will need a lot more testing to prove 
> stability before then.
>
> On system updates, I plan to have the ability for the LCC to 
> supporting updates to the DE firmware from the SBC. In my case, I was 
> thinking manual update by connecting the SBC to a web page and letting 
> the user pick an update from a list. The Remote C&C could implement an 
> external prcedure to virtualize a firmware update to the DE via the 
> LCC interface. It could even be made automatic or a push update from 
> the SWSS, with proper authentication.
>
> 73,
> Scotty WA2DFI
>
> On 2019-05-12 06:52, Tom McDermott wrote:
>
>     Hi Bill - thanks for generating the spec.  It's good to see
>     various pieces of the project starting
>
>     to get documentation. It's a nice document.  Can I borrow the nice
>     HAMSCI and TAPR logos?
>
>     Here are some comments on the spec - it may be too early to
>     address most of them (perhaps
>
>     put them in the parking lot and get back to them later?).
>
>     1.User Interface.  Would it be useful to have an optional status /
>     eye-candy display
>     of space weather, propagation, or measurements?   This might be a
>     selling
>     point for people to acquire a PSWS. Would it need to download
>     something
>     from the central server to do this?
>
>     2. Does the system need a way to do unattended recovery /
>     restart?  Once the
>     system has been configured for unattended operation and
>     measurement, should
>     the system have a GUI settable configuration to enable the ability
>     to auto
>     discover radios, program them to the current observational needs
>     (frequency, band,
>     etc.), and establish server reporting?  Essentially, get back to
>     what it was doing
>     before power failed, or the node was rebooted, etc.
>
>     3. One of the key issues will be the time required to upload data
>     to the central
>     server. For example:  a dual-receiver 8-band 192 ks/s 15-minute
>     observation would
>     be about 20 GB of data. Assuming a 1 Mbit/s upload speed it would
>     take about 2 days
>     to upload (assuming near 100% efficiency).  During that upload the
>     24-hour buffer
>     would be over-written.
>
>     4. Could the data to be uploaded to the server be compressed
>     effectively?  Lossless
>
>     compression might not achieve much reduction in data size.  Lossy
>     compression might
>
>     obscure science data. Can the SBC compress data while doing other
>     tasks (concern
>
>     about CPU performance).
>
>     5. Should Gnuradio support be optional?  It is a lot of overhead,
>     there may be
>
>     lower-resource approaches to data processing.  Does Gnuradio have
>     reliability
>     issues for long-running / continuous tasks?
>
>     -- Tom, N5EG
>
>     On Sat, May 11, 2019 at 12:46 PM Engelke, Bill
>     <bill.engelke at ua.edu <mailto:bill.engelke at ua.edu>> wrote:
>
>         See attached, first draft of Functional Spec for the Local
>         Host (SBC).   Hope to discuss at Dayton.
>
>         -73- Bill AB4EJ
>
>         W. D. Engelke (Bill), Asst. Research Engr.
>
>         Center for Advanced Public Safety
>
>         Cyber Hall
>
>         The University of Alabama
>
>         Tuscaloosa, AL 35487
>
>         Desk: (205) 348-7244
>
>         Mobile: (205) 764-3099
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190513/a54e74ed/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 45516 bytes
Desc: not available
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190513/a54e74ed/attachment-0001.png>