<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.gmail-m-3467446938405794070msolistparagraph, li.gmail-m-3467446938405794070msolistparagraph, div.gmail-m-3467446938405794070msolistparagraph
{mso-style-name:gmail-m_-3467446938405794070msolistparagraph;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1254434921;
mso-list-template-ids:228650760;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1
{mso-list-id:1723671539;
mso-list-template-ids:1258572106;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Gerry, I replied to you using the email address you have in qrz.com – 73- Bill<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>From:</b> TangerineSDR <tangerinesdr-bounces@lists.tapr.org>
<b>On Behalf Of </b>Gerry Creager - NOAA Affiliate via TangerineSDR<br>
<b>Sent:</b> Tuesday, June 9, 2020 1:14 PM<br>
<b>To:</b> TAPR TangerineSDR Modular Software Defined Radio <tangerinesdr@lists.tapr.org><br>
<b>Cc:</b> Gerry Creager - NOAA Affiliate <gerry.creager@noaa.gov><br>
<b>Subject:</b> Re: [TangerineSDR] Uploading of spectrum data from TangerineSDR to Central Control<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Bill, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">lftp, sftp, rsync, etc. are viable alternatives. As usual, in my world, there's something that might work for sending data and allowing its request by multiple hosts, as well. It's used to push weather data all over the place and is widely
used world-wide. Let's talk offline a little and let me get a slightly better sense of the data you're moving and if the LDM system might be useful or might just be, to borrow Phil's comment in another thread, a "hammer looking for a nail".<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">73<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Gerry N5JXS<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Jun 9, 2020 at 4:38 PM Engelke, Bill via TangerineSDR <<a href="mailto:tangerinesdr@lists.tapr.org">tangerinesdr@lists.tapr.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hello All:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I have a prototype of a process working for moving (raw) spectrum data from the TangerineSDR to the Central Host; I would like any interested colleagues to review and comment on
it, since it is such an important piece of the system. I have a mockup running in Ubuntu on a virtual server, but the final Central Host process will run under CentOS and Django. Here’s how the process works:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<ol start="1" type="1">
<li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
When in Ringbuffer mode, the TangerineSDR collects one to 16 bands of spectrum data, saving it in a ring buffer in Digital RF format. (Typical actual use would probably be the ~ 5 bands of WWV).<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
Every so often (say, 120 seconds), the TangerineSDR sends a heartbeat packet to the Central Host.
<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
If the Central Host has been given a data collection command (by a superuser), it will reply to the heartbeat with a Data Request that has a start and end time.<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
The TangerineSDR finds all the files and metadata collected during the requested time frame and uses tar to put into a single file (optionally compressed).
<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
The upload file gets named according to the (still being refined) CWRU naming convention (except that band info will not appear in the file name since a DRF file will usually contain multiple bands).<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
The TangerineSDR optionally sets the maximum upload rate (“throttle”) and uses the utility lftp to send the file to the sftp server running on the Central Host. Each user has their own “incoming” directory.
<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l1 level1 lfo1">
The Central Host checks the incoming directories at certain intervals; when it finds a file package, it moves it to a central collection and puts a cross-reference into the relational database.<o:p></o:p></li></ol>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Notes: I thought about a number of different ways of moving the data from TangerineSDR to the Central Host; this way seems promising because it offers the following benefits –<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<ul type="disc">
<li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l0 level1 lfo2">
<span style="font-family:"Courier New"">lftp</span> is restartable in case the upload is interrupted. Some uploads will run for a long time.<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l0 level1 lfo2">
Using sftp as the protocol protects the data in transit.<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l0 level1 lfo2">
The Central Host can handle a lot of uploads at once without burdening the website.<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l0 level1 lfo2">
Since each user has a unique UID and password, and data travels encrypted, we should be able to minimize hackers uploading garbage.<o:p></o:p></li><li class="gmail-m-3467446938405794070msolistparagraph" style="mso-list:l0 level1 lfo2">
The TangerineSDR has another mode (“snapshotter”) which pre-processes raw spectrum into frequency domain data – this will be handled differently (more to come on that).<o:p></o:p></li></ul>
<p class="gmail-m-3467446938405794070msolistparagraph"> <o:p></o:p></p>
<p class="gmail-m-3467446938405794070msolistparagraph" style="margin-left:.25in">
I’m sure I have not thought of everything though; if you have a suggestion on how to do this better, it is most welcome – thanks for your help! -73- Bill AB4EJ
<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">-- <br>
TangerineSDR mailing list<br>
<a href="mailto:TangerineSDR@lists.tapr.org" target="_blank">TangerineSDR@lists.tapr.org</a><br>
<a href="http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org" target="_blank">http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">Gerry Creager<o:p></o:p></p>
<div>
<p class="MsoNormal">NSSL/CIMMS<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(C) 979.229.5301 <--- NOTE THAT MY OFFICE NUMBER HAS CHANGED<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">++++++++++++++++++++++<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><i>The way to get started is to quit talking and begin doing.</i><o:p></o:p></p>
</div>
<p class="MsoNormal"><i> Walt Disney</i><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>