[TangerineSDR] Filename structure, Node info contents, other stuff to ponder

Thu May 7 13:47:07 EDT 2020

Hi all,

  What is the relevance for attaching the Eric Nichols article from 2010?

Cheers
Phil

On Thu, May 7, 2020 at 1:45 PM Kristina Collins <kvc2 at case.edu> wrote:

> Some notes from this morning's telecon and following discussion... There's
> more not covered here which John will be incorporating into the next
> version of the spec. Email me if you'd like to join the weekly telecon for
> the low-cost PSWS system, which is held Thursdays at 10am EDT.
>
> 1) We should clarify what a node is, and is not. In current conception, a
> node is an experiment being conducted at a location. A user may have
> multiple different experiments running - for example, one TangerineSDR and
> one GrapeSSR (custom-built low-cost PSWS) at one location. In this case,
> the TangerineSDR is a node and the GrapeSSR is a node. All the datasets
> being collected by the TangerineSDR - magnetometer, temp, lightning, etc. -
> are associated with the node number for that station. Similarly, all of the
> frequencies being measured by the GrapeSSR - all WWV and CHU frequencies -
> will be associated with the node number for that GrapeSSR.
>
> 2) It should be noted for those not hanging out on the low-cost/Doppler
> side of the PSWS project that there are two types of system (so far)
> collecting Doppler shift data: (A) John's custom board (the aforementioned
> GrapeSSR, so named because it comes in bunches and because the data
> ferments into something very nice) and (B) ham rigs collecting data by a
> method similar to we did for the Festival of Frequency Measurement. This
> latter set of stations is essentially its own experiment type; we have a
> few pilot stations running now. As a group, I've been referring to these
> ham rigs as the Frequency Analysis Network (FAN). A FAN page will be added
> to the HamSCI website soon.
>
> 3) There will be more characters added to the node naming description in
> order to address the different types of nodes. Again, so far there are
> three: the Tangerine, the Grape, and regular ol' ham rigs. Rather than
> these having just node numbers, we can give them alphanumeric strings: for
> example, NT00087, NG00094, NH00189 might indicate a Tangerine, a Grape and
> a ham rig respectively. Additionally, we could add more resolution in these
> letters: I could see using NHG00189 to indicate a ham rig collecting
> Doppler data which is connected to a GPSDO, and NHN00189 to indicate one
> which does not have a GPSDO, for example. Adding a D for "development"
> would also be a way to avoid some of the issues discussed above.
> Eventually, we will likely want to incorporate more types of experiments.
> For instance, a station set up to measure X and O propagation (like the one
> described in KL7AJ's Dec 2010 QST article, attached) could be added in with
> its own alphanumeric designation in order to set apart its node ID. Bottom
> line: we have to stay flexible, as we'll be adding experiments and metadata
> later on that we won't think of now.
>
> 4) Important: We will not be using fldigi in the final version of the
> GrapeSSR. There will be a dedicated program for doing the data collection.
> FAN stations may still use fldigi for data collection (might not be a bad
> thing to standardize around, Spectrumlab being another option for taking
> frequency estimation data) but I don't anticipate any nodes using fldigi to
> collect data from multiple frequencies at once.
>
> 73,
> -KC
>
> On Thu, May 7, 2020 at 7:49 AM Phil Erickson via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
>> Hi Bill,
>>
>>   I'll reply off-list with Digital RF information, so as not to use
>> bandwidth here.
>>
>> 73
>> Phil W1PJE
>>
>> On Thu, May 7, 2020 at 7:47 AM Bill Liles via TangerineSDR <
>> tangerinesdr at lists.tapr.org> wrote:
>>
>>> Thanks for all the replies.
>>>
>>> My interest is doing oblique ionosondes from transmitters of
>>> opportunity, including ionosondes. Thus, I want to separate the X and O
>>> wave signals. Also, I want to see how the path direction (AZ and EL)
>>> changes with time and from the great circle path. This helps with
>>> understanding TIDs, especially if I can have multiple sites join in
>>> collects.
>>>
>>> Thus the requirement for a vector sensor antenna, so actually have 6
>>> antennas.
>>>
>>> I guess I will be using the DRF format since I would like to collect I/Q
>>> and time from each antenna at each site. Do DRF files have a separate file
>>> naming convention?
>>>
>>> It is important to associate which data is associated with which antenna
>>> so can combine the data properly to get polarization, AZ and EL.
>>>
>>> Thank you.
>>>
>>> Bill NQ6Z
>>>
>>>
>>> On Thu, May 7, 2020 at 12:40 AM John Gibbons via TangerineSDR <
>>> tangerinesdr at lists.tapr.org> wrote:
>>>
>>>>
>>>> Being we don't know the nature of Bill's data collection yet, let's get
>>>> all the data collection requirements straight before we jump into the
>>>> solution space to solve the problem...
>>>>
>>>> That's the intent of this document - to flush out and to understand ALL
>>>> of the experiments going on and create a database that can ultimately
>>>> handle them.  No one understands them all yet.
>>>>
>>>> I don't know anything that has been decided for the Tangerine Project
>>>> which is why they will define their portion of the project.
>>>>
>>>> Bill's project is another example of this.  Multiple nodes at the
>>>> same location may fix one problem, but it creates others.  Not appropriate
>>>> at this stage to do that yet.
>>>>
>>>> Let's get the data collection document to a level of completion that
>>>> we're all happy with first. Then we can understand and solve the overall
>>>> data collection problem.
>>>>
>>>> John N8OBJ
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> John C. Gibbons
>>>> Director - Sears Undergraduate Design Laboratory
>>>> Dept. of Electrical Engineering and Computer Science
>>>> Case Western Reserve University
>>>> 10900 Euclid Ave, Glennan 314
>>>> Cleveland, Ohio  44106-7071
>>>> Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
>>>> E-mail: jcg66 at case.edu
>>>>
>>>>
>>>>
>>>> On Thu, May 7, 2020 at 12:14 AM Rob Wiesler via TangerineSDR <
>>>> tangerinesdr at lists.tapr.org> wrote:
>>>>
>>>>> On Wed, May 06, 2020 at 23:25:41 -0400, John Gibbons wrote:
>>>>> > Your station is given a node number to define its location. So yes,
>>>>> you
>>>>> > have a single node number.
>>>>> >
>>>>> > If the receivers are coordinated to take their data all at the same
>>>>> time,
>>>>> > you would have a single file with (I'm guessing) 6 frequency
>>>>> measurements
>>>>> > and 6 amplitude measurements  for the 6 receivers per sample
>>>>> instance (with
>>>>> > a timestamp that goes with it).
>>>>> > This would be contained all within 1 file for the given (WWV)
>>>>> frequency
>>>>> > being measured and submitted daily.
>>>>>
>>>>> If 6 copies of fldigi (or any program) are writing to the same file,
>>>>> you're going to have data corruption.  There are four reasonable ways
>>>>> to
>>>>> proceed:
>>>>>
>>>>> - Add some field to the name of the file that fldigi's writing to, so
>>>>>   you're writing to 6 files per day, one from each process, then
>>>>> somehow
>>>>>   tell fldigi what value of that field to use.  The best way to do that
>>>>>   probably involves an environment variable that fldigi reads.  Then
>>>>>   relax or work around the one-upload-per-day-per-node restriction
>>>>>   somehow.
>>>>> - Have 6 different nodes, and somehow tell fldigi which node it is.
>>>>> The
>>>>>   best way to do that probably involves an environment variable that
>>>>>   fldigi reads.  Since all nodes have exactly the same metadata
>>>>>   (right?), I don't think you'd even have to modify John's metadata
>>>>>   directory scheme.  This is probably saner than the previous option.
>>>>> - Instead of having fldigi write to a file directly, have it send its
>>>>>   measurements to a daemon via a UNIX socket (or a TCP socket bound to
>>>>>   127.0.0.1).  The daemon then takes care of reading the metadata,
>>>>>   creating the .csv files, and rotating them each UTC day.  This
>>>>> doesn't
>>>>>   involve any environment variables (so long as only one node is used
>>>>>   and the metadata is all the same), but would effectively undo all the
>>>>>   effort John went through to get his changes upstreamed.  Still, I
>>>>>   think this is the sanest option so far.  Actually, I proposed
>>>>>   something like this very early in the project, when David was setting
>>>>>   up the first prototype of the system, and I was shot down because it
>>>>>   seemed too complicated.  If you want to go this route, I can write
>>>>> the
>>>>>   code - it'll take a fraction of a Saturday or Sunday to go from zero
>>>>>   to having a finished pair of packages (one for the modifications to
>>>>>   fldigi, another for the daemon).
>>>>> - Run fldigi as 6 different users, all belonging to the same set of
>>>>>   groups, each with their own unique home directory.  Then they can
>>>>>   share or not share metadata and/or node IDs as desired.  This is the
>>>>>   least sane option, but (when using different nodes) requires no
>>>>>   changes to existing and planned code.
>>>>>
>>>>> I wrote the above list under the assumption that it is okay for the
>>>>> antenna that generated each record not to be associated with that
>>>>> record.  I did that intentionally for the sake of argument only.
>>>>> Actually, I don't think that's okay at all, and there needs to be a
>>>>> different node for each antenna (because the metadata must be
>>>>> different).  Probably Phil and/or Nathaniel should jump in and tell us
>>>>> what they want/need.
>>>>>
>>>>> --
>>>>> TangerineSDR mailing list
>>>>> TangerineSDR at lists.tapr.org
>>>>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>>>>
>>>> --
>>>> TangerineSDR mailing list
>>>> TangerineSDR at lists.tapr.org
>>>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>>>
>>> --
>>> TangerineSDR mailing list
>>> TangerineSDR at lists.tapr.org
>>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>>
>>
>>
>> --
>> ----
>> Phil Erickson
>> phil.erickson at gmail.com
>> --
>> TangerineSDR mailing list
>> TangerineSDR at lists.tapr.org
>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>
>

-- 
----
Phil Erickson
phil.erickson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200507/00aa33ca/attachment-0001.html>