[TangerineSDR] Filename structure, Node info contents, other stuff to ponder

[John Gibbons]

Being we don't know the nature of Bill's data collection yet, let's get all
the data collection requirements straight before we jump into the
solution space to solve the problem...

That's the intent of this document - to flush out and to understand ALL of
the experiments going on and create a database that can ultimately handle
them.  No one understands them all yet.

I don't know anything that has been decided for the Tangerine Project which
is why they will define their portion of the project.

Bill's project is another example of this.  Multiple nodes at the
same location may fix one problem, but it creates others.  Not appropriate
at this stage to do that yet.

Let's get the data collection document to a level of completion that we're
all happy with first. Then we can understand and solve the overall data
collection problem.

John N8OBJ







John C. Gibbons
Director - Sears Undergraduate Design Laboratory
Dept. of Electrical Engineering and Computer Science
Case Western Reserve University
10900 Euclid Ave, Glennan 314
Cleveland, Ohio  44106-7071
Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
E-mail: jcg66 at case.edu



On Thu, May 7, 2020 at 12:14 AM Rob Wiesler via TangerineSDR <
tangerinesdr at lists.tapr.org> wrote:

> On Wed, May 06, 2020 at 23:25:41 -0400, John Gibbons wrote:
> > Your station is given a node number to define its location. So yes, you
> > have a single node number.
> >
> > If the receivers are coordinated to take their data all at the same time,
> > you would have a single file with (I'm guessing) 6 frequency measurements
> > and 6 amplitude measurements  for the 6 receivers per sample instance
> (with
> > a timestamp that goes with it).
> > This would be contained all within 1 file for the given (WWV) frequency
> > being measured and submitted daily.
>
> If 6 copies of fldigi (or any program) are writing to the same file,
> you're going to have data corruption.  There are four reasonable ways to
> proceed:
>
> - Add some field to the name of the file that fldigi's writing to, so
>   you're writing to 6 files per day, one from each process, then somehow
>   tell fldigi what value of that field to use.  The best way to do that
>   probably involves an environment variable that fldigi reads.  Then
>   relax or work around the one-upload-per-day-per-node restriction
>   somehow.
> - Have 6 different nodes, and somehow tell fldigi which node it is.  The
>   best way to do that probably involves an environment variable that
>   fldigi reads.  Since all nodes have exactly the same metadata
>   (right?), I don't think you'd even have to modify John's metadata
>   directory scheme.  This is probably saner than the previous option.
> - Instead of having fldigi write to a file directly, have it send its
>   measurements to a daemon via a UNIX socket (or a TCP socket bound to
>   127.0.0.1).  The daemon then takes care of reading the metadata,
>   creating the .csv files, and rotating them each UTC day.  This doesn't
>   involve any environment variables (so long as only one node is used
>   and the metadata is all the same), but would effectively undo all the
>   effort John went through to get his changes upstreamed.  Still, I
>   think this is the sanest option so far.  Actually, I proposed
>   something like this very early in the project, when David was setting
>   up the first prototype of the system, and I was shot down because it
>   seemed too complicated.  If you want to go this route, I can write the
>   code - it'll take a fraction of a Saturday or Sunday to go from zero
>   to having a finished pair of packages (one for the modifications to
>   fldigi, another for the daemon).
> - Run fldigi as 6 different users, all belonging to the same set of
>   groups, each with their own unique home directory.  Then they can
>   share or not share metadata and/or node IDs as desired.  This is the
>   least sane option, but (when using different nodes) requires no
>   changes to existing and planned code.
>
> I wrote the above list under the assumption that it is okay for the
> antenna that generated each record not to be associated with that
> record.  I did that intentionally for the sake of argument only.
> Actually, I don't think that's okay at all, and there needs to be a
> different node for each antenna (because the metadata must be
> different).  Probably Phil and/or Nathaniel should jump in and tell us
> what they want/need.
>
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200507/540568a1/attachment.html>