[TangerineSDR] Filename structure, Node info contents, other stuff to ponder

John Gibbons jcg66 at case.edu
Wed May 6 16:42:44 EDT 2020


Update to spec - tried to be concise but specific for definitions so there
is no confusion.

John N8OBJ

John C. Gibbons
Director - Sears Undergraduate Design Laboratory
Dept. of Electrical Engineering and Computer Science
Case Western Reserve University
10900 Euclid Ave, Glennan 314
Cleveland, Ohio  44106-7071
Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
E-mail: jcg66 at case.edu



On Wed, May 6, 2020 at 2:46 PM Dr. Nathaniel A. Frissell Ph.D. <
nathaniel.frissell at scranton.edu> wrote:

> Hi John,
>
>
>
> Two thoughts:
>
>    1. While I think the filename is important, I think it should be more
>    for convenience than for actual storage of real information. That is, the
>    file itself should also contain all of the data/metadata to recreate the
>    filename. It is the data inside of the file that should be considered the
>    definitive record.
>    2. Regarding node numbers, perhaps an alpha-numeric approach is
>    better, than just numeric? It gives a lot more flexibility in the naming
>    space.
>
>
>
> 73 de Nathaniel W2NAF
>
>
>
> *From:* TangerineSDR <tangerinesdr-bounces at lists.tapr.org> *On Behalf Of *John
> Gibbons via TangerineSDR
> *Sent:* Wednesday, May 6, 2020 11:54 AM
> *To:* TAPR TangerineSDR Modular Software Defined Radio <
> tangerinesdr at lists.tapr.org>
> *Cc:* John Gibbons <jcg66 at case.edu>; David Kazdan <dxk10 at cwru.edu>
> *Subject:* Re: [TangerineSDR] Filename structure, Node info contents,
> other stuff to ponder
>
>
>
> Rob,
>
>
>
> Any node can be collecting multiple files - that's what the TYP and MOD
> fields are for.  When finished, the LC PSWS will be collecting 6 files (4
> FREQ [with freq modifier added] data sets, 1 MAG and 1 TMP) per day per
> node for any given node per given UTC day.
>
>
>
> If data collection is interrupted, the system just continues adding data
> for that UTC day in the already existing file for that data set [i.e.
> appends and does not start over].
>
> That was one of the major mods I did to FLDIGI to accommodate this
> situation.
>
> Collecting the exact same data set on two different machines at the same
> node on the same UTC day makes no sense and is not allowed.  A node will
> only be allowed to submit one file per specific data set per UTC day.
>
> There will be no merging of files upstream.
>
>
>
> As far as the indication of UTC day (using the ISO format) in the
> filename, I'm already doing that and there is no simpler way to indicate
> this and anyone using the system already understands this.  Adding anything
> here just makes the filename longer.... and it's not necessary for the
> developers.
>
> As I've stated in the spec, data sets are based on UTC day and labeled (in
> ISO format) as such.  It doesn't get much simpler than this.
>
>
>
> The file naming scheme (using the ISO date) is intended to help someone
> who knows what they are doing to visually determine the UTC day's contents
> of the file (per Nathaniel's request).  I believe I've achieved this as
> simply as possible.
>
>
>
> Semantics about user root and user pi are why I did the system call -
> someone may choose a different username and I've already accommodated this
> situation.  And it does work (David Kazdan tested this).
>
> And any user should NOT be running as root - breaks all the protections
> and is bad practice in general.  Nuf' said.
>
>
>
> Using a system call in PYTHON to get the ENV variable for the home
> directory is an acceptable practice to determine this.  We're going to be
> setting up the systems, so this is not going to be a problem long term.
>
>
>
> We have to be able to have our files usable on WIN7/10 systems, so the use
> of special characters in the filename that linux can handle is
> simply not allowed.  We have to be compatible.
>
>
>
> And now that I think about it, the - [minus] sign IS usable in WIN7/10, so
> I'll fix that in the spec.  (And I'm using it in the filename - OOPS!)
>
>
>
> John N8OBJ
>
>
>
>
>
>
>
> John C. Gibbons
>
> Director - Sears Undergraduate Design Laboratory
>
> Dept. of Electrical Engineering and Computer Science
>
> Case Western Reserve University
>
> 10900 Euclid Ave, Glennan 314
>
> Cleveland, Ohio  44106-7071
> Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
> E-mail: jcg66 at case.edu
>
>
>
>
>
> On Tue, May 5, 2020 at 11:50 PM Rob Wiesler via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
> On Tue, May 05, 2020 at 23:01:15 -0400, Phil Erickson via TangerineSDR
> wrote:
> > On Tue, May 5, 2020 at 10:05 PM John Gibbons via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
> >> On Tue, May 5, 2020 at 8:38 PM Rob Wiesler via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
> >>> (David is probably on the TangerineSDR list, but I don't know that for
> >>> sure, so he may get an extra copy of this message (sorry).)
>
> David is *not* on the list.  That's good to know.  Maybe I should stop
> my habit of trimming the Cc header so aggressively for this list.
>
> >>> Does the date in the filename refer to:
> >>> - the beginning of the record, or
> >>> - the end of the record, or
> >>> - a single day in its entirety, or
> >>> - at most a single day, but no (significant) part of any other day?
> >
> >   Your files only have the YYYY-MM-DD form of the ISO date in them.  This
> > seems to imply that there can be only one file per day.  What if the
> > instrument dies for a while and then restarts on the same day?  I guess
> I'm
> > wondering why you wouldn't use the fully qualified ISO (e.g.
> > 2020-05-05T03:00:00).  Maybe I'm missing something obvious.
>
> This hasn't been answered yet exactly, but the current low-cost PSWS
> fldigi scheme rotates the file at the end of the UTC day, so there's a
> one-to-one correspondence between UTC day and filename.  This is
> entirely unclear from the filename specification, though.
>
> In any case, I find it entirely too likely that we'll see two different
> files from nonoverlapping times with the same name from the same node.
> Imagine that someone sets up a system, starts it logging, forgets the
> root password an hour later, and sets it back up again.  Are we going to
> merge the two files upstream?  I would rather not have to.
> Misconfigured systems (for instance, cloned systems containing duplicate
> node IDs) are something we have to look out for, but this is a case of a
> correctly-configured system that just happens to have lost its memory.
>
> Using the full ISO date (minus the (to us) useless time zone offset
> portion) corresponding to the time of the first record in the file would
> fix this problem.
>
> >>> Let's at least specify WWVdata/ as existing relative to "the user"'s
> >>> home directory, instead of /home/pi (it can't hurt to be explicit).
> >>> Also, you have a typo, where you say that "~" is the "root filesystem
> >>> for user", which is not a thing (you mean "home directory").
> >>
> >> For the RasPi OS (Linux), I use a system call to define ~ which is the
> >> BASE of the user's directory structure (root was a BAD choice here - not
> >> intended to confuse it with root user)
>
> It would be even worse to get it confused with the real root directory,
> which is "/".  Or with "user root"'s directory, which is typically
> "/root" (pronounced "slash-root" to disambiguate).
>
> Don't call it a "base", please.  This one one of those cases where
> there's one right term with no competitors.  Just call it the user's
> "home directory".
>
> For today's dose of useless pedantry from Rob, ~ is expanded to the real
> path of the user's home directory by a globbing function, based on the
> value of the HOME environment variable.  Since environment variables are
> loaded into the process's memory space literally before they get a
> chance to start, this doesn't involve a system call at all (system calls
> are a way for userspace processes to invoke functions provided by the
> kernel, not a generic term for functions provided by the system at
> large).
>
> >>> Is there a reason you're avoiding a second '.' in the filename?  It's a
> >>> little awkward to use 2p5 for 2.5, and that second period isn't going
> to
> >>> confuse any software or upend the sorting.
> >>
> >> I stayed away from . and - in the filename as Windows users would get
> into
> >> trouble here (I stuck to just the _ char) and I think we have to cater
> to
> >> that limitation of Windows 7/10/whatever.
>
> Only when renaming files from the GUI does this become difficult on
> Windows, and in any case, we don't want users doing that.  Not exactly a
> feature, but not a bug, either.
>
> >>>> I have the original .doc that created this - let me how we should
> handle
> >>>> version control from here.  (Nathaniel?)
> >>>
> >>> Please turn this into a plain text file.  I can read PDFs, but it's not
> >>> ideal, and I'm getting sick and tired of specification documents in
> >>> other non-textual formats.  A plain text specification file has these
> >>> properties:
> >>>
> >>> - Small file size (because this is 2020 and it totally matters)
> >>> - Less wasted visual space when the document isn't all that long
> (again,
> >>>   wishlist-grade)
> >>> - Universally readable
> >>> - Universally modifiable by the recipient (so recipients can offer
> >>>   suggestions formatted as a pull request)
> >>> - Diffs between revisions can be generated trivially, so that
> recipients
> >>>   can:
> >>>   - offer suggestions formatted as a patch
> >>>   - figure out what changed without scanning the entire document for
> >>>     thin red/yellow lines on a white background (very important to me)
> >>> - Mergeable when put in version control (in addition to all the other
> >>>   properties above you would want for version-controlled documents)
> >>
> >> This will be available online when we get it into version control.
>
> So long as it's not a PDF in version control, please.  The only thing
> sadder than a sourceless binary in version control is one that also has
> the wrong permission bits set.
>
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200506/4f1b2e54/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PSWS_FileNameRequirementsV0_04.pdf
Type: application/pdf
Size: 61381 bytes
Desc: not available
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200506/4f1b2e54/attachment-0001.pdf>


More information about the TangerineSDR mailing list