<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">David Witten</strong> <span dir="auto"><<a href="mailto:wittend@wwrinc.com">wittend@wwrinc.com</a>></span><br>Date: Mon, Jan 18, 2021 at 10:42 AM<br>Subject: Re: Fwd: magnetometer data file<br>To: John Gibbons <<a href="mailto:jcg66@case.edu">jcg66@case.edu</a>>, Dr. Nathaniel A. Frissell Ph.D. <<a href="mailto:nathaniel.frissell@scranton.edu">nathaniel.frissell@scranton.edu</a>><br>Cc: Julius Madey <<a href="mailto:hillfox@fairpoint.net">hillfox@fairpoint.net</a>>, Dave Larsen <<a href="mailto:kv0s.dave@gmail.com">kv0s.dave@gmail.com</a>><br></div><br><br><div dir="ltr">John, et al,<div><br></div><div>Part of the 'problem' here is that we do not have uniform terminology, and also that we are discussing two different programs that I'm writing. </div><div>None of this impacts the design of the current Grape project.</div><div><br></div><div>I have one program which creates an executable called 'runMag' that is the original testing code. </div><div>The other program is 'runRM3100' and makes an effort to follow the Grape project metadata and file storage scheme.</div><div>Both are command line only and are written in pretty portable C. </div><div>They are meant to be straightforward, and I deliberately have avoided any cleverness.</div><div><br></div><div>The runMag program makes no pretense of storing significant metadata and uses a different file name format for the saved data. </div><div>This is what Jules, Dave Larsen, Hyomin, and I are using to capture output 'logs'.</div><div>I am making very few changes to this at the present time, because I'm busy elsewhere.</div><div>But I did push a version 0.0.15 to GitHub last night to try to clean up the CSV output format somewhat for Jules and others.</div><div>The JSON code definitely needs to be re-tested.</div><div>And i might have even introduced an error in the way the data is adjusted for averaging in the ASIC device. </div><div>This needs to be checked, but I don't have time today. </div><div><br></div><div>The runRM3100 program is almost done, but has a number of 'fiddly' aspects that take time to implement and test. </div><div><br></div><div>LATER, both programs would benefit from some rework on the i2c code and on the way that measurement interval is implemented. </div><div>In particular, jitter in the measurement interval needs to be investigated under high CPU load.</div><div><br></div><div>Metadata in this context seems to refer to a number of different things:</div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"></blockquote><ul><li>The information that is concatenated into the filename used for storing the daily files that capture measurements. This is different between the two programs, but not wildly incompatible.</li><li>The layout of a group of folders or directories that hold all of the data related to the Grape project. </li><ul><li>These folders contain input to the program to direct its operation.</li><li>They also contain information about a specific site or node that is incorporated into the header of the output.</li><li>They also include work space where data is collected and stored.</li></ul><li>The layout of header info and data within the files that contain the collected values.</li></ul><blockquote style="margin:0 0 0 40px;border:none;padding:0px"></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"></blockquote><div>To respond again to Jules's original request, all of these issues in the runMag program either have been or will be addressed promptly, however you wish them to be done.<br></div><div><br></div><div>All of the work on the runRM3100 is done with the intent that it will maximize compatibility with the current and future Grape architectures.</div><div>All of this work is done with full knowledge that most of these matters will be mooted by the use of a Digital RF format for all output from the project. Eventually.<br></div><div><br></div><div>Dave Witten, KD0EAG<br><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div><div><br></div></blockquote><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jan 17, 2021 at 11:53 PM John Gibbons <<a href="mailto:jcg66@case.edu" target="_blank">jcg66@case.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">David & Jules,</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Formats are NOT set in stone - if this msg is in reference to the column headers I sent you as a suggestion then you misinterpreted what I was saying.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">You set the format of the data up to what the experiment produces and needs. YOU control that. Nathaniel wants the full ISO timestamp, but other than that you do what you want.<br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">The metadata that is attached to the head of each day's Grape data file is for Bill E. to be able to read in as a standard format. This also may change if Bill needs something else.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">It also follows along with the data file so it can be identified easier as a standard format as to what the data is from.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">For Bill's Tangerine files, he doesn't even use this - he is using a Digital RF format.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">If you elaborate on your problem maybe I can be of more assistance.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">John N8OBJ</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br clear="all"></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div><div><div><div><div><div>John C. Gibbons<br></div>Director - Sears Undergraduate Design Laboratory<br></div>Dept. of Electrical Engineering and Computer Science</div></div>Case Western Reserve University <br></div><div>10900 Euclid Ave, <span style="font-size:12.8px">Glennan 314</span><span style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px"><br></span></div></div><div><span style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px">Cleveland, Ohio 44106-7071</span><br style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px"><span style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px">Phone </span><a href="tel:216-368-2816" value="+12163684572" style="color:rgb(17,85,204);font-family:Helvetica;font-size:12.8px" target="_blank">(216) 368-2816</a><span style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px"> FAX </span><a href="tel:216-368-6888" value="+12163686888" style="color:rgb(17,85,204);font-family:Helvetica;font-size:12.8px" target="_blank">(216) 368-6888</a><br style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px"><span style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px">E-mail: </span><a href="mailto:jcg66@case.edu" style="color:rgb(17,85,204);font-family:Helvetica;font-size:12.8px" target="_blank">jcg66@case.edu</a><br style="color:rgb(0,0,0);font-family:Helvetica;font-size:12.8px"><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jan 17, 2021 at 9:18 PM David Witten <<a href="mailto:wittend@wwrinc.com" target="_blank">wittend@wwrinc.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Jules,<div><br></div><div>No, it is mostly set in stone. Trouble is that no one ever asked us (or anyone else, as far as I know).</div><div><br></div><div>Dave</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jan 17, 2021 at 7:48 PM Julius Madey <<a href="mailto:hillfox@fairpoint.net" target="_blank">hillfox@fairpoint.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<font face="Arial">Hmm, if the metadata describes the data format in
the file, then, in theory, different formats are allowed.?????<br>
<br>
</font><br>
<div>On 1/17/2021 7:19 PM, David Witten
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Jules,
<div><br>
</div>
<div>I entirely understand.
<div><br>
</div>
<div>I have a patch from Dave that apparently addresses most
of those things. I've tried to adjust the averages, but I'm
not sure. I made most of these changes to the newer
program, but I'm, still working through creating and
assembling all to the metadata there. I've kind of lost the
plot for the moment.
<div><br>
</div>
<div>But try version 0.0.15 that I've just committed to
github.</div>
<div><br>
</div>
<div>Dave Witten</div>
<div><br>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">---------- Forwarded
message ---------<br>
From: <strong class="gmail_sendername" dir="auto">Dave
Larsen</strong> <span dir="auto"><<a href="mailto:kv0s.dave@gmail.com" target="_blank">kv0s.dave@gmail.com</a>></span><br>
Date: Sun, Jan 17, 2021 at 5:22 PM<br>
Subject: Re: magnetometer data file<br>
To: David Witten <<a href="mailto:wittend@wwrinc.com" target="_blank">wittend@wwrinc.com</a>><br>
</div>
<br>
<br>
<div dir="auto">Dave
<div dir="auto"><br>
</div>
<div dir="auto">They address many of his issues an
make data analysis easier. </div>
<div dir="auto"><br>
</div>
<div dir="auto">I did not address averages other than
your quadratic mean. (Total)</div>
<div dir="auto"><br>
</div>
<div dir="auto">I changes are only in the CSV switch.</div>
<div dir="auto"><br>
</div>
<div dir="auto">I did not try to find all places I
needed to change.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Let know if you have
specific questions.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Dave</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun, Jan 17,
2021, 5:01 PM David Witten <<a href="mailto:wittend@wwrinc.com" target="_blank">wittend@wwrinc.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Dave,
<div><br>
Does your 'patch' accomplish these things?
<div>My mind has been entirely on other things
than runMag.</div>
<div>If it does, I'll make the shift and push it
to Github.</div>
<div><br>
</div>
<div>Dave</div>
<div><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">----------
Forwarded message ---------<br>
From: <strong class="gmail_sendername" dir="auto">Julius Madey</strong> <span dir="auto"><<a href="mailto:hillfox@fairpoint.net" rel="noreferrer" target="_blank">hillfox@fairpoint.net</a>></span><br>
Date: Sun, Jan 17, 2021 at 4:38 PM<br>
Subject: magnetometer data file<br>
To: John Gibbons <<a href="mailto:jcg66@case.edu" rel="noreferrer" target="_blank">jcg66@case.edu</a>>,
David Witten <<a href="mailto:wittend@wwrinc.com" rel="noreferrer" target="_blank">wittend@wwrinc.com</a>><br>
</div>
<br>
<br>
<div> <font face="Arial">Help me out here
guys.<br>
The version of RunMag I'm using includes
a text label for each component: <br>
time:<br>
rTemp:<br>
lTemp:<br>
x:<br>
y:<br>
z:<br>
rx:<br>
ry:<br>
rz:<br>
<br>
While that format is very handy for
getting use to the magnetometer
operation and general experiments, what
about reducing the file size by using
the same order but with only comma
separated values for the working data
repository.<br>
<br>
And, either use the calculated x,y,z
values from known cycle count and
average count<br>
or<br>
only the raw values with cycle count and
average count in the metadata for the
file<br>
<br>
Any thoughts?<br>
<br>
Thanks,<br>
73,<br>
Jules<br>
<br>
<br>
</font> </div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</div></div>