[TangerineSDR] IGRF Python Package

Phil Erickson phil.erickson at gmail.com
Wed Apr 1 14:19:11 EDT 2020


Hi Bill,

  There is something quick that comes to mind.  There is a large difference
between data provision through an interface (of whatever flavor) and data
archiving as the long-term record.  At MIT, Madrigal serves both functions
for the ground based community, but that's merely one case.  The project
will most definitely need separate and sustained funding to keep the long
term archive in place; it isn't going to happen by default, as you
indicate, and Phase 1 is not going to have the resources to do that.

  As an example, in Madrigal's case, we are now about to deploy the
following for the central long term archive:

- server nodes in two physical locations, one of them a continuously manned
industrial quality vault type location with zero downtime at the financial
institution level;
- each node itself is fully redundant with RAID 6 at a minimum and instant
fail-over
- each node exchanges updates every evening
- network access on major back-haul pipes

  This of course is exactly what Google does for their Google File System
or equivalent, except they tend to put their massive server farms very
close to hydropower for cheap energy.  As you said, there are many, many
options out there.  But the best one is one you control and maintain, with
the associated cost of that maintenance - it's the only way to guarantee
long term preservation.  I simply do not trust any companies any longer in
their promises.  Nothing is really for free.  (Look at the AWS price per
byte now, and you pay for input AND output and storage.  Gets
non-economical in a hurry.)

73
Phil


On Wed, Apr 1, 2020 at 1:05 PM Engelke, Bill via TangerineSDR <
tangerinesdr at lists.tapr.org> wrote:

> This does raise several issues we will need to deal with at some point.
> Some notes…
>
>
>
>    - I have been pushing the software I am developing for the small board
>    computer (to go into the TangerineSDR) to my repository on Github simply
>    because it is the most straightforward way at this moment. It gives me the
>    ability to recover without major loss of work if the Odroid I am using were
>    to catastrophically fail; plus, if I were to drop dead next week due to
>    coronavirus or something (a realistic possibility, unfortunately), somebody
>    in the team may be able to make use of what I have done so far. The
>    software is heavily commented.
>    - It would indeed be a good idea for TAPR to have its own github for
>    this and related software to ensure its continued  access to it.
>    - On a related note, we (here at UA) have extensively discussed how to
>    store data that will be collected from Personal Space WX Stations once we
>    have the network running.  For Phase 1, we will simply put it on a UA
>    server and make it publicly available (and cross-referenced by the database
>    we are developing with NSF funding). It’s the longer term plan (Phase 2 and
>    beyond) that is tricky. Many options have been floated: Zenodo, Madrigal,
>    Grafana, Dropbox, OpenScienceDataCloud, Amazon Web Services, Microsoft
>    Azure, Box, etc. Each one has its pro and con, as well as a chorus of
>    supporters and detractors. A complicating factor on that is that we’d hope
>    to be able to enable some Big Data analysis approach; not to mention that
>    many of these solutions will get quite expensive at scale; which is OK *as
>    long as we have funding*… but this is part of the equation. As part of
>    the current project (albeit in year 2022), we plan to evaluate the issues
>    and make some recommendations (in what might become a Phase 2 NSF proposal
>    if all goes well). *We are most interested to hear everyone’s thoughts
>    on this *as we go forward; still, one thing we don’t want to do is end
>    up with a “horse-designed-by-a-committee” (i.e., a camel) result. (Also,
>    based on past experience, some neat new solution could become available in
>    the next 2 to 3 years that would be even better than current options.)
>
>
>
> -73- Bill AB4EJ
>
>
>
>
>
> *From:* TangerineSDR <tangerinesdr-bounces at lists.tapr.org> *On Behalf Of *Phil
> Erickson via TangerineSDR
> *Sent:* Wednesday, April 1, 2020 10:50 AM
> *To:* TAPR TangerineSDR Modular Software Defined Radio <
> tangerinesdr at lists.tapr.org>
> *Cc:* Phil Erickson <phil.erickson at gmail.com>; Stephen Roland Kaeppler <
> skaeppl at clemson.edu>
> *Subject:* Re: [TangerineSDR] IGRF Python Package
>
>
>
> Hi all,
>
>
>
>   I strongly recommend that this project
>
>
>
> a) use revision control
>
> b) host its own SVN / git / etc. server
>
>
>
>   Relying on things like GitHub is not safe in the long term, because
> corporate decisions can change abruptly on what is offered free to the
> community and what is not.  Also, many of the "free" services make
> absolutely no guarantees about where any of the data is backed up, what the
> backup schedule is, etc.
>
>
>
>   I relate these pieces of advice from painful experience at MIT
> Haystack.  Our group believed Dropbox when they rolled out a storage plan
> with very large upper bound limits for MIT.  A couple years ago, they
> abruptly cancelled that policy and assigned a very large per year fee for
> the storage we had been using.  I'm still trying to figure out how to
> extract 100s of TB of data from them, and some of it may be lost for good.
>
>
>
> 73
>
> Phil W1PJE
>
>
>
> On Wed, Apr 1, 2020 at 11:45 AM Tom McDermott via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
> Hi Steve - thanks for the offer !
>
>
>
> This brings up a general question:  should TangerineSDR / PSWS host this
> kind of software someplace?
>
> One could imagine that over time there may be an increasing number of
> software utilities and packages of
>
> interest to the group, it would be nice to have them accessible through
> one link or portal (ideally with revision control ! ).
>
>
>
> Is this something the group should consider?  If so, is github the right
> solution?, a page of links?, something else?
>
>
>
> -- Tom, N5EG
>
>
>
>
>
>
>
>
>
> On Wed, Apr 1, 2020 at 5:57 AM Stephen Roland Kaeppler via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
> Hi All-
>
> I will add a comment if it is of interest, that I wrote some code a while
> back in python that directly calculates the magnetic field using the IGRF
> coefficients and derivatives (useful in a ray tracer).  If you guys want
> that, I could package it up and send it over to anyone interested.
>
> Thanks,
> Steve
>
> ------------------------------------------------------------
> Stephen R. Kaeppler, Ph.D.
> Assistant Professor
> Department of Physics and Astronomy
> Clemson University
> Clemson, SC 29634
> Email: skaeppl at clemson.edu
> Phone: 864-656-4275
> Web: http://science.clemson.edu/kaeppler/
> Amateur Radio Callsign: AD0AE
> ------------------------------------------------------------
>
>
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>
>
>
>
> --
>
> ----
> Phil Erickson
> phil.erickson at gmail.com
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>


-- 
----
Phil Erickson
phil.erickson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200401/665c7bb2/attachment.html>


More information about the TangerineSDR mailing list