[TangerineSDR] Notes from PSWS / TangerineSDR call of 04-27-2020

Tue Apr 28 14:39:15 EDT 2020

Hi John,

  I used the very latest "fldigi", downloaded 24 hours before the FMT.  The
key difference perhaps is that I did the analysis in retrospective mode on
already recorded data - not live from the antenna. That is a different use
case from yours, and different use cases are not of course going to use the
same methods.  Not suggesting either that a Python module be inserted
real-time into anyone's flow on a resource limited platform (how did we get
there?), but that Python is the language of choice for back-end,
non-realtime, iterative, retrospective analysis.  That frankly is where I
spend 100% of my time as a scientist.

  So a scenario where a C language, fast algorithm is used for all the
realtime needs, and that this very same algorithm is then bundled into an
auto-generated Python module for backend non-realtime retrospective work,
serves all customers from what I can tell.

Cheers
Phil W1PJE

On Tue, Apr 28, 2020 at 11:57 AM John Gibbons via TangerineSDR <
tangerinesdr at lists.tapr.org> wrote:

> All,
>
> With my mods, FLDigi has been working flawlessly for use by us.  There are
> some things in the user interface you need to be aware of, and
> unfortunately Phil got bit by a few of them.
> I could get into them, but it isn't going to apply to us long term.
>
> What version of FLDigi were you using?
> FLdigi-4.1.12 (just released last Monday) was the first version that has
> my improvements in it (that came from this WWV project), and I did use it
> for the
> FMT (and - shameful brag alert.....  had an error of 0.01 hHZ on 40M and
> 0.05Hz on 80M)  😵
> It does work.  But as usual, the devil is in the details....
>
> I'll be pushing out a report on a test I just completed on the Gen1 Radio
> I designed for the low cost PSWS and using FLDigi shows how well it
> performs for frequency estimation.
> We've been using it since July 2019, and a lot of our data collection has
> been done through this radio receiver (ALL of the data collection has been
> using my modified version of FLDIGI-4-1.09).
>
> My test was a COMPLETE 'system' test - from antenna input to file numbers
> output.  I wanted to see how well the whole system was doing including the
> freq analysis algorithm.
>
> The 'final' version will be the algorithm extracted and customized for our
> use.  *FLDIGI is NOT our long term solution* (it never was) - it's a
> stepping stone.
>
> With the processor / memory limitations of the low cost PWSW we will try
> to do python code, but coming from 35+ years of designing embedded
> real-time systems it will more likely be C / C++ code.
> The reason is that an interpreted language (i.e. python) runs so much
> slower (10x - 100X) and uses up so much more system resources that it bogs
> down (and usually fails).
> Also, depending on the interpreter to (not) getting an OS upgrade that
> breaks our code is just a bad idea.         Been there.... been burnt by
> it... don't want to do it again.
> Having compiled code means WE control the updates and not the latest
> release of an interpreter.
>
> And Gen 2 of my radio will attempt to do 4 channels of WWV ( or some on
> CHU) at the same time.. Running 4 instances of FLdigi simply is not an
> option - it just will not work.
>
> BUT,  the final software is being started with the tests being run now in
> python, and concurrently I'm creating tests in C++.  Again, the OS overhead
> of running an interpreted language for a real time
> data acquisition is not done in industry for lots of good reasons with
> lots of test cases to back it up.  If you want a reliable system, using an
> interpretive language is simply not the way to go.
>
> We have made some advancement in the frequency determination algorithm(s),
> and even had one of our helpers try/work on implementing one in a MAX10
> FPGA implementation.
> He got interrupted this semester by having to teach a new embedded systems
> IOT class, but hope to get that development back on track soon (stop
> drooling Scotty😁🤔😃).
>
> More to come later...
>
> John N8OBJ
>
>
>
>
> John C. Gibbons
> Director - Sears Undergraduate Design Laboratory
> Dept. of Electrical Engineering and Computer Science
> Case Western Reserve University
> 10900 Euclid Ave, Glennan 314
> Cleveland, Ohio  44106-7071
> Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
> E-mail: jcg66 at case.edu
>
>
>
> On Tue, Apr 28, 2020 at 8:07 AM Dr. Nathaniel A. Frissell Ph.D. via
> TangerineSDR <tangerinesdr at lists.tapr.org> wrote:
>
>> Hi Phil,
>>
>>
>>
>> Thanks for the comments. I think that is very good to consider. In light
>> of review, I think it is a good idea to postpone FLDigi integration at this
>> time. As Phil said, we should pull out the frequency estimation routine for
>> separate testing and possible direct use. I believe Skylar Dannhoff is
>> working on this. For the WWV monitoring program, we also do need to have
>> the ability to save both the raw data and the derived frequency estimation
>> data product.
>>
>>
>>
>> 73 de Nathaniel W2NAF
>>
>>
>>
>> *From:* TangerineSDR <tangerinesdr-bounces at lists.tapr.org> *On Behalf Of
>> *Phil Erickson via TangerineSDR
>> *Sent:* Tuesday, April 28, 2020 7:12 AM
>> *To:* TAPR TangerineSDR Modular Software Defined Radio <
>> tangerinesdr at lists.tapr.org>
>> *Cc:* Phil Erickson <phil.erickson at gmail.com>
>> *Subject:* Re: [TangerineSDR] Notes from PSWS / TangerineSDR call of
>> 04-27-2020
>>
>>
>>
>> Hi all,
>>
>>
>>
>>   Sorry that I haven't been able to make these discussions much lately.
>> COVID-19 has added a huge amount of work here.
>>
>>
>>
>>   I'm fairly concerned about using "fldigi".  In the recent Frequency
>> Measurement Test a few days ago, I decided to use its frequency algorithm
>> in the "ANALYZE" portion in order to track the unknown carrier that K5CM
>> was putting out.  I had a bunch of data already recorded through things
>> like Gnuradio stacks; for this discussion, let's assume I had I/Q data in
>> some kind of format similar to what Tangerine would put out.  (It was
>> DigitalRF to be specific.)
>>
>>
>>
>>   Frankly, it was a disaster.  The only way you can input data that I
>> could find is through WAV files, so I had to convert all my data to that
>> format.  When you start up the program, it immediately starts generating
>> bogus data on nothing while you fumble around getting to the menu that
>> plays back a WAV file.  When the WAV file ends (I chose non-loop for
>> playback), you get another bunch of junk data while you are fumbling around
>> with literally quitting the entire program as the only way I could figure
>> out to stop the analysis.  While the WAV file is playing and the algorithm
>> is analyzing, I found that there are occasionally jumps/glitches in the
>> time stamp because the time stamp has nothing to do with the WAV file and
>> everything to do with how your computer analysis burden is keeping up (or
>> not) with realtime.
>>
>>
>>
>>   The resulting files needed a lot of hand editing afterward to remove
>> junk before I could profitably do anything with them.  This took a lot of
>> time.  I considered myself lucky that I had 2 minutes of data to look at
>> for the tone analysis that were clear enough that I could spot them on a
>> very simple uniform-X-axis vs frequency plot.  If I hadn't been able to see
>> that, it would have been game over.
>>
>>
>>
>>   So I would definitely not use it again for any kind of serious
>> analysis.  In fact, that is why we've separately requested that the
>> statistical frequency measurement algorithm buried within "fldigi" be
>> broken out into a separate Python routine. I believe CWRU is working on
>> it.  I looked at the fldigi code at one point to see if I could do it, but
>> it's an interconnected quasi-monolithic series of routines with unclear
>> cross-connections and internal assumptions.  That was a research project
>> that I ultimately passed on for lack of time.
>>
>>
>>
>>   Sorry to be negative, but this recent experience is I think quite
>> relevant to Tom's (1) item.
>>
>>
>>
>> 73
>>
>> Phil W1PJE
>>
>>
>>
>>
>>
>> On Tue, Apr 28, 2020 at 12:27 AM Tom McDermott via TangerineSDR <
>> tangerinesdr at lists.tapr.org> wrote:
>>
>>
>> Notes from PSWS / TangerineSDR call of 04-27-2020
>>
>>
>>
>> 1. Discussion of bundling / interfacing FLDIGI into the TangerineSDR
>> software. Tangerine produces IQ data, while FLDIGI may need WAV data (does
>> it accept other formats?). Having the ability to use the various FLDIGI
>> capabilities would be nice to have.  There would be a need to reject the
>> positive or negative frequency spectra to convert from complex to real,
>> multiple approaches are available.
>>
>>
>>
>> 2. Discussion of a lightning detector for the magnetometer board. Does it
>> need an I2C interrupt?  Scotty mentioned that he may want to abandon the
>> PMOD connector and turn the magnetometer board into a Raspberry-PI HAT.
>> Dave discussed some potential issues with the I2C interface on this
>> particular part, he is still trying to find some details on the issue.
>> Another question: what is the science need for lightning detection in terms
>> of range and sensitivity? Gerry Creager might be a good person to ask about
>> the science requirement.
>>
>>
>>
>> -- Tom, N5EG
>>
>>
>>
>> --
>> TangerineSDR mailing list
>> TangerineSDR at lists.tapr.org
>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>
>>
>>
>>
>> --
>>
>> ----
>> Phil Erickson
>> phil.erickson at gmail.com
>> --
>> TangerineSDR mailing list
>> TangerineSDR at lists.tapr.org
>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>

-- 
----
Phil Erickson
phil.erickson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200428/1e6a2e70/attachment-0001.html>