[TangerineSDR] Notes from PSWS / TangerineSDR call of 04-27-2020

Tue Apr 28 16:48:59 EDT 2020

Phil,

Thanks! - I think we are in total agreement.  That is why this group's
diverse experience is critical for this project to succeed.

FLdigi is tricky with subtle nuances the first 10 times you use it and get
burned by it - and I have the wounds to prove it as well.

And this also is the reason I rewrote parts of the Fldigi's analysis
package to address some of these issues (coded in C++).

Ultimately we want a robust and fast (once per second) freq estimation
algorithm that is proven to work in noise (my next set of tests I will be
running after I complete my 1st test report will be exactly that).

We have to know how it performs in practice, and that's what I'm trying to
flush out with these system tests (antenna input to file output).

More to follow as progress is made...

Stay healthy!

John N8OBJ

John C. Gibbons
Director - Sears Undergraduate Design Laboratory
Dept. of Electrical Engineering and Computer Science
Case Western Reserve University
10900 Euclid Ave, Glennan 314
Cleveland, Ohio  44106-7071
Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
E-mail: jcg66 at case.edu

On Tue, Apr 28, 2020 at 2:39 PM Phil Erickson <phil.erickson at gmail.com>
wrote:

> Hi John,
>
>   I used the very latest "fldigi", downloaded 24 hours before the FMT.
> The key difference perhaps is that I did the analysis in retrospective mode
> on already recorded data - not live from the antenna. That is a different
> use case from yours, and different use cases are not of course going to use
> the same methods.  Not suggesting either that a Python module be inserted
> real-time into anyone's flow on a resource limited platform (how did we get
> there?), but that Python is the language of choice for back-end,
> non-realtime, iterative, retrospective analysis.  That frankly is where I
> spend 100% of my time as a scientist.
>
>   So a scenario where a C language, fast algorithm is used for all the
> realtime needs, and that this very same algorithm is then bundled into an
> auto-generated Python module for backend non-realtime retrospective work,
> serves all customers from what I can tell.
>
> Cheers
> Phil W1PJE
>
>
>
> On Tue, Apr 28, 2020 at 11:57 AM John Gibbons via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
>> All,
>>
>> With my mods, FLDigi has been working flawlessly for use by us.  There
>> are some things in the user interface you need to be aware of, and
>> unfortunately Phil got bit by a few of them.
>> I could get into them, but it isn't going to apply to us long term.
>>
>> What version of FLDigi were you using?
>> FLdigi-4.1.12 (just released last Monday) was the first version that has
>> my improvements in it (that came from this WWV project), and I did use it
>> for the
>> FMT (and - shameful brag alert.....  had an error of 0.01 hHZ on 40M and
>> 0.05Hz on 80M)  😵
>> It does work.  But as usual, the devil is in the details....
>>
>> I'll be pushing out a report on a test I just completed on the Gen1 Radio
>> I designed for the low cost PSWS and using FLDigi shows how well it
>> performs for frequency estimation.
>> We've been using it since July 2019, and a lot of our data collection has
>> been done through this radio receiver (ALL of the data collection has been
>> using my modified version of FLDIGI-4-1.09).
>>
>> My test was a COMPLETE 'system' test - from antenna input to file numbers
>> output.  I wanted to see how well the whole system was doing including the
>> freq analysis algorithm.
>>
>> The 'final' version will be the algorithm extracted and customized for
>> our use.  *FLDIGI is NOT our long term solution* (it never was) - it's a
>> stepping stone.
>>
>> With the processor / memory limitations of the low cost PWSW we will try
>> to do python code, but coming from 35+ years of designing embedded
>> real-time systems it will more likely be C / C++ code.
>> The reason is that an interpreted language (i.e. python) runs so much
>> slower (10x - 100X) and uses up so much more system resources that it bogs
>> down (and usually fails).
>> Also, depending on the interpreter to (not) getting an OS upgrade that
>> breaks our code is just a bad idea.         Been there.... been burnt by
>> it... don't want to do it again.
>> Having compiled code means WE control the updates and not the latest
>> release of an interpreter.
>>
>> And Gen 2 of my radio will attempt to do 4 channels of WWV ( or some on
>> CHU) at the same time.. Running 4 instances of FLdigi simply is not an
>> option - it just will not work.
>>
>> BUT,  the final software is being started with the tests being run now in
>> python, and concurrently I'm creating tests in C++.  Again, the OS overhead
>> of running an interpreted language for a real time
>> data acquisition is not done in industry for lots of good reasons with
>> lots of test cases to back it up.  If you want a reliable system, using an
>> interpretive language is simply not the way to go.
>>
>> We have made some advancement in the frequency determination
>> algorithm(s), and even had one of our helpers try/work on implementing one
>> in a MAX10 FPGA implementation.
>> He got interrupted this semester by having to teach a new embedded
>> systems IOT class, but hope to get that development back on track soon
>> (stop drooling Scotty😁🤔😃).
>>
>> More to come later...
>>
>> John N8OBJ
>>
>>
>>
>>
>> John C. Gibbons
>> Director - Sears Undergraduate Design Laboratory
>> Dept. of Electrical Engineering and Computer Science
>> Case Western Reserve University
>> 10900 Euclid Ave, Glennan 314
>> Cleveland, Ohio  44106-7071
>> Phone (216) 368-2816 <216-368-2816> FAX (216) 368-6888 <216-368-6888>
>> E-mail: jcg66 at case.edu
>>
>>
>>
>> On Tue, Apr 28, 2020 at 8:07 AM Dr. Nathaniel A. Frissell Ph.D. via
>> TangerineSDR <tangerinesdr at lists.tapr.org> wrote:
>>
>>> Hi Phil,
>>>
>>>
>>>
>>> Thanks for the comments. I think that is very good to consider. In light
>>> of review, I think it is a good idea to postpone FLDigi integration at this
>>> time. As Phil said, we should pull out the frequency estimation routine for
>>> separate testing and possible direct use. I believe Skylar Dannhoff is
>>> working on this. For the WWV monitoring program, we also do need to have
>>> the ability to save both the raw data and the derived frequency estimation
>>> data product.
>>>
>>>
>>>
>>> 73 de Nathaniel W2NAF
>>>
>>>
>>>
>>> *From:* TangerineSDR <tangerinesdr-bounces at lists.tapr.org> *On Behalf
>>> Of *Phil Erickson via TangerineSDR
>>> *Sent:* Tuesday, April 28, 2020 7:12 AM
>>> *To:* TAPR TangerineSDR Modular Software Defined Radio <
>>> tangerinesdr at lists.tapr.org>
>>> *Cc:* Phil Erickson <phil.erickson at gmail.com>
>>> *Subject:* Re: [TangerineSDR] Notes from PSWS / TangerineSDR call of
>>> 04-27-2020
>>>
>>>
>>>
>>> Hi all,
>>>
>>>
>>>
>>>   Sorry that I haven't been able to make these discussions much lately.
>>> COVID-19 has added a huge amount of work here.
>>>
>>>
>>>
>>>   I'm fairly concerned about using "fldigi".  In the recent Frequency
>>> Measurement Test a few days ago, I decided to use its frequency algorithm
>>> in the "ANALYZE" portion in order to track the unknown carrier that K5CM
>>> was putting out.  I had a bunch of data already recorded through things
>>> like Gnuradio stacks; for this discussion, let's assume I had I/Q data in
>>> some kind of format similar to what Tangerine would put out.  (It was
>>> DigitalRF to be specific.)
>>>
>>>
>>>
>>>   Frankly, it was a disaster.  The only way you can input data that I
>>> could find is through WAV files, so I had to convert all my data to that
>>> format.  When you start up the program, it immediately starts generating
>>> bogus data on nothing while you fumble around getting to the menu that
>>> plays back a WAV file.  When the WAV file ends (I chose non-loop for
>>> playback), you get another bunch of junk data while you are fumbling around
>>> with literally quitting the entire program as the only way I could figure
>>> out to stop the analysis.  While the WAV file is playing and the algorithm
>>> is analyzing, I found that there are occasionally jumps/glitches in the
>>> time stamp because the time stamp has nothing to do with the WAV file and
>>> everything to do with how your computer analysis burden is keeping up (or
>>> not) with realtime.
>>>
>>>
>>>
>>>   The resulting files needed a lot of hand editing afterward to remove
>>> junk before I could profitably do anything with them.  This took a lot of
>>> time.  I considered myself lucky that I had 2 minutes of data to look at
>>> for the tone analysis that were clear enough that I could spot them on a
>>> very simple uniform-X-axis vs frequency plot.  If I hadn't been able to see
>>> that, it would have been game over.
>>>
>>>
>>>
>>>   So I would definitely not use it again for any kind of serious
>>> analysis.  In fact, that is why we've separately requested that the
>>> statistical frequency measurement algorithm buried within "fldigi" be
>>> broken out into a separate Python routine. I believe CWRU is working on
>>> it.  I looked at the fldigi code at one point to see if I could do it, but
>>> it's an interconnected quasi-monolithic series of routines with unclear
>>> cross-connections and internal assumptions.  That was a research project
>>> that I ultimately passed on for lack of time.
>>>
>>>
>>>
>>>   Sorry to be negative, but this recent experience is I think quite
>>> relevant to Tom's (1) item.
>>>
>>>
>>>
>>> 73
>>>
>>> Phil W1PJE
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 28, 2020 at 12:27 AM Tom McDermott via TangerineSDR <
>>> tangerinesdr at lists.tapr.org> wrote:
>>>
>>>
>>> Notes from PSWS / TangerineSDR call of 04-27-2020
>>>
>>>
>>>
>>> 1. Discussion of bundling / interfacing FLDIGI into the TangerineSDR
>>> software. Tangerine produces IQ data, while FLDIGI may need WAV data (does
>>> it accept other formats?). Having the ability to use the various FLDIGI
>>> capabilities would be nice to have.  There would be a need to reject the
>>> positive or negative frequency spectra to convert from complex to real,
>>> multiple approaches are available.
>>>
>>>
>>>
>>> 2. Discussion of a lightning detector for the magnetometer board. Does
>>> it need an I2C interrupt?  Scotty mentioned that he may want to abandon the
>>> PMOD connector and turn the magnetometer board into a Raspberry-PI HAT.
>>> Dave discussed some potential issues with the I2C interface on this
>>> particular part, he is still trying to find some details on the issue.
>>> Another question: what is the science need for lightning detection in terms
>>> of range and sensitivity? Gerry Creager might be a good person to ask about
>>> the science requirement.
>>>
>>>
>>>
>>> -- Tom, N5EG
>>>
>>>
>>>
>>> --
>>> TangerineSDR mailing list
>>> TangerineSDR at lists.tapr.org
>>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>>
>>>
>>>
>>>
>>> --
>>>
>>> ----
>>> Phil Erickson
>>> phil.erickson at gmail.com
>>> --
>>> TangerineSDR mailing list
>>> TangerineSDR at lists.tapr.org
>>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>>
>> --
>> TangerineSDR mailing list
>> TangerineSDR at lists.tapr.org
>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>
>
>
> --
> ----
> Phil Erickson
> phil.erickson at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20200428/becdfa13/attachment-0001.html>