[TangerineSDR] Magnetometer/Grape Raspberry Pi Image

[Rob Wiesler]

On Mon, Nov 01, 2021 at 08:51:13 -0400, Jonathan wrote:
> On Nov 1, 2021, at 5:55 AM, Rob Wiesler <robert.wiesler at case.edu> wrote:
>> On Sun, Oct 31, 2021 at 21:00:42 -0400, Jonathan via TangerineSDR wrote:
>>> If I remember correctly, you used a GPS hat for the Raspberry Pi on
>>> the Magnetometer/Grape system. Is this true? If so, which one do you
>>> use?
>>>
>>> The reason I ask is that there was a bug in gpsd that caused the GPS
>>> date to be erroneously reported as March of 2002 once October 23th
>>> 2021 has passed.
>>
>> To the best of my understanding, this is not exactly true.  The bug
>> affects test cases (Gary refers to them as "regressions"), not live
>> code.  Live code uses the current time and date to disambiguate
>> rollover, but the regressions run on old NMEA captures and do not have
>> that luxury.  So, if you were to take a buggy version of gpsd and run
>> the regressions with "scons check", you would encounter the bug then,
>> but not in normal operation.
>>
>> (If you're running gpsd on a RPi with no RTC and no guarantee that ntp
>> -g steps the time *before* gpsd starts, you have a buggy installation
>> regardless of which version of gpsd you're using.)
>>
>>>                 This has been fixed and you are encouraged to update
>>> to gpsd version 3.23.1. To do so, do a git pull of the gpsd
>>> repository, then type "git checkout release-3.23.1", then "scons -c &&
>>> scons && scons install".
>>
>> Please don't do this.  This bug report is months old, and has been
>> addressed.  The maintainer of the gpsd package in your operating system
>> should have backported this fix if it's relevant to the packaged
>> version.  For Debian and Raspian (and likely Ubuntu), this patch is
>> included in 3.22-4 (the -4 suffix is the Debian revision of the
>> packaging), and a simple system update is sufficient to bring it in for
>> bullseye and buster-backports (and bookworm, but I doubt you're running
>> that).  Older versions of the package (in buster and older) are not
>> patched, as this bug was caused by a commit first seen in version 3.20 .
>>
>> The Debian packaging also includes several fixes that are pertinent for
>> Debian installations.  If you do end up installing from source, I
>> recommend pulling in the patches from Debian.
>>
>>> Details about the bug can be found at:
>>> https://gitlab.com/gpsd/gpsd/-/issues/144 or LWN, El Reg, or CERT
>>> write ups.
>>
>> I recommend reading Gary's comment in that bug report that starts with
>> "GPS, by design, is a 1024 week time warp waiting to happen.".  This is
>> the best place to start reading to verify if my initial statement in
>> this email is correct or not.
>>
>> (It's a good thread in general.  I was absolutely floored to learn that
>> there was a bug in (a test case of) production software due to global
>> warming causing a scheduled/expected leap second to be rescinded.)
>>
>>> As of this email, version 3.32.2 is the most current, but there is a
>>> bug that prevents using the GPIO PPS with another serial device than
>>> /dev/ttyAMA0. Most GPS hats use /dev/ttyAMA0, so you can do a git
>>> checkout and build/install of gpsd, but since I use /dev/ttyAMA0 for a
>>> serial console (I still believe in out-of-band access), I utilize
>>> /dev/ttyUSB0 from a FT230 USB serial interface IC that my GPS serial
>>> port connects to, which then connects to the Pi USB port. In this
>>> configuration, version 3.23.2 of gpsd cannot use the /dev/ttyUSB0
>>> /dev/pps0 runtime option and capture PPS asserts/clears from the
>>> pps_gpio driver. But, most GPS hats and users don't use this
>>> configuration very often, so you won't be affected. A fix for this
>>> issue will be pushed to the gpsd repository soon.
>>
>> 3.22 should be unaffected by this.  In general, when an upstream
>> maintainer recommends jumping straight to the bleeding edge, you might
>> want to take that with a grain of salt.
>
> I was running an old version of 3.20 on both my Raspbian and FreeBSD
> systems and was directly affected by this bug. The fix for the bug
> affected the regressions but Gary was able to work through that. As in
> my previous email, my systems showed a GPS date that was 20 years
> behind. I first noticed it on my Pi this past Friday when I went to
> check my ntp peers and the GPS peer had a huge offset. Updating to the
> latest (3.23.2) fixed the problem, however on my Pi where a use a
> separate PPS source and a USB serial adapter, it failed to do shm puts
> from the /dev/pps0 source. Using 3.23.1 fixed that problem for now.

I don't know about FreeBSD, but it sounds like you were already on a
custom from-source installation, just from you mentioning version 3.20,
which is not one of the versions that made it into Debian (and therefore
Raspian, which pulls from Debian for anything that doesn't require a
custom version).

It also sounds like your lack of an RTC contributed to the bug you
encountered, which was *not* the one you linked to.  Please re-read my
post (and the bug report you linked to), which explains that the bug you
were concerned about does not affect running code, but only test cases.
gpsd uses the current system date and time to disambiguate the GPS epoch
when the 1024-week counter rolls over (1000 weeks on certain miserable
receivers).  If your system clock thinks the time is January 1 1970 UTC
because you bought a computer that didn't come with an RTC and then
failed to make sure gpsd did not start until ntp -g had stepped the
time, gpsd cannot help you even when bug-free.

> On the Pi, I use “tinker panic 0” in /etc/ntp.conf so ntpd will still
> discipline the system clock after it receives the current time from
> one of its peers.

That's a mistake, and it's well-documented as one in many places online.
Your Pi is not waking up from suspend and needing to step the time,
right?  That's the only case I can think of where it would be remotely
appropriate.  NTP still disciplines the clock as it runs, but it slews
it rather than stepping it, and if a timeserver is too far off what is
known to be correct, it should reject that timeserver ("panic") and move
on to another one.  If you need to step the clock at boot (because you
don't have an RTC), use "-g", which disables this panic when gpsd first
finds a timeserver, but not thereafter.  Take another look at the NTP
documentation - "tinker" is filed under "modify sacred system parameters
(dangerous)".  It is almost always wrong to touch these, especially when
you think it's right.

>                   I don’t use an RTC because I did not see a need for
> it, especially since it gets the time of day from the GPS peer if the
> network were to be down.

Well, you just found a need for it.

If your system does not have the correct time at boot, i.e. before NTP
and/or GPS can set the time, lots of things break subtly.  One of those
things can be gpsd, because it uses the current time to disambiguate
when the 1024-week epoch bug rolls around (which can be at different
points in the 1024-week count depending on the GPS receiver).

There are horrible, crufty ways to work around this, generally involving
faking an RTC by setting the system clock before NTP and gpsd start,
then (ideally) sequencing gpsd to start after NTP has obtained the time
and stepped the clock.

>                          Plus, adding another hat, more power
> consumption, and driver requirements seemed like it would add more
> unnecessary complexity.

Lots of applications for which the Pi is well-suited don't need to know
the actual time, which is why it doesn't have one built in, but if
you're just using the Pi as a small, cheap computer, you probably
actually do need one.  Otherwise you get to deal with the fallout from
timestamps going backwards, x509 (HTTPS) certs and other security
mechanisms throwing a fit because the NotBefore constraint is not
satisfied, replay attacks, etc.  It is certainly somewhat obnoxious to
add an RTC to a Pi, but trust me, it solves a lot of avoidable problems
(like yours!).

> Many package maintainers fail to keep gpsd updated and many
> installations do not get updated on a regular basis. Back in March of
> 2020, Raspbian’s gpsd package was stuck at 3.17 when the latest was
> 3.20.

Yes, that's how Debian works, and therefore how Raspian works.  When
3.20 came out, it was packaged and landed in the unstable repository,
and when 3.22 came out, it got the same treatment.  When the next
version of Debian came out, the version of gpsd that was in unstable was
copied to stable, and supported from there.  Any bugfixes that were made
in more recent versions of gpsd that were found to be in 3.20 in Debian
were backported in order to fix those specific issues without jumping
straight to the bleeding edge and breaking other software.  Do not
mistake an old version number in Debian for broken software (which is
possible, but generally not for such widely-used software as gpsd).

There's only one package maintainer for gpsd in Debian, and that's Bernd
Zeimetz, who seems to do just fine.  On Debian (and therefore Raspian),
gpsd's regressions were patched pretty much immediately (the changelog
entry is for Sun, 01 Aug 2021 21:58:54 +0200, which was *before* the
final release of gpsd 3.23, which fixed the bug you linked to).

>       If your distribution offers an up-to-date package, then you
> should use it, but if it doesn’t, the best option is to compile from
> source from a major version tarball or git pull and build from there.
> As far as distribution-specific patches, I usually leave it up to user
> discretion.

Again, "up-to-date" is not necessarily what you want.  "Not broken" is
usually what you want, and that involves some sort of compromise between
running the most recent versions of everything and backporting fixes to
older versions.

In this case, the distribution relevant to the thread has a patched
version of gpsd, and it is highly disrecommended to yeet it in favor of
a self-compiled version that will add to the system's maintenance burden
just because of a no-longer-applicable bug that never had anything to do
with screwing up the time of a running system.

If you want bleeding edge, Arch will run on an RPi, and even in that
case running a custom from-source version of gpsd is *still* the wrong
way to go in the vast majority of cases.

> 3.22 is unaffected by the alternate Pi UART and PPS device bug that I
> experienced, but Gary recommended to me that I use 3.23.1. So far,
> it’s working reliably on my VLF SDR.

Most upstreams want people to run the newest thing they wrote.  They're
not completely wrong, but it's also not feasible to do that for every
bit of system software in nearly all cases.

I'm glad to hear that 3.23.1 works for you.  Good luck keeping it
patched by yourself, given that all gpsd bugs are security bugs.