[TangerineSDR] Updated Detailed Design Specification for PSWS Local Host (v0.1)
Engelke, Bill
bill.engelke at ua.edu
Mon Nov 18 11:38:17 EST 2019
Rob - thanks for all the help on this, there is a lot to digest, and I am going to go thru everything. In the meantime, a few notes, answers and questions for you...
- I had been figuring we would have a distributable image with everything we need included in it. People could build their system by downloading this and putting on eMMC/SDchip, or buy a pre-made one from TAPR. You mentioned that we should have a version of this for desktop that is exactly the same as for the Odroid; seems fine, but don't we have at least the difference that the Odroid is ARM and the desktop would be x86/x64? Is everything we need going to work the same on both architectures?
- On the FT8 decoder, I just picked the initial approach based on how it is done in the Red Pitaya (Pavel Demin's "Notebook" code). If we have plenty of RAM, RAMdisk is fine. The FT8 files are about 2 MB each, so decoding 8 bands would take 16 MB which is not that much for the Odroid. BTW, I have the FT8 decoder source (mix of C and Fortran), and it could probably be enhanced to use pipe or socket or other more state-of-the-art method, but I would make that a lower priority for Phase 1, since we do have a working solution.
- On Gmail, I have tried on several occasions to push outgoing mail to it (and to AT&T also) , trying various methods with good SMTP credentials, but have never been successful; always ended up using SMTP2GO. If you can provide detailed guidance on how to actually make that work, it would be great.
- On frequency of clock updating, I based that merely on the guidelines for setting up a system to update clock with Dimension 4; updating clock every 15 or 30 minutes is quite sufficient for FT8 or WSPR. We can have it update as often as deemed necessary.
- Regarding "- You mention in the "Remote System Updates" section, "Need to select a package
for enabling this." What do you mean by that?" - - - Not knowing any better I had assumed that there would be more than one way to distribute and apply system updates from the Central Control system; this just refers to deciding how to apply these. I gather that you have a good handle on best practice here, and can recommend how to do it.
I appreciate you pointing us in the right direction, as I have minimal exposure to distributing a system like this for a SBC. More feedback is definitely most welcome.
- tnx es 73 - Bill, AB4EJ
-----Original Message-----
From: TangerineSDR <tangerinesdr-bounces at lists.tapr.org> On Behalf Of Rob Wiesler via TangerineSDR
Sent: Saturday, November 16, 2019 4:23 PM
To: TAPR TangerineSDR Modular Software Defined Radio <tangerinesdr at lists.tapr.org>
Cc: Rob Wiesler <robert.wiesler at case.edu>
Subject: Re: [TangerineSDR] Updated Detailed Design Specification for PSWS Local Host (v0.1)
On Fri, Nov 15, 2019 at 22:25:00 +0000, Engelke, Bill via TangerineSDR wrote:
> To all - Attached please find 2 files. This is the latest update
> (Version 0.1) of the PSWS Local Host (SBC) Detailed Design
> Specification, both with all markup and no markup.
Thanks, Bill.
> I tried to create a version with "simple markup" where it just puts a
> red line next to anything that was changed; but MS Word will not
> produce a pdf with that markup style. Strange. I tried all kinds of
> ways.
I actually quite like the existing style. It makes it quite obvious what I have to pay attention to.
> There is a lot of new stuff in this version, you can see a summary of
> the changes on p. 2. Please review and comment - thanks -
I have a lot of comments:
- Remember, to achieve our goals relative to software distribution,
maintenance, updates, and installation, we basically cannot use HardKernel's
Ubuntu image as-is, and we definitely cannot pile a heap of manual builds of
unpackaged software on top of that.
- Security updates are not something we can leave to HardKernel, even for the
subset of packages they modified for the image.
- We want the simplest possible installation. That means everything must
come preinstalled, or modification of an existing Debian-based operating
system must be limited to the addition of a sources.list.d entry and
an archive keyring, and subsequent installation of packages (which
is scriptable).
- The HardKernel image uses an ext4 (I believe) root filesystem. Since we
want atomic security updates and minimal downtime (and all in-tree
filesystem drivers), we'll need to convert that to a btrfs filesystem (and
move /boot to vfat, as Das U-Boot isn't that great with btrfs just yet).
- The HardKernel distribution is aging. Fortunately, part of why they
haven't kept it up-to-date is because they've been upstreaming their kernel
changes, i.e. the only reason their disk image even exists is because
officially-distributed kernels from that point in time needed patching
because their changes did not get backported upstream (because that's how
stable distributions work). I'm still verifying this, so take it with a
grain of salt.
- We want as much of this as possible to run on a desktop in *exactly* the
same form as it does on the LH, for lots of reasons.
- Obviously, Phase 1 will probably still use the HardKernel image as a base,
but it will have to be modified to convert partition 2 to btrfs, move /boot
to partition 1, configure our APT repo, and install our packages by default.
- gnuradio is already packaged for Debian/Ubuntu. If there's a specific
reason why that version (or the version in backports) is unsuitable,
I don't remember anybody mentioning why. Additionally, if it
actually is unsuitable, we should instead distribute a modified
version of the package via our repo instead of building from source
on each end device.
- I now plan to package digital-rf for Debian. It looks to be in relatively
good shape, and the dependencies all appear to be in Debian already. Any
package I create can live in our repo (which doesn't yet exist) until it
makes it to Ubuntu.
- The RedPitaya out-of-tree module should also make it into a real package.
- Getting the time via NTP is not a network operation if a GPSDO is
installed (the diagram is misleading on this point). NTP will still
be used, but it will act as a stratum 1 time server, with its inputs
coming from the GPSDO. Additionally, modern computers do not udpate
their system clocks in the way you seem to think - the requirement
that "the LH system clock should be updated no more frequently than
once every 15 minutes" is bogus and harmful. Modern system clocks are
slewed continuously, not stepped, and this is for the benefit of WSPR,
not its detriment. Again, this is something to let NTP handle and not
think too hard about.
- The FT8 decoder should be able to decode our samples without needing them to
be written to a disk. It's conceivable that the FT8 decoder is not
sophisticated enough to read from stdin or a pipe, but in that case we still
need to save the sample to a file in tmpfs (RAM disk) instead of involving
flash memory.
- I believe that callsign monitoring is totally something we should use
a local event bus for, and MQTT is perfect for that (and fot the other
event types you mention that might generate email). That means that
each observed callsign is published locally (and probably discarded
due to lack of subscriptions), and our event-to-email process
subscribes to these events. I don't see a limit of 8 callsigns as
being either reasonable or useful - the system should be designed to
listen for as many callsigns as the user wants. There shouldn't be
any technical reason against this, and any interim implementation that
imposes a cap will necessarily be more complicated.
- A comment in the callsign-to-email section mentions that Google Mail
et. al. no longer accepts "this kind of email". That is not
necessarily true. After having spent way too long last week
configuring my own mail server and offline IMAP storage for my
existing Gmail account (and not too long configuring a lab-bound RasPi
to send mail for Kristina Collins (KD8OXT)), I can say confidently
that our options and constraints are these:
- The listed corporate email providers don't like mail sent without a DKIM
signature. That's basically the easiest part of setting up a mail server
(including picking which packages to install), but it means that mail
should be sent through a private SMTP relay (such as Gmail's servers) that
performs this task, because we're never going to get DKIM working on
systems in the field.
- Providing credentials (username and password) for a correctly-configured
SMTP account are sufficient. If you have a Gmail account, your normal
login will work (and you can make a new Gmail account trivially).
Receiving mail from a Gmail account, on the other hand, is a complete
nightmare, but thankfully we don't have to deal with that.
- Python's smtplib module is enough for this task. I believe it's part of
the standard Python distribution. Sending an email using it takes about a
dozen lines of code (example on request, but the online
documentation isn't too bad).
- The software update mechanism is not present in the LH System Architecture
diagram. The additions I would expect to see look like this:
┏━━━━━━━━━┓
┃ MAINCTL ┃
┗━━━━━━━━━┛
↓ ↑
┌───────────────────────────────────────────────────┐ ┌───────────────────────┐
│ Local apt/dpkg/apt-get/aptitude/etc. (Unmodified) │ ← │ TangerineSDR APT repo │
└───────────────────────────────────────────────────┘ └───────────────────────┘
↑ ↑
↑ TangerineSDR sources.list.d entry, TangerineSDR archive keyring
↑
Hook scripts for before/after package installation (btrfs snapshot management)
↑
initrd hook script to recover from incomplete updates by reverting to
last btrfs snapshot (i.e. runs from initramfs before real rootfs is
mounted read-write and init (systemd) runs)
- You mention in the "Remote System Updates" section, "Need to select a package
for enabling this." What do you mean by that?
- Timestamps are still listed as being 128-bit. I believe Ryan gave a good
argument as to why this is too many. Even if 64 is too few (and it probably
isn't), 128 is definitely overkill. That's not to say that this is
not acceptable - I'm just trying to point out something that appears
not to be part of what I understand as our consensus.
--
73 de AC8YV
--
TangerineSDR mailing list
TangerineSDR at lists.tapr.org
http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
More information about the TangerineSDR
mailing list