[TangerineSDR] PSWS System Specification preliminary Ver 0.1

Scotty Cowling scotty at tonks.com
Mon May 6 19:31:21 EDT 2019


Hi Bill,

I think it is unworkable.

The HPSDR protocol is hardwired to talk from exactly one SDR (referred 
to as "SDR hardware") to exactly one Host (referred to as "PC").

No provision is made for multiple discovery requests, and you cannot 
stream data to more than one IP address. So all 192K slice receivers 
would have to stream to the same hardware (same IP address). Command and 
control definitions are so married to the HPSDR and ANAN hardware that 
we would probably not be compatible with any off-the-shelf radios anyway.

While I think we can base the protocol loosely on the HPSDR protocol as 
a starting point, I think it is too restricted to be of general use. I 
am working on a more general-purpose protocol for LCC that would 
eliminate most, if not all, hardware-specific restrictions. We can use 
some SDR hardware that I have already built to develop the protocol, or 
use existing TAPR or Apache Labs Hermes boards or even ANAN radios. They 
all have FPGAs in them, so we can program them to do the SDR end of the 
protocol for testing. I would be willing to work on the FPGA end if 
someone wants to work on the Host end so we can get a system up and running.

My goal is a hardware-agnostic, multiple client, multiple server 
protocol that would handle streaming of I/Q data, raw sample data, RX 
audio, TX audio, and control commands, as well as re-programming of the 
hardware over Ethernet. HPSDR protocol 2 is far from this goal.

Do you have a networking expert (maybe that would be you?) that we can 
get in on the definition?

I don't want to interfere with your progress, but the above were 
features that I wanted to be included in protocol 2 that were not. I 
think that the use of different (changeable) ports for every RX stream, 
but no changeable ports (or limited ports) for other types of streams 
makes too many assumptions on the hardware that is on each end.

So can we start with a clean slate? Pretty please? :-)

73,
Scotty WA2DFI

On 2019-05-06 14:29, Engelke, Bill wrote:
>
> Hello Tom & Scotty:
>
> Nathaniel has given me the go-ahead to start working on the specs for 
> the software, which I understand to include:
>
> -Functional Specifications for the Central Control system and related 
> database
>
> -Remote Command and Control Protocol
>
> -Functional Specifications for the local control system (this is the 
> SBC often referred to as the Host)
>
> I guess a part of this would touch on the Local Command and Control 
> Protocol (between the Host and Data Engine). Here’s my 2 cents:
>
> If I would suggest  to use openHPDSR protocol (or at least a 
> subset/superset of it). Reasons:
>
> ·We can stand on the shoulders of the existing, proven design.
>
> ·If somebody has another SDR (e.g., Red Pitaya, Hermes, etc.) they 
> could use these as the radio for the PSWS.  (Note that the central 
> database will know what kind of radio every observation came from, and 
> if you are concerned about the quality of data from a different type 
> of radio, you can simply exclude from any analysis that is sensitive 
> to that factor. WE WILL NOT SUPPORT any of these different radios 
> except to conform to our selected ported of the spec).
>
> ·We may be able to re-use some existing working code.
>
> ·Maybe a user could use their PSWS as a radio (a receiver, at least) 
> under PowerSDR if they wanted to.
>
> ·We can get started with software development even before the new 
> hardware is ready, because there are radios that conform to the spec.
>
> ·I have found the openHPSDR-Protocol-2 online at 
> https://github.com/TAPR/OpenHPSDR-Firmware/blob/master/Protocol%202/Documentation/openHPSDR%20Ethernet%20Protocol%20v3.8.pdf
>
> If anyone thinks this is definitely not workable, please let me know, 
> so that I can learn; I expect there are aspects of this that I am not 
> aware of, so I am open to all feedback… thanks….
>
> -73- Bill, AB4EJ
>
> *From:*Tom McDermott <tom.n5eg at gmail.com>
> *Sent:* Sunday, May 5, 2019 8:36 PM
> *To:* Engelke, Bill <bill.engelke at ua.edu>
> *Cc:* TAPR TangerineSDR Modular Software Defined Radio 
> <tangerinesdr at lists.tapr.org>
> *Subject:* Re: [TangerineSDR] PSWS System Specification preliminary 
> Ver 0.1
>
> Hi Bill - one of the problems when writing a new specification is that 
> the author has
>
> undocumented assumptions in their mind.  The review process helps get 
> those discovered
>
> and then written into the spec or some other document.
>
> This thread is good because it has uncovered several of those that 
> need writing out.
>
> For this project, it is apparent that we need a use case document, 
> covering the two cases,
>
> and perhaps more later on.
>
> -- Tom, N5EG
>
> On Sun, May 5, 2019 at 1:16 PM Engelke, Bill <bill.engelke at ua.edu 
> <mailto:bill.engelke at ua.edu>> wrote:
>
>     Tom – OK, I get it.  Thanks for being patient with me, I’m trying
>     to understand as much as possible about this so that I can
>     properly guide the software development. There are indeed people /
>     organizations that have major local horsepower, and it is
>     certainly advisable to ensure they can roll their own system if
>     they wish to.  -73- Bill AB4EJ
>
>     *From:*Tom McDermott <tom.n5eg at gmail.com <mailto:tom.n5eg at gmail.com>>
>     *Sent:* Sunday, May 5, 2019 10:23 AM
>     *To:* Engelke, Bill <bill.engelke at ua.edu <mailto:bill.engelke at ua.edu>>
>     *Cc:* TAPR TangerineSDR Modular Software Defined Radio
>     <tangerinesdr at lists.tapr.org <mailto:tangerinesdr at lists.tapr.org>>
>     *Subject:* Re: [TangerineSDR] PSWS System Specification
>     preliminary Ver 0.1
>
>     Hi Bill - Your understanding is one particular implementation,
>     it's the same implementation that Scotty is assuming.
>
>     Given today's silicon technology it may be the only practical
>     implementation. Given tomorrow's technology maybe not.
>
>     There could be several use cases:  the one you are assuming, where
>     the remote PSWS is accessed over the Internet. and
>
>     where remote-->server bandwidth is a significant limitation.  For
>     that case Nathaniel laid out the need for the remote station to
>
>     stream to local non-volatile storage, then be triggered during an
>     event of interest to send a small data subset back to a central
>
>     server.
>
>     But there is another use case where the PSWS is located on the
>     same LAN as the server. Phil Erickson expressed interest
>
>     in this case.  For this case perhaps folks would want to stream
>     continuously over the LAN to that server, providing greater
>
>     data capture capability.  For this use case an inexpensive Host
>     function might not be able to keep up.  Essentially, it's replacing
>
>     the low-cost host function with one that's a lot more powerful,
>     perhaps integrated with a local server.   With software modularity
>
>     perhaps the host software could be portable, or maybe that's phase
>     2 (or maybe phase never).
>
>     So the spec is drafted in an attempt to try not to limit the use case.
>
>     -- Tom, N5EG
>
>     On Sun, May 5, 2019 at 7:23 AM Engelke, Bill <bill.engelke at ua.edu
>     <mailto:bill.engelke at ua.edu>> wrote:
>
>         Hi Tom – a couple of notes – My understanding is that the Host
>         Computer (local control system, SBC) will be connected to the
>         DE vu gigabit Ethernet.
>
>         Also I have a concern about the statement in System Spec rev
>         0.2 -
>
>         If the DE and Host are physically separate devices, then the
>         DE may need to stream data directly to the central server if
>         the Host cannot process fast enough.
>
>         There will not be sufficient capacity either in the central
>         sever nor the user’s internet bandwidth to stream directly.
>         Very high speed upload is the exception rather than the rule
>         in the US, and we can’t assume we will have the funding for a
>         central host able to receive this data rate from lots of PSWS
>         units at once.  I think we have to *ensure that the local
>         control system is able to process fast enough* by how we
>         design the whole system. If I am missing something, please let
>         me know.
>
>         -73- Bill AB4EJ
>
>         *From:*TangerineSDR <tangerinesdr-bounces at lists.tapr.org
>         <mailto:tangerinesdr-bounces at lists.tapr.org>> *On Behalf Of
>         *Tom McDermott via TangerineSDR
>         *Sent:* Sunday, May 5, 2019 8:13 AM
>         *To:* TAPR TangerineSDR Modular Software Defined Radio
>         <tangerinesdr at lists.tapr.org <mailto:tangerinesdr at lists.tapr.org>>
>         *Cc:* Tom McDermott <tom.n5eg at gmail.com
>         <mailto:tom.n5eg at gmail.com>>
>         *Subject:* Re: [TangerineSDR] PSWS System Specification
>         preliminary Ver 0.1
>
>         Hi Scotty
>
>         The spec says that there is a DE function and a Host Computer
>         function - it doesn't say how they are
>
>         physically connected or implemented.  If that is unclear then
>         the spec is not written well
>
>         enough and other folks will likely make the same
>         (mis)interpretation, so it should be reworded.
>
>         Thanks for idea about the synthesizer.  You are right, the SL
>         spec does say that the output dividers are all
>
>         synchronously reset at power up and can also be reset by the
>         programmer to re-sync. It also says they
>
>         will maintain phase-sync across outputs that are derived from
>         the same synthesizer.  I will make some
>
>         measurements on the Eval Board across the 4 outputs to see how
>         well they stay aligned.    I hope it's
>
>         not susceptible to glitches un-syncing them. That would be a
>         dog to debug in the field.
>
>         -- Tom, N5EG
>
>         On Sat, May 4, 2019 at 6:27 PM Scott Cowling via TangerineSDR
>         <tangerinesdr at lists.tapr.org
>         <mailto:tangerinesdr at lists.tapr.org>> wrote:
>
>             Hi Tom,
>
>             OK, I was thinking that the four outputs of the AD5344
>             would be our
>             Clock Module outputs. One to the FPGA, one to each RF
>             Module, one
>             external. If you wanted two or more of them
>             phase-coherent, just drive
>             them from the same AD5344 synthesizer. Then the AD5344
>             output block
>             would perform our clock distribution, making an additional
>             chip
>             unnecessary. The variable frequency provided by the AD5344
>             synthesizers
>             would be used to configure clock outputs for different
>             kinds of RFMs.
>             This would enable us to build a low-cost LowFER RFM
>             (100kHz-500KHz) that
>             samples at, say, 10MHz if we want to. NCOs are totally
>             under the control
>             of the DE, so they can be designed as one shared NCO or
>             multiple NCOs as
>             desired.
>
>             Yes, the whole point it to have the hardware that is best
>             suited to each
>             task, do that task.
>
>             Implementing a full TCP/IP stack or SSH in the FPGA is an
>             expensive use
>             of hardware (so let the SBC do it). Implementing
>             decimation and slice
>             filtering in the SBC puts an unreasonable burden on a
>             low-cost
>             general-purpose SBC (so let the FPGA do it).
>
>             Tom, it just seemed like you were picking one way over
>             another in your
>             document by having a DE section 6 and a Host PC section 7.
>             I agree that
>             we need to specify the tasks, but not necessarily the
>             hardware that does
>             each one. Hopefully it will become obvious where each task
>             is best
>             handled when we document the needed steps. And I am trying
>             really hard
>             to keep the SBC out of the main flow, if possible. (Maybe
>             it is not.)
>
>             It was never my intention to allow the user to pick any
>             old SBC for
>             PSWS. My intent was to be flexible enough that *we* could
>             pick from many
>             alternatives and pick the one that works best for us.
>
>             All these options will get out of control very quickly if
>             we let users
>             pick whichever ones they want. It will turn out exactly as
>             you describe!
>             Just look at the early days of Flex, when users got to
>             pick their own
>             sound card to use with the SDR-1000. Total mayhem, even
>             after Flex TOLD
>             them to use "brand X, model Y" cards.
>
>             The options are for US to more easily build something that
>             meets the
>             requirements that this specification is enumerating (and
>             others that we
>             decide to support). And if users want to play, that is
>             fine. They do so
>             at their own skill level. We support only those
>             configurations that we
>             have engineered to meet specific targets.
>
>             OK, I'm off my soapbox now. :-)
>
>             73,
>             Scotty WA2DFI
>
>
>             On 5/3/19 8:36 AM, Tom McDermott wrote:
>             > Hi Scotty  - yes, having one synthesizer output feed two
>             clock drivers
>             > (one per RFM) would be fine.
>             > It's not a problem to have a phase offset, provided that
>             offset does not
>             > change.
>             >
>             > I don't think it's a problem to have fixed frequency on
>             the ADC clocks.
>             > That clock is not used to tune the receiver,
>             > the NCO in the DE is used to tune the receiver. So the
>             DE can implement
>             > one NCO for the synchronous case,
>             > and two NCOs for the case where we want the two
>             receivers to tune to
>             > different frequencies.
>             >
>             > While the spec describes SBC and DE, those two functions
>             could
>             > theoretically co-exist on on module (for
>             > example the Red Pitaya) where the FPGA does both the DE
>             function, and
>             > via the CPU-block does the SBC
>             > function.  A concern with having the FPGA do both is the
>             lack of
>             > sufficient computational capability in the
>             > limited CPU performance that's implemented on the FPGA
>             die.   So the
>             > spec tries to separate the functionality
>             > without saying where it's physically implemented.
>             >
>             > I do hope we restrict ourselves to one implementation. 
>             If folks can
>             > choose any SBC, any Receiver, etc. then there
>             > will be no end of "The software doesn't work on my
>             combination",  "My
>             > hardware receivers aren't coherent",
>             > "My hardware is overflowing buffers", "My hardware
>             doesn't support
>             > amplitude calibration"    ad nauseum.
>             > The data that is produced risks being largely garbage.
>             >
>             > -- Tom, N5EG
>             >
>             >
>             >
>             > On Fri, May 3, 2019 at 7:08 AM Scotty Cowling via
>             TangerineSDR
>             > <tangerinesdr at lists.tapr.org
>             <mailto:tangerinesdr at lists.tapr.org>
>             <mailto:tangerinesdr at lists.tapr.org
>             <mailto:tangerinesdr at lists.tapr.org>>> wrote:
>             >
>             >     Hi Tom,
>             >
>             >     Regarding the clocks, it seems that the AD5344
>             addresses this
>             >     problem with the cross-point switch and synchronous
>             dividers.
>             >
>             >     We can feed two outputs with one synthesizer if we
>             like, for a fixed
>             >     (and very small) phase offset. Routing one clock
>             output to two RF
>             >     Modules is problematic, especially with LVDS
>             outputs. Wouldn't this
>             >     work?
>             >
>             >     Alternatively, we could put a clock distribution
>             chip on the DE to
>             >     take one Clock Module output and distribute it to
>             FPGA, RFM1 and
>             >     RFM2, but then we lose the capability to clock the
>             RFMs at different
>             >     frequencies (for example, when one is a TX and one
>             is an RX).
>             >
>             >     One thing to keep in mind is that the FPGA has
>             multiple clock
>             >     inputs, so we can route one from each RFM, one from
>             the Clock Module
>             >     and one from a local oscillator and let the FPGA
>             code decide which
>             >     one to use. Of course, we would have to either let
>             the FPGA generate
>             >     the RFM clocks or route multiple clocks to each RFM
>             to clock the
>             >     ADCs directly. The reason I bring this up is that I
>             want to use an
>             >     on-board inexpensive oscillator (and no Clock Module
>             at all) as the
>             >     inexpensive, entry-level SDR. No, the performance
>             would not be as
>             >     good. But the cost would be much lower than with any
>             clock module,
>             >     and it would be selectable by simply loading the
>             FPGA with a
>             >     different image and unplugging the Clock Module.
>             >
>             >     The section 7 comment was not intended to dictate a
>             data flow path.
>             >     I just wanted to show that we need (and will have)
>             two GbE ports to
>             >     use that path in case we want to handle data in this
>             way (for
>             >     example, if the SBC cannot keep up). We can word it
>             in whatever way
>             >     makes it clear that we can do it either way. Isn't
>             your description
>             >     implementation specific also (i.e., data flows
>             through the SBC)?
>             >
>             >     Maybe for PSWS, the data has to flow through the
>             SBC? That is the
>             >     impression I got, and I am not sure we want to
>             restrict the
>             >     architecture that way, but we could if that is our
>             intent.
>             >
>             >     73,
>             >     Scotty WA2DFI
>             >
>             >
>             >     On 2019-05-03 04:09, Tom McDermott wrote:
>             >>     Thanks for the good comments, Scotty !
>             >>
>             >>     Section 3:  the ADC clocks MUST come from the same
>             one synthesizer
>             >>     output. If they
>             >>     come from two different outputs there is a big
>             problem...
>             >>
>             >>     The clocks to the two ADCs must be (and remain)
>             phase coherent.
>             >>     If the ADC clocks come
>             >>     from different outputs of the synthesizer, then
>             each time the
>             >>     synthesizer starts, there could
>             >>     be any phase difference between the two. Further
>             they could drift
>             >>     back and forth relative to one another
>             >>     a little bit because of the way synthesizers work. 
>             Thus the phase
>             >>     between them under
>             >>     the best case would be +/- 180 degrees.  At 30 MHz
>             that would
>             >>     represent +/- 45 degrees. This
>             >>     means that the baseband sampled signal at 30 MHz
>             would also have
>             >>     an unknown phase
>             >>     difference of +/- 45 degrees.  That completely
>             wrecks the ability
>             >>     to discriminate polarization.
>             >>
>             >>     There can be a difference in phase between the two
>             ADC clocks, but
>             >>     it must remain constant over time.
>             >>     Any difference will be calibrated out when the
>             antennas and
>             >>     feedlines are calibrated for phase delay.
>             >>     But then the phase differences cannot change.  The
>             only way I see
>             >>     to do that is to have one
>             >>     synthesizer output that is distributed to the two
>             ADC clocks.  If
>             >>     the ADCs are on different modules
>             >>     then one clock signal has to be routed to the two
>             of them in some
>             >>     manner (parallel, daisy-chained, etc.)
>             >>     Maybe there is some other way but it's difficult to
>             see.
>             >>
>             >>     This is one of those things that makes phase
>             coherent receivers
>             >>     very difficult, and why off-the-shelf units
>             >>     have to be carefully evaluated, as virtually none
>             of the ones I've
>             >>     seen address this problem properly.
>             >>     It's the Radio Astronomy problem.
>             >>
>             >>     Section 7:  The spec attempts to be
>             implementation-non-specific.
>             >>     Forcing a particular method
>             >>     to distribute Ethernet data may eliminate all other
>             potential
>             >>     solutions.  The approach you outline
>             >>     is a really good and elegant solution, but the spec
>             should not
>             >>     mandate it (it should allow it).
>             >>
>             >>     Section 8:  great comment.  I will restructure as
>             you outline.
>             >>
>             >>     -- Tom, N5EG
>             >>
>             >>
>             >>
>             >>     On Thu, May 2, 2019 at 5:27 PM Scotty Cowling via
>             TangerineSDR
>             >>     <tangerinesdr at lists.tapr.org
>             <mailto:tangerinesdr at lists.tapr.org>
>             <mailto:tangerinesdr at lists.tapr.org
>             <mailto:tangerinesdr at lists.tapr.org>>>
>             >>     wrote:
>             >>
>             >>         Hi Tom,
>             >>
>             >>         Here are my comments on your excellent document.
>             >>
>             >>         73,
>             >>         Scotty WA2DFI
>             >>
>             >>         Notes on PSWS Specification V 0.1 5 May 2019
>             >>
>             >>         Section 3.0
>             >>         Clock module will have 4 *programmable clock*
>             outputs:
>             >>         1. FPGA
>             >>         2. RF Module #1
>             >>         3. RF Module #2
>             >>         4. High-speed Reference Clock
>             >>
>             >>         In addition, two more outputs:
>             >>         5. 1 PPS timing
>             >>         6. 10MHz fixed reference
>             >>
>             >>         Clock outputs should be differential LVDS.
>             Single ended clocks
>             >>         will be
>             >>         too noisy, especially across a connector boundary.
>             >>
>             >>         The FPGA should not provide the ADC clocks,
>             they should come
>             >>         directly
>             >>         from the clock module. The ADC may supply a
>             source-synchronous
>             >>         data
>             >>         clock to the FPGA.
>             >>
>             >>         Section 4.0
>             >>         Magnetometer interface can be I2C, SPI, serial
>             UART or RS-485.
>             >>
>             >>         The magnetometer will almost certainly need to
>             be remotely
>             >>         mounted. In
>             >>         this case, RS-485 is recommended. We can
>             specify two-wires for
>             >>         communictions and two wires for power
>             (typically 5V).
>             >>
>             >>         Section 5.0
>             >>         It is not immediately clear that *each* RF
>             Module has two
>             >>         synchronous
>             >>         channels.
>             >>
>             >>         Note that the *pluggable filter* can be
>             bypassed with a
>             >>         jumper, making
>             >>         it optional. Maybe call it "optional pluggable
>             filter".
>             >>
>             >>         Section 6.
>             >>         The DE shall also be capable of sourcing or
>             sinking UDP data
>             >>         streams
>             >>         to/from any IP address, under direction of the
>             host processor.
>             >>
>             >>         The DE will have a three-port GbE switch,
>             connecting the FPGA,
>             >>         host PC
>             >>         and external network. How do you want to
>             explain this?
>             >>
>             >>         Section 7.0
>             >>         Processing the stream from the DE is optional,
>             since it may
>             >>         not be able
>             >>         to process this much data. The metadata tasks
>             will fall to the
>             >>         DE in
>             >>         this case.
>             >>
>             >>         The host will not always transmit the data to
>             the central
>             >>         server. The
>             >>         host may direct the DE to stream data directly
>             to the central
>             >>         server if
>             >>         it cannot process the volume or rate of data.
>             >>
>             >>         Section 8.
>             >>         I have been using "Command and Control
>             Protocol" for the
>             >>         protocol used
>             >>         between the central server and the host PC. I
>             have been using
>             >>         "Local SDR
>             >>         Protocol" for the protocol between the host PC
>             and the DE.
>             >>
>             >>         We could use "Remote Command and Control", or
>             "RCC" for the
>             >>         host<-->central server communications, and
>             "Local Command and
>             >>         Control"
>             >>         or "LCC" for host<-->DE communications.
>             Whatever we use, we
>             >>         should
>             >>         define it.
>             >>
>             >>         The Remote Command and Control section seems to
>             be missing
>             >>         (although you
>             >>         refer to it once in section 10). Even though we
>             don't know
>             >>         what the
>             >>         protocol    is, I think it should be mentioned
>             (as defined an
>             >>         a separate
>             >>         document?)
>             >>
>             >>         I think we need to make a clear distinction
>             between Remote C&C
>             >>         and Local
>             >>         SDR C&C. We will need security and maybe
>             encryption on the
>             >>         Remote C&C,
>             >>         but not so much on the Local SDR protocol.
>
>
>
>             -- 
>             TangerineSDR mailing list
>             TangerineSDR at lists.tapr.org
>             <mailto:TangerineSDR at lists.tapr.org>
>             http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190506/9c2005ee/attachment-0001.html>


More information about the TangerineSDR mailing list