[TangerineSDR] PSWS Time Stamping Concept: maybe we should use the 128-bit time

Engelke, Bill bill.engelke at ua.edu
Tue Oct 22 12:18:01 EDT 2019


Phil,
Here’s how we got into this discussion… a suggestion was made to use NTP Standard time (see: https://tools.ietf.org/html/rfc5905 ).   (I guess, another classic example of there being multiple “standards.”)   Then there is the additional question of when a 64-bit time will roll over, and what you do then. If you use the 128-bit NTP standard, the rollover problem goes away. Using NTP time also lets you use the NTP system which (as I understand) lets you set your system clock very accurately.

When I go back and look at the code to create a Digital RF file, it seems to depend on getting Unix time. There are algorithms to convert NTP to Unix time, and this would not be computationally expensive in our case, because we would do this conversion only at start of data collection. We could also have the convention that we always start data collection exactly at the top of a second, so we don’t have to deal with fractional seconds.

Having said that, it seems that there will be a requirement down the road for Digital RF to deal with the 64-bit rollover issue, and this doesn’t speak to that. So, a valid approach could be that for Phase 1, we simply use Unix time for everything, with the idea that we deal with all rollover issues in a later phase. I don’t recall that this question came up during the telco; and I don’t have a strong preference either way.

Any other thoughts?   Anyone?


-73- Bill







From: TangerineSDR <tangerinesdr-bounces at lists.tapr.org> On Behalf Of Phil Erickson via TangerineSDR
Sent: Tuesday, October 22, 2019 10:18 AM
To: TAPR TangerineSDR Modular Software Defined Radio <tangerinesdr at lists.tapr.org>
Cc: Phil Erickson <phil.erickson at gmail.com>
Subject: Re: [TangerineSDR] PSWS Time Stamping Concept: maybe we should use the 128-bit time

Hi all,

   My experience is that floating point times can cause problems, and that integer time stamps are much easier to deal with.  Are these times integer or float?

  Also, is there a compelling reason why there is one time stamp format in the data packet, and another one for the HDF5 DigitalRF recorded data?  Seems like extra work that might be unnecessary, since both have to keep as coherent a time stamp as manageable.  That was part of our discussion on the telecon regarding the data time epoch - the idea on the phone would be that time is kept as (64 bit) time as seconds since the Unix epoch, plus a fractional second say in picoseconds or nanoseconds or .. .

73
Phil W1PJE

On Tue, Oct 22, 2019 at 11:13 AM Tom McDermott via TangerineSDR <tangerinesdr at lists.tapr.org<mailto:tangerinesdr at lists.tapr.org>> wrote:
Hi All,     I think the time/date is already coded into the HDF-5 metadata.
Thus the packet time stamping does not need to resolve that.

However having at least whole seconds in the data packet is  probably good to
avoid one-second boundary issues.  Thus, 64-bit time (32-bits seconds +
32 bits fractional seconds) within the data packet itself should be adequate.

-- Tom, N5EG



On Mon, Oct 21, 2019 at 4:05 PM Scotty Cowling via TangerineSDR <tangerinesdr at lists.tapr.org<mailto:tangerinesdr at lists.tapr.org>> wrote:
Hi Bill,

128-bit format should be no problem. Do you know what format VITA-49 uses?

I don't like rollovers either. And I use lots more memory than 1MB.

73,
Scotty WA2DFI
On 2019-10-21 12:56, Engelke, Bill via TangerineSDR wrote:
Tom – thanks for bringing this up, I definitely should have included it.  Referring to https://tools.ietf.org/html/rfc5905#section-6 .

After thinking about this, I’m wondering:   should we consider using the full 128-bit NTP date format (instead of the NTP Short format of 64 bits), with the idea that we then don’t have to cope with the year 2036 rollover?  The Era value will take care of it.  Since our time stamps don’t occur all that often, it doesn’t seem to me like a big cost.  Our experience has been that these software projects take on a life of their own and last much longer than anyone ever expects; thinking of it in that way, 2037 is right around the corner.

Would anyone else like to weigh in on this matter?

73- Bill, AB4EJ



From: TangerineSDR <tangerinesdr-bounces at lists.tapr.org><mailto:tangerinesdr-bounces at lists.tapr.org> On Behalf Of Tom McDermott via TangerineSDR
Sent: Friday, October 18, 2019 4:29 PM
To: TAPR TangerineSDR Modular Software Defined Radio <tangerinesdr at lists.tapr.org><mailto:tangerinesdr at lists.tapr.org>
Cc: Tom McDermott <tom.n5eg at gmail.com><mailto:tom.n5eg at gmail.com>
Subject: Re: [TangerineSDR] PSWS Time Stamping Concept

HI Bill - thanks for documenting this.   The format of the 64-bit time stamp
should be referenced.  My recommendation is NTP format.  We cannot use all of
the precision that affords, and thus may want to consider setting the LSBs to zero below
some accuracy limit.

The format is documented in: IETF RFC 5905 Section 6, "NTP Timestamp format", figure 3.
page 13.
        https://tools.ietf.org/html/rfc5905
There are standard library tools to manipulate it.  Whilst UNIX time will rollover in 2037,
my view is that standard methods, procedures, and standard code will emerge to consistently handle it.
If we adopt a proprietary format, then custom code would otherwise need to be crafted.

The fractional portion of one second is encoded as a 32-bit integer. The LSB represents 0.233 nanoseconds.
We could consider setting the 5 LSBs to zero for the 50 nanosecond-ish accuracy.  Then use those LSBs
later if some GPS receiver in the future had better accuracy.

-- Tom, N5EG









On Fri, Oct 18, 2019 at 1:09 PM Scotty Cowling via TangerineSDR <tangerinesdr at lists.tapr.org<mailto:tangerinesdr at lists.tapr.org>> wrote:
Hi Bill,

Just to be clear, if the number of channels does not divide evenly into 1024, then a packet might not start with channel 0 I/Q samples.

Is there a requirement that the time stamp immediately precede channel 0 I/Q data? For example, a packet could look like this:

CH0_I(0), CH0_Q(0), CH1_I(0), CH1_Q(0), CH2_I(0), CH2_Q(0), CH0_I(1), CH0_Q(1), CH1_1(1), CH1_Q(1), CH2_I(1), CH2_Q(1), CH0_I(2)...
...CH2_I(339), CH2_Q(339), CH0_I(340), CH0_Q(340)

So you would start the next packet like this:
CH1_I(340), CH1_Q(340), CH2_I(340), CH2_Q(340), CH0_I(341), CH0_Q(341)...

If I put the time stamp at the beginning:
<sync><time stamp>CH1_I(340), CH1_Q(340), CH2_I(340), CH2_Q(340), CH0_I(341), CH0_Q(341)...

Then the time stamp would apply to the first and second I/Q pairs (CH1 and CH2) as well as to the last I/Q pair of the previous packet (CH0).

If I always put the time stamp before CH0, then the time stamp would apply to the last I/Q pair of one packet and also to the first two I/Q pairs of the next packet.

So are the time stamps always before CH0, or can they be anywhere?  I think for proper synchronization, they will have to be before CH0 only.

Also, while it is clear that time stamps are sent periodically, that period is not specified anywhere. I think we need to specify that, don't we? Maximum count between timestamps? Maximum number of packets?

Did you want to expand on the two commands (or methods) used by the SBC to set the two times (GPSDO and "best effort")?  We talked about an "arm" command that causes the time to be set on the next 1 PPS transition and an "immediate" command that sets the time immediately upon reception of the command.

73,
Scotty WA2DFI
On 2019-10-17 17:37, Engelke, Bill via TangerineSDR wrote:
To all:
Attached is our proposed concept for Time Stamping for PSWS data – for your review and comment.
Note that this is primarily for the case where raw I/Q data is being stored in Digital RF format.
Data recording will be a bit different in the low-bandwidth case where the I/Q data is to be processed by GNURadio running on the SBC, and FFT (waterfall) results are uploaded to the database.

Dave: please post to TangerineSDR.com

TNX ES 73 - W. D. Engelke (Bill), AB4EJ
Center for Advanced Public Safety
Cyber Hall
The University of Alabama
Tuscaloosa, AL 35487
Desk: (205) 348-7244
Mobile: (205) 764-3099



--
TangerineSDR mailing list
TangerineSDR at lists.tapr.org<mailto:TangerineSDR at lists.tapr.org>
http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org


--
TangerineSDR mailing list
TangerineSDR at lists.tapr.org<mailto:TangerineSDR at lists.tapr.org>
http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
--
TangerineSDR mailing list
TangerineSDR at lists.tapr.org<mailto:TangerineSDR at lists.tapr.org>
http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org


--
----
Phil Erickson
phil.erickson at gmail.com<mailto:phil.erickson at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20191022/34786e9f/attachment-0001.html>


More information about the TangerineSDR mailing list