[TangerineSDR] Benchmarking Digital RF (HDF5)

[Tom McDermott]

I updated digital_rf (the MIT HDF5 based package) in gnuradio to
the latest, 2.6.1  running under gnuradio 3.7.13.5  and ran some tests.

The gr-hpsdr module tries to do a good job of accounting for dropped
packets.

1. SATA drive: 4ch x 192k seems to keep up with HDF-5 coding without packet
loss.
2. SATA drive:  4ch x 384k drops packets at about a 0.06% loss rate.
3. I set up a 8G ramdisk in Linux, then streamed 4ch x 384k to that, and
the loss rate is zero
for a few minutes.  Stopped at 7.1G before exceeding the ramdisk size.

So the updated HDF5 coder seems to work better than the V1.8 version that I
had.
But the SATA disk does seem to be causing some backups long enough to lose
data when
running at a 12.5 MB/s rate.

-- Tom, N5EG






On Wed, May 29, 2019 at 7:43 AM Tom McDermott <tom.n5eg at gmail.com> wrote:

> Hi Bill -- thank you for running those benchmarks !
> Hopefully they hold up for sustained writes.
>
> I need to figure out why my system with HDF5 + gnuradio performs so poorly
> in comparison.
>
> -- Tom, N5EG
>
>
>
>
> On Wed, May 29, 2019 at 7:08 AM Engelke, Bill via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
>> Benchmarking Digital_rf on Odroid
>>
>>
>>
>> There have been some deep concerns about the ability of a SBC to keep up
>> with saving spectrum data using HDF5 (here using the MIT Digital_rf
>> package, which uses HDF5 internally). I installed Digital_rf on my Odroid
>> XU4 and ran the benchmarks that are included in the Examples folder.
>>
>>
>>
>> Writing to MicroSd card in the XU4 (SanDisk Ultra Plus, claimed speed 80
>> MB/sec), benchmark-reported speeds
>>
>> 1.                     Writing raw binary data directly
>>                                    133.36  MB/sec
>>
>> 2.                     Writing “simple single write” in HDF5
>>                                 154.14 MB/sec
>>
>> 3.                     Writing “simple single write” in HDF5 with
>> checksum        71.29 MB/sec
>>
>> 4.                     Writing      “        “    “  with checksum +
>> compression L1        8.01 MB/sec
>>
>> Unexpected results here:
>>
>> a. HDF5 "simple write" is faster than writing raw binary
>>
>> b. top two speeds are faster than the claimed speed of the MicroSD card.
>> Maybe the benchmark is measuring a burst rate during just the HDF5
>> processing (?) Maybe there is some caching  or buffering going on.
>>
>>
>>
>> Writing to Western Digital USB 3 (spinning) external 4TB hard drive
>> (incl. 64 MB internal cache) – benchmark-reported speeds
>>
>> 1.                     Writing raw binary data directly
>>                                    103.66  MB/sec
>>
>> 2.                     Writing “simple single write” in HDF5
>>                          96.04 MB/sec
>>
>> 3.                     Writing “simple single write” in HDF5 with
>> checksum          55.02 MB/sec
>>
>> 4.                     Writing      “        “    “  with checksum +
>> compression L1        7.67 MB/sec
>>
>> These results make a bit more sense than the MicroSD result. I can't find
>> a claimed transfer speed for this hard drive on Western Digital's site.
>>
>>
>>
>> IF these results are representative (a big “if”) – then even a 4 TB
>> spinning hard drive will be able to keep up with the PSWS data write rate.
>> If anyone can confirm, reproduce, add to, or refute these results, I am
>> most interested to know…
>>
>>
>>
>> -73-
>>
>> W. D. Engelke (Bill), Asst. Research Engr. AB4EJ
>>
>> Center for Advanced Public Safety
>>
>> Cyber Hall
>>
>> The University of Alabama
>>
>> Tuscaloosa, AL 35487
>>
>> Desk: (205) 348-7244
>>
>> Mobile: (205) 764-3099
>>
>>
>> --
>> TangerineSDR mailing list
>> TangerineSDR at lists.tapr.org
>> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190529/22acdc89/attachment.html>