<div dir="ltr"><div>I updated digital_rf (the MIT HDF5 based package) in gnuradio to</div><div>the latest, 2.6.1 running under gnuradio 3.7.13.5 and ran some tests.</div><div><br></div><div>The gr-hpsdr module tries to do a good job of accounting for dropped packets.</div><div><br></div><div>1. SATA drive: 4ch x 192k seems to keep up with HDF-5 coding without packet loss.</div><div>2. SATA drive: 4ch x 384k drops packets at about a 0.06% loss rate.</div><div>3. I set up a 8G ramdisk in Linux, then streamed 4ch x 384k to that, and the loss rate is zero</div><div>for a few minutes. Stopped at 7.1G before exceeding the ramdisk size.</div><div><br></div><div>So the updated HDF5 coder seems to work better than the V1.8 version that I had.</div><div>But the SATA disk does seem to be causing some backups long enough to lose data when</div><div>running at a 12.5 MB/s rate.</div><div><br></div><div>-- Tom, N5EG</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Wed, May 29, 2019 at 7:43 AM Tom McDermott <<a href="mailto:tom.n5eg@gmail.com">tom.n5eg@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"><div dir="ltr"><div>Hi Bill -- thank you for running those benchmarks !</div><div>Hopefully they hold up for sustained writes.</div><div><br></div><div>I need to figure out why my system with HDF5 + gnuradio performs so poorly in comparison.</div><div><br></div><div>-- Tom, N5EG</div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Wed, May 29, 2019 at 7:08 AM Engelke, Bill via TangerineSDR <<a href="mailto:tangerinesdr@lists.tapr.org" target="_blank">tangerinesdr@lists.tapr.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid">
<div lang="EN-US">
<div class="gmail-m_1317044697636495356gmail-m_2887597405448876631WordSection1">
<p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt">Benchmarking Digital_rf on Odroid<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt">There have been some deep concerns about the ability of a SBC to keep up with saving spectrum data using HDF5 (here using the MIT Digital_rf package, which uses HDF5 internally).
I installed Digital_rf on my Odroid XU4 and ran the benchmarks that are included in the Examples folder.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt">Writing to MicroSd card in the XU4 (SanDisk Ultra Plus, claimed speed 80 MB/sec), benchmark-reported speeds<u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>1.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing raw binary data directly 133.36 MB/sec</span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>2.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing “simple single write” in HDF5 154.14 MB/sec </span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>3.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing “simple single write” in HDF5 with checksum 71.29 MB/sec</span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>4.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing “ “ “ with checksum + compression L1 8.01 MB/sec </span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Unexpected results here:</span><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">a. HDF5 "simple write" is faster than writing raw binary</span><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">b. top two speeds are faster than the claimed speed of the MicroSD card. Maybe the benchmark is measuring a burst rate during just the HDF5 processing (?) Maybe there is some caching or buffering going on.</span><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Writing to Western Digital USB 3 (spinning) external 4TB hard drive (incl. 64 MB internal cache) – benchmark-reported speeds</span><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>1.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing raw binary data directly 103.66 MB/sec</span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>2.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing “simple single write” in HDF5 96.04 MB/sec </span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>3.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing “simple single write” in HDF5 with checksum 55.02 MB/sec</span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:0in;margin-left:33pt">
<u></u><span style="font-family:"Times New Roman",serif"><span>4.<span style="font:7pt "Times New Roman";font-size-adjust:none;font-stretch:normal">
</span></span></span><u></u><span style="color:rgb(31,73,125)">Writing “ “ “ with checksum + compression L1 7.67 MB/sec </span><span style="font-family:"Times New Roman",serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">These results make a bit more sense than the MicroSD result. I can't find a claimed transfer speed for this hard drive on Western Digital's site.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">IF these results are representative (a big “if”) – then even a 4 TB spinning hard drive will be able to keep up with the PSWS data write rate. If anyone can confirm, reproduce, add to, or refute these results,
I am most interested to know…</span><span style="font-family:"Times New Roman",serif;font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">-73-<u></u><u></u></p>
<p class="MsoNormal">W. D. Engelke (Bill), Asst. Research Engr. AB4EJ<u></u><u></u></p>
<p class="MsoNormal">Center for Advanced Public Safety<u></u><u></u></p>
<p class="MsoNormal">Cyber Hall<u></u><u></u></p>
<p class="MsoNormal">The University of Alabama<u></u><u></u></p>
<p class="MsoNormal">Tuscaloosa, AL 35487<u></u><u></u></p>
<p class="MsoNormal">Desk: (205) 348-7244<u></u><u></u></p>
<p class="MsoNormal">Mobile: (205) 764-3099<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
-- <br>
TangerineSDR mailing list<br>
<a href="mailto:TangerineSDR@lists.tapr.org" target="_blank">TangerineSDR@lists.tapr.org</a><br>
<a href="http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org" target="_blank" rel="noreferrer">http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org</a><br>
</blockquote></div>
</blockquote></div>