[TangerineSDR] Benchmarking the Odroid N2 with Digital RF
Phil Erickson
phil.erickson at gmail.com
Thu Jul 11 20:13:41 EDT 2019
Hi Rick,
Have you been writing continuously? That is the use case I was referring to
73
Phil
On Thu, Jul 11, 2019 at 19:41 Rick W2GPS <w2gps at cnssys.com> wrote:
> Phil,
>
>
>
> Based on comments from the VLBI / VGOS correlator team at Haystack, I
> purchased a set of the following disks for my Network Attached Storage
> device. So far, it has run 24/7 for two years without any problem.
>
>
>
> Hard Disk Drives, HUH728080ALE600 0F23267 8TB 7200 RPM 128MB Cache SATA
> 6.0Gb/s 3.5" Helium Platform Enterprise
>
>
>
> Apparently, the Helium reduces “air resistance” inside the drive, making
> head movement easier and with less wear.
>
>
>
> Rick
>
> W2GPS
>
>
>
> *From:* TangerineSDR <tangerinesdr-bounces at lists.tapr.org> *On Behalf Of *Phil
> Erickson via TangerineSDR
> *Sent:* July 8, 2019 2:26 PM
> *To:* Engelke, Bill <bill.engelke at ua.edu>
> *Cc:* Phil Erickson <phil.erickson at gmail.com>; TAPR TangerineSDR Modular
> Software Defined Radio <tangerinesdr at lists.tapr.org>
> *Subject:* Re: [TangerineSDR] Benchmarking the Odroid N2 with Digital RF
>
>
>
> Hi Bill,
>
>
>
> Agree. One more point: There is no way around the fact that nothing -
> spinning media, SSDs, etc. - is going to survive repeated writes such as
> happens in a ring buffer situation. The RAMdisk for the first buffer
> doesn't solve the problem either - something has to shovel the data out of
> there onto one of these types of media.
>
>
>
> The manufacturers still make you pay > 10X per GB for "enterprise class"
> drives if you want the sustained performance under those stress levels. I
> can't see any other way around it. The social implications for the
> personal space weather station need to be thought on some, and a strategy
> for what will be done needs to be articulated, even if it's "sorry - you'll
> have to replace media every 2 years". But that conclusion allows for
> future technology progress to things like magnetoresistive storage, which
> is supposed to come along Real Soon Now (tm).
>
>
>
> 73
>
> Phil
>
>
>
> On Mon, Jul 8, 2019 at 2:14 PM Engelke, Bill <bill.engelke at ua.edu> wrote:
>
> Phil-
>
>
>
> Yes, I’m sure that would work. That would be the technique used to
> implement my last suggestion (i.e., to write 4 channels to each of 2
> separate HDDs), to overcome the speed limitation in the drive(s). I can
> also code up a test to try writing in 2 write streams (by grouping 4
> channels into each) to a single drive (will try tomorrow); on that, my
> previous experience working with hard drives suggests that this is likely
> to generate quite a bit of head movement, which detracts from speed –
> however, some drives are smart enough and quick enough to sort the writes
> in their buffer and write them in the lowest latency order… so your mileage
> may differ… I’ll publish the results of that test when I complete it.
>
>
>
> -thanks & 73- Bill
>
>
>
> *From:* Phil Erickson <phil.erickson at gmail.com>
> *Sent:* Monday, July 8, 2019 11:57 AM
> *To:* TAPR TangerineSDR Modular Software Defined Radio <
> tangerinesdr at lists.tapr.org>
> *Cc:* Engelke, Bill <bill.engelke at ua.edu>
> *Subject:* Re: [TangerineSDR] Benchmarking the Odroid N2 with Digital RF
>
>
>
> Hi Bill,
>
>
>
> There is an additional point regarding Digital RF that Ryan mentioned to
> me, and I believe to you as well in some earlier discussions.
>
>
>
> The single-threading requirement only holds for each channel
> (==directory) of Digital RF data. So if you're dealing with multiple
> channels, those could each be written in separate threads with (as required
> anyway) separate Digital_rf_write_object instances. So I suggest trying
> that as a way to further optimize the writing process, if you haven't
> already done so. Perhaps you could comment?
>
>
>
> 73
>
> Phil W1PJE
>
>
>
> On Mon, Jul 8, 2019 at 11:56 AM Engelke, Bill via TangerineSDR <
> tangerinesdr at lists.tapr.org> wrote:
>
> I received an Odroid N2 4GB model, shipped direct to me from Korea for $94
> (including shipping). I installed the following:
>
>
>
> 1. Image of Odroid Ubuntu from
> https://wiki.odroid.com/odroid-n2/os_images/ubuntu
> 2. HDF5 development package: libhdf5- serial dev
> 3. Digital RF from github
> 4. A few other packages necessary to run the above
>
>
>
> BENCHMARKS
>
>
>
> Digital rf comes with some examples; the interesting one for this is the c
> benchmark, which writes out about 1500 M of data using raw binary (to check
> write speed independent of Digital rf); then writes data using Digital RF
> simple, with checksum, and finally with compression. I tried writing to a
> Western Digital Elements spinning hard drive, and also writing to a ramdisk
> (the 4 GB Odroid has enough RAM than you can create a 2 GB ram disk and
> still have RAM left).
>
>
>
> Values below are all in MB/sec. I tried 3 runs; each gave different
> results, so the values are the average of the 3 runs.
>
>
>
> Raw binary write Digital RF simple Digital RF with
> checksum Digital RF with checksum/compression
>
> Western Digital HDD 198.53 187.03
> 116.63 11.14
>
> Ramdisk 1320.95 1020.58
> 270.72 11.76
>
>
>
>
>
> TESTING WITH RED PITAYA
>
>
>
> I compiled the John Melton (not to be confused with the 17th century poet John
> Milton <https://en.wikipedia.org/wiki/John_Milton>) program pihpsdr which
> works nicely with the Red Pitaya when it is in HPSDR emulation mode. The
> Red Pitaya runs at 48 ksps, one channel of I & Q, so I duplicate the I & Q
> data across 16 subchannels (8 channels each with 2 antennas), and write it
> out 4 times (4 X 48 = 197) to simulate the planned speed of Tangerine,
> writing out Digital RF.
>
>
>
> The Odroid N2 provides much better results than the Odroid XU4. It is no
> problem to achieve this data rate writing to ramdisk. Writing to the
> Western Digital Elements HDD occasionally does miss 6 to 10 buffers, so it
> appears that the HDD writing speed is slightly lower than needed for
> error-free operation. I have discussed this with the MIT Digital RF team;
> the software does not do internal threading (which makes sense because
> formatting and storing this data is a serial operation). My previous tests
> with forking off threads for the writes led to crashes due to Digital RF
> trying to access the same memory in multiple threads (I interpret this to
> mean the Digital RF is not designed as a thread-safe system). However,
> there are a couple of ways to deal with this:
>
>
>
> 1. Use a solid state drive. While technically possible, I have found
> that SDD in the size we need (~ 4 TB) is very expensive (>$800).
> 2. Set up 2 HDDs and write 4 channels to each. A pair of 2 TB
> spinning HDs can be had for less than $150. This might be a very practical
> solution, which I plan to test. Since the N2 has 4 USB-3 ports, you can put
> 2 HDDs on and still have mouse & keyboard connected.
>
>
>
> Another open question is: how will a cheap spinning HD do, after running
> it in ~ 100% duty cycle for a while? Probably not well; but remember that
> SSDs are not able to handle an infinite number of read/write cycles either;
> they also have a lifetime.
>
>
>
> -73- AB4EJ
>
> W. D. Engelke (Bill), Asst. Research Engr.
>
> Center for Advanced Public Safety
>
> Cyber Hall
>
> The University of Alabama
>
> Tuscaloosa, AL 35487
>
> Desk: (205) 348-7244
>
> Mobile: (205) 764-3099
>
>
>
> --
> TangerineSDR mailing list
> TangerineSDR at lists.tapr.org
> http://lists.tapr.org/mailman/listinfo/tangerinesdr_lists.tapr.org
>
>
>
>
> --
>
> ----
> Phil Erickson
> phil.erickson at gmail.com
>
>
>
>
> --
>
> ----
> Phil Erickson
> phil.erickson at gmail.com
>
--
----
Phil Erickson
phil.erickson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tapr.org/pipermail/tangerinesdr_lists.tapr.org/attachments/20190711/aa5ff175/attachment.html>
More information about the TangerineSDR
mailing list