Gammacam External Storage

Storage prices have dropped drastically since I built my Gamma Camera array. As I mentioned a little while ago, I’m interested in USB storage for the next iteration.

In gammacam P6 I have 6 Raspberry Pi 4B nodes. The microSD card holds the OS and also the imagery data. I have heard that microSD card speeds are slower than USB options. And lately I’ve noticed that prices for some pretty fast storage options have been falling. On the high-end I’ve been paying attention to PCIe 5.0 drive availability and prices. As these start to become available at the top end of the market the previous generations are dropping in price. The maximum bus speed doesn’t tell the whole story, as usual. The market seems to be segmented into high and low performance drives from each generation of PCIe.

What I’m more interested in is the best of the rest. A drive that’s easy to obtain, easy to connect, and low enough cost that users would be able to easily buy one for each node in a camera array. Other authours have done some pretty thorough benchmarks as well and found specific features that make a performance difference for some applications.

PCIe and USB on the RPi 4B

The Raspberry Pi 4B has 2 USB3.0 ports and 2 USB 2.0 ports according to the specifications. These are only the type A ports. We’re going to continue to ignore the USB C port. When it comes to USB 3, the version numbers are very confusing so we have to be specific. USB3.0 Superspeed has been renamed a couple times. It’s also called USB 3.1 Gen 1 or USB 3.2 Gen 1x1 - Wikipedia has the details. This is why PC ports are often just labelled with the speed: USB 5Gbps.

The USB ports are connected to the Raspberry Pi 4B’s PCI Express bus. This bus wouldn’t be able to keep up with anything close to the fastest drives aimed at the PC market though. The Raspberry Pi 4B uses the VL805 USB controller. VIA Labs describes this chip as a USB host controller which uses a single lane PCI Express Gen2 bus to provide USB 3 connection at 5 Gb/s. A single lane of PCIe Gen2 is also 5 Gb/s. So connecting to storage over USB doesn’t cost any theoretical bandwidth.

Write bandwith for camera data

So we can’t hope for more than 5Gb/s - approximately 500 MB/s. Many other applications are strongly affected by read speeds but Gamma Camera tries to write a lot of data to disk. Just how much data does it want to write though? Camera selection, framerate, and resolution are really up to the user application. Ultimately Gamma Camera delegates to the libcamera apps or whatever else the user plugs in. It’s not hard to find an upper limit for data coming from the Raspberry Pi camera though. The Broadcom SoC connects to the camera through the CSI-2 D-PHY. There are two camera connections: the first has two lanes and the second has four. A carrier board for the CM4 can connect all lanes of both CAM0 and CAM1. The RPi 4B CSI connector goes to CAM1 but only connects two of the lanes. Each lane provides 1 Gb/s so we can’t have more than 2 Gb/s on the RPi 4B or 6 Gb/s combined on the RPi CM4. These should be very conservative upper limits. It’s more realistic to consider compressed video or still frames which will be much smaller and likely under 40MB/s or 320 Mb/s . This is kind of a guess based on discussions like this.

USB flash drive prices have been falling but for a long time I wasn’t seeing good options with USB 3.1.

USB flash drives and USB external drives should also be thrown into the mix here since USB is going to be the most common interface.

There’s a handy tracking site called Disk Prices with Amazon Affiliate links to the product listings.

OS and data or just OS

OS performance is impacted by random RW performance more than sequential. Writing out videos or images to the disk should reflect sequential write speeds only. Using external storage only for data recording is a simple upgrade if you’re already running from a fast microSD card. However looking at the cost of materials there’s now a microSD card and the external storage. So there’s an argument to be made that when building from scratch it’s less costly to have only the USB-connected external storage. Then there’s no need to buy a microSD card for each node.

Testing

Before getting started with the USB drive on the RPi I did a couple quick tests with Crystal DiskMark on my desktop. The initial results seem to indicate that the USB drive does indeed perform much faster.

A 32GB Samsung Evo card¹ gave these results:

Sequential 1MB reads came in around 45 MB/s, writes were under 40MB/s. Random reads were around 10MB/s and writes are around 4MB/s.

And the Evo Plus cards really do look to be twice as fast - but only for large sequential reads!

Testing the 128GB Samsung fit USB drive gave these results on my desktop:

This flash drive seems to be substantially faster in a few categories. The biggest numbers are sequential read speeds. The most relevant for our recording application are those write speeds - they’re 2-5 times faster than the microSD card! Random write speed of 200 Mb/s (25 MB/s) is 1/10th the raw CSI-2 data rate on the RPi 4B. Using H.264 for an average compression rate of 1:10 isn’t far fetched so this drive should be able to keep up pretty well.

What’s next

Most of this research happened a few months back but the Raspberry Pi 5 was announced this week. It looks like some factors might change with the next model since the RPi 5 will have faster microSD card support. It’ll be interesting to retest and see what changes and what remains the same.

• gammacam