Anyone want a contract job writing a SSD storage #Linux block device driver

A trusted friend has passed me a jobspec:

On behalf of a client, I’m looking for someone who can write a Linux block device driver. Know anyone suitable?

In particular, we have a need for something which can perform as many read IOPS per second to a SSD array as possible. The application is for read-only random access to a database of a tens of terabytes. The access is truly random and each useful transfer is about 1kbyte, though the structure of the underlying hardware means that we have to fetch a 4k block and discard the bytes we’re not interested in. Our current solution uses a couple of Linux servers, each multicore, each 32GB RAM and fitted with a total of four SATA controllers for a total of up to 128 drives. The bandwidth is easily sufficient for our needs but the IOPS available from the standard SCSI/SATA layer scales very badly. Each disk can support over 30k IOPS per second but we can’t get much more than 300k IOPS per server even when packed solid with disks. We need at least 1M IOPS per second and 3M would be better.

We’d like the driver written quickly and we are willing to pay a reasonable fee for it. Licensing would be at the choice of the author as long as we get unrestricted access to its source code. In practice we’d expect it to be GPLed.

Drop me a line if you’re interested.

Comments

One response to “Anyone want a contract job writing a SSD storage #Linux block device driver”

  1. I’ve seen benchmarks of Linux boxes which substantially exceed what they are getting, especially for random read-only on (apparently) smaller hardware. Of course benchmarks can mislead, and no one is lending me toys like this to play with.

    I’m curious how they figure out that the bottleneck is the SCSI/SATA layer scaling. High IOPS is notoriously temperamental across all OSes, and tends to bottleneck on things you don’t usually expect to be bottlenecks on like anything between the app and the raw device (RAID/LVM/File system), and in some cases the apps themselves lack sufficient parallelism (unlike the benchmarks).

Leave a Reply

Your email address will not be published. Required fields are marked *