If you feel tempted to validate my claim that reading from multiple files shouldn’t be fundamentally slower than reading

To tackle the API issue I have designed Glommio (formerly known as Scipio), a Direct I/O-oriented thread-per-core Rust library. Glommio builds upon io_uring and supports many of its advanced features like registered buffers and poll-based (no interrupts) completions to make Direct I/O shine. For the sake of familiarity Glommio does support buffered files backed by the Linux page cache in a way that resembles the standard Rust APIs (which we will use in this comparison), but it is oriented towards bringing Direct I/O to the limelight.

It is also very low level and raw. For it to be useful you need to accumulate I/O and dispatch in batches. This calls for a policy of when to do it, and some form of event-loop, meaning it works better with a framework that already provides the machinery for that.

http://news24.gruposio.es/ydd/v-ideos-Valencia-Basket-Barca-Lassa-v-en-gb-1dth-8.php

http://go.negronicocktailbar.com/npt/Video-Chicago-Bulls-Kings-v-en-us-1ywh-8.php

http://news7.totssants.com/zwo/video-Bragantino-Sao-Paulo-v-en-gb-1igu-1.php

http://go.negronicocktailbar.com/npt/videos-Fasil-Kenema-Sidama-Bunna-v-en-gb-1qua30122020-.php

http://news7.totssants.com/zwo/Video-barnechea-v-nublense-v-es-cl-1hag-22.php

http://news24.gruposio.es/ydd/Video-valencia-basket-v-barca-lassa-v-es-es-1uon-5.php

http://live-stream.munich.es/exd/videos-cska-moscow-v-saski-baskonia-v-es-es-1egz-18.php

http://go.negronicocktailbar.com/npt/Video-Fasil-Kenema-Sidama-Bunna-v-en-gb-1ohi30122020-19.php

http://go.negronicocktailbar.com/npt/Video-Fasil-Kenema-Sidama-Bunna-v-en-gb-1rsx-22.php

http://news7.totssants.com/zwo/Video-barnechea-v-nublense-v-es-cl-1csl-24.php

http://news24.gruposio.es/ydd/video-valencia-basket-v-barca-lassa-v-es-es-1uai-5.php

http://live-stream.munich.es/exd/Video-cska-moscow-v-saski-baskonia-v-es-es-1ine-18.php

http://go.negronicocktailbar.com/npt/video-Brisbane-Roar-Sydney-FC-v-en-gb-1caz-.php

http://live-stream.munich.es/exd/Video-Valencia-Basket-Barca-Lassa-v-en-gb-1bnj30122020-.php

http://live-stream.munich.es/exd/Video-Valencia-Basket-Barca-Lassa-v-en-gb-1icz30122020-12.php

http://go.negronicocktailbar.com/npt/video-Brisbane-Roar-Sydney-FC-v-en-gb-1frb30122020-24.php

http://news24.gruposio.es/ydd/Video-valencia-basket-v-barca-lassa-v-es-es-1jgs-25.php

http://news7.totssants.com/zwo/video-barnechea-v-nublense-v-es-cl-1eee-24.php

http://go.negronicocktailbar.com/npt/videos-Brisbane-Roar-Sydney-FC-v-en-gb-1qfw-15.php

http://news24.gruposio.es/ydd/v-ideos-valencia-basket-v-barca-lassa-v-es-es-1nts-5.php

ng jokes at this type of mom because “C’mon, let’s be real, nobody’s perfect.” And of course, that’s true. I get it. It’s annoying to see someone like that because, like, where do they find the time? Or maybe it’s annoying that their husband is oh so helpful, or maybe she doesn’t work—the luxury.

And in fact, should we try to emulate similar parameters using the Direct I/O API (4kB buffers, parallelism of one), the results would be disappointing, “confirming” our suspicion that Direct I/O is indeed, much slower.

Using multiple files is a step in the right direction, as it allows more effectively parallelism: while one reader is waiting, another can hopefully proceed. But if you are not careful you just end up amplifying one of the previous problems:

Direct I/O is full of caveats, and io_uring being a raw interface doesn’t even try (nor should it) to hide these problems: For example, memory must be properly aligned, as well as the positions where you are reading from.

To demonstrate these APIs glommio has an example program that issues I/O with various settings using all those APIs (buffered, Direct I/O, random, sequential) and evaluates their performance.

Since the issue is the OS page cache, what happens if you just open a file with Direct I/O, all else being equal? Unfortunately that likely won’t get faster either. But that’s because of our third and last issue:

Glommio’s read-ahead works differently than OS-level read ahead: our goal is to exploit parallelism and not just increase I/O sizes. Instead of consuming the entire read-ahead buffer and only then sending a request for a new batch, glommio dispatches a new request as soon as the contents of a buffer is fully consumed and will always try to keep a fixed number of buffers in-flight, as shown in the image below.

Streams are designed to be mostly familiar to Rust’s default AsyncRead, so it implements the AsyncRead trait and will still read data to a user buffer. All the benefits of Direct I/O-based scans are still there, but there is a copy between our internal read ahead buffers and the user buffer. That’s a tax on the convenience of using the standard API.

I have written extensively in the past about how revolutionary io_uring is. But being a fairly low level interface it is really just one piece of the API puzzle. Here’s why:

A file is seen as a sequential stream of bytes, and whether data is in-memory or not is transparent to the reader. Traditional APIs will wait until you touch data that is not resident to issue an I/O operation. The I/O operation may be larger than what the user requested due to read-ahead, but is still just one.

But as we discussed, glommio’s Direct I/O file streams can take an explicit read-ahead parameter. If active glommio will issue I/O requests in advance of the position being currently read, to exploit the device’s parallelism.

Random access files take a position as an argument, meaning there is no need to maintain a seek cursor. But more importantly: they don’t take a buffer as a parameter. Instead, they use io_uring’s pre-registered buffer area to allocate a buffer and return to the user. That means no memory mapping, no copying to the user buffer — there is only a copy from the device to the glommio buffer and the user get a reference counted pointer to that. And because we know this is random I/O, there is no need to read more data than what was requested.

That is certainly not bad considering that this file doesn’t fit in memory, but here the merits are all on Intel Optane’s out-of-this world performance and the io_uring backend. It still has an effective parallelism of one whenever I/O is dispatched and although the OS page size is 4kB, read-ahead allow us to effectively increase the I/O size.