The operating system reads data in page granularity, meaning it can only read at a minimum 4kB at a time. That means if

I have written extensively in the past about how revolutionary io_uring is. But being a fairly low level interface it is really just one piece of the API puzzle. Here’s why:

Streams are designed to be mostly familiar to Rust’s default AsyncRead, so it implements the AsyncRead trait and will still read data to a user buffer. All the benefits of Direct I/O-based scans are still there, but there is a copy between our internal read ahead buffers and the user buffer. That’s a tax on the convenience of using the standard API.

Using multiple files is a step in the right direction, as it allows more effectively parallelism: while one reader is waiting, another can hopefully proceed. But if you are not careful you just end up amplifying one of the previous problems:

A file is seen as a sequential stream of bytes, and whether data is in-memory or not is transparent to the reader. Traditional APIs will wait until you touch data that is not resident to issue an I/O operation. The I/O operation may be larger than what the user requested due to read-ahead, but is still just one.

But as we discussed, glommio’s Direct I/O file streams can take an explicit read-ahead parameter. If active glommio will issue I/O requests in advance of the position being currently read, to exploit the device’s parallelism.

http://live-stream.munich.es/twr/video-Al-Rayyan-Al-Gharafa-v-en-gb-1ykx30122020-24.php

http://live07.colomboserboli.com/niy/video-Al-Rayyan-Al-Gharafa-v-en-gb-1khz-23.php

http://news24.gruposio.es/ktn/Video-Qatar-SC-Al-Kharaitiyat-v-en-gb-1rxf-4.php

http://news24.gruposio.es/ktn/video-Al-Rayyan-Al-Gharafa-v-en-gb-1ztg-.php

http://go.acaps.cat/vac/Video-Al-Rayyan-Al-Gharafa-v-en-gb-1upd30122020-24.php

http://live-stream.munich.es/twr/video-Al-Rayyan-Al-Gharafa-v-en-gb-1sfb-16.php

http://live07.colomboserboli.com/niy/Video-fenerbahce-v-alanyaspor-v-tr-tr-1hvo-5.php

http://news24.gruposio.es/ktn/video-Al-Rayyan-Al-Gharafa-v-en-gb-1ail-3.php

http://news24.gruposio.es/ktn/v-ideos-Al-Rayyan-Al-Gharafa-v-en-gb-1fbe-16.php

http://live07.colomboserboli.com/niy/video-fenerbahce-v-alanyaspor-v-tr-tr-1zsl-3.php

http://go.acaps.cat/vac/Video-fenerbahce-v-alanyaspor-v-tr-tr-1bji-1.php

http://live-stream.munich.es/twr/video-fenerbahce-v-alanyaspor-v-tr-tr-1glu-16.php

http://live-stream.munich.es/twr/videos-fenerbahce-v-alanyaspor-v-tr-tr-1xhw-1.php

http://go.acaps.cat/vac/videos-fenerbahce-v-alanyaspor-v-tr-tr-1rlx-18.php

http://live07.colomboserboli.com/niy/Video-fenerbahce-v-alanyaspor-v-tr-tr-1ktc-19.php

http://news24.gruposio.es/ktn/Video-fenerbahce-v-alanyaspor-v-tr-tr-1zyu-7.php

http://go.acaps.cat/vac/v-ideos-fenerbahce-v-alanyaspor-v-tr-tr-1xoy-1.php

http://main.dentisalut.com/mqk/video-Al-Wehda-Al-Fateh-v-en-gb-1plz-24.php

http://news24.gruposio.es/ktn/v-ideos-fenerbahce-v-alanyaspor-v-tr-tr-1jlk-5.php

http://live07.colomboserboli.com/niy/videos-fenerbahce-v-alanyaspor-v-tr-tr-1efo-7.php

re today in part because of what you’ve been saying about yourself. Words are like seeds. When you speak something out, you give life to what you’re saying. If you continue to say it, eventually that can become a reality . . . you are prophesying your future.”

Direct I/O is full of caveats, and io_uring being a raw interface doesn’t even try (nor should it) to hide these problems: For example, memory must be properly aligned, as well as the positions where you are reading from.

To tackle the API issue I have designed Glommio (formerly known as Scipio), a Direct I/O-oriented thread-per-core Rust library. Glommio builds upon io_uring and supports many of its advanced features like registered buffers and poll-based (no interrupts) completions to make Direct I/O shine. For the sake of familiarity Glommio does support buffered files backed by the Linux page cache in a way that resembles the standard Rust APIs (which we will use in this comparison), but it is oriented towards bringing Direct I/O to the limelight.

If you feel tempted to validate my claim that reading from multiple files shouldn’t be fundamentally slower than reading from a single file, you may end up proving yourself right, but only because read amplification increased by a lot the amount of data effectively read.

And in fact, should we try to emulate similar parameters using the Direct I/O API (4kB buffers, parallelism of one), the results would be disappointing, “confirming” our suspicion that Direct I/O is indeed, much slower.

It is also very low level and raw. For it to be useful you need to accumulate I/O and dispatch in batches. This calls for a policy of when to do it, and some form of event-loop, meaning it works better with a framework that already provides the machinery for that.

To demonstrate these APIs glommio has an example program that issues I/O with various settings using all those APIs (buffered, Direct I/O, random, sequential) and evaluates their performance.

Random access files take a position as an argument, meaning there is no need to maintain a seek cursor. But more importantly: they don’t take a buffer as a parameter. Instead, they use io_uring’s pre-registered buffer area to allocate a buffer and return to the user. That means no memory mapping, no copying to the user buffer — there is only a copy from the device to the glommio buffer and the user get a reference counted pointer to that. And because we know this is random I/O, there is no need to read more data than what was requested.

That is certainly not bad considering that this file doesn’t fit in memory, but here the merits are all on Intel Optane’s out-of-this world performance and the io_uring backend. It still has an effective parallelism of one whenever I/O is dispatched and although the OS page size is 4kB, read-ahead allow us to effectively increase the I/O size.

Since the issue is the OS page cache, what happens if you just open a file with Direct I/O, all else being equal? Unfortunately that likely won’t get faster either. But that’s because of our third and last issue: