qiurui
/
Centos-kernel-stream-9
mirror of https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

io_uring/rw: ensure poll based multishot read retries appropriately 

JIRA: https://issues.redhat.com/browse/RHEL-27755

commit c79f52f0656eeb3e4a12f7f358f760077ae111b6
Author: Jens Axboe <axboe@kernel.dk>
Date:   Sat Jan 27 13:44:58 2024 -0700

    io_uring/rw: ensure poll based multishot read retries appropriately
    
    io_read_mshot() always relies on poll triggering retries, and this works
    fine as long as we do a retry per size of the buffer being read. The
    buffer size is given by the size of the buffer(s) in the given buffer
    group ID.
    
    But if we're reading less than what is available, then we don't always
    get to read everything that is available. For example, if the buffers
    available are 32 bytes and we have 64 bytes to read, then we'll
    correctly read the first 32 bytes and then wait for another poll trigger
    before we attempt the next read. This next poll trigger may never
    happen, in which case we just sit forever and never make progress, or it
    may trigger at some point in the future, and now we're just delivering
    the available data much later than we should have.
    
    io_read_mshot() could do retries itself, but that is wasteful as we'll
    be going through all of __io_read() again, and most likely in vain.
    Rather than do that, bump our poll reference count and have
    io_poll_check_events() do one more loop and check with vfs_poll() if we
    have more data to read. If we do, io_read_mshot() will get invoked again
    directly and we'll read the next chunk.
    
    io_poll_multishot_retry() must only get called from inside
    io_poll_issue(), which is our multishot retry handler, as we know we
    already "own" the request at this point.
    
    Cc: stable@vger.kernel.org
    Link: https://github.com/axboe/liburing/issues/1041
    Fixes: fc68fcda0491 ("io_uring/rw: add support for IORING_OP_READ_MULTISHOT")
    Signed-off-by: Jens Axboe <axboe@kernel.dk>

Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
This commit is contained in:
Jeff Moyer 2024-01-27 13:44:58 -07:00
parent fb1b2f09ee
commit c76e4594b6
2 changed files with 18 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
io_uring/poll.h
View File

@ -24,6 +24,15 @@ struct async_poll {
struct io_poll *double_poll;
};
/*
* Must only be called inside issue_flags & IO_URING_F_MULTISHOT, or
* potentially other cases where we already "own" this poll request.
*/
static inline void io_poll_multishot_retry(struct io_kiocb *req)
{
atomic_inc(&req->poll_refs);
}
int io_poll_add_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe);
int io_poll_add(struct io_kiocb *req, unsigned int issue_flags);

10
io_uring/rw.c
View File

@ -18,6 +18,7 @@
#include "opdef.h"
#include "kbuf.h"
#include "rsrc.h"
#include "poll.h"
#include "rw.h"
struct io_rw {
@ -962,8 +963,15 @@ int io_read_mshot(struct io_kiocb *req, unsigned int issue_flags)
if (io_fill_cqe_req_aux(req,
issue_flags & IO_URING_F_COMPLETE_DEFER,
ret, cflags | IORING_CQE_F_MORE)) {
if (issue_flags & IO_URING_F_MULTISHOT)
if (issue_flags & IO_URING_F_MULTISHOT) {
/*
* Force retry, as we might have more data to
* be read and otherwise it won't get retried
* until (if ever) another poll is triggered.
*/
io_poll_multishot_retry(req);
return IOU_ISSUE_SKIP_COMPLETE;
}
return -EAGAIN;
}
}