2014-05-14 18:06:36 +00:00
|
|
|
/* Uncancelable versions of cancelable interfaces. Linux/NPTL version.
|
2025-01-01 18:14:45 +00:00
|
|
|
Copyright (C) 2003-2025 Free Software Foundation, Inc.
|
2003-04-19 16:57:17 +00:00
|
|
|
This file is part of the GNU C Library.
|
|
|
|
|
|
|
|
The GNU C Library is free software; you can redistribute it and/or
|
|
|
|
modify it under the terms of the GNU Lesser General Public
|
|
|
|
License as published by the Free Software Foundation; either
|
|
|
|
version 2.1 of the License, or (at your option) any later version.
|
|
|
|
|
|
|
|
The GNU C Library is distributed in the hope that it will be useful,
|
|
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
Lesser General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU Lesser General Public
|
2012-02-09 23:18:22 +00:00
|
|
|
License along with the GNU C Library; if not, see
|
Prefer https to http for gnu.org and fsf.org URLs
Also, change sources.redhat.com to sourceware.org.
This patch was automatically generated by running the following shell
script, which uses GNU sed, and which avoids modifying files imported
from upstream:
sed -ri '
s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g
s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g
' \
$(find $(git ls-files) -prune -type f \
! -name '*.po' \
! -name 'ChangeLog*' \
! -path COPYING ! -path COPYING.LIB \
! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \
! -path manual/texinfo.tex ! -path scripts/config.guess \
! -path scripts/config.sub ! -path scripts/install-sh \
! -path scripts/mkinstalldirs ! -path scripts/move-if-change \
! -path INSTALL ! -path locale/programs/charmap-kw.h \
! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \
! '(' -name configure \
-execdir test -f configure.ac -o -f configure.in ';' ')' \
! '(' -name preconfigure \
-execdir test -f preconfigure.ac ';' ')' \
-print)
and then by running 'make dist-prepare' to regenerate files built
from the altered files, and then executing the following to cleanup:
chmod a+x sysdeps/unix/sysv/linux/riscv/configure
# Omit irrelevant whitespace and comment-only changes,
# perhaps from a slightly-different Autoconf version.
git checkout -f \
sysdeps/csky/configure \
sysdeps/hppa/configure \
sysdeps/riscv/configure \
sysdeps/unix/sysv/linux/csky/configure
# Omit changes that caused a pre-commit check to fail like this:
# remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines
git checkout -f \
sysdeps/powerpc/powerpc64/ppc-mcount.S \
sysdeps/unix/sysv/linux/s390/s390-64/syscall.S
# Omit change that caused a pre-commit check to fail like this:
# remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline
git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S
2019-09-07 05:40:42 +00:00
|
|
|
<https://www.gnu.org/licenses/>. */
|
2003-04-19 16:57:17 +00:00
|
|
|
|
2014-09-24 16:38:59 +00:00
|
|
|
#ifndef NOT_CANCEL_H
|
|
|
|
# define NOT_CANCEL_H
|
|
|
|
|
2017-07-03 14:11:24 +00:00
|
|
|
#include <fcntl.h>
|
2003-04-19 16:57:17 +00:00
|
|
|
#include <sysdep.h>
|
2014-09-24 16:38:59 +00:00
|
|
|
#include <errno.h>
|
|
|
|
#include <unistd.h>
|
arc4random: simplify design for better safety
Rather than buffering 16 MiB of entropy in userspace (by way of
chacha20), simply call getrandom() every time.
This approach is doubtlessly slower, for now, but trying to prematurely
optimize arc4random appears to be leading toward all sorts of nasty
properties and gotchas. Instead, this patch takes a much more
conservative approach. The interface is added as a basic loop wrapper
around getrandom(), and then later, the kernel and libc together can
work together on optimizing that.
This prevents numerous issues in which userspace is unaware of when it
really must throw away its buffer, since we avoid buffering all
together. Future improvements may include userspace learning more from
the kernel about when to do that, which might make these sorts of
chacha20-based optimizations more possible. The current heuristic of 16
MiB is meaningless garbage that doesn't correspond to anything the
kernel might know about. So for now, let's just do something
conservative that we know is correct and won't lead to cryptographic
issues for users of this function.
This patch might be considered along the lines of, "optimization is the
root of all evil," in that the much more complex implementation it
replaces moves too fast without considering security implications,
whereas the incremental approach done here is a much safer way of going
about things. Once this lands, we can take our time in optimizing this
properly using new interplay between the kernel and userspace.
getrandom(0) is used, since that's the one that ensures the bytes
returned are cryptographically secure. But on systems without it, we
fallback to using /dev/urandom. This is unfortunate because it means
opening a file descriptor, but there's not much of a choice. Secondly,
as part of the fallback, in order to get more or less the same
properties of getrandom(0), we poll on /dev/random, and if the poll
succeeds at least once, then we assume the RNG is initialized. This is a
rough approximation, as the ancient "non-blocking pool" initialized
after the "blocking pool", not before, and it may not port back to all
ancient kernels, though it does to all kernels supported by glibc
(≥3.2), so generally it's the best approximation we can do.
The motivation for including arc4random, in the first place, is to have
source-level compatibility with existing code. That means this patch
doesn't attempt to litigate the interface itself. It does, however,
choose a conservative approach for implementing it.
Cc: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Cristian Rodríguez <crrodriguez@opensuse.org>
Cc: Paul Eggert <eggert@cs.ucla.edu>
Cc: Mark Harris <mark.hsj@gmail.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: linux-crypto@vger.kernel.org
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2022-07-26 19:58:22 +00:00
|
|
|
#include <sys/poll.h>
|
2014-09-24 16:38:59 +00:00
|
|
|
#include <sys/syscall.h>
|
2017-07-03 18:43:51 +00:00
|
|
|
#include <sys/wait.h>
|
2017-07-03 18:54:02 +00:00
|
|
|
#include <time.h>
|
linux: Add support for getrandom vDSO
Linux 6.11 has getrandom() in vDSO. It operates on a thread-local opaque
state allocated with mmap using flags specified by the vDSO.
Multiple states are allocated at once, as many as fit into a page, and
these are held in an array of available states to be doled out to each
thread upon first use, and recycled when a thread terminates. As these
states run low, more are allocated.
To make this procedure async-signal-safe, a simple guard is used in the
LSB of the opaque state address, falling back to the syscall if there's
reentrancy contention.
Also, _Fork() is handled by blocking signals on opaque state allocation
(so _Fork() always sees a consistent state even if it interrupts a
getrandom() call) and by iterating over the thread stack cache on
reclaim_stack. Each opaque state will be in the free states list
(grnd_alloc.states) or allocated to a running thread.
The cancellation is handled by always using GRND_NONBLOCK flags while
calling the vDSO, and falling back to the cancellable syscall if the
kernel returns EAGAIN (would block). Since getrandom is not defined by
POSIX and cancellation is supported as an extension, the cancellation is
handled as 'may occur' instead of 'shall occur' [1], meaning that if
vDSO does not block (the expected behavior) getrandom will not act as a
cancellation entrypoint. It avoids a pthread_testcancel call on the fast
path (different than 'shall occur' functions, like sem_wait()).
It is currently enabled for x86_64, which is available in Linux 6.11,
and aarch64, powerpc32, powerpc64, loongarch64, and s390x, which are
available in Linux 6.12.
Link: https://pubs.opengroup.org/onlinepubs/9799919799/nframe.html [1]
Co-developed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Tested-by: Jason A. Donenfeld <Jason@zx2c4.com> # x86_64
Tested-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> # x86_64, aarch64
Tested-by: Xi Ruoyao <xry111@xry111.site> # x86_64, aarch64, loongarch64
Tested-by: Stefan Liebler <stli@linux.ibm.com> # s390x
2024-09-18 14:01:22 +00:00
|
|
|
#include <sys/random.h>
|
2003-04-19 16:57:17 +00:00
|
|
|
|
2017-07-03 14:11:24 +00:00
|
|
|
/* Non cancellable open syscall. */
|
not-cancel.h: Support testing fortify build with Clang
When Clang is used to test fortify glibc build configured with
--enable-fortify-source=N
clang issues errors like
In file included from tst-rfc3484.c:60:
In file included from ./getaddrinfo.c:81:
../sysdeps/unix/sysv/linux/not-cancel.h:36:10: error: reference to overloaded function could not be resolved; did you mean to call it?
36 | __typeof (open64) __open64_nocancel;
| ^~~~~~~~
../include/bits/../../io/bits/fcntl2.h:127:1: note: possible target for call
127 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag,
| ^
../include/bits/../../io/bits/fcntl2.h:118:1: note: possible target for call
118 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag)
| ^
../include/bits/../../io/bits/fcntl2.h:114:1: note: possible target for call
114 | open64 (const char *__path, int __oflag, mode_t __mode, ...)
| ^
../io/fcntl.h:219:12: note: possible target for call
219 | extern int open64 (const char *__file, int __oflag, ...) __nonnull ((1));
| ^
because clang fortify support for functions with variable arguments relies
on function overload. Update not-cancel.h to avoid __typeof on functions
with variable arguments.
Co-Authored-By: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
2025-01-01 23:22:36 +00:00
|
|
|
extern int __open_nocancel (const char *, int, ...);
|
2017-07-03 14:11:24 +00:00
|
|
|
|
|
|
|
/* Non cancellable open syscall (LFS version). */
|
not-cancel.h: Support testing fortify build with Clang
When Clang is used to test fortify glibc build configured with
--enable-fortify-source=N
clang issues errors like
In file included from tst-rfc3484.c:60:
In file included from ./getaddrinfo.c:81:
../sysdeps/unix/sysv/linux/not-cancel.h:36:10: error: reference to overloaded function could not be resolved; did you mean to call it?
36 | __typeof (open64) __open64_nocancel;
| ^~~~~~~~
../include/bits/../../io/bits/fcntl2.h:127:1: note: possible target for call
127 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag,
| ^
../include/bits/../../io/bits/fcntl2.h:118:1: note: possible target for call
118 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag)
| ^
../include/bits/../../io/bits/fcntl2.h:114:1: note: possible target for call
114 | open64 (const char *__path, int __oflag, mode_t __mode, ...)
| ^
../io/fcntl.h:219:12: note: possible target for call
219 | extern int open64 (const char *__file, int __oflag, ...) __nonnull ((1));
| ^
because clang fortify support for functions with variable arguments relies
on function overload. Update not-cancel.h to avoid __typeof on functions
with variable arguments.
Co-Authored-By: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
2025-01-01 23:22:36 +00:00
|
|
|
extern int __open64_nocancel (const char *, int, ...);
|
2006-02-15 17:20:33 +00:00
|
|
|
|
2017-07-03 18:00:26 +00:00
|
|
|
/* Non cancellable openat syscall. */
|
not-cancel.h: Support testing fortify build with Clang
When Clang is used to test fortify glibc build configured with
--enable-fortify-source=N
clang issues errors like
In file included from tst-rfc3484.c:60:
In file included from ./getaddrinfo.c:81:
../sysdeps/unix/sysv/linux/not-cancel.h:36:10: error: reference to overloaded function could not be resolved; did you mean to call it?
36 | __typeof (open64) __open64_nocancel;
| ^~~~~~~~
../include/bits/../../io/bits/fcntl2.h:127:1: note: possible target for call
127 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag,
| ^
../include/bits/../../io/bits/fcntl2.h:118:1: note: possible target for call
118 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag)
| ^
../include/bits/../../io/bits/fcntl2.h:114:1: note: possible target for call
114 | open64 (const char *__path, int __oflag, mode_t __mode, ...)
| ^
../io/fcntl.h:219:12: note: possible target for call
219 | extern int open64 (const char *__file, int __oflag, ...) __nonnull ((1));
| ^
because clang fortify support for functions with variable arguments relies
on function overload. Update not-cancel.h to avoid __typeof on functions
with variable arguments.
Co-Authored-By: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
2025-01-01 23:22:36 +00:00
|
|
|
extern int __openat_nocancel (int fd, const char *, int, ...);
|
2017-07-03 18:00:26 +00:00
|
|
|
|
|
|
|
/* Non cacellable openat syscall (LFS version). */
|
not-cancel.h: Support testing fortify build with Clang
When Clang is used to test fortify glibc build configured with
--enable-fortify-source=N
clang issues errors like
In file included from tst-rfc3484.c:60:
In file included from ./getaddrinfo.c:81:
../sysdeps/unix/sysv/linux/not-cancel.h:36:10: error: reference to overloaded function could not be resolved; did you mean to call it?
36 | __typeof (open64) __open64_nocancel;
| ^~~~~~~~
../include/bits/../../io/bits/fcntl2.h:127:1: note: possible target for call
127 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag,
| ^
../include/bits/../../io/bits/fcntl2.h:118:1: note: possible target for call
118 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag)
| ^
../include/bits/../../io/bits/fcntl2.h:114:1: note: possible target for call
114 | open64 (const char *__path, int __oflag, mode_t __mode, ...)
| ^
../io/fcntl.h:219:12: note: possible target for call
219 | extern int open64 (const char *__file, int __oflag, ...) __nonnull ((1));
| ^
because clang fortify support for functions with variable arguments relies
on function overload. Update not-cancel.h to avoid __typeof on functions
with variable arguments.
Co-Authored-By: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
2025-01-01 23:22:36 +00:00
|
|
|
extern int __openat64_nocancel (int fd, const char *, int, ...);
|
2017-07-03 18:00:26 +00:00
|
|
|
|
2017-07-03 17:20:46 +00:00
|
|
|
/* Non cancellable read syscall. */
|
|
|
|
__typeof (__read) __read_nocancel;
|
2014-09-24 16:38:59 +00:00
|
|
|
|
2019-10-02 16:42:28 +00:00
|
|
|
/* Non cancellable pread syscall (LFS version). */
|
|
|
|
__typeof (__pread64) __pread64_nocancel;
|
|
|
|
|
2014-09-24 16:38:59 +00:00
|
|
|
/* Uncancelable write. */
|
2017-07-03 17:39:52 +00:00
|
|
|
__typeof (__write) __write_nocancel;
|
2014-05-14 18:06:36 +00:00
|
|
|
|
2003-04-19 16:57:17 +00:00
|
|
|
/* Uncancelable close. */
|
2017-07-03 18:22:58 +00:00
|
|
|
__typeof (__close) __close_nocancel;
|
|
|
|
|
2025-06-24 20:17:25 +00:00
|
|
|
/* Uncancellable close that does not also set errno in case of failure. */
|
|
|
|
void __close_nocancel_nostatus (int);
|
|
|
|
|
2022-10-24 19:12:15 +00:00
|
|
|
/* Uncancelable fcntl. */
|
not-cancel.h: Support testing fortify build with Clang
When Clang is used to test fortify glibc build configured with
--enable-fortify-source=N
clang issues errors like
In file included from tst-rfc3484.c:60:
In file included from ./getaddrinfo.c:81:
../sysdeps/unix/sysv/linux/not-cancel.h:36:10: error: reference to overloaded function could not be resolved; did you mean to call it?
36 | __typeof (open64) __open64_nocancel;
| ^~~~~~~~
../include/bits/../../io/bits/fcntl2.h:127:1: note: possible target for call
127 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag,
| ^
../include/bits/../../io/bits/fcntl2.h:118:1: note: possible target for call
118 | open64 (__fortify_clang_overload_arg (const char *, ,__path), int __oflag)
| ^
../include/bits/../../io/bits/fcntl2.h:114:1: note: possible target for call
114 | open64 (const char *__path, int __oflag, mode_t __mode, ...)
| ^
../io/fcntl.h:219:12: note: possible target for call
219 | extern int open64 (const char *__file, int __oflag, ...) __nonnull ((1));
| ^
because clang fortify support for functions with variable arguments relies
on function overload. Update not-cancel.h to avoid __typeof on functions
with variable arguments.
Co-Authored-By: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
2025-01-01 23:22:36 +00:00
|
|
|
int __fcntl64_nocancel (int, int, ...);
|
2022-10-24 19:12:15 +00:00
|
|
|
|
|
|
|
#if IS_IN (libc) || IS_IN (rtld)
|
|
|
|
hidden_proto (__open_nocancel)
|
|
|
|
hidden_proto (__open64_nocancel)
|
|
|
|
hidden_proto (__openat_nocancel)
|
|
|
|
hidden_proto (__openat64_nocancel)
|
|
|
|
hidden_proto (__read_nocancel)
|
|
|
|
hidden_proto (__pread64_nocancel)
|
|
|
|
hidden_proto (__write_nocancel)
|
|
|
|
hidden_proto (__close_nocancel)
|
2025-06-24 20:17:25 +00:00
|
|
|
hidden_proto (__close_nocancel_nostatus)
|
2022-10-24 19:12:15 +00:00
|
|
|
hidden_proto (__fcntl64_nocancel)
|
|
|
|
#endif
|
|
|
|
|
2017-07-03 18:33:23 +00:00
|
|
|
/* Non cancellable writev syscall that does not also set errno in case of
|
|
|
|
failure. */
|
|
|
|
static inline void
|
|
|
|
__writev_nocancel_nostatus (int fd, const struct iovec *iov, int iovcnt)
|
|
|
|
{
|
2020-01-29 20:38:36 +00:00
|
|
|
INTERNAL_SYSCALL_CALL (writev, fd, iov, iovcnt);
|
2017-07-03 18:33:23 +00:00
|
|
|
}
|
2003-07-15 07:52:52 +00:00
|
|
|
|
2022-09-29 19:15:20 +00:00
|
|
|
static inline ssize_t
|
linux: Add support for getrandom vDSO
Linux 6.11 has getrandom() in vDSO. It operates on a thread-local opaque
state allocated with mmap using flags specified by the vDSO.
Multiple states are allocated at once, as many as fit into a page, and
these are held in an array of available states to be doled out to each
thread upon first use, and recycled when a thread terminates. As these
states run low, more are allocated.
To make this procedure async-signal-safe, a simple guard is used in the
LSB of the opaque state address, falling back to the syscall if there's
reentrancy contention.
Also, _Fork() is handled by blocking signals on opaque state allocation
(so _Fork() always sees a consistent state even if it interrupts a
getrandom() call) and by iterating over the thread stack cache on
reclaim_stack. Each opaque state will be in the free states list
(grnd_alloc.states) or allocated to a running thread.
The cancellation is handled by always using GRND_NONBLOCK flags while
calling the vDSO, and falling back to the cancellable syscall if the
kernel returns EAGAIN (would block). Since getrandom is not defined by
POSIX and cancellation is supported as an extension, the cancellation is
handled as 'may occur' instead of 'shall occur' [1], meaning that if
vDSO does not block (the expected behavior) getrandom will not act as a
cancellation entrypoint. It avoids a pthread_testcancel call on the fast
path (different than 'shall occur' functions, like sem_wait()).
It is currently enabled for x86_64, which is available in Linux 6.11,
and aarch64, powerpc32, powerpc64, loongarch64, and s390x, which are
available in Linux 6.12.
Link: https://pubs.opengroup.org/onlinepubs/9799919799/nframe.html [1]
Co-developed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Tested-by: Jason A. Donenfeld <Jason@zx2c4.com> # x86_64
Tested-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> # x86_64, aarch64
Tested-by: Xi Ruoyao <xry111@xry111.site> # x86_64, aarch64, loongarch64
Tested-by: Stefan Liebler <stli@linux.ibm.com> # s390x
2024-09-18 14:01:22 +00:00
|
|
|
__getrandom_nocancel_direct (void *buf, size_t buflen, unsigned int flags)
|
2024-01-04 13:41:20 +00:00
|
|
|
{
|
|
|
|
return INLINE_SYSCALL_CALL (getrandom, buf, buflen, flags);
|
|
|
|
}
|
|
|
|
|
linux: Add support for getrandom vDSO
Linux 6.11 has getrandom() in vDSO. It operates on a thread-local opaque
state allocated with mmap using flags specified by the vDSO.
Multiple states are allocated at once, as many as fit into a page, and
these are held in an array of available states to be doled out to each
thread upon first use, and recycled when a thread terminates. As these
states run low, more are allocated.
To make this procedure async-signal-safe, a simple guard is used in the
LSB of the opaque state address, falling back to the syscall if there's
reentrancy contention.
Also, _Fork() is handled by blocking signals on opaque state allocation
(so _Fork() always sees a consistent state even if it interrupts a
getrandom() call) and by iterating over the thread stack cache on
reclaim_stack. Each opaque state will be in the free states list
(grnd_alloc.states) or allocated to a running thread.
The cancellation is handled by always using GRND_NONBLOCK flags while
calling the vDSO, and falling back to the cancellable syscall if the
kernel returns EAGAIN (would block). Since getrandom is not defined by
POSIX and cancellation is supported as an extension, the cancellation is
handled as 'may occur' instead of 'shall occur' [1], meaning that if
vDSO does not block (the expected behavior) getrandom will not act as a
cancellation entrypoint. It avoids a pthread_testcancel call on the fast
path (different than 'shall occur' functions, like sem_wait()).
It is currently enabled for x86_64, which is available in Linux 6.11,
and aarch64, powerpc32, powerpc64, loongarch64, and s390x, which are
available in Linux 6.12.
Link: https://pubs.opengroup.org/onlinepubs/9799919799/nframe.html [1]
Co-developed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Tested-by: Jason A. Donenfeld <Jason@zx2c4.com> # x86_64
Tested-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> # x86_64, aarch64
Tested-by: Xi Ruoyao <xry111@xry111.site> # x86_64, aarch64, loongarch64
Tested-by: Stefan Liebler <stli@linux.ibm.com> # s390x
2024-09-18 14:01:22 +00:00
|
|
|
__typeof (getrandom) __getrandom_nocancel attribute_hidden;
|
|
|
|
|
2024-01-04 13:41:20 +00:00
|
|
|
/* Non cancellable getrandom syscall that does not also set errno in case of
|
|
|
|
failure. */
|
|
|
|
static inline ssize_t
|
linux: Add support for getrandom vDSO
Linux 6.11 has getrandom() in vDSO. It operates on a thread-local opaque
state allocated with mmap using flags specified by the vDSO.
Multiple states are allocated at once, as many as fit into a page, and
these are held in an array of available states to be doled out to each
thread upon first use, and recycled when a thread terminates. As these
states run low, more are allocated.
To make this procedure async-signal-safe, a simple guard is used in the
LSB of the opaque state address, falling back to the syscall if there's
reentrancy contention.
Also, _Fork() is handled by blocking signals on opaque state allocation
(so _Fork() always sees a consistent state even if it interrupts a
getrandom() call) and by iterating over the thread stack cache on
reclaim_stack. Each opaque state will be in the free states list
(grnd_alloc.states) or allocated to a running thread.
The cancellation is handled by always using GRND_NONBLOCK flags while
calling the vDSO, and falling back to the cancellable syscall if the
kernel returns EAGAIN (would block). Since getrandom is not defined by
POSIX and cancellation is supported as an extension, the cancellation is
handled as 'may occur' instead of 'shall occur' [1], meaning that if
vDSO does not block (the expected behavior) getrandom will not act as a
cancellation entrypoint. It avoids a pthread_testcancel call on the fast
path (different than 'shall occur' functions, like sem_wait()).
It is currently enabled for x86_64, which is available in Linux 6.11,
and aarch64, powerpc32, powerpc64, loongarch64, and s390x, which are
available in Linux 6.12.
Link: https://pubs.opengroup.org/onlinepubs/9799919799/nframe.html [1]
Co-developed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Tested-by: Jason A. Donenfeld <Jason@zx2c4.com> # x86_64
Tested-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> # x86_64, aarch64
Tested-by: Xi Ruoyao <xry111@xry111.site> # x86_64, aarch64, loongarch64
Tested-by: Stefan Liebler <stli@linux.ibm.com> # s390x
2024-09-18 14:01:22 +00:00
|
|
|
__getrandom_nocancel_nostatus_direct (void *buf, size_t buflen, unsigned int flags)
|
stdlib: Add arc4random, arc4random_buf, and arc4random_uniform (BZ #4417)
The implementation is based on scalar Chacha20 with per-thread cache.
It uses getrandom or /dev/urandom as fallback to get the initial entropy,
and reseeds the internal state on every 16MB of consumed buffer.
To improve performance and lower memory consumption the per-thread cache
is allocated lazily on first arc4random functions call, and if the
memory allocation fails getentropy or /dev/urandom is used as fallback.
The cache is also cleared on thread exit iff it was initialized (so if
arc4random is not called it is not touched).
Although it is lock-free, arc4random is still not async-signal-safe
(the per thread state is not updated atomically).
The ChaCha20 implementation is based on RFC8439 [1], omitting the final
XOR of the keystream with the plaintext because the plaintext is a
stream of zeros. This strategy is similar to what OpenBSD arc4random
does.
The arc4random_uniform is based on previous work by Florian Weimer,
where the algorithm is based on Jérémie Lumbroso paper Optimal Discrete
Uniform Generation from Coin Flips, and Applications (2013) [2], who
credits Donald E. Knuth and Andrew C. Yao, The complexity of nonuniform
random number generation (1976), for solving the general case.
The main advantage of this method is the that the unit of randomness is not
the uniform random variable (uint32_t), but a random bit. It optimizes the
internal buffer sampling by initially consuming a 32-bit random variable
and then sampling byte per byte. Depending of the upper bound requested,
it might lead to better CPU utilization.
Checked on x86_64-linux-gnu, aarch64-linux, and powerpc64le-linux-gnu.
Co-authored-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Yann Droneaud <ydroneaud@opteya.com>
[1] https://datatracker.ietf.org/doc/html/rfc8439
[2] https://arxiv.org/pdf/1304.1916.pdf
2022-07-21 13:04:59 +00:00
|
|
|
{
|
2022-09-29 19:18:06 +00:00
|
|
|
return INTERNAL_SYSCALL_CALL (getrandom, buf, buflen, flags);
|
stdlib: Add arc4random, arc4random_buf, and arc4random_uniform (BZ #4417)
The implementation is based on scalar Chacha20 with per-thread cache.
It uses getrandom or /dev/urandom as fallback to get the initial entropy,
and reseeds the internal state on every 16MB of consumed buffer.
To improve performance and lower memory consumption the per-thread cache
is allocated lazily on first arc4random functions call, and if the
memory allocation fails getentropy or /dev/urandom is used as fallback.
The cache is also cleared on thread exit iff it was initialized (so if
arc4random is not called it is not touched).
Although it is lock-free, arc4random is still not async-signal-safe
(the per thread state is not updated atomically).
The ChaCha20 implementation is based on RFC8439 [1], omitting the final
XOR of the keystream with the plaintext because the plaintext is a
stream of zeros. This strategy is similar to what OpenBSD arc4random
does.
The arc4random_uniform is based on previous work by Florian Weimer,
where the algorithm is based on Jérémie Lumbroso paper Optimal Discrete
Uniform Generation from Coin Flips, and Applications (2013) [2], who
credits Donald E. Knuth and Andrew C. Yao, The complexity of nonuniform
random number generation (1976), for solving the general case.
The main advantage of this method is the that the unit of randomness is not
the uniform random variable (uint32_t), but a random bit. It optimizes the
internal buffer sampling by initially consuming a 32-bit random variable
and then sampling byte per byte. Depending of the upper bound requested,
it might lead to better CPU utilization.
Checked on x86_64-linux-gnu, aarch64-linux, and powerpc64le-linux-gnu.
Co-authored-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Yann Droneaud <ydroneaud@opteya.com>
[1] https://datatracker.ietf.org/doc/html/rfc8439
[2] https://arxiv.org/pdf/1304.1916.pdf
2022-07-21 13:04:59 +00:00
|
|
|
}
|
|
|
|
|
arc4random: simplify design for better safety
Rather than buffering 16 MiB of entropy in userspace (by way of
chacha20), simply call getrandom() every time.
This approach is doubtlessly slower, for now, but trying to prematurely
optimize arc4random appears to be leading toward all sorts of nasty
properties and gotchas. Instead, this patch takes a much more
conservative approach. The interface is added as a basic loop wrapper
around getrandom(), and then later, the kernel and libc together can
work together on optimizing that.
This prevents numerous issues in which userspace is unaware of when it
really must throw away its buffer, since we avoid buffering all
together. Future improvements may include userspace learning more from
the kernel about when to do that, which might make these sorts of
chacha20-based optimizations more possible. The current heuristic of 16
MiB is meaningless garbage that doesn't correspond to anything the
kernel might know about. So for now, let's just do something
conservative that we know is correct and won't lead to cryptographic
issues for users of this function.
This patch might be considered along the lines of, "optimization is the
root of all evil," in that the much more complex implementation it
replaces moves too fast without considering security implications,
whereas the incremental approach done here is a much safer way of going
about things. Once this lands, we can take our time in optimizing this
properly using new interplay between the kernel and userspace.
getrandom(0) is used, since that's the one that ensures the bytes
returned are cryptographically secure. But on systems without it, we
fallback to using /dev/urandom. This is unfortunate because it means
opening a file descriptor, but there's not much of a choice. Secondly,
as part of the fallback, in order to get more or less the same
properties of getrandom(0), we poll on /dev/random, and if the poll
succeeds at least once, then we assume the RNG is initialized. This is a
rough approximation, as the ancient "non-blocking pool" initialized
after the "blocking pool", not before, and it may not port back to all
ancient kernels, though it does to all kernels supported by glibc
(≥3.2), so generally it's the best approximation we can do.
The motivation for including arc4random, in the first place, is to have
source-level compatibility with existing code. That means this patch
doesn't attempt to litigate the interface itself. It does, however,
choose a conservative approach for implementing it.
Cc: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Cristian Rodríguez <crrodriguez@opensuse.org>
Cc: Paul Eggert <eggert@cs.ucla.edu>
Cc: Mark Harris <mark.hsj@gmail.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: linux-crypto@vger.kernel.org
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2022-07-26 19:58:22 +00:00
|
|
|
static inline int
|
|
|
|
__poll_infinity_nocancel (struct pollfd *fds, nfds_t nfds)
|
|
|
|
{
|
|
|
|
return INLINE_SYSCALL_CALL (ppoll, fds, nfds, NULL, NULL, 0);
|
|
|
|
}
|
stdlib: Add arc4random, arc4random_buf, and arc4random_uniform (BZ #4417)
The implementation is based on scalar Chacha20 with per-thread cache.
It uses getrandom or /dev/urandom as fallback to get the initial entropy,
and reseeds the internal state on every 16MB of consumed buffer.
To improve performance and lower memory consumption the per-thread cache
is allocated lazily on first arc4random functions call, and if the
memory allocation fails getentropy or /dev/urandom is used as fallback.
The cache is also cleared on thread exit iff it was initialized (so if
arc4random is not called it is not touched).
Although it is lock-free, arc4random is still not async-signal-safe
(the per thread state is not updated atomically).
The ChaCha20 implementation is based on RFC8439 [1], omitting the final
XOR of the keystream with the plaintext because the plaintext is a
stream of zeros. This strategy is similar to what OpenBSD arc4random
does.
The arc4random_uniform is based on previous work by Florian Weimer,
where the algorithm is based on Jérémie Lumbroso paper Optimal Discrete
Uniform Generation from Coin Flips, and Applications (2013) [2], who
credits Donald E. Knuth and Andrew C. Yao, The complexity of nonuniform
random number generation (1976), for solving the general case.
The main advantage of this method is the that the unit of randomness is not
the uniform random variable (uint32_t), but a random bit. It optimizes the
internal buffer sampling by initially consuming a 32-bit random variable
and then sampling byte per byte. Depending of the upper bound requested,
it might lead to better CPU utilization.
Checked on x86_64-linux-gnu, aarch64-linux, and powerpc64le-linux-gnu.
Co-authored-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Yann Droneaud <ydroneaud@opteya.com>
[1] https://datatracker.ietf.org/doc/html/rfc8439
[2] https://arxiv.org/pdf/1304.1916.pdf
2022-07-21 13:04:59 +00:00
|
|
|
|
2014-09-24 16:38:59 +00:00
|
|
|
#endif /* NOT_CANCEL_H */
|