A string tunable value usually references the GLIBC_TUNABLES (or alias)
environment string, which lives in the environment block the kernel places
on the initial stack. That memory is owned by the application, which may
overwrite it (e.g. setproctitle), so the reference is only safe while no
application code has run (a value coming from the system-wide tunables
cache is a copy instead, but the rule is applied uniformly).
This patch make the lifetime explicit and enforced without copying the value
or allocating any memory by adding __tunable_seal_strings, which drops every
string tunable reference once early startup is complete.
The seal is applied after the only string tunable consumer and before any
code outside of the startup sequence runs.
Checked on aarch64-linux-gnu and x86_64-linux-gnu. I also run the elf
tests on powerpc64le-linux-gnu, loongarch64-linux-gnuf64, and
s390x-linux-gnu.
The WRDE_APPEND path duplicates the caller's we_wordv array, which
already holds we_offs + we_wordc + 1 pointers. Follow-up to commit
e2cefe16c3.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Commit 24d188a2a1 left a stray #endif in
the powerpc32 soft-float __longjmp-common.S. Remove it, matching the
fpu variant.
Checked with a build for powerpc32-linux-gnu-soft.
Commits 6deadd4eb6 and ade9f30ce2 changed m68k fmod to call
__m81_u(fmod), instead of the mathimpl.h inline
__m81_u(__ieee754_fmod) (that wraps the m68k fmod instruction).
This leads to infinite recursion.
Tested-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
clang-23 enables -Wunused-but-set-global by default, which flags
variables that are set but never read. Also remove the fstat call,
which was only used to set the removed variable.
Two issues with clang-23:
1. -std=gnu11 triggers warnings with the 'uwb' suffix due to
-Wc23-extensions being enabled by default.
2. an ICE with __builtin_stdc_bit_ceil with a non-constant
unsigned _BitInt(1) argument [1]
This patch disables the warning for 1. and disables the affected
stdc_bit_ceil tests for clang.
[1] https://github.com/llvm/llvm-project/issues/214478
clang-23 warns that the variable 'ext' set but not used:
dl-cache.c:42:47: error: variable 'ext' set but not used
[-Werror,-Wunused-but-set-global]
The variable is set unconditionally by _dl_load_cache_lookup, but it
is only read by glibc_hwcaps_priorities_init (which is compiled only
for SHARED).
clang enables -Wsingle-bit-bitfield-constant-conversion with -Werror
and it triggers:
tunconf.c:338:32: error: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion]
338 | entry->value_is_negative = 1;
Change both value_is_negative and value_was_parsed to unsigned.
Checked on x86_64-linux-gnu and i686-linux-gnu.
glob splits the pattern at its rightmost slash and calls itself on
the part before it, so a pattern needs one stack frame per directory
component. It also calls itself once per brace expression. Either can
be made as deep as the pattern is long, so glob overflows the stack
before it can answer. The descent is on the pattern alone, so the
leading component, which is what decides whether anything can match at
all, is only reached at the bottom of the recursion:
glob ("__nonexistent__/*/*/.../*/x", 0, NULL, &g)
with a few thousand components crashes with an default stack (usually
8MB on Linux).
Expand the components in a loop instead. glob_dir_pattern collects
what each component has to do into a heap-allocated array, then matches
them from left to right, and glob_brace walks the brace expansions with
an explicit stack. Both arrays are sized from the pattern up front:
there is no more than one step per slash and no more than one brace
level per brace, since each consumes one.
Matching left to right also means a leading directory that does not
exist ends the expansion at the first component rather than after
descending through all of them.
Stack usage no longer depends on the pattern: a pattern with 100000
components now resolves on a 64 KiB thread stack, where before 4096
components overflowed 8 MiB.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, and i686-linux-gnu.
Reviewed-by: Collin Funk <collin.funk1@gmail.com>
What is left are the name used to stat a component without
metacharacters and the blocks holding the matched names. With those on
the heap the alloca budget can go as well.
Also treat the size overflow as an error. It used to fall through
to malloc with the wrapped size and then copy the full length
into it.
alloca is now used only by the MSDOS and Windows paths, which glibc
does not build, so move its header out of the way as well.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, and i686-linux-gnu.
Reviewed-by: Collin Funk <collin.funk1@gmail.com>
The last alloca in __glob is the buffer holding one expansion of a
brace expression. As with the directory and user names, the stack it
takes is not bounded by the call itself.
Use malloc unconditionally. __glob no longer uses alloca; glob_in_dir
still does, so the accounting stays for now.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, and i686-linux-gnu.
Reviewed-by: Collin Funk <collin.funk1@gmail.com>
Use malloc unconditionally. The name is only needed for the passwd
lookup that follows, which is far more expensive than the allocation.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, and i686-linux-gnu.
Reviewed-by: Collin Funk <collin.funk1@gmail.com>
Expanding "~" or "~user" needs a struct scratch_buffer to call
getpwnam_r through, where the code might reserve extra stack in
every glob frame (around 1224 bytes on x86_64). Even though the
lookups only run when the caller passed GLOB_TILDE or
GLOB_TILDE_CHECK.
Move the two lookups into glob_current_home_dir and glob_user_home_dir,
which return the directory as a malloc'ed string. The frame of each
glob call drops to around 184 bytes.
This also fixes a small leak: the ~user path returned GLOB_NOSPACE
without freeing user_name when scratch_buffer_grow failed.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, and i686-linux-gnu.
Reviewed-by: Collin Funk <collin.funk1@gmail.com>
Use malloc unconditionally instead. These are one-off allocations
whose cost is dwarfed by the readdir and fnmatch work that follows.
The amount of stack this can take is not bounded by these calls alone,
glob recursively calls itself per pattern component, and each call
starts a fresh alloca budget.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, and i686-linux-gnu.
Reviewed-by: Collin Funk <collin.funk1@gmail.com>
The 63b31c05a8 split relocation processing must agree for the lazy
flag, a mismatch would change the .rel.plt handling.
This is not an issue for any port currently, but on hppa it may return
a different value: if hppa implements IFUNC support, the second pass would
route PLT entries to its empty lazy handler and leave the descriptors
unrelocated, silently.
Make ELF_DYNAMIC_RELOCATE_PASS take lazy as an int lvalue and store the
effective mode back into it, so the DL_RELOC_IRELATIVE call reuses the
same variable instead of a separately threaded copy. The two passes can no
longer disagree about the partitioning. elf_machine_runtime_setup has side
effects, so it must stay a single call.
Checked on x86_64-linux-gnu, and built for all supported architectures.
Reviewed-by: Sam James <sam@gentoo.org>
Commit 63b31c05a8 ("elf: Defer all IRELATIVE relocations until after PLT
setup") dropped the skip_ifunc argument from elf_dynamic_do_Rel, assuming
the new deferred elf_dynamic_do_Rel_irelative pass handles every relocation
that may run an IFUNC resolver. That only holds for IFUNC symbols defined
in the object being relocated: a reference to an IFUNC in another object is
an ordinary JMP_SLOT or GLOB_DAT against an undefined symbol, and its IFUNC
nature is only known after symbol resolution inside elf_machine_rel. Those
relocations stay in the regular pass, which no longer propagated
skip_ifunc, so __RTLD_NOIFUNC was ignored for them.
ldd -u forces non-lazy binding (GLRO(dl_lazy) = 0 for DL_DEBUG_UNUSED), so
the resolver was called and the diagnostic emitted:
$ ldd -u /bin/ls
/bin/ls: Relink `' with `/usr/lib64/libc.so.6' for IFUNC symbol `__mempcpy_chk'
ldd -r with LD_BIND_NOW is affected in the same way.
Restore the skip_ifunc parameter and thread it through _ELF_DYNAMIC_DO_RELOC.
This new semantic shows that ELF_DYNAMIC_RELOCATE_NOIFUNC naming is misleading
(it reads as "do not process IFUNC", yet it takes a skip_ifunc
argument). Replace it to:
DL_RELOC_BOTH -> DL_RELOC_ALL
DL_RELOC_NOIFUNC -> DL_RELOC_NORMAL
DL_RELOC_IFUNC -> DL_RELOC_IRELATIVE
ELF_DYNAMIC_RELOCATE_NOIFUNC and ELF_DYNAMIC_RELOCATE_IFUNC become a single
ELF_DYNAMIC_RELOCATE_PASS taking the pass as its first argument, and
ELF_DYNAMIC_DO_REL/ELF_DYNAMIC_DO_RELA take the pass instead of having three
near-identical variants each.
Checked on x86_64-linux-gnu, and built for all supported architectures.
Reviewed-by: Sam James <sam@gentoo.org>
Rewrite the BZ#15339 test to use the resolv_test framework instead of
querying the network, so it can run as a regular test.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The librtld.map and librtld.os link recipes use $(gnulib), which on arm
contains libgcc-stubs.a through gnulib-arch. But the archive is only a
prerequisite of lib-noranlib so the rtld link can run before the archive
exists:
ld.bfd: cannot find .../elf/libgcc-stubs.a: No such file or directory
The race seems to predates the parallel subdirectory recursion, which
only made it observable.
Add the order-only dependency in sysdeps/arm/Makefile rather than in
elf/Makefile. Theprerequisite lists expand when the rule is parsed,
and gnulib-arch is only defined once Makerules includes the sysdeps
makefiles.
Verified with a build for arm-linux-gnueabihf.
Reviewed-by: Sam James <sam@gentoo.org>
The archive rules in Makerules list every stamp file as a prerequisite,
including the top level's own, and the elf sub-make evaluates them to
build libc_pic.a for the librtld.map link. A sub-make can only create
the stamp files of its own directory, so when the top-level ones do not
exist yet it fails with:
make[2]: *** No rule to make target '.../stamp.os', needed by
'.../libc_pic.a'. Stop.
The serial recursion created them before the subdirectories through the
prerequisite order of subdir_lib; the parallel recursion (commit
7cac99621e) does not. Add them as prerequisites of the object-building
per-subdirectory targets.
Reviewed-by: Sam James <sam@gentoo.org>
The %.d: %.dt rule seds its input into a fixed temporary name, renames
it into place and removes the input. Two makes converting the same
file trip over each other:
mv: cannot stat '.../test-double-libmvec-sincos-avx512f.o.T': No such file or directory
sed: can't read .../test-float-libmvec-acosf-avx512f.o.dt: No such file or directory
That happens because the elf rtld-Rules recursion runs a sub-make over
every $(rtld-subdirs) directory, which converts that directory's .dt
files, and the parallel subdirectory recursion (commit 7cac99621e)
runs it concurrently with those subdirectories' own sub-makes.
Add the PID of the shell to the temporary name and claim the input with
a rename: only the run that wins converts and installs the target.
Reviewed-by: Sam James <sam@gentoo.org>
The build-only first pass of the two-pass 'make check' still runs the
static checks (abi, conformtest, installed headers, etc.), and the
top-level tests recipe merged and summarized their results.
An unexpected FAIL there (e.g. check-abi) aborted 'check' before the
second pass ran any built test, and even a clean run printed a misleading
partial summary.
Pass tests-summary=no in the first pass to skip the merge and summary;
the .test-result files persist, so the second pass folds those results
into the one complete summary at the end, restoring the single-pass
reporting behavior.
Reviewed-by: Sam James <sam@gentoo.org>
The parallel subdirectory recursion (commit 7cac99621e) only orders
csu (and mach/hurd on Hurd) before the parallel fan-out plus the edges
the Depend files request. A header generated from gen-as-const-headers
is only ordered before the compilations of the subdirectory that
adds the .sym (through before-compile), so a header consumed by a
different subdirectory may not exist yet when its consumer is
compiled.
That is the case for <sigaltstack-offsets.h>: it is generated when
building misc, while its only consumer, ____longjmp_chk.S (x86_64 and
sh), is built in debug. The serial recursion always ran misc before
debug in the sorted order, hiding the missing dependency.
Move the generate the header to 'debug' instead.
The same class of problem exists on Hurd: jmp_buf-ssp.h that is used
by ____longjmp_chk.S in debug, and signal-defines.h that is sued
by debug and setjmp.
Deterministically reproduced with 'make debug/subdir_lib' from a clean
build tree (which orders only csu before debug), and verified with
builds for x86_64-linux-gnu, sh4-linux-gnu, i686-gnu, and x86_64-gnu.
Reviewed-by: Sam James <sam@gentoo.org>
The cancellable syscall wrappers end with a tail call to __syscall_cancel,
the wrapper frame is then elided, so when the syscall executes the wrapper
is no longer present on the stack. Tools that unwind from CFI alone, such
as valgrind, perf and sampling profilers, cannot observe it. On gdb, it
only recovers it from DWARF call site information, which reduced-debuginfo
libc builds usually omit.
The behaviour is target dependent: for a shared (PIC) the tail call is
emitted on aarch64, arc, loongarch and riscv. It is not emitted on i386,
x86_64, arm, s390x, sparc and alpha, where the seventh argument is passed
on the stack or fewer argument registers are available, nor on powerpc
and mips, where the TOC/GOT pointer must be restored after the call.
This is why the problem was originally reported as aarch64 specific while
x86_64 was unaffected.
Rather than only inhibiting the tail call [1] (which keeps the wrapper frame
but still leaves the __syscall_cancel and __internal_syscall_cancel
frames), move the cancellation logic back into the wrappers. In the
single-threaded case the syscall is now issued directly from the wrapper;
only the multi-threaded path still calls the out-of-line __syscall_cancel_arch.
This keeps the wrapper observable and removes the extra frames, mimicking
how cancellation was handled before 89b53077d2.
The result is a small libc.so .text increase (size, first column):
ABI master patched diff increase
aarch64 1635880 1647424 11544 0.71%
x86_64 1981081 1992257 11176 0.56%
powerpc64le 2364336 2376964 12628 0.53%
riscv64 1368386 1376704 8318 0.61%
loongarch64 1741385 1755601 14216 0.82%
The tst-backtrace5 was suppose to track this issue, but due wrong
loop variable check it does not take this in account. This patch also fixes
it.
Checked on aarch64-linux-gnu, x86_64-linux-gnu, i686-linux-gnu,
arm-linux-gnueabihf, and powerpc64le-linux-gnu.
[1] https://sourceware.org/pipermail/libc-alpha/2025-March/165395.html
Commit 6758def717 changed the generic ftw{64}.c to include
kernel_stat.h, which is Linux specific.
Add a Hurd version of kernel_stat.h defining XSTAT_IS_XSTAT64 to 0,
since struct stat and struct stat64 never share a layout on Hurd: on
32-bit ABIs st_ino, st_size, and st_blocks are narrower in struct stat,
and on 64-bit ABIs the two structures still differ in size because
_SPARE_SIZE in bits/stat.h reserves three more ints of spare space in
struct stat than in struct stat64.
This keeps the ftw/ftw64 symbols exactly as before the change, where
the aliasing check on __OFF_T_MATCHES_OFF64_T was always false because
the Hurd bits/typesizes.h does not define it.
Checked with a full build for i686-gnu and x86_64-gnu.
The generic implementation emits libm_alias_double unconditionally, so
tanhf32x and tanhf64 bind directly to __tanh_fma.
Guard the alias with '#ifndef __tanh' and emit it from the dispatcher,
as sin. Also remove the stale __expm1 defines, unused since tanh moved
to CORE-MATH.
Checked on x86_64-linux-gnu, and with 'qemu-x86_64 -cpu Nehalem'.
Reported-by: Michael Brunnbauer <brunni@netestate.de>
The $(objpfx)bench-%.c rule writes its output into $(objpfx) without
ensuring that directory exists. Serial builds happened to satisfy
that ordering, with parallel builds the generation recipe can
run before the directory is created, failing with:
cannot create .../benchtests/bench-xxx.c-tmp: Directory nonexistent
Add the standard $(make-target-directory).
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The CORE-MATH import mistranslated the accurate path result scaling
'th *= sp.f' as 'th *= asuint64 (sp)' (commit 106f8c2ed6), and two of
the 51 exceptional-case table entries were dropped when the table was
moved to e_sinh_data.c (commit f05c4907a2).
Checked on x86_64-linux-gnu and aarch64-linux-gnu.
The Payne-Hanek range reducer __branred delivers the reduced argument
as a double-double with only about 93 significant bits. For arguments
extremely close to a multiple of pi/2 the true reduced argument can be
as small as 2^-61, so most of those bits cancel and sin/cos/tan can be
wrong by up to ~143000 ulp. This inaccuracy used to be handled by the
multiple-precision slow paths, which was removed by commit
649095838b ("sin/cos slow paths: remove slow paths from huge range
reduction") and commit 476d692e8a ("math: Remove slow paths in tan
[BZ #15267]").
Restore the e_rem_pio2.c (removed as unused by commit ca3aac57ef
"Remove unused math files") and use __ieee754_rem_pio2 for
the huge-argument reduction instead of __branred, which is removed.
It also does not depend on precise IEEE double rounding, so the nofma
and vector-width workarounds for branred.c are no longer needed.
The file is restored trimmed to its huge-argument path, the callers
reduce smaller arguments themselves and handle non-finite inputs, so
only 1e8 < |x| < 2^1024 reaches __ieee754_rem_pio2.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, armv7a-linux-gnueabihf,
and i686-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The tan input list spans the full binary64 range, so a single number mixes
the kernel, and the i__branred reductions in one average. Add two named
workloads that isolate the ends of that spread, so each can be measured
separately:
- workload-fast.wrf: uniform random inputs in [-pi, pi].
- workload-slow.wrf: |x| in [2^27, 2^1024), log-uniform over binades.
The existing full-range inputs are replaced as the default workload.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The sin input list spans the full binary64 range, so a single number mixes
the kernel, the reduce_sincos reduction , and the __branred reduction in
one average. Add three named workloads that isolate the paths __cos
actually dispatches to, so each can be measured separately:
- workload-fast.wrf: uniform random inputs in [-pi, pi].
- workload-moderate.wrf: |x| in [4, 6.7e7], log-uniform over binades.
- workload-slow.wrf: |x| in [2^27, 2^1024), log-uniform over binades.
The existing full-range inputs are replaced as the default workload.
Reviewed-by: Carlos O'Donell <carlos@redaht.com>
The cos input list spans the full binary64 range, so a single number mixes
the kernel, the reduce_sincos reduction , and the __branred reduction in one
average. Add three named workloads that isolate the paths __cos actually
dispatches to, so each can be measured separately:
- workload-fast.wrf: uniform random inputs in [0, 2*pi].
- workload-moderate.wrf: |x| in [4, 6.7e7], log-uniform over binades.
- workload-slow.wrf: |x| in [2^27, 2^1024), log-uniform over binades.
The existing full-range inputs are replaced as the default workload.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The sparc64 uses an 8KB base page and 8MB PMD transparent huge page.
It fixes the following regression on sparc:
FAIL: elf/tst-thp-1
FAIL: elf/tst-thp-1-no-s-code
FAIL: elf/tst-thp-1-no-s-code-pde
FAIL: elf/tst-thp-1-no-s-code-static
FAIL: elf/tst-thp-1-pde
FAIL: elf/tst-thp-1-static
FAIL: elf/tst-thp-align
Checked with the elf tests on qemu sparc64.
Tested-by: Andreas K. Hüttel <dilfridge@gentoo.org>
The split introduced by commit 63b31c05a8 does not handle sparc
and (R_SPARC_JMP_IREL) powerpc64 (ELFv1, R_PPC64_JMP_IREL), which
are emited in some constructions. Handle such cases on
elf_dynamic_is_Rel_irelative.
It fixes elf/tst-ifunc-fault-bindnow and elf/tst-ifunc-fault-lazy on
sparc64 (powerpc64 emits R_PPC64_IRELATIVE in both cases,
R_PPC64_JMP_IREL is emitted only when the ifunc is called, not just
referenced).
Checked with the elf tests on qemu sparc64 and powerpc64.
Tested-by: Andreas K. Hüttel <dilfridge@gentoo.org>
After commit b75ad99d45, __libc_setup_tls -> _dl_allocate_tls_init copies
the TLS init image with the IFUNC __mempcpy before ARCH_SETUP_IREL resolves
it, so on a multi-arch sparcv9/sparc64 static build the call jumps through an
unrelocated PLT slot and the process dies with SIGILL.
The sparc dl-symbol-redir-ifunc.h only redirected memset. Redirect
memcpy, memmove, mempcpy and __mempcpy to the ultra1 routines as well,
and merge the identical sparc64 and sparc32/sparcv9 copies into a single
sysdeps/sparc file. The redirection is guarded by __sparc_v9__ &&
USE_MULTIARCH so that sparcv8 (leon) and --disable-multi-arch builds,
which have no __*_ultra1 routines, are left untouched.
Checked with elf tests for sparc32 and sparc64 on qemu system.
Tested-by: Andreas K. Hüttel <dilfridge@gentoo.org>
The previous implementation saved a copy of the wordexp_t struct at
entry and blindly restored it on error via (*pwordexp = old_word).
This is incorrect when WRDE_APPEND is set because w_addword may have
called realloc on we_wordv during partial processing before the error
was detected. If realloc relocated the buffer, the saved we_wordv
pointer is dangling; restoring it causes a use-after-free in the
caller (e.g. via wordfree), and the relocated buffer is leaked.
Fix this by duplicating the we_wordv pointer array at entry when
WRDE_APPEND is set, so that all subsequent realloc calls inside
w_addword operate on the copy.
This change also fixes a POSIX conformance issue: if the WRDE_APPEND
flag is specified, pwordexp->we_wordc and pwordexp->we_wordv shall
not be modified.
Also fix two pre-existing error return paths in the '"' and '\'' cases
that returned directly from w_addword failures instead of going through
do_error, which would leak the saved array (and previously would also
skip the word cleanup).
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
The unconditional '.NOTPARALLEL' in benchtests/Makefile forced the whole
subdirectory to build serially, even though its only purpose is to keep
the benchmark *runs* from perturbing each other's timing.
Replace it with ordering that serializes only the benchmark runs, and
only when more than one benchmark group will actually run. The combined
'bench' goal builds every benchmark program in parallel (through
bench-build) and then runs the bench-set, bench-func and bench-malloc
groups strictly one after another.
Reviewed-by: Sam James <sam@gentoo.org>
For the default --enable-default-pie, $(pic-default) adds -DPIC to
CPPFLAGS-.o so. However, -fPIE ($(pie-default)) is only added to
CFLAGS-.o, which does not affect assembler (.S) sources
On SPARC the GOT register setup in SETUP_PIC_REG references
_GLOBAL_OFFSET_TABLE_ through %hi/%lo, and the assembler only rewrite
those into the required PC-relative relocations (R_SPARC_PC22 and
R_SPARC_PC10) when it is in *PIC* mode; otherwise it emits absolute
R_SPARC_HI22/R_SPARC_LO10. With the absolute relocations the
__sparc_get_pc_thunk sequence adds the run-time PC to an already-absolute
GOT address, so the computed GOT register is wrong. In _start this makes
the address of main come out bogus, and __libc_start_main jumps to an
unmapped address.
This removes the requirement of the --disable-default-pie for sparc
to build static binaries correctly.
Checked some tests (mainly the elf/ one) on a sparc64-linux-gnu
qemu system.
Reviewed-by: Sam James <sam@gentoo.org>
The per-tunable security level is no longer part of struct _tunable and
no tunable in dl-tunables.list declares one.
Reviewed-by: DJ Delorie <dj@redhat.com>
A system-wide tunable without an onlysecure/nonsecure/anysecure prefix
defaults to "nonsecure", i.e. it is not applied to AT_SECURE processes.
This is a deliberate, conservative default but was not documented.
Reviewed-by: DJ Delorie <dj@redhat.com>
The environment-variable alias loop in __tunables_init skipped every tunable
whose "initialized" flag was set, which was originally meant only to give
the canonical GLIBC_TUNABLES form precedence over the legacy MALLOC_*
aliases.
Now that the cache also sets "initialized", a legacy alias could no longer
override an *overridable* cache default, even though the canonical
GLIBC_TUNABLES form still could.
Track separately the tunables that were set from GLIBC_TUNABLES during this
call and skip only those in the alias loop
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
_dl_check_ldsocache_needs_loading only stored the stat fields it
compares (mtime, ino, size, dev) on the path where a cache was already
loaded. On the very first call CACHE is NULL and the function returned
"needs loading" without recording those fields, leaving
new_cache_file_time zero. The next call then copied that zero value
into cache_file_time and compared it against the freshly stat'd values,
which always differed, forcing a second, unnecessary load (munmap +
mmap + re-parse) of an unchanged cache at every startup.
It can be shown with repro:
$ cat << EOF > repro.c
#include <dlfcn.h>
int main (void) { dlopen ("does-not-exist-xyz.so.99", RTLD_NOW); return 0; }
EOF
$ gcc repro.c -o repro
$ strace -f -e trace=openat elf/ld.so --library-path . ./repro 2>&1 | grep -c "/etc/ld.so.cache"
The result should be 1, instead of 2.
Record the stat fields as soon as the stat succeeds, before the
CACHE == NULL early return, so the following call has an accurate
baseline and does not spuriously reload.
Reviewed-by: DJ Delorie <dj@redhat.com>
The tunable header signature and version are written by ldconfig but
never checked them on read, so the version field was inert. Reject
the section unless both match.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
_dl_load_cache_tunables bounds each entry's string offsets against
[s_start, start + cache_new->len_strings], but len_strings is an
unvalidated 32-bit field from ld.so.cache and s_start/s_end were int. A
corrupt cache with an oversized len_strings could make s_end exceed the
mapping (or overflow), letting an offset point outside the mmap; the
following strcmp/__strdup would then read unmapped memory.
Compute the offsets as size_t and clamp s_end to cachesize, matching how
the regular library lookup bounds string offsets against the mapping size.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
Add the TCP_RTO_MAX_MS and TCP_RTO_MIN_US socket options (commit
54a378f43425085d0684679d99735696b69165bc, Linux 6.15) and TCP_DELACK_MAX_US
(commit 9552f90835ef3552d0af327e48dc360717777d62, Linux 6.15).
Commit 7e46c2aae4 synced the accurate ECN
additions but encoded the trailing bitfield word of struct tcp_info
incorrectly as two uint16_t fields (tcpi_accecn_fail_mode and
tcpi_accecn_opt_seen), omitting tcpi_ecn_mode and tcpi_options2. Restore
the kernel layout:
uint32_t tcpi_ecn_mode:2,
tcpi_accecn_opt_seen:2,
tcpi_accecn_fail_mode:4,
tcpi_options2:24;
The overall structure size is unchanged. Also add the TCPI_ECN_MODE_*
and TCP_ACCECN_* value constants for these fields, introduced together
with them by Linux commit 4fa4ac5e5848 (Linux 7.0).
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Add OPEN_TREE_NAMESPACE (commit 9b8a0ba68246a61d903ce62c35c303b1501df28b,
Linux 7.0) and FSMOUNT_NAMESPACE (commit
5e8969bd192712419aae511dd5ba26855c2c78db, Linux 7.1).
THREAD_SET_STACK_GUARD reads the thread pointer via THREAD_SELF
(__builtin_thread_pointer), which the compiler treats as invariant. In the
static startup path the thread pointer is installed by the __libc_setup_tls
call that immediately precedes the guard store, so the read must stay below
it. The existing barrier only clobbered the access registers a0/a1, which
creates no dependency on the call, so the compiler could move the whole
barrier and read above __libc_setup_tls.
This is sensitive to instruction scheduling and recent TLS startup
changes exposed it on s390x.
Add a "memory" clobber to the barrier so it is tied to the call's memory
effects and cannot be hoisted above it. The macro is shared with the
dynamic loader, so both startup paths are covered.
I checked on s390x-linux-gnu build for arch5, arch8, arch9, and arch11
by running the elf tests on qemu system (kernel 6.1.0).
Reviewed-by: Stefan Liebler <stli@linux.ibm.com>
There are no new constants covered by tst-mman-consts.py or
tst-openat2-consts.py in Linux 6.18, 6.19, 7.0, or 7.1.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Linux 6.18 adds the uprobe syscall (x86_64 only), Linux 6.19 adds listns,
and Linux 7.0 adds rseq_slice_yield. Linux 7.0 also wires up clone3 on
sparc and memfd_secret on loongarch. Update syscall-names.list and
regenerate the arch-syscall.h headers with build-many-glibcs.py
update-syscalls.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
It was added by commit 2e7af192697ef2a71c76fd57860b0fcd02754e14, which
introduced the flags argument for sched_getattr.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The pidfd_info interface was extended to report coredump information:
* PIDFD_INFO_SUPPORTED_MASK, along with the supported_mask field, so
userspace can check which flags the running kernel supports (commit
dfd78546c95330db2252e0d7e937a15ab5eddb4e, Linux 6.19).
* PIDFD_INFO_COREDUMP_SIGNAL, along with the coredump_signal field
(commit 036375522be8425874e9e0f907c7127e315c7a52, Linux 6.19).
* PIDFD_INFO_COREDUMP_CODE, along with the coredump_code field (commit
701f7f4fbabbf4989ba6fbf033b160dd943221d5, Linux 7.1).
The struct pidfd_info is extended accordingly and PIDFD_INFO_SIZE_VER1,
PIDFD_INFO_SIZE_VER2, and PIDFD_INFO_SIZE_VER3 are added to reflect the
new struct sizes.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Add the FD_NSFS_ROOT (commit e83f0b5d10dcf62833008327cb661c7d118bca85,
Linux 6.18), and F_GETDELEG and F_SETDELEG (commit
1602bad16d7df82faca6d7c70821117684a66f49, Linux 6.19).
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The tests-container rule launches support/test-container, which is always
dynamically linked, through $(test-via-rtld-prefix). That prefix is empty
for tests listed in tests-static/xtests-static, so for a static container
test the helper ran under the system loader/libc instead of the newly
built one.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The default 'make check' serializes the timing-sensitive test runs: the
threading (nptl, or htl on Hurd) and realtime (rt) subdirectories run
with .NOTPARALLEL and are ordered after the rest of the test run, so they
are not perturbed by competing machine load.
Add a check-parallel (and xcheck-parallel) variant for when that is not
wanted -- an idle machine, or a run where the extra throughput is worth
the risk of flakiness in the timing-sensitive tests (check the
libc-alpha discussion [1] for more context why this approach was proposed).
A new serialize-tests flag (default yes, defined in Makeconfig) gates both
the per-subdirectory .NOTPARALLEL and the top-level run-time ordering;
check-parallel just runs the suite with serialize-tests=no, so every test
program builds and runs at full concurrency in a single pass.
'make check' and its default behavior are unchanged.
[1] https://inbox.sourceware.org/libc-alpha/lhutsr3khz7.fsf@oldenburg.str.redhat.com/
Reviewed-by: Sam James <sam@gentoo.org>
The check-installed-headers-c/-cxx tests ran one script invocation per
subdirectory over all of its installed headers, performing about 80
compiler invocations per header serially.
Give each header its own intermediate target so the compiler
invocations parallelize under the make jobserver, recording the
per-header script exit status next to the output. The .out target
concatenates the per-header outputs in the original $(headers) order
and fails if any recorded status is non-zero, so both the .out contents
(verified byte-identical for all 76 files) and the tests.sum results
are unchanged.
Results on a x86_64 machine [1] from a make check with run-built-tests=no
show neutral results, and on aarch64 machine [2] it improves from 241.574s
to 182.405.
[1] Ryzen 5900x, 12c/24t, gcc 16.1.1, binutils 2.26, and GNU make 4.3
[2] N1, 80c, gcc 15.1.1, binutils 2.25, GNU make 4.3
Reviewed-by: Sam James <sam@gentoo.org>
Every subdirectory tests target depends on
$(objpfx)testroot.pristine/install.stamp, whose recipe performs a full
DESTDIR installation (about 12s and 154MB on x86_64). The testroot is
only consumed by the container tests, which do not run when
run-built-tests is no, so skip it entirely in that case.
This saves about 10s on x86_64 [1] and 30s on aarch64 [2].
[1] Ryzen 5900x, 12c/24t, gcc 16.1.1, binutils 2.26, and GNU make 4.3
[2] N1, 80c, gcc 15.1.1, binutils 2.25, GNU make 4.3
Reviewed-by: Sam James <sam@gentoo.org>
The requirement that the elf subdirectory comes last in the
subdirectory ordering stems from its lib pass: the rtld link consumes
$(common-objpfx)libc_pic.a, which aggregates every other
subdirectory's objects.
The others, tests, and xtests classes have no such dependency: everything
they consume from other subdirectories is provided by the pass barriers
(others after lib, tests after others). Keep elf last only for the
object-building classes and let its others and tests sub-makes run
concurrently with the other subdirectories.
With elf no longer forced last for those classes, the Depend edges
pointing to elf (e.g. support/Depend) no longer create a cycle there,
so honor them instead of dropping them.
This improves the make check with run-built-tests=no, specially on
machine with many cores. Results on a x86_64 machine [1] it improves
from 190s to 181s, while on a aarch64 machine [2] it improves from
298.726s to 243.098s.
Build results remain bit-identical and the tests.sum failure sets are
unchanged.
[1] Ryzen 5900x, 12c/24t, gcc 16.1.1, binutils 2.26, and GNU make 4.3
[2] N1, 80c, gcc 15.1.1, binutils 2.25, GNU make 4.3
Reviewed-by: Sam James <sam@gentoo.org>
The top-level makefile was marked .NOTPARALLEL and ran the
per-subdirectory sub-makes strictly one at a time in the topological
order computed by scripts/gen-sorted.awk. Only the compilations inside
a single subdirectory could run in parallel, so on wide machines every
subdirectory's compile tail and link steps left most cores idle, once
per subdirectory per pass.
Drop .NOTPARALLEL and encode the ordering the serial recursion relied
on as explicit dependencies between the per-subdirectory targets:
* The subdirectories that generate shared files in $(common-objpfx)
consumed by the rest of the build without explicit dependencies run
serially, in their sorted order, before the rest fan out: csu
provides the tree-wide gen-as-const headers, and on Hurd the mach
and hurd directories generate the MiG RPC headers (every other
subdirectory otherwise runs a nested make in hurd/ to create them,
racing under parallel recursion; see sysdeps/mach/hurd/Makefile).
The first of them also materializes the other shared generated files
(abi-versions.h, sysd-syscalls, before-compile headers).
* The edges requested by the Depend files (now emitted by
gen-sorted.awk as subdir-deps-*) are preserved. Edges pointing to
elf are dropped, as the sorted list already overrides them by
forcing elf last.
* The tests and xtests classes only run the per-directory test
programs, which are mutually independent once the others pass has
built the tree. They therefore carry only the others pass barrier
below and none of the csu-first or Depend edges (+ordered_parallel_-
subdir_targets excludes them); otherwise "make subdir/tests" would
also run the tests of every subdirectory reachable through those
edges, rather than just the requested one.
* elf stays last: its rtld link consumes $(common-objpfx)libc_pic.a,
which aggregates every other subdirectory's objects, and its
rtld-Rules recursion compiles into the other subdirectories' object
directories.
* Pass barriers replace the implicit pass ordering: others after lib
(a subdirectory others sub-make would otherwise race to link
libc.so itself), tests/xtests after others, and the testroot
install behind others.
* The threading (nptl, or htl on Hurd) and realtime (rt) tests are
timing-sensitive and were previously shielded from system load by
the global .NOTPARALLEL. With the recursion now parallel, a full
test run ('make check'/'tests', run-built-tests=yes) orders them
after the rest of the test run and one group at a time -- the
threading subdirectory, then rt -- and each serializes its own run
via a .NOTPARALLEL in its Makefile. A targeted 'make subdir/tests'
is not ordered.
The serialization (the per-subdirectory .NOTPARALLEL and the ordering
above) constrains only the test run, not the build of the test
programs; but building and running a subdirectory's tests are fused
in its sub-make, so under run-built-tests=yes the serialized
subdirectories would also build their test programs serially. To
avoid that, the top-level 'make check' (in Makerules) now runs two
passes: it builds every test program with run-built-tests=no, where
the recursion is fully parallel and none of the serialization
applies, and then runs the tests with run-built-tests=yes. 'make
tests' and a subdirectory's own 'check' stay single pass.
* The subdirectory-built files that the top-level libc.so and
linkobj/libc_pic.a rules list as prerequisites (elf/ld.so,
interp.os, sofini.os, sunrpc/librpc_compat_pic.a, and on Hurd
mach/libmachuser_pic.a and hurd/libhurduser_pic.a, from which the
lib*user-link.so inputs of libc.so are built) get order-only edges
on the corresponding sub-make with an explicit empty recipe. A
prerequisite-only rule would trigger an implicit rule search and
this level would compile them itself in the wrong context.
* The install, clean, abi, and stubs target classes keep the
previous total order via a serial dependency chain.
* The elf DSO sorting test recipes, run when make remakes the
included generated makefiles at parse time, create the elf object
directory before writing into it; the serial recursion no longer
guarantees another rule created it first.
* catgets builds locale-specific message catalogs (and tst-catgets
reads one) by running gencat under de_DE.ISO-8859-1, hr_HR.ISO-8859-2
and ja_JP.SJIS, but never declared those locales as prerequisites: it
relied on localedata running before it in the serial order. Under
the parallel recursion gencat could run before localedata generated
the locale, fall back to C, and fail. catgets/Makefile now pulls the
locales in via gen-locales.mk, like the other subdirectories that use
locales in their tests.
Results on a x86_64 machine [1] with default configuration [3]: a
from-scratch build improves from 78.728s to 61s, and check with
run-built-tests=no from 374s to 190s.
On a 80-core aarch64 machine [2] with default configuration [3]: a
from-scratch build improves from 105.251s to 56.703s, and check with
run-built-tests=no from 886.183s to 298.726s.
Build results are unchanged: all 8919 built objects, archives, and
shared objects are bit-identical to the serial build across 7 clean
parallel builds, the installed tree layout is identical, and the
tests.sum failure sets are identical. i686-gnu was verified with
repeated from-scratch builds.
[1] Ryzen 5900x, 12c/24t, gcc 16.1.1, binutils 2.26, and GNU make 4.3
[2] N1, 80c, gcc 15.1.1, binutils 2.25, GNU make 4.3
[3] --enable-stack-protector=all --enable-bind-now=yes --enable-profile=yes
--enable-fortify-source=2 --enable-hardcoded-path-in-tests
Reviewed-by: Sam James <sam@gentoo.org>
gen-sorted.awk collects the inter-subdirectory dependencies from the
Depend files and the sysdeps Subdirs 'first' directives, but only uses
them to compute the flattened sorted-subdirs list.
This change also emit the raw edges as subdir-deps-<dir> make variables
so the parent makefile can express the same ordering as explicit
dependencies between the per-subdirectory recursion targets and run
independent subdirectories in parallel.
For instance, on x86_64-linux-gnu build the 'sysd-sorted' now shows:
[...]
subdir-deps-assert += iconvdata
subdir-deps-assert += localedata
subdir-deps-catgets += intl
subdir-deps-debug += localedata
subdir-deps-iconvdata += iconv
subdir-deps-iconvdata += localedata
subdir-deps-intl += iconvdata
subdir-deps-intl += localedata
subdir-deps-libio += localedata
subdir-deps-localedata += locale
[...]
Reviewed-by: Sam James <sam@gentoo.org>
Add a new section to the Dynamic Linker chapter describing the
STT_GNU_IFUNC mechanism: the resolver calling convention for each
supported architecture, when resolvers run, and the functionality
supported in resolvers (TLS access, stack protector, cross-object
references) along with the remaining restrictions, reflecting the
recent ordering fixes (BZ 20680, BZ 23240, BZ 28817, BZ 34164,
BZ 34170). Also cross-reference it from the hardening section.
Reviewed-by: Yury Khrustalev <yury.khrustalev@arm.com>
The parse_tilde function previously used strndupa to allocate memory
for the parsed username on the stack, and since the input is
user-defined, this can lead to a stack overflow.
This patch fixes the issue by replacing strndupa with scratch_buffer,
by reusing the buffer used in the __getpwnam_r call.
The new “tst-wordexp-tilde.c” test is a test-container to avoid using
system-defined NSS modules.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Commit a5ec880f80 ("Consolidate pointer
guard to use a relro variable instead of the TCB") moved the pointer
guard to the module-local relro variable __pointer_chk_guard_local and
dropped the THREAD_{SET,COPY}_POINTER_GUARD macros, which are no longer
defined by any target.
Checked on a build for all afftected targets.
_dl_reseed_random zeroed the AT_RANDOM bytes before refilling them with
getrandom (GRND_NONBLOCK). That call is best-effort and can write nothing
(e.g. before the kernel entropy pool is initialized early at boot), leaving
the 16 bytes as a constant zero.
Since the value is exposed through getauxval (AT_RANDOM), this traded a
potential guard leak for a guaranteed predictable value.
Drop the memset and overwrite the bytes in place instead. The generic
version now just drops the pointer (it is not used anywhere).
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Once the pointer and stack guards have been derived from AT_RANDOM, scrub
the bytes and refill them with new random data unrelated to the guards.
On Linux, it uses getrandom syscall (as for tcache_key_initialize), and
fallback to zero the memory if the syscall is not avaiable.
This keeps AT_RANDOM useful to applications while ensuring those bytes no
longer reveal the guards.
The work is done by _dl_reseed_random, called once the guards are in place
and before any ELF constructor can observe AT_RANDOM: in security_init for
the dynamic loader and in __libc_start_main for statically linked programs.
Checked on aarch64-linux-gnu, x86_64-linux-gnu, and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
The generic and the Linux _dl_setup_stack_chk_guard and
_dl_setup_pointer_guard are identical when the kernel provides the
AT_RANDOM auxiliary vector, which is always the case on the supported
Linux kernels; the generic version only adds a fallback for a missing
AT_RANDOM. Remove the Linux-specific dl-osinfo.h and use the generic
one for all targets.
The removed header pulled in <dl-sysdep.h>, which is what transitively
declared __libc_enable_secure for dl-parse_auxv.h; include <unistd.h>
there directly so it no longer relies on that. Include <string.h> in
the generic header so it is self-contained.
Reviewed-by: DJ Delorie <dj@redhat.com>
The per-architecture pointer_guard.h files all implemented some variant
of C PTR_MANGLE/PTR_DEMANGLE: the i386 and x86_64 versions did an
exclusive-or by the guard followed by a rotate left by
2 * sizeof (uintptr_t) + 1 bits (9 on 32-bit, 17 on 64-bit), while the
other targets did the exclusive-or only. The assembly macros likewise
rotated on i386 and x86_64 but not on the other targets.
Consolidate everything to use XOR + rotate uniformly: collapse the
per-architecture C headers into a single generic implementation, and
add the matching rotate to the assembly implementations that lacked it.
Targets with an assembly rotate instruction (aarch64, arm, loongarch,
powerpc, s390, csky) just add it. Those without one (alpha, sparc, sh)
synthesize the rotation: alpha and sparc with a shift/shift/or sequence,
which needs an extra scratch register added to the PTR_MANGLE/PTR_DEMANGLE
macros, sh with single-bit rotates, which needs no scratch. The s390
__longjmp and ____longjmp_chk demangle the return address and stack
pointer in C, so update those too.
Checked with setjmp, ptrguard and longjmp_chk tests for all supported
ABIs that can run under qemu (all but csky), and with builds for all
supported ABIs.
Reviewed-by: DJ Delorie <dj@redhat.com>
arc, m68k, microblaze, mips, and riscv store the raw stack pointer in
setjmp, while their jmpbuf-unwind.h applies PTR_DEMANGLE to it in
_jmpbuf_sp. This is currently harmless because these ABIs use the
generic pointer_guard.h, where the C PTR_DEMANGLE is a no-op.
No functional change.
Reviewed-by: DJ Delorie <dj@redhat.com>
Move the assembly PTR_MANGLE/PTR_DEMANGLE definitions out of each
pointer_guard.h into a new sibling pointer_guard-asm.h, guarded by
__ASSEMBLER__. pointer_guard.h now includes pointer_guard-asm.h and
keeps only the C definitions.
No functional change: the .S and .c files still include pointer_guard.h
and get the same macros, and <pointer_guard-asm.h> resolves per-arch
through the usual sysdeps include path.
Checked with a build for all affected ABIs.
Reviewed-by: DJ Delorie <dj@redhat.com>
i386, powerpc, s390, sh and sparc fell back to the generic no-op
pointer_guard.h in ld.so, so the loader's own setjmp/longjmp and mangled
function pointers were left unprotected (the other targets already
mangle in rtld). ld.so is position-independent and sets up
__pointer_chk_guard_local early in dl_main, so the loader can reach it
the same way a statically linked program does:
- powerpc, s390, sh, sparc: select __pointer_chk_guard_local for
IS_IN (rtld) and reuse the existing PIC guard load.
- i386: load it via @GOTOFF off a GOT pointer set up with LOAD_PIC_REG
in the caller-provided scratch register
(ld.so does not keep the GOT in %ebx at the setjmp/longjmp entry).
- s390: drop the IS_IN (rtld) "#undef PTR_DEMANGLE" in __longjmp.c so
the loader's longjmp demangles to match its setjmp.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, i686-linux-gnu, and
armv7a-linux-gnueabihf. I also run elf and setjmp checks on qemu
loongarch64, powerpc64, powerpc64le, and s390x systems.
Reviewed-by: DJ Delorie <dj@redhat.com>
For the targets that still read the pointer guard from the thread
descriptor (i386, x86_64, powerpc, s390, sparc, and sh), mangle and
demangle pointers using the module-local __pointer_chk_guard_local (in
ld.so and static binaries) or the global __pointer_chk_guard provided by
the dynamic loader (in shared objects), matching the scheme already used
by aarch64, alpha, arm, csky and loongarch.
On i386 the guard is no longer reachable as a single %gs-relative memory
operand, so the assembly PTR_MANGLE/PTR_DEMANGLE PIC variants need a
register to hold the GOT pointer and the guard address.
This removes the dependency on the TCB pointer_guard slot: drop
THREAD_{GET,SET,COPY}_POINTER_GUARD and the POINTER_GUARD tcb-offset,
and point the POINTER_CHK_GUARD test macro at the relro variable. The
tcbhead_t slot is kept (renamed to an unused reserved field, except on
s390 which reused the stack guard) to preserve the ABI. s390 gains a
distinct pointer guard rather than aliasing the stack guard.
Checked on x86_64-linux-gnu, aarch64-linux-gnu, i686-linux-gnu, and
armv7a-linux-gnueabihf. I also run elf and setjmp checks on qemu
loongarch64, powerpc64, powerpc64le, and s390x systems.
Reviewed-by: DJ Delorie <dj@redhat.com>
The static-dlopen does not initialize the pointer guard for ABIs that
define THREAD_SET_POINTER_GUARD. Besides not properly guard the
pointer if a libc.so symbol is called, this can lead to setjmp
failures (a jmp_buf set up by the loaded libc.so.6 cannot be restored
by the static program's __longjmp, and vice versa).
Seed the just-mapped loader's __pointer_chk_guard from the program's
__pointer_chk_guard_local in __rtld_static_init, next to the other
runtime values copied there.
Checked on aarch64-linux-gnu, x86_64-linux-gnu, and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
It syncs with CORE-MATH commit 887cab6f. The new tests are from
the CORE-MATH acosh worst-cases inputs that trigger the FE_TOWARDZERO
issue with the previous implementation, along with the hard-to-round
cases from CORE-MATH commits 72febed6 and ce9e99f6.
Checked on x86_64-linux-gnu, x86_64-linux-gnu with --disable-multi-arch
(so soft FMA implementation is used), i686-linux-gnu (which uses its
own acosh implementation), aarch64-linux-gnu, and
armv7a-linux-gnueabihf.
Reviewed-by: Paul Zimmermann <Paul.Zimmermann@inria.fr>
_dl_runtime_resolve and _dl_runtime_profile only preserved the integer
argument registers (r0-r3) across the inner call to _dl_fixup /
_dl_profile_fixup. Two related ABI requirements demand more:
* Under AAPCS-VFP, d0-d7 hold the caller's double arguments to the
function being resolved. Recent GCC emits VFP instructions inside
the fixup routines, clobbering them, so the resolved function sees
corrupted arguments (BZ 34144).
* Per RTABI32, the __aeabi_mem* helpers (and similar runtime helpers
reachable through the dynamic linker) must only corrupt integer
core registers. IFUNC resolvers, audit modules, and interposed
malloc invoked during symbol resolution may also use VFP, even on
softfp ABI builds (BZ 15792).
Save all call-clobbered VFP state -- d0-d15 unconditionally, d16-d31
when HWCAP_ARM_VFPD32 is set, and fpscr -- around the inner fixup
call. Whether VFP is usable is a property of the hardware, not of
the ABI glibc was built with, so the decision is gated on AT_HWCAP at
runtime in both hardfp and softfp builds; hardfp builds will always
find HWCAP_ARM_VFP set, while softfp builds running on a non-VFP CPU
correctly skip the save.
For _dl_runtime_profile the save area is slipped in just before the
bl to _dl_profile_fixup; the outgoing framesizep argument is
recomputed to account for the extra frame, and both the fast path
(no audit framesize) and the slow path (audit wraps with
pltenter/pltexit) traverse the restore before splitting.
Checked on arm-linux-gnueabihf.
Tested-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
ARCH_INIT_CPU_FEATURES is called from __libc_start_main before
_dl_relocate_static_pie, so any code it runs must not rely on
relocations being applied yet. Two issues break this under static PIE:
1. The cpu_list table held 'const char *name' members, whose
addresses are not link-time constants under PIE and thus require
base-relative relocations that are not yet in place. Replace the
array of structs with a flat NUL-separated name string and a
parallel midr array, removing the pointer relocations.
2. tunable_strcmp pulls in memcmp, which is an IFUNC and is likewise
unresolved at this point. Redirect memcmp to __memcmp_generic in
the aarch64 dl-symbol-redir-ifunc.h and include it from
csu/libc-start.c.
Checked on aarch64-linux-gnu.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
With 01964c3ec8e fix ifunc resolvers can be fully instrumented with
stack protector.
Checked on aarch64-linux-gnu, x86_64-linux-gnu, and i686-linux-gnu
built with --enable-stack-protector=all.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
An IFUNC resolver in a DSO that is being loaded by dlopen is allowed
to read its own TLS storage during the resolver call. After
af34b1376a ("elf: Initialize static TLS before relocation processing
(BZ 34164)") that works for the initial-exec model on every supported
architecture.
However, it does not work for the dynamic-TLS path (on both -mtls-dialect mode,
if the ABI supports both). Both lookup paths index the calling thread's
DTV by the new module's l_tls_modid and, on miss, walk
GL(dl_tls_dtv_slotinfo_list) to discover the module and lazily allocate
its TLS block. The just-loaded DSO is however not yet in that list when
its resolver fires, so the lookup faults inside dlopen. This is the
direct dlopen analog of BZ 34164.
The solution is to reorder dl_open_worker_begin so the slotinfo install
happens before the relocation pass. The new order is:
1. resize_scopes, resize_tls_slotinfo, add_to_global_resize
(unchanged, still recoverable).
2. update_tls_slotinfo: register the new modules in slotinfo, bump
dl_tls_generation, initialise their static TLS images.
3. Relocate the new objects. IFUNC resolvers can now read their
own DSO's __thread storage via any TLS model.
4. Demarcation point.
5. update_scopes, _dl_find_object_update.
Checked on aarch64-linux-gnu and x86_64-linux-gnu.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
The glibc.malloc.hugetlb=1 is redundant with kernel is n 'always'
mode, but the madvise does work, and the tunable should not fail.
Suggested-by: Yury Khrustalev <yury.khrustalev@arm.com>
Reviewed-by: Yury Khrustalev <yury.khrustalev@arm.com>
Commit c2d6afb4a0 changed the PIC && !SHARED path of start.S (used by
crt1.o and rcrt1.o) to load main via a R_ARM_REL32 data relocation. When
main is provided by a shared object, BFD ld synthesizes a canonical PLT
entry for it, but other linkers (e.g. mold) do not, producing a broken
binary that crashes at startup.
Follow the aarch64 approach and reference main through a local __wrap_main
that tail-calls it with a branch relocation, which every linker turns into
a PLT entry.
Checked on armv7a-linux-gnueabihf with and without --enable-default-pie.
The test failed with --disable-default-pie because its primary check read
the resolver's diagnostic side effect before the resolver had run.
In a non-PIE executable the references to the IFUNC (fptr and ifunc_tls)
are satisfied through a canonical IPLT entry in the executable itself.
Under the default lazy binding that IPLT is resolved on first use, not
during startup relocation, so the resolver had not yet run when
'check_sentinel' inspected that value. With a PIE executable
(or LD_BIND_NOW=1) the resolver runs eagerly at startup and the check
passed. The dlopen path was unaffected because dlopen resolves the
data relocation eagerly.
This is a test ordering issue: the resolver always reads the initial-exec
TLS correctly whenever it runs, so the BZ 34164 fix is not involved.
Reorder test_tls_ifunc so that fptr/ifunc_tls force the IFUNC to be
resolved before the last_seen_sentinel value is inspected.
Checked on x86_64-linux-gnu and i686-linux-gnu with --disable-default-pie.
Reported-by: Yury Khrustalev <yury.khrustalev@arm.com>
Reviewed-by: Yury Khrustalev <yury.khrustalev@arm.com>
After "elf: Initialize TCB and stack-protector before static IFUNC
resolvers (BZ 20680, BZ 27582, BZ 28817)", early loader and static
initialization code may run before the IFUNC resolvers are processed.
The armv7-a supports memcpy, so add a dl-symbol-redir-ifunc.h to
redirect the memcpy to to __memcpy_arm implementation.
Checked on armv7a-linux-gnueabihf.
The swallow argument in the INT_FIELD and STRING_FIELD macros is used as a
boolean, change all callers to use false and true instead of 0 and 1.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
auto-libm-test-in shares inputs between narrowing and non-narrowing
functions, and some carry arg-format conditions (e.g. "arg-ibm128").
When auto-libm-test-out-fma is regenerated, gen-libm-test.py turns these
into TEST_COND_arg_ibm128, which expands via ARG_MANT_DIG to ARG_PREFIX.
ARG_PREFIX is only defined for TEST_NARROW, so the non-narrowing fma test
failed to build with "ARG_PREFIX_MANT_DIG undeclared".
Define TEST_COND_arg_ibm128 to 0 when ARG_FLOAT is not defined, mirroring
the existing guard for TEST_COND_ibm128_libgcc; in the non-narrowing case
there is no separate argument format, so the condition is always false.
Regenerate auto-libm-test-out-fma accordingly.
When d (the exponent difference between z and x*y) is exactly 64,
the alignment path shifts z left by 64 bits via (zhi = nz.m, zlo = 0)
and decrements d to 0, then takes the inner 'if (d < 64)' branch
which evaluates 'rhi << (64 - d)' with d == 0. A shift by 64 of a
64-bit value is UB in C.
Add the explicit 'if (d == 0)' empty branch (present in the
original musl implementation).
Checked on x86_64-linux-gnu with --disable-multi-arch and
arm-linux-gnueabihf.
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Adhemerval Zanella