This avoids a regression in the omnetpp and xalancbmk benchmarks
of SPEC. Apparently, these benchmarks are very sensitive to the
fill rate in malloc.
Suggested-by: Wilco Dijkstra <wilco.dijkstra@arm.com>
Tested-by: Wilco Dijkstra <wilco.dijkstra@arm.com>
Batched frees amortize the locking overhead once the tcache is full.
Previously, once the tcache slot was full, each free acquired the
arena lock. With this change, neighboring tcache allocations in the
same arena re-use an arena lock that already exists.
Pass the tcache pointer to __libc_free_batch so that it is not
necessary to reload the tcache pointer after the _int_free_chunk
calls.
Both ld.so and malloc track kernel THP mode and THP page size when THP
in ld.so is enabled by
GLIBC_TUNABLES=glibc.elf.thp=1
and THP in malloc is enabled by
GLIBC_TUNABLES=glibc.malloc.hugetlb=1
But DL_MAP_DEFAULT_THP_PAGESIZE and MALLOC_DEFAULT_THP_PAGESIZE may be
different when they are defined in <hugepages.h>.
If THP in ld.so is enabled, change malloc to use kernel THP mode from
ld.so, instead of calling __get_thp_mode, and use THP page size from
ld.so if it came from __get_thp_size. This avoids calling __get_thp_mode
and __get_thp_size again, which are quite expensive:
1. Initialize mp_.thp_mode and GL(dl_thp_mode) to thp_mode_unknown.
2. Set mp_.thp_mode to GL(dl_thp_mode) if GL(dl_thp_mode) isn't
thp_mode_unknown. Otherwise call __get_thp_mode to set mp_.thp_mode.
3. GL(dl_elf_thp_pagesize) is set to DL_MAP_DEFAULT_THP_PAGESIZE without
calling __get_thp_size and THP page size for malloc may be different from
THP page size for ld.so. Set mp_.thp_pagesize to GL(dl_elf_thp_pagesize)
if DL_MAP_DEFAULT_THP_PAGESIZE is defined. Otherwise call __get_thp_size
to set mp_.thp_pagesize.
4. THP page size in malloc is capped to MAX_THP_PAGESIZE. If THP page
size is above MAX_THP_PAGESIZE, THP in malloc is disabled.
These result in when glibc.elf.thp is set to 1, malloc uses the actual
kernel THP mode instead of defaulting to madvise mode and madvise_thp
will stop issuing MADV_HUGEPAGE if kernel THP mode is always.
This fixes BZ #34083.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
When scanf parses special floating-point inputs like "nan", "inf", it
may read one extra character to check whether the input still matches
the expected word.
If that extra character does not match, the conversion should fail, but
the character has already been read from the stream. Some error paths
did not push it back before reporting the conversion error, so later input
started at the wrong position.
Fix this by keeping EOF and width errors separate from character mismatch
errors. Push the character back before calling conv_error for mismatch
errors.
Add narrow and wide scanf tests using the same test body. The test cases
focus on the paths changed by this patch: mismatches while matching "nan",
"nan(...)", "inf", and "infinity".
Tested on x86_64 Fedora 42.
The following tests passed without regressions:
make test t=stdio-common/tst-scanf-bz34208
make test t=stdio-common/tst-wscanf-bz34208
make subdirs="stdio-common libio" check
Signed-off-by: Xiang Gao <gaoxiang@kylinos.cn>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Since the NSS reorganization in glibc 2.33 the --enable-static-nss
option no longer changes the build: the files and dns services are built
into libc unconditionally, and there is no longer any mechanism to bake
the remaining services into libc.a. After removing the dead
-DSTATIC_NSS define and the <gnu/lib-names.h> include (commits
82b710c190 and
2169401438), the option's only remaining
effect was to define DO_STATIC_NSS, which suppresses the static link
warnings on the NSS interface functions in libc.a.
With DO_STATIC_NSS gone, nss_interface_function emits the static link
warning unconditionally; static_nss and build-static-nss are removed
along with the configure option.
tst-nss-static, previously built only when build-static-nss was set, is
now added to tests-static unconditionally so the static NSS path retains
test coverage.
Signed-off-by: Michael Ford <fanquake@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Add a new section to the Dynamic Linker chapter describing the
STT_GNU_IFUNC mechanism: the resolver calling convention for each
supported architecture, when resolvers run, and the functionality
supported in resolvers (TLS access, stack protector, cross-object
references) along with the remaining restrictions, reflecting the
recent ordering fixes (BZ 20680, BZ 23240, BZ 28817, BZ 34164,
BZ 34170). Also cross-reference it from the hardening section.
Reviewed-by: Yury Khrustalev <yury.khrustalev@arm.com>
Restore the POWER10 memchr implementation reverted in commit
a7877bb668 [BZ #33059], addressing the
actual cause:
The non-volatile vector register v20 used as the
zero vector (M_VREG_ZERO) is replaced with the volatile register v17.
Additionally, the assembler workaround used earlier for the
instructions lxvp and vextractbm is replaced with actual
instructions.
The RISC-V libthread_db THREAD_SELF descriptor currently assumes 64-bit
general-register slots:
REGISTER (64, 64, 4 * 8, ...)
That is correct for RV64, where tp/x4 is at byte offset 4 * 8 in the
general register set. On RV32, general-register slots are 32-bit wide,
so tp/x4 is at byte offset 4 * 4.
Using the RV64 layout on RV32 makes libthread_db read the wrong thread
pointer value and prevents GDB from enabling pthread-aware thread
debugging.
Use a helper macro to pass the RISC-V register size into the
DB_THREAD_SELF definition.
Signed-off-by: Shengwen Cheng <shengwen1997.tw@gmail.com>
Reviewed-by: Peter Bergner <bergner@tenstorrent.com>
Since R_ARC_S25W_PCREL range is limited, 32MB THP code padding leads to
relocation overflow:
.../support/libsupport_nonshared.a(support_test_main.oS):
in function `support_test_main':
.../support/support_test_main.c:396:(.text+0x650): relocation truncated to fit: R_ARC_S25W_PCREL_PLT against symbol `atexit' defined in .text section in
../glibc/libc_nonshared.a(atexit.oS)
.../install/compilers/arc-linux-gnu/arc-glibc-linux-gnu/bin/ld: final link failed
collect2: error: ld returned 1 exit status
Limit THP code padding to 2 * THP page size to avoid R_ARC_S25W_PCREL
relocation overflow.
Tested natively for Linux/x86-64 and Linux/i686 as well as with
build-many-glibcs.py for arc-linux-gnu.
This fixes BZ #34311
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The new structure matches that of Arm Optimized-routines,
to facilitate files traceability.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The feupdateenv function is supposed to raise exceptions after
installing the environment represented by its envp argument.
This was accidentally missed on hppa.
The failure to raise exceptions was noticed by the failure of
the math/test-narrowing-trap test.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Add a test for the buffer overflow in gethostbyname_r (Bug 18287), which
occurs when alignment padding is not subtracted from the remaining buffer
length.
The test uses binary search to find the exact required buffer size
for a mocked DNS response. It then tests all pointer misalignments
(1-7 bytes) using a custom 0xAA guard region. This guarantees
deterministic detection of out-of-bounds writes, avoiding false
negatives caused by malloc chunk rounding.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Change the definition of _PATH_MOUNTED to "/etc/mtab". This is the
value used on Linux, and BSD also uses /etc/mtab.
The change is motivated by the fact that on Debian /etc/mtab is a
symbolic link to /proc/mounts. This patch adjusts the macro for
non-linux systems such as Hurd. Changing this using
sysdeps/mach/hurd/paths.h causes build problems because
/usr/include/hurd/paths.h is shadowed by this file. This change is
proposed in the hope that aligning the non-linux targets with the
glibc for Linux is perceived as a good thing while fixing this problem
on Debian/Hurd along the way.
The parse_tilde function previously used strndupa to allocate memory
for the parsed username on the stack, and since the input is
user-defined, this can lead to a stack overflow.
This patch fixes the issue by replacing strndupa with scratch_buffer,
by reusing the buffer used in the __getpwnam_r call.
The new “tst-wordexp-tilde.c” test is a test-container to avoid using
system-defined NSS modules.
Checked on x86_64-linux-gnu and i686-linux-gnu.
This patch adds an RVV-optimized implementation of memmove for RISC-V and
enables it for both multiarch (IFUNC) and non-multiarch builds.
The implementation integrates Hau Hsu's 2023 RVV work under a unified
ifunc-based framework. A vectorized version (__memmove_vector) is added
alongside the generic fallback (__memmove_generic). The runtime resolver
selects the RVV variant when RISCV_HWPROBE_KEY_IMA_EXT_0 reports vector
support (RVV).
Currently, the resolver still selects the RVV variant even when the RVV
extension is disabled via prctl(). As a consequence, any process that
has RVV disabled via prctl() will receive SIGILL when calling memmove().
Co-authored-by: Hau Hsu <hau.hsu@sifive.com>
Co-authored-by: Jerry Shih <jerry.shih@sifive.com>
Signed-off-by: Yao Zihong <zihong.plct@isrc.iscas.ac.cn>
Reviewed-by: Peter Bergner <bergner@tenstorrent.com>
The current THP segment load approach works only when THP is enabled
with always in the kernel. If THP is enabled with madvise in the
kernel, to enable THP segment load in an application, madvise should
be called with MADV_HUGEPAGE on all THP eligible PT_LOAD segments:
1. Define DL_MAP_DEFAULT_THP_PAGESIZE in hugepages.h and default it to 0.
If DL_MAP_DEFAULT_THP_PAGESIZE is defined, assume kernel THP madvise mode.
If kernel THP mode is always or never, there is an extra madvise call
which has no impact. DL_MAP_DEFAULT_THP_PAGESIZE is defined for x86-64
and 64-bit loongarch.
2. Update _dl_map_segment_align to support madvise THP mode. This fixes
BZ #34079.
3. Call _dl_executable_postprocess in rtld_setup_main_map for dynamic
executables and in LIBC_START_MAIN for static executables, which calls
madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments in
executable. This fixes BZ #34080 for both dynamic and static executables.
4. Call _dl_postprocess_loadcmd_extra in _dl_postprocess_loadcmd, which
calls madvise with MADV_HUGEPAGE on all THP eligible PT_LOAD segments
when loading an object after they have been mapped in. This fixes
BZ #34080 for shared objects.
5. Set the maximum page alignment on THP tests to THP page size as the
default maximum page alignment may be smaller than THP page size.
6. Add tests to verify that large executable PT_LOAD segments in
executables are mapped at addresses aligned to THP page size when the
kernel is configured to use THP in "always" mode or "madvise" mode by
inspecting /proc/self/maps to check that the mapping address is aligned
to THP page size reported by the kernel. Also verify that madvise is
called with MADV_HUGEPAGE when the glibc tunable glibc.elf.thp=1 is used
and madvise isn't called with MADV_HUGEPAGE when the glibc tunable
glibc.elf.thp=0 is used.
Skip these tests if THP page size cannot be determined or if THP is not
enabled in "always" mode nor "madvise" mode.
Quote WANG Rui <wangrui@loongson.cn>:
From benchmarking a clang build of the Linux kernel on x86_64 with
your patch in THP madvise mode, I observed that iTLB misses were
reduced, similar to what we see in THP always mode.
NB: Some THP tests fail on arm due to limitations of arm32 kABI:
https://sourceware.org/bugzilla/show_bug.cgi?id=34096
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Commit a5ec880f80 ("Consolidate pointer
guard to use a relro variable instead of the TCB") moved the pointer
guard to the module-local relro variable __pointer_chk_guard_local and
dropped the THREAD_{SET,COPY}_POINTER_GUARD macros, which are no longer
defined by any target.
Checked on a build for all afftected targets.
This test case covers both input buffer overreads and output buffer
overflows. It should systematically cover these issues.
I used code auto-generation for updating the test expectations for
truncated RDATA in TXT, ISDN records, after writing the rest
of the test by hand.
Assisted-by: LLM
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Check that the RDATA payload does not require more than RDATALEN
bytes while processing it. The fixes cover A6, LOC records.
(CERT, TKEY, TSIG were fixed before, by switching to the generic
formatter.)
The vulnerable LOC record handling was first introduced before
glibc 2.0, in commit ee188d555b.
CERT, TSIG, TKEY handling came with commit
b43b13ac25, released with glibc 2.2.
A6 record handling was introduced in commit
9163381643 ("* resolv/ns_print.c
(ns_sprintrrf): Handle ns_t_a6 and ns_t_opt."), which went into glibc
2.7.
This fixes bug 34069.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This makes the output more consistent (either failure or complete
output) and helps with systematic testing with varying buffer
sizes.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Do not add the "unknown RR type" comment. After adding the TYPE
prefix, the number is largely redundant.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The p_class and p_type results could overwrite each other if both
were unknown. Format unknown values with CLASS and TYPE prefixes,
as in RFC 3597. Handle A6 separately because it cannot be added
to __p_type_syms for ABI reasons.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
It turns out there is a missing inner length check in it, too.
Also fix the vulnerable commit. It predates the glibc 2.0 release
because the old stream-based formatting code in resolv/res_debug.c had
the same bug in its LOC handling.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Add matches before the start to ensure the memchr test checks they do not
result in false matches or missed matches.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Implemented double precision vector fallback for SVE sin/cos.
Includes rewrite of sin to use trig instructions.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Implemented single precision vector fallback for SVE sinf/cosf.
Includes a rewrite of SVE sinf to use the trig instructions.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Implemented single precision vector fallback for AdvSIMD sinf/cosf.
Includes cleanup of the fast pass to remove old styling and conventions.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
_dl_reseed_random zeroed the AT_RANDOM bytes before refilling them with
getrandom (GRND_NONBLOCK). That call is best-effort and can write nothing
(e.g. before the kernel entropy pool is initialized early at boot), leaving
the 16 bytes as a constant zero.
Since the value is exposed through getauxval (AT_RANDOM), this traded a
potential guard leak for a guaranteed predictable value.
Drop the memset and overwrite the bytes in place instead. The generic
version now just drops the pointer (it is not used anywhere).
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Update memchr description to indicate it cannot overread and the size
may be larger than the object passed. This is required since C11.
Suggested-by: Paul Eggert <eggert@cs.ucla.edu>
Reviewed-by: Paul Eggert <eggert@cs.ucla.edu>
Once the pointer and stack guards have been derived from AT_RANDOM, scrub
the bytes and refill them with new random data unrelated to the guards.
On Linux, it uses getrandom syscall (as for tcache_key_initialize), and
fallback to zero the memory if the syscall is not avaiable.
This keeps AT_RANDOM useful to applications while ensuring those bytes no
longer reveal the guards.
The work is done by _dl_reseed_random, called once the guards are in place
and before any ELF constructor can observe AT_RANDOM: in security_init for
the dynamic loader and in __libc_start_main for statically linked programs.
Checked on aarch64-linux-gnu, x86_64-linux-gnu, and i686-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
The generic and the Linux _dl_setup_stack_chk_guard and
_dl_setup_pointer_guard are identical when the kernel provides the
AT_RANDOM auxiliary vector, which is always the case on the supported
Linux kernels; the generic version only adds a fallback for a missing
AT_RANDOM. Remove the Linux-specific dl-osinfo.h and use the generic
one for all targets.
The removed header pulled in <dl-sysdep.h>, which is what transitively
declared __libc_enable_secure for dl-parse_auxv.h; include <unistd.h>
there directly so it no longer relies on that. Include <string.h> in
the generic header so it is self-contained.
Reviewed-by: DJ Delorie <dj@redhat.com>
The per-architecture pointer_guard.h files all implemented some variant
of C PTR_MANGLE/PTR_DEMANGLE: the i386 and x86_64 versions did an
exclusive-or by the guard followed by a rotate left by
2 * sizeof (uintptr_t) + 1 bits (9 on 32-bit, 17 on 64-bit), while the
other targets did the exclusive-or only. The assembly macros likewise
rotated on i386 and x86_64 but not on the other targets.
Consolidate everything to use XOR + rotate uniformly: collapse the
per-architecture C headers into a single generic implementation, and
add the matching rotate to the assembly implementations that lacked it.
Targets with an assembly rotate instruction (aarch64, arm, loongarch,
powerpc, s390, csky) just add it. Those without one (alpha, sparc, sh)
synthesize the rotation: alpha and sparc with a shift/shift/or sequence,
which needs an extra scratch register added to the PTR_MANGLE/PTR_DEMANGLE
macros, sh with single-bit rotates, which needs no scratch. The s390
__longjmp and ____longjmp_chk demangle the return address and stack
pointer in C, so update those too.
Checked with setjmp, ptrguard and longjmp_chk tests for all supported
ABIs that can run under qemu (all but csky), and with builds for all
supported ABIs.
Reviewed-by: DJ Delorie <dj@redhat.com>
arc, m68k, microblaze, mips, and riscv store the raw stack pointer in
setjmp, while their jmpbuf-unwind.h applies PTR_DEMANGLE to it in
_jmpbuf_sp. This is currently harmless because these ABIs use the
generic pointer_guard.h, where the C PTR_DEMANGLE is a no-op.
No functional change.
Reviewed-by: DJ Delorie <dj@redhat.com>
Florian Weimer
H.J. Lu
Jiri Stransky
Xiang Gao
Michael Ford
Adhemerval Zanella
Sachin Monga
Shengwen Cheng
Mike FABIAN
Pierre Blanchard
John David Anglin
Sergey Kolosov
Justus Winter
Carlos O'Donell
Wilco Dijkstra
Dylan Fleming
Yury Khrustalev
Samuel Thibault