Add support for MTE to strncmp. Regression tested with xcheck and benchmarked
with glibc's benchtests on the Cortex-A53, Cortex-A72, and Neoverse N1.
The existing implementation assumes that any access to the pages in which the
string resides is safe. This assumption is not true when MTE is enabled. This
patch updates the algorithm to ensure that accesses remain within the bounds
of an MTE tag (16-byte chunks) and improves overall performance.
Co-authored-by: Branislav Rankov <branislav.rankov@arm.com>
Co-authored-by: Wilco Dijkstra <wilco.dijkstra@arm.com>
(cherry picked from commit 03e1378f94)
The variant PCS support was ineffective because in the common case
linkmap->l_mach.plt == 0 but then the symbol table flags were ignored
and normal lazy binding was used instead of resolving the relocs early.
(This was a misunderstanding about how GOT[1] is setup by the linker.)
In practice this mainly affects SVE calls when the vector length is
more than 128 bits, then the top bits of the argument registers get
clobbered during lazy binding.
Fixes bug 26798.
(cherry picked from commit 558251bd87)
This enables searching shared libraries in atomics/ when the hardware
supports LSE atomics of armv8.1 so one can provide optimized variants
of libraries in a portable way.
LSE atomics does not affect library abi, the new instructions can
interoperate with old ones.
I considered the earlier comments on the patch
https://sourceware.org/ml/libc-alpha/2018-04/msg00400.htmlhttps://sourceware.org/ml/libc-alpha/2018-04/msg00625.html
It turns out that the way glibc dynamic linker decides on the search
path is not very flexible: it wants to use hwcap bits and associated
strings. So some targets reuse hwcap bits for glibc internal purposes
to affect the search logic. But hwcap is an interface with the kernel,
glibc should not allocate bits in it for its internal logic as that
limits future hwcap extensions and confusing to users who expect to see
hwcap bits in ifunc resolvers. Instead of rewriting the dynamic linker
path logic (which affects all targets) this patch just uses the existing
mechanism, however this means that the path name has to be the hwcap
name "atomics" and cannot be changed to something more meaningful to
users.
It is hard to tell how much performance benefit this can give, in
principle armv8.1 atomics can be better optimized in the hardware, so it
can make a difference for synchronization heavy code. On some systems
such multilib setup may be the only viable way to get optimized
libraries used.
* sysdeps/unix/sysv/linux/aarch64/dl-procinfo.h (HWCAP_IMPORTANT): Add
HWCAP_ATOMICS.
(cherry picked from commit 397c54c1af)
This partially reverts
commit f82e9672ad
Author: Siddhesh Poyarekar <siddhesh@sourceware.org>
aarch64: Allow overriding HWCAP_CPUID feature check using HWCAP_MASK
The idea was to make it possible to disable cpuid based ifunc resolution
in glibc by changing the hwcap mask which the user could already control.
However the hwcap mask has an orthogonal role: it specifies additional
library search paths for the dynamic linker. So "cpuid" got added to
the search paths when it was set in the default mask (HWCAP_IMPORTANT),
which is not useful behaviour, the hwcap masking should not be reused
in the cpu features code.
Meanwhile there is a tunable to set the cpu explicitly so it is possible
to disable the cpuid based dispatch without using a hwcap mask:
GLIBC_TUNABLES=glibc.tune.cpu=generic
* sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features):
Use dl_hwcap without masking.
* sysdeps/unix/sysv/linux/aarch64/dl-procinfo.h (HWCAP_IMPORTANT):
Remove HWCAP_CPUID.
(cherry picked from commit d0cd798071)
The value of `end_name' points into the value of `dirname', thus don't
deallocate the latter before the last use of the former.
(cherry picked from commit ddc650e9b3 with
changes from commit d711a00f93)
When unwinding through a signal frame the backtrace function on PowerPC
didn't check array bounds when storing the frame address. Fixes commit
d400dcac5e ("PowerPC: fix backtrace to handle signal trampolines").
(cherry picked from commit d937694059)
Commit c402355dfa ("libio: Disable
vtable validation in case of interposition [BZ #23313]") only covered
the interposable glibc 2.1 handles, in libio/stdfiles.c. The
parallel code in libio/oldstdfiles.c needs similar detection logic.
Fixes (again) commit db3476aff1
("libio: Implement vtable verification [BZ #20191]").
Change-Id: Ief6f9f17e91d1f7263421c56a7dc018f4f595c21
(cherry picked from commit cb61630ed7)
The problem was introduced in glibc 2.23, in commit
b9eb92ab05
("Add Prefer_MAP_32BIT_EXEC to map executable pages with MAP_32BIT").
(cherry picked from commit d5dfad4326)
Linux/Mips kernels prior to 4.8 could potentially crash the user
process when doing FPU emulation while running on non-executable
user stack.
Currently, gcc doesn't emit .note.GNU-stack for mips, but that will
change in the future. To ensure that glibc can be used with such
future gcc, without silently resulting in binaries that might crash
in runtime, this patch forces RWX stack for all built objects if
configured to run against minimum kernel version less than 4.8.
* sysdeps/unix/sysv/linux/mips/Makefile
(test-xfail-check-execstack):
Move under mips-has-gnustack != yes.
(CFLAGS-.o*, ASFLAGS-.o*): New rules.
Apply -Wa,-execstack if mips-force-execstack == yes.
* sysdeps/unix/sysv/linux/mips/configure: Regenerated.
* sysdeps/unix/sysv/linux/mips/configure.ac
(mips-force-execstack): New var.
Set to yes for hard-float builds with minimum_kernel < 4.8.0
or minimum_kernel not set at all.
(mips-has-gnustack): New var.
Use value of libc_cv_as_noexecstack
if mips-force-execstack != yes, otherwise set to no.
(cherry picked from commit 33bc9efd91)
This patch significantly improves performance of memmem using a novel
modified Horspool algorithm. Needles up to size 256 use a bad-character
table indexed by hashed pairs of characters to quickly skip past mismatches.
Long needles use a self-adapting filtering step to avoid comparing the whole
needle repeatedly.
By limiting the needle length to 256, the shift table only requires 8 bits
per entry, lowering preprocessing overhead and minimizing cache effects.
This limit also implies worst-case performance is linear.
Small needles up to size 2 use a dedicated linear search. Very long needles
use the Two-Way algorithm (to avoid increasing stack size or slowing down
the common case, inlining is disabled).
The performance gain is 6.6 times on English text on AArch64 using random
needles with average size 8.
Tested against GLIBC testsuite and randomized tests.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* string/memmem.c (__memmem): Rewrite to improve performance.
(cherry picked from commit 680942b016)
This patch significantly improves performance of strstr using a novel
modified Horspool algorithm. Needles up to size 256 use a bad-character
table indexed by hashed pairs of characters to quickly skip past mismatches.
Long needles use a self-adapting filtering step to avoid comparing the whole
needle repeatedly.
By limiting the needle length to 256, the shift table only requires 8 bits
per entry, lowering preprocessing overhead and minimizing cache effects.
This limit also implies worst-case performance is linear.
Small needles up to size 3 use a dedicated linear search. Very long needles
use the Two-Way algorithm.
The performance gain using the improved bench-strstr on Cortex-A72 is 5.8
times basic_strstr and 3.7 times twoway_strstr.
Tested against GLIBC testsuite, randomized tests and the GNULIB strstr test
(https://git.savannah.gnu.org/cgit/gnulib.git/tree/tests/test-strstr.c).
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* string/str-two-way.h (two_way_short_needle): Add inline to avoid
warning.
(two_way_long_needle): Block inlining.
* string/strstr.c (strstr2): Add new function.
(strstr3): Likewise.
(STRSTR): Completely rewrite strstr to improve performance.
(cherry picked from commit 5e0a7ecb66)
The generic strstr in GLIBC 2.28 fails to match huge needles. The optimized
AVAILABLE macro reads ahead a large fixed amount to reduce the overhead of
repeatedly checking for the end of the string. However if the needle length
is larger than this, two_way_long_needle may confuse this as meaning the end
of the string and return NULL. This is fixed by adding the needle length to
the amount to read ahead.
[BZ #23637]
* string/test-strstr.c (pr23637): New function.
(test_main): Add tests with longer needles.
* string/strcasestr.c (AVAILABLE): Fix readahead distance.
* string/strstr.c (AVAILABLE): Likewise.
(cherry picked from commit 83a552b0bb)
As done in commit 284f42bc77, memcmp
can be used after memchr to avoid the initialization overhead of the
two-way algorithm for the first match. This has shown improvement
>40% for first match.
(cherry picked from commit c8dd67e7c9)
Looking at the benchtests, both strstr and strcasestr spend a lot of time
in a slow initialization loop handling one character per iteration.
This can be simplified and use the much faster strlen/strnlen/strchr/memcmp.
Read ahead a few cachelines to reduce the number of strnlen calls, which
improves performance by ~3-4%. This patch improves the time taken for the
full strstr benchtest by >40%.
* string/strcasestr.c (STRCASESTR): Simplify and speedup first match.
* string/strstr.c (AVAILABLE): Likewise.
(cherry picked from commit 284f42bc77)
Improve strstr performance. Strstr tends to be slow because it uses
many calls to memchr and a slow byte loop to scan for the next match.
Performance is significantly improved by using strnlen on larger blocks
and using strchr to search for the next matching character. strcasestr
can also use strnlen to scan ahead, and memmem can use memchr to check
for the next match.
On the GLIBC bench tests the performance gains on Cortex-A72 are:
strstr: +25%
strcasestr: +4.3%
memmem: +18%
On a 256KB dataset strstr performance improves by 67%, strcasestr by 47%.
Reviewd-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit 3ae725dfb6)
Add Ares to the midr_el0 list and support ifunc dispatch. Since Ares
supports 2 128-bit loads/stores, use Neon registers for memcpy by
selecting __memcpy_falkor by default (we should rename this to
__memcpy_simd or similar).
* manual/tunables.texi (glibc.cpu.name): Add ares tunable.
* sysdeps/aarch64/multiarch/memcpy.c (__libc_memcpy): Use
__memcpy_falkor for ares.
* sysdeps/unix/sysv/linux/aarch64/cpu-features.h (IS_ARES):
Add new define.
* sysdeps/unix/sysv/linux/aarch64/cpu-features.c (cpu_list):
Add ares cpu.
(cherry picked from commit 02f440c1ef)
Vector registers perform better than scalar register pairs for copying
data so prefer them instead. This results in a time reduction of over
50% (i.e. 2x speed improvemnet) for some smaller sizes for memcpy-walk.
Larger sizes show improvements of around 1% to 2%. memcpy-random shows
a very small improvement, in the range of 1-2%.
* sysdeps/aarch64/multiarch/memcpy_falkor.S (__memcpy_falkor):
Use vector registers.
(cherry picked from commit 0aec4c1d18)
For smaller and medium sized copies, the effect of hardware
prefetching are not as dominant as instruction level parallelism.
Hence it makes more sense to load data into multiple registers than to
try and route them to the same prefetch unit. This is also the case
for the loop exit where we are unable to latch on to the same prefetch
unit anyway so it makes more sense to have data loaded in parallel.
The performance results are a bit mixed with memcpy-random, with
numbers jumping between -1% and +3%, i.e. the numbers don't seem
repeatable. memcpy-walk sees a 70% improvement (i.e. > 2x) for 128
bytes and that improvement reduces down as the impact of the tail copy
decreases in comparison to the loop.
* sysdeps/aarch64/multiarch/memcpy_falkor.S (__memcpy_falkor):
Use multiple registers to copy data in loop tail.
(cherry picked from commit db725a458e)
Binutils 2.26.* and older do not support moves with shifted registers,
so use a separate shift instruction instead.
(cherry picked from commit d46f84de74)
The mutually misaligned inputs on aarch64 are compared with a simple
byte copy, which is not very efficient. Enhance the comparison
similar to strcmp by loading a double-word at a time. The peak
performance improvement (i.e. 4k maxlen comparisons) due to this on
the strncmp microbenchmark is as follows:
falkor: 3.5x (up to 72% time reduction)
cortex-a73: 3.5x (up to 71% time reduction)
cortex-a53: 3.5x (up to 71% time reduction)
All mutually misaligned inputs from 16 bytes maxlen onwards show
upwards of 15% improvement and there is no measurable effect on the
performance of aligned/mutually aligned inputs.
* sysdeps/aarch64/strncmp.S (count): New macro.
(strncmp): Store misaligned length in SRC1 in COUNT.
(mutual_align): Adjust.
(misaligned8): Load dword at a time when it is safe.
(cherry picked from commit 7108f1f944)
I accidentally set the loop jump back label as misaligned8 instead of
do_misaligned. The typo is harmless but it's always nice to not have
to unnecessarily execute those two instructions.
* sysdeps/aarch64/strcmp.S (do_misaligned): Jump back to
do_misaligned, not misaligned8.
(cherry picked from commit 6ca24c4348)
Replace the simple byte-wise compare in the misaligned case with a
dword compare with page boundary checks in place. For simplicity I've
chosen a 4K page boundary so that we don't have to query the actual
page size on the system.
This results in up to 3x improvement in performance in the unaligned
case on falkor and about 2.5x improvement on mustang as measured using
bench-strcmp.
* sysdeps/aarch64/strcmp.S (misaligned8): Compare dword at a
time whenever possible.
(cherry picked from commit 2bce01ebba)
I goofed up when changing the loop8 name to loop16 and missed on out
the branch instance. Fixed and actually build tested this time.
* sysdeps/aarch64/memcmp.S (more16): Fix branch target loop16.
(cherry picked from commit 4e54d91863)
This improved memcmp provides a fast path for compares up to 16 bytes
and then compares 16 bytes at a time, thus optimizing loads from both
sources. The glibc memcmp microbenchmark retains performance (with an
error of ~1ns) for smaller compare sizes and reduces up to 31% of
execution time for compares up to 4K on the APM Mustang. On Qualcomm
Falkor this improves to almost 48%, i.e. it is almost 2x improvement
for sizes of 2K and above.
* sysdeps/aarch64/memcmp.S: Widen comparison to 16 bytes at a
time.
(cherry picked from commit 30a81dae5b)
This is an optimized memcmp for AArch64. This is a complete rewrite
using a different algorithm. The previous version split into cases
where both inputs were aligned, the inputs were mutually aligned and
unaligned using a byte loop. The new version combines all these cases,
while small inputs of less than 8 bytes are handled separately.
This allows the main code to be sped up using unaligned loads since
there are now at least 8 bytes to be compared. After the first 8 bytes,
align the first input. This ensures each iteration does at most one
unaligned access and mutually aligned inputs behave as aligned.
After the main loop, process the last 8 bytes using unaligned accesses.
This improves performance of (mutually) aligned cases by 25% and
unaligned by >500% (yes >6 times faster) on large inputs.
* sysdeps/aarch64/memcmp.S (memcmp):
Rewrite of optimized memcmp.
(cherry picked from commit 922369032c)
The fix for BZ#21270 (commit 158d5fa0e1) added a mask to avoid offset larger
than 1^44 to be used along __NR_mmap2. However mips64n32 users __NR_mmap,
as mips64n64, but still defines off_t as old non-LFS type (other ILP32, such
x32, defines off_t being equal to off64_t). This leads to use the same
mask meant only for __NR_mmap2 call for __NR_mmap, thus limiting the maximum
offset it can use with mmap64.
This patch fixes by setting the high mask only for __NR_mmap2 usage. The
posix/tst-mmap-offset.c already tests it and also fails for mips64n32. The
patch also change the test to check for an arch-specific header that defines
the maximum supported offset.
Checked on x86_64-linux-gnu, i686-linux-gnu, and I also tests tst-mmap-offset
on qemu simulated mips64 with kernel 3.2.0 kernel for both mips-linux-gnu and
mips64-n32-linux-gnu.
[BZ #24699]
* posix/tst-mmap-offset.c: Mention BZ #24699.
(do_test_bz21270): Rename to do_test_large_offset and use
mmap64_maximum_offset to check for maximum expected offset value.
* sysdeps/generic/mmap_info.h: New file.
* sysdeps/unix/sysv/linux/mips/mmap_info.h: Likewise.
* sysdeps/unix/sysv/linux/mmap64.c (MMAP_OFF_HIGH_MASK): Define iff
__NR_mmap2 is used.
(cherry picked from commit a008c76b56)
Backport of commit 82bc69c012
and commit 30ba037546
without using DT_AARCH64_VARIANT_PCS for optimizing the symbol table check.
This is needed so the internal abi between ld.so and libc.so is unchanged.
Avoid lazy binding of symbols that may follow a variant PCS with different
register usage convention from the base PCS.
Currently the lazy binding entry code does not preserve all the registers
required for AdvSIMD and SVE vector calls. Saving and restoring all
registers unconditionally may break existing binaries, even if they never
use vector calls, because of the larger stack requirement for lazy
resolution, which can be significant on an SVE system.
The solution is to mark all symbols in the symbol table that may follow
a variant PCS so the dynamic linker can handle them specially. In this
patch such symbols are always resolved at load time, not lazily.
So currently LD_AUDIT for variant PCS symbols are not supported, for that
the _dl_runtime_profile entry needs to be changed e.g. to unconditionally
save/restore all registers (but pass down arg and retval registers to
pltentry/exit callbacks according to the base PCS).
This patch also removes a __builtin_expect from the modified code because
the branch prediction hint did not seem useful.
* sysdeps/aarch64/dl-machine.h (elf_machine_lazy_rel): Check
STO_AARCH64_VARIANT_PCS and bind such symbols at load time.
STO_AARCH64_VARIANT_PCS is a non-visibility st_other flag for marking
symbols that reference functions that may follow a variant PCS with
different register usage convention from the base PCS.
DT_AARCH64_VARIANT_PCS is a dynamic tag that marks ELF modules that
have R_*_JUMP_SLOT relocations for symbols marked with
STO_AARCH64_VARIANT_PCS (i.e. have variant PCS calls via a PLT).
* elf/elf.h (STO_AARCH64_VARIANT_PCS): Define.
(DT_AARCH64_VARIANT_PCS): Define.
The tcache counts[] array is a char, which has a very small range and thus
may overflow. When setting tcache_count tunable, there is no overflow check.
However the tunable must not be larger than the maximum value of the tcache
counts[] array, otherwise it can overflow when filling the tcache.
[BZ #24531]
* malloc/malloc.c (MAX_TCACHE_COUNT): New define.
(do_set_tcache_count): Only update if count is small enough.
* manual/tunables.texi (glibc.malloc.tcache_count): Document max value.
(cherry picked from commit 5ad533e8e6)
When computing the length of the converted part of the stdio buffer, use
the number of consumed wide characters, not the (negative) distance to the
end of the wide buffer.
(cherry picked from commit 32ff397533)
While debugging a kernel warning, Thomas Gleixner, Sebastian Sewior and
Heiko Carstens found a bug in pthread_mutex_trylock due to misordered
instructions:
140: a5 1b 00 01 oill %r1,1
144: e5 48 a0 f0 00 00 mvghi 240(%r10),0 <--- THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
14a: e3 10 a0 e0 00 24 stg %r1,224(%r10) <--- last THREAD_SETMEM of ENQUEUE_MUTEX_PI
vs (with compiler barriers):
140: a5 1b 00 01 oill %r1,1
144: e3 10 a0 e0 00 24 stg %r1,224(%r10)
14a: e5 48 a0 f0 00 00 mvghi 240(%r10),0
Please have a look at the discussion:
"Re: WARN_ON_ONCE(!new_owner) within wake_futex_pi() triggerede"
(https://lore.kernel.org/lkml/20190202112006.GB3381@osiris/)
This patch is introducing the same compiler barriers and comments
for pthread_mutex_trylock as introduced for pthread_mutex_lock and
pthread_mutex_timedlock by commit 8f9450a0b7
"Add compiler barriers around modifications of the robust mutex list."
ChangeLog:
[BZ #24180]
* nptl/pthread_mutex_trylock.c (__pthread_mutex_trylock):
Add compiler barriers and comments.
(cherry picked from commit 823624bdc4)
Since the size argument is unsigned. we should use unsigned Jcc
instructions, instead of signed, to check size.
Tested on x86-64 and x32, with and without --disable-multi-arch.
[BZ #24155]
CVE-2019-7309
* NEWS: Updated for CVE-2019-7309.
* sysdeps/x86_64/memcmp.S: Use RDX_LP for size. Clear the
upper 32 bits of RDX register for x32. Use unsigned Jcc
instructions, instead of signed.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-memcmp-2.
* sysdeps/x86_64/x32/tst-size_t-memcmp-2.c: New test.
(cherry picked from commit 3f635fb433)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes strnlen/wcsnlen for x32. Tested on x86-64 and x32. On
x86-64, libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/multiarch/strlen-avx2.S: Use RSI_LP for length.
Clear the upper 32 bits of RSI register.
* sysdeps/x86_64/strlen.S: Use RSI_LP for length.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-strnlen
and tst-size_t-wcsnlen.
* sysdeps/x86_64/x32/tst-size_t-strnlen.c: New file.
* sysdeps/x86_64/x32/tst-size_t-wcsnlen.c: Likewise.
(cherry picked from commit 5165de69c0)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes strncpy for x32. Tested on x86-64 and x32. On x86-64,
libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/multiarch/strcpy-sse2-unaligned.S: Use RDX_LP
for length.
* sysdeps/x86_64/multiarch/strcpy-ssse3.S: Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-strncpy.
* sysdeps/x86_64/x32/tst-size_t-strncpy.c: New file.
(cherry picked from commit c7c54f65b0)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes the strncmp family for x32. Tested on x86-64 and x32.
On x86-64, libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/multiarch/strcmp-sse42.S: Use RDX_LP for length.
* sysdeps/x86_64/strcmp.S: Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-strncasecmp,
tst-size_t-strncmp and tst-size_t-wcsncmp.
* sysdeps/x86_64/x32/tst-size_t-strncasecmp.c: New file.
* sysdeps/x86_64/x32/tst-size_t-strncmp.c: Likewise.
* sysdeps/x86_64/x32/tst-size_t-wcsncmp.c: Likewise.
(cherry picked from commit ee915088a0)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes memset/wmemset for x32. Tested on x86-64 and x32. On
x86-64, libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/multiarch/memset-avx512-no-vzeroupper.S: Use
RDX_LP for length. Clear the upper 32 bits of RDX register.
* sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-wmemset.
* sysdeps/x86_64/x32/tst-size_t-memset.c: New file.
* sysdeps/x86_64/x32/tst-size_t-wmemset.c: Likewise.
(cherry picked from commit 82d0b4a4d7)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes memrchr for x32. Tested on x86-64 and x32. On x86-64,
libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/memrchr.S: Use RDX_LP for length.
* sysdeps/x86_64/multiarch/memrchr-avx2.S: Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-memrchr.
* sysdeps/x86_64/x32/tst-size_t-memrchr.c: New file.
(cherry picked from commit ecd8b842cf)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes memcpy for x32. Tested on x86-64 and x32. On x86-64,
libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/multiarch/memcpy-ssse3-back.S: Use RDX_LP for
length. Clear the upper 32 bits of RDX register.
* sysdeps/x86_64/multiarch/memcpy-ssse3.S: Likewise.
* sysdeps/x86_64/multiarch/memmove-avx512-no-vzeroupper.S:
Likewise.
* sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:
Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-memcpy.
tst-size_t-wmemchr.
* sysdeps/x86_64/x32/tst-size_t-memcpy.c: New file.
(cherry picked from commit 231c56760c)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes memcmp/wmemcmp for x32. Tested on x86-64 and x32. On
x86-64, libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/multiarch/memcmp-avx2-movbe.S: Use RDX_LP for
length. Clear the upper 32 bits of RDX register.
* sysdeps/x86_64/multiarch/memcmp-sse4.S: Likewise.
* sysdeps/x86_64/multiarch/memcmp-ssse3.S: Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-memcmp and
tst-size_t-wmemcmp.
* sysdeps/x86_64/x32/tst-size_t-memcmp.c: New file.
* sysdeps/x86_64/x32/tst-size_t-wmemcmp.c: Likewise.
(cherry picked from commit b304fc201d)
On x32, the size_t parameter may be passed in the lower 32 bits of a
64-bit register with the non-zero upper 32 bits. The string/memory
functions written in assembly can only use the lower 32 bits of a
64-bit register as length or must clear the upper 32 bits before using
the full 64-bit register for length.
This pach fixes memchr/wmemchr for x32. Tested on x86-64 and x32. On
x86-64, libc.so is the same with and withou the fix.
[BZ #24097]
CVE-2019-6488
* sysdeps/x86_64/memchr.S: Use RDX_LP for length. Clear the
upper 32 bits of RDX register.
* sysdeps/x86_64/multiarch/memchr-avx2.S: Likewise.
* sysdeps/x86_64/x32/Makefile (tests): Add tst-size_t-memchr and
tst-size_t-wmemchr.
* sysdeps/x86_64/x32/test-size_t.h: New file.
* sysdeps/x86_64/x32/tst-size_t-memchr.c: Likewise.
* sysdeps/x86_64/x32/tst-size_t-wmemchr.c: Likewise.
(cherry picked from commit 97700a34f3)
On POWER9, cbrtf128 fails by 1 ULP.
* sysdeps/powerpc/fpu/libm-test-ulps: Regenerate.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
(cherry picked from commit 428fc49eaa)
Some math functions have to be distributed in libc because they're
required by printf.
libc and libm require their own builds of these functions, e.g. libc
functions have to call __stack_chk_fail_local in order to bypass the
PLT, while libm functions have to call __stack_chk_fail.
While math/Makefile treat the generic cases, i.e. s_isinff, the
multiarch Makefile has to treat its own files, i.e. s_isinff-ppc64.
[BZ #21745]
* sysdeps/powerpc/powerpc64/fpu/multiarch/Makefile:
[$(subdir) = math] (sysdep_calls): New variable. Has the
previous contents of sysdep_routines, but re-sorted..
[$(subdir) = math] (sysdep_routines): Re-use the contents from
sysdep_calls.
[$(subdir) = math] (libm-sysdep_routines): Remove the functions
defined in sysdep_calls and replace by the respective m_* names.
* sysdeps/powerpc/powerpc64/fpu/multiarch/s_isnan-ppc64.S:
(compat_symbol): Undefine to avoid duplicated compat symbols in
libc.
(cherry picked from commit 61c45f2505)
This commit removes the custom memcpy implementation from _int_realloc
for small chunk sizes. The ncopies variable has the wrong type, and
an integer wraparound could cause the existing code to copy too few
elements (leaving the new memory region mostly uninitialized).
Therefore, removing this code fixes bug 24027.
(cherry picked from commit b50dd3bc8c)
Th commit 'Disable TSX on some Haswell processors.' (2702856bf4) changed the
default flags for Haswell models. Previously, new models were handled by the
default switch path, which assumed a Core i3/i5/i7 if AVX is available. After
the patch, Haswell models (0x3f, 0x3c, 0x45, 0x46) do not set the flags
Fast_Rep_String, Fast_Unaligned_Load, Fast_Unaligned_Copy, and
Prefer_PMINUB_for_stringop (only the TSX one).
This patch fixes it by disentangle the TSX flag handling from the memory
optimization ones. The strstr case cited on patch now selects the
__strstr_sse2_unaligned as expected for the Haswell cpu.
Checked on x86_64-linux-gnu.
[BZ #23709]
* sysdeps/x86/cpu-features.c (init_cpu_features): Set TSX bits
independently of other flags.
(cherry picked from commit c3d8dc45c9)
nptl/tst-attr3 fails to build with GCC mainline because of
(deliberate) aliasing between the second (attributes) and fourth
(argument to thread start routine) arguments to pthread_create.
Although both those arguments are restrict-qualified in POSIX,
pthread_create does not actually dereference its fourth argument; it's
an opaque pointer passed to the thread start routine. Thus, the
aliasing is actually valid in this case, and it's deliberate in the
test. So this patch makes the test disable -Wrestrict for the two
pthread_create calls in question. (-Wrestrict was added in GCC 7,
hence the __GNUC_PREREQ conditions, but the particular warning in
question is new in GCC 8.)
Tested compilation with build-many-glibcs.py for aarch64-linux-gnu.
* nptl/tst-attr3.c: Include <libc-diag.h>.
(do_test) [__GNUC_PREREQ (7, 0)]: Ignore -Wrestrict for two tests.
(cherry picked from commit 40c4162df6)
GCC 8 warns about strncat calls with truncated output.
string/bug-strncat1.c tests such a call; this patch disables the
warning for it.
Tested (compilation) with GCC 8 for x86_64-linux-gnu with
build-many-glibcs.py (in conjunction with Martin's patch to allow
glibc to build).
* string/bug-strncat1.c: Include <libc-diag.h>.
(main): Disable -Wstringop-truncation for strncat call for GCC 8.
(cherry picked from commit ec72135e5f)
With current GCC mainline, one strncat test involving a size close to
SIZE_MAX results in a -Wrestrict warning that that buffer size would
imply that the two buffers must overlap. This patch fixes the build
by adding disabling of -Wrestrict (for GCC versions supporting that
option) to the already-present disabling of -Wstringop-overflow= and
-Warray-bounds for this test.
Tested with build-many-glibcs.py that this restores the testsuite
build with GCC mainline for aarch64-linux-gnu.
* string/tester.c (test_strncat) [__GNUC_PREREQ (7, 0)]: Also
ignore -Wrestrict for one test.
(cherry picked from commit 35ebb6b0c4)
Some strncat tests fail to build with GCC 8 because of -Warray-bounds
warnings. These tests are deliberately test over-large size arguments
passed to strncat, and already disable -Wstringop-overflow warnings,
but now the warnings for these tests come under -Warray-bounds so that
option needs disabling for them as well, which this patch does (with
an update on the comments; the DIAG_IGNORE_NEEDS_COMMENT call for
-Warray-bounds doesn't need to be conditional itself, because that
option is supported by all versions of GCC that can build glibc).
Tested compilation with build-many-glibcs.py for aarch64-linux-gnu.
* string/tester.c (test_strncat): Also disable -Warray-bounds
warnings for two tests.
(cherry picked from commit 1421f39b7e)
GCC 8 warns about more cases of string functions truncating their
output or not copying a trailing NUL byte.
This patch fixes testsuite build failures caused by such warnings in
string/tester.c. In general, the warnings are disabled around the
relevant calls using DIAG_* macros, since the relevant cases are being
deliberately tested. In one case, the warning is with
-Wstringop-overflow= instead of -Wstringop-truncation; in that case,
the conditional is __GNUC_PREREQ (7, 0) (being the version where
-Wstringop-overflow= was introduced), to allow the conditional to be
removed sooner, since it's harmless to disable the warning for a
GCC version where it doesn't actually occur. In the case of warnings
for strncpy calls in test_memcmp, the calls in question are changed to
use memcpy, as they don't copy a trailing NUL and the point of that
code is to test memcmp rather than strncpy.
Tested (compilation) with GCC 8 for x86_64-linux-gnu with
build-many-glibcs.py (in conjunction with Martin's patch to allow
glibc to build).
* string/tester.c (test_stpncpy): Disable -Wstringop-truncation
for stpncpy calls for GCC 8.
(test_strncat): Disable -Wstringop-truncation warning for strncat
calls for GCC 8. Disable -Wstringop-overflow= warning for one
strncat call for GCC 7.
(test_strncpy): Disable -Wstringop-truncation warning for strncpy
calls for GCC 8.
(test_memcmp): Use memcpy instead of strncpy for calls not copying
trailing NUL.
(cherry picked from commit 2e64ec9c9e)
Current GCC mainline detects that nscd calls readlink with the same
buffer for both input and output, which is not valid (those arguments
are both restrict-qualified in POSIX). This patch makes it use a
separate buffer for readlink's input (with a size that is sufficient
to avoid truncation, so there should be no problems with warnings
about possible truncation, though not strictly minimal, but much
smaller than the buffer for output) to avoid this problem.
Tested compilation for aarch64-linux-gnu with build-many-glibcs.py.
[BZ #22446]
* nscd/connections.c (handle_request) [SO_PEERCRED]: Use separate
buffers for readlink input and output.
(cherry picked from commit 49b036bce9)
The resolv/res_debug.c function p_secstodate (which is a public
function exported from libresolv, taking an unsigned long argument)
does:
struct tm timebuf;
time = __gmtime_r(&clock, &timebuf);
time->tm_year += 1900;
time->tm_mon += 1;
sprintf(output, "%04d%02d%02d%02d%02d%02d",
time->tm_year, time->tm_mon, time->tm_mday,
time->tm_hour, time->tm_min, time->tm_sec);
If __gmtime_r returns NULL (because the year overflows the range of
int), this will dereference a null pointer. Otherwise, if the
computed year does not fit in four characters, this will cause a
buffer overrun of the fixed-size 15-byte buffer. With current GCC
mainline, there is a compilation failure because of the possible
buffer overrun.
I couldn't find a specification for how this function is meant to
behave, but Paul pointed to RFC 4034 as relevant to the cases where
this function is called from within glibc. The function's interface
is inherently problematic when dates beyond Y2038 might be involved,
because of the ambiguity in how to interpret 32-bit timestamps as such
dates (the RFC suggests interpreting times as being within 68 years of
the present date, which would mean some kind of interface whose
behavior depends on the present date).
This patch works on the basis of making a minimal fix in preparation
for obsoleting the function. The function is made to handle times in
the interval [0, 0x7fffffff] only, on all platforms, with <overflow>
used as the output string in other cases (and errno set to EOVERFLOW
in such cases). This seems to be a reasonable state for the function
to be in when made a compat symbol by a future patch, being compatible
with any existing uses for existing timestamps without trying to work
for later timestamps. Results independent of the range of time_t also
simplify the testcase.
I couldn't persuade GCC to recognize the ranges of the struct tm
fields by adding explicit range checks with a call to
__builtin_unreachable if outside the range (this looks similar to
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80776>), so having added
a range check on the input, this patch then disables the
-Wformat-overflow= warning for the sprintf call (I prefer that to the
use of strftime, as being more transparently correct without knowing
what each of %m and %M etc. is).
I do not know why this build failure should be new with mainline GCC
(that is, I don't know what GCC change might have introduced it, when
the basic functionality for such warnings was already in GCC 7).
I do not know if this is a security issue (that is, if there are
plausible ways in which a date before -999 or after 9999 from an
untrusted source might end up in this function). The system clock is
arguably an untrusted source (in that e.g. NTP is insecure), but
probably not to that extent (NTP can't communicate such wild
timestamps), and uses from within glibc are limited to 32-bit inputs.
Tested with build-many-glibcs.py that this restores the build for arm
with yesterday's mainline GCC. Also tested for x86_64 and x86.
[BZ #22463]
* resolv/res_debug.c: Include <libc-diag.h>.
(p_secstodate): Assert time_t at least as wide as u_long. On
overflow, use integer seconds since the epoch as output, or use
"<overflow>" as output and set errno to EOVERFLOW if integer
seconds since the epoch would be 14 or more characters.
(p_secstodate) [__GNUC_PREREQ (7, 0)]: Disable -Wformat-overflow=
for sprintf call.
* resolv/tst-p_secstodate.c: New file.
* resolv/Makefile (tests): Add tst-p_secstodate.
($(objpfx)tst-p_secstodate): Depend on $(objpfx)libresolv.so.
(cherry picked from commit f120cda607)
The -Wstringop-truncation option new in GCC 8 detects common misuses
of the strncat and strncpy function that may result in truncating
the copied string before the terminating NUL. To avoid false positive
warnings for correct code that intentionally creates sequences of
characters that aren't guaranteed to be NUL-terminated, arrays that
are intended to store such sequences should be decorated with a new
nonstring attribute. This change add this attribute to Glibc and
uses it to suppress such false positives.
ChangeLog:
* misc/sys/cdefs.h (__attribute_nonstring__): New macro.
* sysdeps/gnu/bits/utmp.h (struct utmp): Use it.
* sysdeps/unix/sysv/linux/s390/bits/utmp.h (struct utmp): Same.
(cherry picked from commit 7532837d7b)
Building glibc with current mainline GCC fails, among other reasons,
because of an error for use of strlen on the nonstring ut_user field.
This patch changes the problem code in getlogin_r to use __strnlen
instead. It also needs to set the trailing NUL byte of the result
explicitly, because of the case where ut_user does not have such a
trailing NUL byte (but the result should always have one).
Tested for x86_64. Also tested that, in conjunction with
<https://sourceware.org/ml/libc-alpha/2017-11/msg00797.html>, it fixes
the build for arm with mainline GCC.
[BZ #22447]
* sysdeps/unix/getlogin_r.c (__getlogin_r): Use __strnlen not
strlen to compute length of ut_user and set trailing NUL byte of
result explicitly.
(cherry picked from commit 4bae615022)
The fallback code of Linux wrapper for preadv2/pwritev2 executes
regardless of the errno code for preadv2, instead of the case where
the syscall is not supported.
This fixes it by calling the fallback code iff errno is ENOSYS. The
patch also adds tests for both invalid file descriptor and invalid
iov_len and vector count.
The only discrepancy between preadv2 and fallback code regarding
error reporting is when an invalid flags are used. The fallback code
bails out earlier with ENOTSUP instead of EINVAL/EBADF when the syscall
is used.
Checked on x86_64-linux-gnu on a 4.4.0 and 4.15.0 kernel.
[BZ #23579]
* misc/tst-preadvwritev2-common.c (do_test_with_invalid_fd): New
test.
* misc/tst-preadvwritev2.c, misc/tst-preadvwritev64v2.c (do_test):
Call do_test_with_invalid_fd.
* sysdeps/unix/sysv/linux/preadv2.c (preadv2): Use fallback code iff
errno is ENOSYS.
* sysdeps/unix/sysv/linux/preadv64v2.c (preadv64v2): Likewise.
* sysdeps/unix/sysv/linux/pwritev2.c (pwritev2): Likewise.
* sysdeps/unix/sysv/linux/pwritev64v2.c (pwritev64v2): Likewise.
(cherry picked from commit 7a16bdbb9f)
If glibc is built with gcc 8 and -march=z900,
the testcase posix/tst-spawn4-compat crashes with a segfault.
In function maybe_script_execute, the new_argv array is dynamically
initialized on stack with (argc + 1) elements.
The function wants to add _PATH_BSHELL as the first argument
and writes out of bounds of new_argv.
There is an off-by-one because maybe_script_execute fails to count
the terminating NULL when sizing new_argv.
ChangeLog:
* sysdeps/unix/sysv/linux/spawni.c (maybe_script_execute):
Increment size of new_argv by one.
(cherry picked from commit 28669f86f6)
cpu-features.h has
#define bit_cpu_LZCNT (1 << 5)
#define index_cpu_LZCNT COMMON_CPUID_INDEX_1
#define reg_LZCNT
But the LZCNT feature bit is in COMMON_CPUID_INDEX_80000001:
Initial EAX Value: 80000001H
ECX Extended Processor Signature and Feature Bits:
Bit 05: LZCNT available
index_cpu_LZCNT should be COMMON_CPUID_INDEX_80000001, not
COMMON_CPUID_INDEX_1. The VMX feature bit is in COMMON_CPUID_INDEX_1:
Initial EAX Value: 01H
Feature Information Returned in the ECX Register:
5 VMX
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
[BZ #23456]
* sysdeps/x86/cpu-features.h (index_cpu_LZCNT): Set to
COMMON_CPUID_INDEX_80000001.
(cherry picked from commit 65d87ade1e)
This suppresses Perl warnings like these:
Unescaped left brace in regex is deprecated here (and will be fatal in
Perl 5.32), passed through in regex; marked by <-- HERE in m/^element
*({ <-- HERE ([^}]*)}|([^{ ]*)) *({([^}]*)}|([^{ ]*)) *([A-Za-z0-9_]*)
*(.*)/ at conformtest.pl line 370.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
(cherry picked from commit ddb3c626b0)
A lookup operation in map_newlink could turn into an insert because of
holes in the interface part of the map. This leads to incorrectly set
the name of the interface to NULL when the interface is not present
for the address being processed (most likely because the interface was
added between the RTM_GETLINK and RTM_GETADDR calls to the kernel).
When such changes are detected by the kernel, it'll mark the dump as
"inconsistent" by setting NLM_F_DUMP_INTR flag on the next netlink
message.
This patch checks this condition and retries the whole operation.
Hopes are that next time the interface corresponding to the address
entry is present in the list and correct name is returned.
(cherry picked from commit c1f86a33ca)
After commit d76d370355 ("Fix missing
timespec definition for sys/stat.h (BZ #21371)") in combination with
kernel UAPI changes, GCC sanitizer builds start to fail due to a
conflicting definition of struct timespec in <linux/time.h>. Use
_STRUCT_TIMESPEC as the header file inclusion guard, which is already
checked in the kernel header, to support including <linux/time.h> and
<sys/stat.h> in the same translation unit.
(cherry picked from commit c1c2848b57)
The commit
commit c85e54ac6c
Author: Gabriel F. T. Gomes <gabriel@inconstante.eti.br>
Date: Fri Nov 3 10:44:36 2017 -0200
Provide a C++ version of iseqsig (bug 22377)
mistakenly used double parameters in the long double version of iseqsig,
thus causing spurious conversions to double, as reported on bug 23171.
Tested for powerpc64le and x86_64.
(cherry picked from commit fb0e10b8eb)
These unmangled function pointers reside on the heap and could
be targeted by exploit writers, effectively bypassing libio vtable
validation. Instead, we ignore these pointers and always call
malloc or free.
In theory, this is a backwards-incompatible change, but using the
global heap instead of the user-supplied callback functions should
have little application impact. (The old libstdc++ implementation
exposed this functionality via a public, undocumented constructor
in its strstreambuf class.)
(cherry picked from commit 4e8a6346cd)
When compiled as mempcpy, the return value is the end of the destination
buffer, thus it cannot be used to refer to the start of it.
(cherry picked from commit 9aaaab7c6e)
Integer addition overflow may cause stack buffer overflow
when realpath() input length is close to SSIZE_MAX.
2018-05-09 Paul Pluzhnikov <ppluzhnikov@google.com>
[BZ #22786]
* stdlib/canonicalize.c (__realpath): Fix overflow in path length
computation.
* stdlib/Makefile (test-bz22786): New test.
* stdlib/test-bz22786.c: New test.
(cherry picked from commit 5460617d15)
A PT_NOTE in a binary could be arbitratily large, so using alloca
for it may cause stack overflow. If the note is larger than
__MAX_ALLOCA_CUTOFF, use dynamically allocated memory to read it in.
2018-05-05 Paul Pluzhnikov <ppluzhnikov@google.com>
[BZ #20419]
* elf/dl-load.c (open_verify): Fix stack overflow.
* elf/Makefile (tst-big-note): New test.
* elf/tst-big-note-lib.S: New.
* elf/tst-big-note.c: New.
(cherry picked from commit 0065aaaaae)
On s390 (31bit) if glibc is build with -Os, pthread_join sometimes
blocks indefinitely. This is e.g. observable with
testcase intl/tst-gettext6.
pthread_join is calling lll_wait_tid(tid), which performs the futex-wait
syscall in a loop as long as tid != 0 (thread is alive).
On s390 (and build with -Os), tid is loaded from memory before
comparing against zero and then the tid is loaded a second time
in order to pass it to the futex-wait-syscall.
If the thread exits in between, then the futex-wait-syscall is
called with the value zero and it waits until a futex-wake occurs.
As the thread is already exited, there won't be a futex-wake.
In lll_wait_tid, the tid is stored to the local variable __tid,
which is then used as argument for the futex-wait-syscall.
But unfortunately the compiler is allowed to reload the value
from memory.
With this patch, the tid is loaded with atomic_load_acquire.
Then the compiler is not allowed to reload the value for __tid from memory.
ChangeLog:
[BZ #23137]
* sysdeps/nptl/lowlevellock.h (lll_wait_tid):
Use atomic_load_acquire to load __tid.
(cherry picked from commit 1660901840)
Bug 17343 reports that stdlib/random_r.c has code with undefined
behavior because of signed integer overflow on int32_t. This patch
changes the code so that the possibly overflowing computations use
unsigned arithmetic instead.
Note that the bug report refers to "Most code" in that file. The
places changed in this patch are the only ones I found where I think
such overflow can occur.
Tested for x86_64 and x86.
[BZ #17343]
* stdlib/random_r.c (__random_r): Use unsigned arithmetic for
possibly overflowing computations.
(cherry picked from commit 8a07b0c43c)
This patch fixes the i386 sa_restorer field initialization for sigaction
syscall for kernel with vDSO. As described in bug report, i386 Linux
(and compat on x86_64) interprets SA_RESTORER clear with nonzero
sa_restorer as a request for stack switching if the SS segment is 'funny'.
This means that anything that tries to mix glibc's signal handling with
segmentation (for instance through modify_ldt syscall) is randomly broken
depending on what values lands in sa_restorer.
The testcase added is based on Linux test tools/testing/selftests/x86/ldt_gdt.c,
more specifically in do_multicpu_tests function. The main changes are:
- C11 atomics instead of plain access.
- Remove x86_64 support which simplifies the syscall handling and fallbacks.
- Replicate only the test required to trigger the issue.
Checked on i686-linux-gnu.
[BZ #21269]
* sysdeps/unix/sysv/linux/i386/Makefile (tests): Add tst-bz21269.
* sysdeps/unix/sysv/linux/i386/sigaction.c (SET_SA_RESTORER): Clear
sa_restorer for vDSO case.
* sysdeps/unix/sysv/linux/i386/tst-bz21269.c: New file.
(cherry picked from commit 68448be208)
Unlike other nscd caches, the netgroup cache contains two types of
records - those for "iterate through a netgroup" (i.e. setnetgrent())
and those for "is this user in this netgroup" (i.e. innetgr()),
i.e. full and partial records. The timeout code assumes these records
have the same key for the group name, so that the collection of records
that is "this netgroup" can be expired as a unit.
However, the keys are not the same, as the in-netgroup key is generated
by nscd rather than being passed to it from elsewhere, and is generated
without the trailing NUL. All other keys have the trailing NUL, and as
noted in the linked BZ, debug statements confirm that two keys for the
same netgroup are added to the cache with two different lengths.
The result of this is that as records in the cache expire, the purge
code only cleans out one of the two types of entries, resulting in
stale, possibly incorrect, and possibly inconsistent cache data.
The patch simply includes the existing NUL in the computation for the
key length ('key' points to the char after the NUL, and 'group' to the
first char of the group, so 'key-group' includes the first char to the
NUL, inclusive).
[BZ #22342]
* nscd/netgroupcache.c (addinnetgrX): Include trailing NUL in
key value.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
(cherry picked from commit 1c81d55fc4)
When there is no login uid Linux sets /proc/self/loginid to the sentinel
value of, (uid_t) -1. If this is set we can return early and avoid
needlessly looking up the sentinel value in any configured nss
databases.
Checked on aarch64-linux-gnu.
* sysdeps/unix/sysv/linux/getlogin_r.c (__getlogin_r_loginuid): Return
early when linux sentinel value is set.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit cc8a1620eb)
When posix_memalign is called with an alignment less than MALLOC_ALIGNMENT
and a requested size close to SIZE_MAX, it falls back to malloc code
(because the alignment of a block returned by malloc is sufficient to
satisfy the call). In this case, an integer overflow in _int_malloc leads
to posix_memalign incorrectly returning successfully.
Upon fixing this and writing a somewhat thorough regression test, it was
discovered that when posix_memalign is called with an alignment larger than
MALLOC_ALIGNMENT (so it uses _int_memalign instead) and a requested size
close to SIZE_MAX, a different integer overflow in _int_memalign leads to
posix_memalign incorrectly returning successfully.
Both integer overflows affect other memory allocation functions that use
_int_malloc (one affected malloc in x86) or _int_memalign as well.
This commit fixes both integer overflows. In addition to this, it adds a
regression test to guard against false successful allocations by the
following memory allocation functions when called with too-large allocation
sizes and, where relevant, various valid alignments:
malloc, realloc, calloc, reallocarray, memalign, posix_memalign,
aligned_alloc, valloc, and pvalloc.
(cherry picked from commit 8e448310d7)
The tunables framework needs to execute syscall early in process
initialization, before the TCB is available for consumption. This
behavior conflicts with powerpc{|64|64le}'s lock elision code, that
checks the TCB before trying to abort transactions immediately before
executing a syscall.
This patch adds a powerpc-specific implementation of __access_noerrno
that does not abort transactions before the executing syscall.
Tested on powerpc{|64|64le}.
[BZ #22685]
* sysdeps/powerpc/powerpc32/sysdep.h (ABORT_TRANSACTION_IMPL): Renamed
from ABORT_TRANSACTION.
(ABORT_TRANSACTION): Redirect to ABORT_TRANSACTION_IMPL.
* sysdeps/powerpc/powerpc64/sysdep.h (ABORT_TRANSACTION,
ABORT_TRANSACTION_IMPL): Likewise.
* sysdeps/unix/sysv/linux/powerpc/not-errno.h: New file. Reuse
Linux code, but remove the code that aborts transactions.
Signed-off-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
Tested-by: Aurelien Jarno <aurelien@aurel32.net>
(cherry picked from commit 4612268a0a)
In C++ mode, __MATH_TG cannot be used for defining iseqsig, because
__MATH_TG relies on __builtin_types_compatible_p, which is a C-only
builtin. This is true when float128 is provided as an ABI-distinct type
from long double.
Moreover, the comparison macros from ISO C take two floating-point
arguments, which need not have the same type. Choosing what underlying
function to call requires evaluating the formats of the arguments, then
selecting which is wider. The macro __MATH_EVAL_FMT2 provides this
information, however, only the type of the macro expansion is relevant
(actually evaluating the expression would be incorrect).
This patch provides a C++ version of iseqsig, in which only the type of
__MATH_EVAL_FMT2 (__typeof or decltype) is used as a template parameter
for __iseqsig_type. This function calls the appropriate underlying
function.
Tested for powerpc64le and x86_64.
[BZ #22377]
* math/Makefile [C++] (tests): Add test for iseqsig.
* math/math.h [C++] (iseqsig): New implementation, which does
not rely on __MATH_TG/__builtin_types_compatible_p.
* math/test-math-iseqsig.cc: New file.
* sysdeps/powerpc/powerpc64le/Makefile
(CFLAGS-test-math-iseqsig.cc): New variable.
(cherry picked from commit c85e54ac6c)
This patch rewrites aarch64 elf_machine_load_address to use special _DYNAMIC
symbol instead of _dl_start.
The static address of _DYNAMIC symbol is stored in the first GOT entry.
Here is the change which makes this solution work (part of binutils 2.24):
https://sourceware.org/ml/binutils/2013-06/msg00248.html
i386, x86_64 targets use the same method to do this as well.
The original implementation relies on a trick that R_AARCH64_ABS32 relocation
being resolved at link time and the static address fits in the 32bits.
However, in LP64, normally, the address is defined to be 64 bit.
Here is the C version one which should be portable in all cases.
* sysdeps/aarch64/dl-machine.h (elf_machine_load_address): Use
_DYNAMIC symbol to calculate load address.
(cherry picked from commit a68ba2f3cd)
_dl_runtime_profile calls _dl_call_pltexit, passing a pointer to
La_x86_64_retval which is allocated on stack. The lrv_vector0
field in La_x86_64_retval must be aligned to size of vector register.
When allocating stack space for La_x86_64_retval, we need to make sure
that the address of La_x86_64_retval + RV_VECTOR0_OFFSET is aligned to
VEC_SIZE. This patch checks the alignment of the lrv_vector0 field
and pads the stack space if needed.
Tested with x32 and x86-64 on SSE4, AVX and AVX512 machines. It fixed
FAIL: elf/tst-audit10
FAIL: elf/tst-audit4
FAIL: elf/tst-audit5
FAIL: elf/tst-audit6
FAIL: elf/tst-audit7
on x32 AVX512 machine.
(cherry picked from commit 207a72e298)
[BZ #22715]
* sysdeps/x86_64/dl-trampoline.h (_dl_runtime_profile): Properly
align La_x86_64_retval to VEC_SIZE.
We do not need thread_local support in the C++11 comiler, and the
minimum GCC version for glibc has C++11 support (if it has C++ support).
(cherry picked from commit 10d200dbac)
I verified that without the guard accounting change in commit
630f4cc3aa (Fix stack guard size
accounting) and RTLD_NOW for libgcc_s introduced by commit
f993b87540 (nptl: Open libgcc.so with
RTLD_NOW during pthread_cancel), the tst-minstack-cancel test fails on
an AVX-512F machine. tst-minstack-exit still passes, and either of
the mentioned commit by itself frees sufficient stack space to make
tst-minstack-cancel pass, too.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
(cherry picked from commit d8b778907e)
Disabling lazy binding reduces stack usage during unwinding.
Note that RTLD_NOW only makes a difference if libgcc.so has not
already been loaded, so this is only a partial fix.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit f993b87540)
Previously if user requested S stack and G guard when creating a
thread, the total mapping was S and the actual available stack was
S - G - static_tls, which is not what the user requested.
This patch fixes the guard size accounting by pretending the user
requested S+G stack. This way all later logic works out except
when reporting the user requested stack size (pthread_getattr_np)
or when computing the minimal stack size (__pthread_get_minstack).
Normally this will increase thread stack allocations by one page.
TLS accounting is not affected, that will require a separate fix.
[BZ #22637]
* nptl/descr.h (stackblock, stackblock_size): Update comments.
* nptl/allocatestack.c (allocate_stack): Add guardsize to stacksize.
* nptl/nptl-init.c (__pthread_get_minstack): Remove guardsize from
stacksize.
* nptl/pthread_getattr_np.c (pthread_getattr_np): Likewise.
(cherry picked from commit 630f4cc3aa)
GCC PR 83641 results in a miscompilation of libpthread, which
causes pthread_exit not to restore callee-saved registers before
running destructors for objects on the stack. This test detects
this situation:
info: unsigned int, direct pthread_exit call
tst-thread-exit-clobber.cc:80: numeric comparison failure
left: 4148288912 (0xf741dd90); from: value
right: 1600833940 (0x5f6ac994); from: magic_values.v2
info: double, direct pthread_exit call
info: unsigned int, indirect pthread_exit call
info: double, indirect pthread_exit call
error: 1 test failures
(cherry picked from commit 579396ee08)
Currently getcwd(3) can succeed without returning an absolute path
because the underlying getcwd syscall, starting with linux commit
v2.6.36-rc1~96^2~2, may succeed without returning an absolute path.
This is a conformance issue because "The getcwd() function shall
place an absolute pathname of the current working directory
in the array pointed to by buf, and return buf".
This is also a security issue because a non-absolute path returned
by getcwd(3) causes a buffer underflow in realpath(3).
Fix this by checking the path returned by getcwd syscall and falling
back to generic_getcwd if the path is not absolute, effectively making
getcwd(3) fail with ENOENT. The error code is chosen for consistency
with the case when the current directory is unlinked.
[BZ #22679]
CVE-2018-1000001
* sysdeps/unix/sysv/linux/getcwd.c (__getcwd): Fall back to
generic_getcwd if the path returned by getcwd syscall is not absolute.
* io/tst-getcwd-abspath.c: New test.
* io/Makefile (tests): Add tst-getcwd-abspath.
(cherry picked from commit 52a713fdd0)
Current optimized ia64 memchr uses a strategy to check for last address
by adding the input one with expected size. However it does not take
care for possible overflow.
It was triggered by 3038145ca2 where default rawmemchr now uses memchr
(p, c, (size_t)-1).
This patch fixes it by implement a satured addition where overflows
sets the maximum pointer size to UINTPTR_MAX.
Checked on ia64-linux-gnu where it fixes both stratcliff and
test-rawmemchr failures.
Adhemerval Zanella <adhemerval.zanella@linaro.org>
James Clarke <jrtc27@jrtc27.com>
[BZ #22603]
* sysdeps/ia64/memchr.S (__memchr): Avoid overflow in pointer
addition.
(cherry picked from commit 3bb1ef58b9)
* sysdeps/unix/sysv/linux/tst-ttyname.c (do_in_chroot_1): Skip the
test instead of failing in case of ENOENT returned by posix_openpt.
(cherry picked from commit d7ff3f11b6)
POSIX explicitly says that applications should check errno only after
failure, so the errno value can be clobbered on success as long as it
is not set to zero.
Changelog:
[BZ #22611]
* malloc/tst-realloc.c (do_test): Remove the test checking that errno
is unchanged on success.
(cherry picked from commit f8aa69be44)
The fillin_rpath function in elf/dl-load.c loops over each RPATH or
RUNPATH tokens and interprets empty tokens as the current directory
("./"). In practice the check for empty token is done *after* the
dynamic string token expansion. The expansion process can return an
empty string for the $ORIGIN token if __libc_enable_secure is set
or if the path of the binary can not be determined (/proc not mounted).
Fix that by moving the check for empty tokens before the dynamic string
token expansion. In addition, check for NULL pointer or empty strings
return by expand_dynamic_string_token.
The above changes highlighted a bug in decompose_rpath, an empty array
is represented by the first element being NULL at the fillin_rpath
level, but by using a -1 pointer in decompose_rpath and other functions.
Changelog:
[BZ #22625]
* elf/dl-load.c (fillin_rpath): Check for empty tokens before dynamic
string token expansion. Check for NULL pointer or empty string possibly
returned by expand_dynamic_string_token.
(decompose_rpath): Check for empty path after dynamic string
token expansion.
(cherry picked from commit 3e3c904dae)
Starting with commit
glibc-2.18.90-470-g2a939a7e6d81f109d49306bc2e10b4ac9ceed8f9 that
introduced substitution of dynamic string tokens in fillin_rpath,
_dl_init_paths invokes _dl_dst_substitute for $LD_LIBRARY_PATH twice:
the first time it's called directly, the second time the result
is passed on to fillin_rpath which calls expand_dynamic_string_token
which in turn calls _dl_dst_substitute, leading to the following
behaviour:
$ mkdir -p /tmp/'$ORIGIN' && cd /tmp/'$ORIGIN' &&
echo 'int main(){}' |gcc -xc - &&
strace -qq -E LD_LIBRARY_PATH='$ORIGIN' -e /open ./a.out
open("/tmp//tmp/$ORIGIN/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/tmp//tmp/$ORIGIN/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/tmp//tmp/$ORIGIN/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/tmp//tmp/$ORIGIN/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
Fix this by removing the direct _dl_dst_substitute invocation.
* elf/dl-load.c (_dl_init_paths): Remove _dl_dst_substitute preparatory
code and invocation.
(cherry picked from commit bb195224ac)
Add a new tst-ttyname test that includes several named sub-testcases.
This patch is ordered after the patches with the fixes that it tests for (to
avoid breaking `git bisect`), but for reference, here's how each relevant change
so far affected the testcases in this commit, starting with
15e9a4f378:
| | before | | make checks | don't |
| | 15e9a4f | 15e9a4f | consistent | bail |
|---------------------------------+---------+---------+-------------+-------|
| basic smoketest | PASS | PASS | PASS | PASS |
| no conflict, no match | PASS[1] | PASS | PASS | PASS |
| no conflict, console | PASS | FAIL! | FAIL | PASS! |
| conflict, no match | FAIL | PASS! | PASS | PASS |
| conflict, console | FAIL | FAIL | FAIL | PASS! |
| with readlink target | PASS | PASS | PASS | PASS |
| with readlink trap; fallback | FAIL | FAIL | FAIL | PASS! |
| with readlink trap; no fallback | FAIL | PASS! | PASS | PASS |
| with search-path trap | FAIL | FAIL | PASS! | PASS |
|---------------------------------+---------+---------+-------------+-------|
| | 4/9 | 5/9 | 6/9 | 9/9 |
[1]: 15e9a4f introduced a semantic that, under certain failure
conditions, ttyname sets errno=ENODEV, where previously it didn't
set errno; it's not quite fair to hold "before 15e9a4f" ttyname to
those new semantics. This testcase actually fails, but would have
passed if we tested for the old the semantics.
Each of the failing tests before 15e9a4f are all essentially the same bug: that
it returns a PTY slave with the correct minor device number, but from the wrong
devpts filesystem instance.
15e9a4f sought to fix this, but missed several of the cases that can cause this
to happen, and also broke the case where both the erroneous PTY and the correct
PTY exist.
Acked-by: Christian Brauner <christian.brauner@ubuntu.com>
(cherry picked from commit d9611e3085)
Commit 15e9a4f378 introduced logic for ttyname()
sending back ENODEV to signal that we can't get a name for the TTY because we
inherited it from a different mount namespace.
However, just because we inherited it from a different mount namespace and it
isn't available at its original path, doesn't mean that its name is unknowable;
we can still try to find it by allowing the normal fall back on iterating
through devices.
An example scenario where this happens is with "/dev/console" in containers.
It's a common practice among container managers to allocate a PTY master/slave
pair in the host's mount namespace (the slave having a path like "/dev/pty/$X"),
bind mount the slave to "/dev/console" in the container's mount namespace, and
send the slave FD to a process in the container. Inside of the
container, the slave-end isn't available at its original path ("/dev/pts/$X"),
since the container mount namespace has a separate devpts instance from the host
(that path may or may not exist in the container; if it does exist, it's not the
same PTY slave device). Currently ttyname{_r} sees that the file at the
original "/dev/pts/$X" path doesn't match the FD passed to it, and fails early
and gives up, even though if it kept searching it would find the TTY at
"/dev/console". Fix that; don't have the ENODEV path force an early return
inhibiting the fall-back search.
This change is based on the previous patch that adds use of is_mytty in
getttyname and getttyname_r. Without that change, this effectively reverts
15e9a4f, which made us disregard the false similarity of file pointed to by
"/proc/self/fd/$Y", because if it doesn't bail prematurely then that file
("/dev/pts/$X") will just come up again anyway in the fall-back search.
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
(cherry picked from commit a09dfc19ed)
In the ttyname and ttyname_r routines on Linux, at several points it needs to
check if a given TTY is the TTY we are looking for. It used to be that this
check was (to see if `maybe` is `mytty`):
__xstat64(_STAT_VER, maybe_filename, &maybe) == 0
#ifdef _STATBUF_ST_RDEV
&& S_ISCHR(maybe.st_mode) && maybe.st_rdev == mytty.st_rdev
#else
&& maybe.st_ino == mytty.st_ino && maybe.st_dev == mytty.st_dev
#endif
This check appears in several places.
Then, one of the changes made in commit 15e9a4f378
was to change that check to:
__xstat64(_STAT_VER, maybe_filename, &maybe) == 0
#ifdef _STATBUF_ST_RDEV
&& S_ISCHR(maybe.st_mode) && maybe.st_rdev == mytty.st_rdev
#endif
&& maybe.st_ino == mytty.st_ino && maybe.st_dev == mytty.st_dev
That is, it made the st_ino and st_dev parts of the check happen even if we have
the st_rdev member. This is an important change, because the kernel allows
multiple devpts filesystem instances to be created; a device file in one devpts
instance may share the same st_rdev with a file in another devpts instance, but
they aren't the same file.
This check appears twice in each file (ttyname.c and ttyname_r.c), once (in
ttyname and __ttyname_r) to check if a candidate file found by inspecting /proc
is the desired TTY, and once (in getttyname and getttyname_r) to check if a
candidate file found by searching /dev is the desired TTY. However, 15e9a4f
only updated the checks for files found via /proc; but the concern about
collisions between devpts instances is just as valid for files found via /dev.
So, update all 4 occurrences the check to be consistent with the version of the
check introduced in 15e9a4f. Make it easy to keep all 4 occurrences of the
check consistent by pulling it in to a static inline function, is_mytty.
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
(cherry picked from commit 2fbce9c203)
is_pty returning a bool is fine since there's no possible outcome other than
true or false, and bool is used throughout the codebase.
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
(cherry picked from commit d10d6cab16)
Linux 4.10 moved many of the documentation files around.
4.10 came out between the time the patch adding the comment (commit
15e9a4f378) was submitted and the time
it was applied (in February, January, and March 2017; respectively).
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
(cherry picked from commit 9b5a87502d)
Commit 15e9a4f378 introduced ENODEV as a possible
error condition for ttyname and ttyname_r. Update the manual to mention this GNU
extension.
Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>
(cherry picked from commit 495a56fdeb)
Regenerate libm-test-ulps for gcc 7 with "-m32 -O2 -march=i586".
* sysdeps/i386/fpu/libm-test-ulps: Regenerated for GCC 7 with
"-O2 -march=i586".
(cherry picked from commit 63d3b468c1)
This patch fixes ia64 failures on thread exit by madvise the required
area taking in consideration its disjoing stacks
(NEED_SEPARATE_REGISTER_STACK). Also the snippet that setup the
madvise call to advertise kernel the area won't be used anymore in
near future is reallocated in allocatestack.c (for consistency to
put all stack management function in one place).
Checked on x86_64-linux-gnu and i686-linux-gnu for sanity (since
it is not expected code changes for architecture that do not
define NEED_SEPARATE_REGISTER_STACK) and also got a report that
it fixes ia64-linux-gnu failures from Sergei Trofimovich
<slyfox@gentoo.org>.
[BZ #21672]
* nptl/allocatestack.c [_STACK_GROWS_DOWN] (setup_stack_prot):
Set to use !NEED_SEPARATE_REGISTER_STACK as well.
(advise_stack_range): New function.
* nptl/pthread_create.c (START_THREAD_DEFN): Move logic to mark
stack non required to advise_stack_range at allocatestack.c
(cherry pick from commit 01b87c656f)
Default semantic for mmap2 syscall is to take the offset in 4096-byte
units. However m68k and ia64 mmap2 implementation take in the
configured pageunit units and for both architecture it can be
different values.
This patch fixes the m68k runtime discover of mmap2 offset unit
and adds the ia64 definition to find it at runtime.
Checked the basic tst-mmap and tst-mmap-offset on m68k (the system
is configured with 4k, so current code is already passing on this
system) and a sanity check on x86_64-linux-gnu (which should not be
affected by this change). Sergei also states that ia64 loader now
work correctly with this change.
Adhemerval Zanella <adhemerval.zanella@linaro.org>
Sergei Trofimovich <slyfox@inbox.ru>
* sysdeps/unix/sysv/linux/m68k/mmap_internal.h (MMAP2_PAGE_SHIFT):
Rename to MMAP2_PAGE_UNIT.
* sysdeps/unix/sysv/linux/mmap.c: Include mmap_internal iff
__OFF_T_MATCHES_OFF64_T is not defined.
* sysdeps/unix/sysv/linux/mmap_internal.h (page_unit): Declare as
uint64_t.
(MMAP2_PAGE_UNIT) [MMAP2_PAGE_UNIT == -1]: Redefine to page_unit.
(page_unit) [MMAP2_PAGE_UNIT != -1]: Remove definition.
(cherry picked from commit 1f14d0c3dd)
When running strace, IPC_64 was set in the command, but ia64 is
an architecture where CONFIG_ARCH_WANT_IPC_PARSE_VERSION *isn't* set
in the kernel, so ipc_parse_version just returns IPC_64 without
clearing the IPC_64 bit in the command.
* sysdeps/unix/sysv/linux/ia64/ipc_priv.h: New file defining
__IPC_64 to 0 to avoid IPC_64 being set.
Signed-off-by: James Clarke <jrtc27@jrtc27.com>
(cherry picked from commit 89bd8016b3)
realloc_check has
unsigned char *magic_p;
...
__libc_lock_lock (main_arena.mutex);
const mchunkptr oldp = mem2chunk_check (oldmem, &magic_p);
__libc_lock_unlock (main_arena.mutex);
if (!oldp)
malloc_printerr ("realloc(): invalid pointer");
...
if (newmem == NULL)
*magic_p ^= 0xFF;
with
static void malloc_printerr(const char *str) __attribute__ ((noreturn));
GCC 7 -O3 warns
hooks.c: In function ‘realloc_check’:
hooks.c:352:14: error: ‘magic_p’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
*magic_p ^= 0xFF;
due to the GCC bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82090
This patch silences GCC 7 by using DIAG_IGNORE_NEEDS_COMMENT.
[BZ #22052]
* malloc/hooks.c (realloc_check): Use DIAG_IGNORE_NEEDS_COMMENT
to silence -O3 -Wall warning with GCC 7.
(cherry picked from commit 8e57c9432a)
When the per-thread cache is enabled, __libc_malloc uses request2size (which
does not perform an overflow check) to calculate the chunk size from the
requested allocation size. This leads to an integer overflow causing malloc
to incorrectly return the last successfully allocated block when called with
a very large size argument (close to SIZE_MAX).
This commit uses checked_request2size instead, removing the overflow.
(cherry picked from commit 34697694e8)
This patch adds a single-threaded fast path to malloc, realloc,
calloc and memalloc. When we're single-threaded, we can bypass
arena_get (which always locks the arena it returns) and just use
the main arena. Also avoid retrying a different arena since
there is just the main arena.
* malloc/malloc.c (__libc_malloc): Add SINGLE_THREAD_P path.
(__libc_realloc): Likewise.
(_mid_memalign): Likewise.
(__libc_calloc): Likewise.
(cherry-picked 3f6bb8a32e)
This patch adds single-threaded fast paths to _int_free.
Bypass the explicit locking for larger allocations.
* malloc/malloc.c (_int_free): Add SINGLE_THREAD_P fast paths.
(cherry-picked from a15d53e2de)
This patch fixes a deadlock in the fastbin consistency check.
If we fail the fast check due to concurrent modifications to
the next chunk or system_mem, we should not lock if we already
have the arena lock. Simplify the check to make it obviously
correct.
* malloc/malloc.c (_int_free): Fix deadlock bug in consistency check.
(cherry-pick d74e6f6c0d)
Clean up calls to malloc_printerr and trim its argument list.
This also removes a few bits of work done before calling
malloc_printerr (such as unlocking operations).
The tunable/environment variable still enables the lightweight
additional malloc checking, but mallopt (M_CHECK_ACTION)
no longer has any effect.
(cherry-picked from ac3ed168d0)
Linux commit ID cba6ac4869e45cc93ac5497024d1d49576e82666 reserved a new
bit for a scenario where transactional memory is available, but the
suspended state is disabled.
* sysdeps/powerpc/bits/hwcap.h (PPC_FEATURE2_HTM_NO_SUSPEND): New
macro.
(cherry picked from commit df0c40ee3a)
Signed-off-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
Unlike the vfork forwarder and like the fork forwarder as in bug 19861,
there won't be a problem when the compiler does not turn this into a tail
call.
(cherry picked from commit fc5ad7024c)
POWER9 DD2.1 and earlier has an issue where some cache inhibited
vector load traps to the kernel, causing a performance degradation. To
handle this in memcpy and memmove, lvx/stvx is used for aligned
addresses instead of lxvd2x/stxvd2x.
Reference: https://patchwork.ozlabs.org/patch/814059/
* sysdeps/powerpc/powerpc64/power7/memcpy.S: Replace
lxvd2x/stxvd2x with lvx/stvx.
* sysdeps/powerpc/powerpc64/power7/memmove.S: Likewise.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit 63da5cd4a0)
When configuring and building GNU libc using the Mozilla NSS library
for cryptography (--enable-nss-crypt option), also include the
NSPR header files along with the Mozilla NSS library header files.
Finally, when running the check-local-headers test, ignore the
Mozilla NSPR library header files (used by the Mozilla NSS library).
(cherry picked from commit 57b4af1955)
Currently free typically uses 2 atomic operations per call. The have_fastchunks
flag indicates whether there are recently freed blocks in the fastbins. This
is purely an optimization to avoid calling malloc_consolidate too often and
avoiding the overhead of walking all fast bins even if all are empty during a
sequence of allocations. However using catomic_or to update the flag is
completely unnecessary since it can be changed into a simple boolean and
accessed using relaxed atomics. There is no change in multi-threaded behaviour
given the flag is already approximate (it may be set when there are no blocks in
any fast bins, or it may be clear when there are free blocks that could be
consolidated).
Performance of malloc/free improves by 27% on a simple benchmark on AArch64
(both single and multithreaded). The number of load/store exclusive instructions
is reduced by 33%. Bench-malloc-thread speeds up by ~3% in all cases.
* malloc/malloc.c (FASTCHUNKS_BIT): Remove.
(have_fastchunks): Remove.
(clear_fastchunks): Remove.
(set_fastchunks): Remove.
(malloc_state): Add have_fastchunks.
(malloc_init_state): Use have_fastchunks.
(do_check_malloc_state): Remove incorrect invariant checks.
(_int_malloc): Use have_fastchunks.
(_int_free): Likewise.
(malloc_consolidate): Likewise.
(cherry-picked from e956075a5a)
The functions tcache_get and tcache_put show up in profiles as they
are a critical part of the tcache code. Inline them to give tcache
a 16% performance gain. Since this improves multi-threaded cases
as well, it helps offset any potential performance loss due to adding
single-threaded fast paths.
* malloc/malloc.c (tcache_put): Inline.
(tcache_get): Inline.
(cherry-picked from commit e4dd4ace56)
Normally, TLS relocations against local symbols are optimised by the linker
to be absolute. However, gold does not do this, and so it is possible to
end up with, for example, R_SPARC_TLS_DTPMOD64 referring to a local symbol.
Since sym_map is left as null in elf_machine_rela for the special local
symbol case, the relocation handling thinks it has nothing to do, and so
the module gets left as 0. Havoc then ensues when the variable in question
is accessed.
Before this fix, the main_local_gold program would receive a SIGBUS on
sparc64, and SIGSEGV on powerpc32. With this fix applied, that test now
passes like the rest of them.
* sysdeps/powerpc/powerpc32/dl-machine.h (elf_machine_rela):
Assign sym_map to be map for local symbols, as TLS relocations
use sym_map to determine whether the symbol is defined and to
extract the TLS information.
* sysdeps/sparc/sparc32/dl-machine.h (elf_machine_rela): Likewise.
* sysdeps/sparc/sparc64/dl-machine.h (elf_machine_rela): Likewise.
(cherry picked from commit 8644588807)
Update libm-test-ulps for AVX512 mathvec tests by running
“make regen-ulps” on Intel Xeon processor with AVX512.
* sysdeps/x86_64/fpu/libm-test-ulps: Regenerated.
(cherry picked from commit fcaaca412f)
This patch adds two new internal defines to set the internal
pthread_mutex_t layout required by the supported ABIS:
1. __PTHREAD_MUTEX_NUSERS_AFTER_KIND which control whether to define
__nusers fields before or after __kind. The preferred value for
is 0 for new ports and it sets __nusers before __kind.
2. __PTHREAD_MUTEX_USE_UNION which control whether internal __spins and
__list members will be place inside an union for linuxthreads
compatibility. The preferred value is 0 for ports and it sets
to not use an union to define both fields.
It fixes the wrong offsets value for __kind value on x86_64-linux-gnu-x32.
Checked with a make check run-built-tests=no on all afected ABIs.
[BZ #22298]
* nptl/allocatestack.c (allocate_stack): Check if
__PTHREAD_MUTEX_HAVE_PREV is non-zero, instead if
__PTHREAD_MUTEX_HAVE_PREV is defined.
* nptl/descr.h (pthread): Likewise.
* nptl/nptl-init.c (__pthread_initialize_minimal_internal):
Likewise.
* nptl/pthread_create.c (START_THREAD_DEFN): Likewise.
* sysdeps/nptl/fork.c (__libc_fork): Likewise.
* sysdeps/nptl/pthread.h (PTHREAD_MUTEX_INITIALIZER): Likewise.
* sysdeps/nptl/bits/thread-shared-types.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION): New
defines.
(__pthread_internal_list): Check __PTHREAD_MUTEX_USE_UNION instead
of __WORDSIZE for internal layout.
(__pthread_mutex_s): Check __PTHREAD_MUTEX_NUSERS_AFTER_KIND instead
of __WORDSIZE for internal __nusers layout and __PTHREAD_MUTEX_USE_UNION
instead of __WORDSIZE whether to use an union for __spins and __list
fields.
(__PTHREAD_MUTEX_HAVE_PREV): Define also for __PTHREAD_MUTEX_USE_UNION
case.
* sysdeps/aarch64/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION): New
defines.
* sysdeps/alpha/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/arm/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/hppa/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/ia64/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/m68k/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/microblaze/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/mips/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/nios2/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/powerpc/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/s390/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/sh/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/sparc/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/tile/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
* sysdeps/x86/nptl/bits/pthreadtypes-arch.h
(__PTHREAD_MUTEX_NUSERS_AFTER_KIND, __PTHREAD_MUTEX_USE_UNION):
Likewise.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit 06be6368da)
This patch adds a new build test to check for internal fields
offsets for user visible internal field. Although currently
the only field which is statically initialized to a non zero value
is pthread_mutex_t.__data.__kind value, the tests also check the
offset of __kind, __spins, __elision (if supported), and __list
internal member. A internal header (pthread-offset.h) is added
to each major ABI with the reference value.
Checked on x86_64-linux-gnu and with a build check for all affected
ABIs (aarch64-linux-gnu, alpha-linux-gnu, arm-linux-gnueabihf,
hppa-linux-gnu, i686-linux-gnu, ia64-linux-gnu, m68k-linux-gnu,
microblaze-linux-gnu, mips64-linux-gnu, mips64-n32-linux-gnu,
mips-linux-gnu, powerpc64le-linux-gnu, powerpc-linux-gnu,
s390-linux-gnu, s390x-linux-gnu, sh4-linux-gnu, sparc64-linux-gnu,
sparcv9-linux-gnu, tilegx-linux-gnu, tilegx-linux-gnu-x32,
tilepro-linux-gnu, x86_64-linux-gnu, and x86_64-linux-x32).
* nptl/pthreadP.h (ASSERT_PTHREAD_STRING,
ASSERT_PTHREAD_INTERNAL_OFFSET): New macro.
* nptl/pthread_mutex_init.c (__pthread_mutex_init): Add build time
checks for internal pthread_mutex_t offsets.
* sysdeps/aarch64/nptl/pthread-offsets.h
(__PTHREAD_MUTEX_NUSERS_OFFSET, __PTHREAD_MUTEX_KIND_OFFSET,
__PTHREAD_MUTEX_SPINS_OFFSET, __PTHREAD_MUTEX_ELISION_OFFSET,
__PTHREAD_MUTEX_LIST_OFFSET): New macro.
* sysdeps/alpha/nptl/pthread-offsets.h: Likewise.
* sysdeps/arm/nptl/pthread-offsets.h: Likewise.
* sysdeps/hppa/nptl/pthread-offsets.h: Likewise.
* sysdeps/i386/nptl/pthread-offsets.h: Likewise.
* sysdeps/ia64/nptl/pthread-offsets.h: Likewise.
* sysdeps/m68k/nptl/pthread-offsets.h: Likewise.
* sysdeps/microblaze/nptl/pthread-offsets.h: Likewise.
* sysdeps/mips/nptl/pthread-offsets.h: Likewise.
* sysdeps/nios2/nptl/pthread-offsets.h: Likewise.
* sysdeps/powerpc/nptl/pthread-offsets.h: Likewise.
* sysdeps/s390/nptl/pthread-offsets.h: Likewise.
* sysdeps/sh/nptl/pthread-offsets.h: Likewise.
* sysdeps/sparc/nptl/pthread-offsets.h: Likewise.
* sysdeps/tile/nptl/pthread-offsets.h: Likewise.
* sysdeps/x86_64/nptl/pthread-offsets.h: Likewise.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit dff91cd45e)
As shown in some buildbot issues on aarch64 and powerpc, calling
clone (VFORK) and waitpid (WNOHANG) does not guarantee the child
is ready to be collected. This patch changes the call back to 0
as before fe05e1cb6d fix.
This change can lead to the scenario 4.3 described in the commit,
where the waitpid call can hang undefinitely on the call. However
this is also a very unlikely and also undefinied situation where
both the caller is trying to terminate a pid before posix_spawn
returns and the race pid reuse is triggered. I don't see how to
correct handle this specific situation within posix_spawn.
Checked on x86_64-linux-gnu, aarch64-linux-gnu and
powerpc64-linux-gnu.
* sysdeps/unix/sysv/linux/spawni.c (__spawnix): Use 0 instead of
WNOHANG in waitpid call.
(cherry picked from commit aa95a2414e)
As noted by Florian Weimer, current Linux posix_spawn implementation
can trigger an assert if the auxiliary process is terminated before
actually setting the err member:
340 /* Child must set args.err to something non-negative - we rely on
341 the parent and child sharing VM. */
342 args.err = -1;
[...]
362 new_pid = CLONE (__spawni_child, STACK (stack, stack_size), stack_size,
363 CLONE_VM | CLONE_VFORK | SIGCHLD, &args);
364
365 if (new_pid > 0)
366 {
367 ec = args.err;
368 assert (ec >= 0);
Another possible issue is killing the child between setting the err and
actually calling execve. In this case the process will not ran, but
posix_spawn also will not report any error:
269
270 args->err = 0;
271 args->exec (args->file, args->argv, args->envp);
As suggested by Andreas Schwab, this patch removes the faulty assert
and also handles any signal that happens before fork and execve as the
spawn was successful (and thus relaying the handling to the caller to
figure this out). Different than Florian, I can not see why using
atomics to set err would help here, essentially the code runs
sequentially (due CLONE_VFORK) and I think it would not be legal the
compiler evaluate ec without checking for new_pid result (thus there
is no need to compiler barrier).
Summarizing the possible scenarios on posix_spawn execution, we
have:
1. For default case with a success execution, args.err will be 0, pid
will not be collected and it will be reported to caller.
2. For default failure case, args.err will be positive and the it will
be collected by the waitpid. An error will be reported to the
caller.
3. For the unlikely case where the process was terminated and not
collected by a caller signal handler, it will be reported as succeful
execution and not be collected by posix_spawn (since args.err will
be 0). The caller will need to actually handle this case.
4. For the unlikely case where the process was terminated and collected
by caller we have 3 other possible scenarios:
4.1. The auxiliary process was terminated with args.err equal to 0:
it will handled as 1. (so it does not matter if we hit the pid
reuse race since we won't possible collect an unexpected
process).
4.2. The auxiliary process was terminated after execve (due a failure
in calling it) and before setting args.err to -1: it will also
be handle as 1. but with the issue of not be able to report the
caller a possible execve failures.
4.3. The auxiliary process was terminated after args.err is set to -1:
this is the case where it will be possible to hit the pid reuse
case where we will need to collected the auxiliary pid but we
can not be sure if it will be expected one. I think for this
case we need to actually change waitpid to use WNOHANG to avoid
hanging indefinitely on the call and report an error to caller
since we can't differentiate between a default failure as 2.
and a possible pid reuse race issue.
Checked on x86_64-linux-gnu.
* sysdeps/unix/sysv/linux/spawni.c (__spawnix): Handle the case where
the auxiliary process is terminated by a signal before calling _exit
or execve.
(cherry picked from commit fe05e1cb6d)
This patch fixes the compat glob implementation consolidation from
commit 116f1c64d with the following changes:
- Add a compat implementation on s390 to avoid the architecture
to build the symbols on default linux oldglob.c by setting
GLOB_NO_OLD_VERSION.
- Remove the duplicate rule to build oldglob on alpha.
Checked on s390-linux-gnu and alpha-linux-gnu using build-many-glibc.py.
* sysdeps/unix/sysv/linux/s390/s390-32/oldglob.c: New file.
* sysdeps/unix/sysv/linux/alpha/Makefile
[$(subdir) = csu] (sysdep_routines): Remove rule.
(cherry picked from commit 3ca622e4d6)
This patch consolidates the glob implementation. The main changes are:
* On Linux all implementation now uses the default one at
sysdeps/unix/sysv/linux/glob{free}{64}.c with the exception
of alpha (which requires specific versioning) and s390-32 (which
different than other 32 bits ports it does not add a compat one
symbol for 2.1 version).
* The default implementation uses XSTAT_IS_XSTAT64 to define whether
both glob{free} and glob{free}64 should be different implementations.
For archictures that define XSTAT_IS_XSTAT64, glob{free} is an alias
to glob{free}64.
* Move i386 olddirent.h header to Linux default directory, since it is
the only header with this name and it is shared among different
architectures (and used on compat glob symbol as well).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
* sysdeps/unix/sysv/linux/arm/glob64.c: Remove file.
* sysdeps/unix/sysv/linux/i386/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/m68k/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/wordsize-64/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/wordsize-64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/glob.c: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/globfree.c: Likewise.
* sysdeps/wordsize-64/glob.c: Likewise.
* sysdeps/wordsize-64/glob64.c: Likewise.
* sysdeps/wordsize-64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/glob.c: New file.
* sysdeps/unix/sysv/linux/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/globfree.c: Likewise.
* sysdeps/unix/sysv/linux/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/glob64.c: Likewise.
* sysdeps/unix/sysv/linux/oldglob.c [SHLIB_COMPAT]: Also
adds !GLOB_NO_OLD_VERSION as an extra condition.
* sysdeps/unix/sysv/linux/i386/alphasort64.c: Include olddirent.h
using relative path instead of absolute one.
* sysdeps/unix/sysv/linux/i386/getdents64.c: Likewise.
* sysdeps/unix/sysv/linux/i386/readdir64.c: Likewise.
* sysdeps/unix/sysv/linux/i386/readdir64_r.c: Likewise.
* sysdeps/unix/sysv/linux/i386/versionsort64.c: Likewise.
* sysdeps/unix/sysv/linux/i386/olddirent.h: Move to ...
* sysdeps/unix/sysv/linux//olddirent.h: ... here.
(cherry picked from commit 116f1c64d8)
Current implementation of tunables does not set arena_max and arena_test
values. Any value provided by glibc.malloc.arena_max and
glibc.malloc.arena_test parameters is ignored.
These tunables have minval value set to 1 (see elf/dl-tunables.list file)
and undefined maxval value. In that case default value (which is 0. see
scripts/gen-tunables.awk) is being used to set maxval.
For instance, generated tunable_list[] entry for arena_max is:
(gdb) p *cur
$1 = {name = 0x7ffff7df6217 "glibc.malloc.arena_max",
type = {type_code = TUNABLE_TYPE_SIZE_T, min = 1, max = 0},
val = {numval = 0, strval = 0x0}, initialized = false,
security_level = TUNABLE_SECLEVEL_SXID_IGNORE,
env_alias = 0x7ffff7df622e "MALLOC_ARENA_MAX"}
As a result, any value of glibc.malloc.arena_max is ignored by
TUNABLE_SET_VAL_IF_VALID_RANGE macro
__type min = (__cur)->type.min; <- initialized to 1
__type max = (__cur)->type.max; <- initialized to 0!
if (min == max) <- false
{
min = __default_min;
max = __default_max;
}
if ((__type) (__val) >= min && (__type) (val) <= max) <- false
{
(__cur)->val.numval = val;
(__cur)->initialized = true;
}
Assigning correct min/max values at a build time fixes a problem.
Plus, a bit of optimization: Setting of default min/max values for the
given type at a run time might be eliminated.
* elf/dl-tunables.c (do_tunable_update_val): Range checking fix.
* scripts/gen-tunables.awk: Set unspecified minval and/or maxval
values to correct default value for given type.
Similar to bug 21987 for SPARC, MIPS64 wrongly installs the ldbl-128
version of bits/long-double.h, meaning incorrect results when using
headers installed from a 64-bit installation for a 32-bit build. (I
haven't actually seen this cause build failures before its interaction
with bits/floatn.h did so - installed headers wrongly expecting
_Float128 to be available in a 32-bit configuration.)
This patch fixes the bug by moving the MIPS header to
sysdeps/mips/ieee754, which comes before sysdeps/ieee754/ldbl-128 in
the sysdeps directory ordering. (bits/floatn.h will need a similar
fix - duplicating the ldbl-128 version for MIPS will suffice - for
headers from a 32-bit installation to be correct for 64-bit builds.)
Tested with build-many-glibcs.py (compilers build for
mips64-linux-gnu, where there was previously a libstdc++ build failure
as at
<https://sourceware.org/ml/libc-testresults/2017-q4/msg00130.html>).
[BZ #22322]
* sysdeps/mips/bits/long-double.h: Move to ....
* sysdeps/mips/ieee754/bits/long-double.h: ... here.
(cherry picked from commit 37bb78cb8c)
When using gcc < 6.x, signbit does not use the type-generic
__builtin_signbit builtin, instead it uses __MATH_TG.
However, when library support for float128 is available, __MATH_TG uses
__builtin_types_compatible_p, which is not available in C++ mode.
On the other hand, libstdc++ undefines (in cmath) many macros from
math.h, including signbit, so that it can provide its own functions.
However, during its configure tests, libstdc++ just tests for the
availability of the macros (it does not undefine them, nor does it
provide its own functions).
Finally, libstdc++ configure tests include math.h and get the definition
of signbit that uses __MATH_TG (and __builtin_types_compatible_p).
Since libstdc++ does not undefine the macros during its configure
tests, they fail.
This patch lets signbit use the builtin in C++ mode when gcc < 6.x is
used. This allows the configure test in libstdc++ to work.
Tested for x86_64.
[BZ #22296]
* math/math.h: Let signbit use the builtin in C++ mode with gcc
< 6.x
Cc: Gabriel F. T. Gomes <gftg@linux.vnet.ibm.com>
Cc: Joseph Myers <joseph@codesourcery.com>
(cherry picked from commit 386e1c26ac)
In _dl_runtime_resolve, use fxsave/xsave/xsavec to preserve all vector,
mask and bound registers. It simplifies _dl_runtime_resolve and supports
different calling conventions. ld.so code size is reduced by more than
1 KB. However, use fxsave/xsave/xsavec takes a little bit more cycles
than saving and restoring vector and bound registers individually.
Latency for _dl_runtime_resolve to lookup the function, foo, from one
shared library plus libc.so:
Before After Change
Westmere (SSE)/fxsave 345 866 151%
IvyBridge (AVX)/xsave 420 643 53%
Haswell (AVX)/xsave 713 1252 75%
Skylake (AVX+MPX)/xsavec 559 719 28%
Skylake (AVX512+MPX)/xsavec 145 272 87%
Ryzen (AVX)/xsavec 280 553 97%
This is the worst case where portion of time spent for saving and
restoring registers is bigger than majority of cases. With smaller
_dl_runtime_resolve code size, overall performance impact is negligible.
On IvyBridge, differences in build and test time of binutils with lazy
binding GCC and binutils are noises. On Westmere, differences in
bootstrap and "makc check" time of GCC 7 with lazy binding GCC and
binutils are also noises.
[BZ #21265]
* sysdeps/x86/cpu-features-offsets.sym (XSAVE_STATE_SIZE_OFFSET):
New.
* sysdeps/x86/cpu-features.c: Include <libc-pointer-arith.h>.
(get_common_indeces): Set xsave_state_size, xsave_state_full_size
and bit_arch_XSAVEC_Usable if needed.
(init_cpu_features): Remove bit_arch_Use_dl_runtime_resolve_slow
and bit_arch_Use_dl_runtime_resolve_opt.
* sysdeps/x86/cpu-features.h (bit_arch_Use_dl_runtime_resolve_opt):
Removed.
(bit_arch_Use_dl_runtime_resolve_slow): Likewise.
(bit_arch_Prefer_No_AVX512): Updated.
(bit_arch_MathVec_Prefer_No_AVX512): Likewise.
(bit_arch_XSAVEC_Usable): New.
(STATE_SAVE_OFFSET): Likewise.
(STATE_SAVE_MASK): Likewise.
[__ASSEMBLER__]: Include <cpu-features-offsets.h>.
(cpu_features): Add xsave_state_size and xsave_state_full_size.
(index_arch_Use_dl_runtime_resolve_opt): Removed.
(index_arch_Use_dl_runtime_resolve_slow): Likewise.
(index_arch_XSAVEC_Usable): New.
* sysdeps/x86/cpu-tunables.c (TUNABLE_CALLBACK (set_hwcaps)):
Support XSAVEC_Usable. Remove Use_dl_runtime_resolve_slow.
* sysdeps/x86_64/Makefile (tst-x86_64-1-ENV): New if tunables
is enabled.
* sysdeps/x86_64/dl-machine.h (elf_machine_runtime_setup):
Replace _dl_runtime_resolve_sse, _dl_runtime_resolve_avx,
_dl_runtime_resolve_avx_slow, _dl_runtime_resolve_avx_opt,
_dl_runtime_resolve_avx512 and _dl_runtime_resolve_avx512_opt
with _dl_runtime_resolve_fxsave, _dl_runtime_resolve_xsave and
_dl_runtime_resolve_xsavec.
* sysdeps/x86_64/dl-trampoline.S (DL_RUNTIME_UNALIGNED_VEC_SIZE):
Removed.
(DL_RUNTIME_RESOLVE_REALIGN_STACK): Check STATE_SAVE_ALIGNMENT
instead of VEC_SIZE.
(REGISTER_SAVE_BND0): Removed.
(REGISTER_SAVE_BND1): Likewise.
(REGISTER_SAVE_BND3): Likewise.
(REGISTER_SAVE_RAX): Always defined to 0.
(VMOV): Removed.
(_dl_runtime_resolve_avx): Likewise.
(_dl_runtime_resolve_avx_slow): Likewise.
(_dl_runtime_resolve_avx_opt): Likewise.
(_dl_runtime_resolve_avx512): Likewise.
(_dl_runtime_resolve_avx512_opt): Likewise.
(_dl_runtime_resolve_sse): Likewise.
(_dl_runtime_resolve_sse_vex): Likewise.
(USE_FXSAVE): New.
(_dl_runtime_resolve_fxsave): Likewise.
(USE_XSAVE): Likewise.
(_dl_runtime_resolve_xsave): Likewise.
(USE_XSAVEC): Likewise.
(_dl_runtime_resolve_xsavec): Likewise.
* sysdeps/x86_64/dl-trampoline.h (_dl_runtime_resolve_avx512):
Removed.
(_dl_runtime_resolve_avx512_opt): Likewise.
(_dl_runtime_resolve_avx): Likewise.
(_dl_runtime_resolve_avx_opt): Likewise.
(_dl_runtime_resolve_sse): Likewise.
(_dl_runtime_resolve_sse_vex): Likewise.
(_dl_runtime_resolve_fxsave): New.
(_dl_runtime_resolve_xsave): Likewise.
(_dl_runtime_resolve_xsavec): Likewise.
(cherry picked from commit b52b0d793d)
Before glibc 2.26, ld.so set dl_platform to "x86_64" and searched the
"x86_64" subdirectory when loading a shared library. ld.so in glibc
2.26 was changed to set dl_platform to "haswell" or "xeon_phi", based
on supported ISAs. This led to shared library loading failure for
shared libraries placed under the "x86_64" subdirectory.
This patch adds "x86_64" to x86-64 dl_hwcap so that ld.so will always
search the "x86_64" subdirectory when loading a shared library.
NB: We can't set x86-64 dl_platform to "x86-64" since ld.so will skip
the "haswell" and "xeon_phi" subdirectories on "haswell" and "xeon_phi"
machines.
Tested on i686 and x86-64.
[BZ #22093]
* sysdeps/x86/cpu-features.c (init_cpu_features): Initialize
GLRO(dl_hwcap) to HWCAP_X86_64 for x86-64.
* sysdeps/x86/dl-hwcap.h (HWCAP_COUNT): Updated.
(HWCAP_IMPORTANT): Likewise.
(HWCAP_X86_64): New enum.
(HWCAP_X86_AVX512_1): Updated.
* sysdeps/x86/dl-procinfo.c (_dl_x86_hwcap_flags): Add "x86_64".
* sysdeps/x86_64/Makefile (tests): Add tst-x86_64-1.
(modules-names): Add x86_64/tst-x86_64mod-1.
(LDFLAGS-tst-x86_64mod-1.so): New.
($(objpfx)tst-x86_64-1): Likewise.
($(objpfx)x86_64/tst-x86_64mod-1.os): Likewise.
(tst-x86_64-1-clean): Likewise.
* sysdeps/x86_64/tst-x86_64-1.c: New file.
* sysdeps/x86_64/tst-x86_64mod-1.c: Likewise.
(cherry picked from commit 45ff34638f)
This patch syncs posix/glob.c implementation with gnulib version
b5ec983 (glob: simplify symlink detection). The only difference
to gnulib code is
* DT_UNKNOWN, DT_DIR, and DT_LNK definition in the case there
were not already defined. Gnulib code which uses
HAVE_STRUCT_DIRENT_D_TYPE will redefine them wrongly because
GLIBC does not define HAVE_STRUCT_DIRENT_D_TYPE. Instead
the patch check for each definition instead.
Also, the patch requires additional globfree and globfree64 files
for compatibility version on some architectures. Also the code
simplification leads to not macro simplification (not need for
NO_GLOB_PATTERN_P anymore).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #1062]
* posix/Makefile (routines): Add globfree, globfree64, and
glob_pattern_p.
* posix/flexmember.h: New file.
* posix/glob_internal.h: Likewise.
* posix/glob_pattern_p.c: Likewise.
* posix/globfree.c: Likewise.
* posix/globfree64.c: Likewise.
* sysdeps/gnu/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/alpha/globfree.c: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/oldglob.c: Likewise.
* sysdeps/unix/sysv/linux/wordsize-64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/globfree.c: Likewise.
* sysdeps/wordsize-64/globfree.c: Likewise.
* sysdeps/wordsize-64/globfree64.c: Likewise.
* posix/glob.c (HAVE_CONFIG_H): Use !_LIBC instead.
[NDEBUG): Remove comments.
(GLOB_ONLY_P, _AMIGA, VMS): Remove define.
(dirent_type): New type. Use uint_fast8_t not
uint8_t, as C99 does not require uint8_t.
(DT_UNKNOWN, DT_DIR, DT_LNK): New macros.
(struct readdir_result): Use dirent_type. Do not define skip_entry
unless it is needed; this saves a byte on platforms lacking d_ino.
(readdir_result_type, readdir_result_skip_entry):
New functions, replacing ...
(readdir_result_might_be_symlink, readdir_result_might_be_dir):
these functions, which were removed. This makes the callers
easier to read. All callers changed.
(D_INO_TO_RESULT): Now empty if there is no d_ino.
(size_add_wrapv, glob_use_alloca): New static functions.
(glob, glob_in_dir): Check for size_t overflow in several places,
and fix some size_t checks that were not quite right.
Remove old code using SHELL since Bash no longer
uses this.
(glob, prefix_array): Separate MS code better.
(glob_in_dir): Remove old Amiga and VMS code.
(globfree, __glob_pattern_type, __glob_pattern_p): Move to
separate files.
(glob_in_dir): Do not rely on undefined behavior in accessing
struct members beyond their bounds. Use a flexible array member
instead
(link_stat): Rename from link_exists2_p and return -1/0 instead of
0/1. Caller changed.
(glob): Fix memory leaks.
* posix/glob64 (globfree64): Move to separate file.
* sysdeps/gnu/glob64.c (NO_GLOB_PATTERN_P): Remove define.
(globfree64): Remove hidden alias.
* sysdeps/unix/sysv/linux/Makefile (sysdeps_routines): Add
oldglob.
* sysdeps/unix/sysv/linux/alpha/glob.c (__new_globfree): Move to
separate file.
* sysdeps/unix/sysv/linux/i386/glob64.c (NO_GLOB_PATTERN_P): Remove
define.
Move compat code to separate file.
* sysdeps/wordsize-64/glob.c (globfree): Move definitions to
separate file.
(cherry picked from commit c66c908230)
Hide internal __old_glob64 function to allow direct access within
libc.so and libc.a without using GOT nor PLT.
[BZ #18822]
* sysdeps/unix/sysv/linux/i386/glob64.c (__old_glob64): Add
libc_hidden_proto and libc_hidden_def.
(cherry picked from commit 2585d7b839)
After commit 37f802f864 (Remove
__need_IOV_MAX and __need_FOPEN_MAX), UIO_MAXIOV is no longer supplied
(indirectly) through <bits/stdio_lim.h>, so sysdeps/posix/sysconf.c no
longer sees the definition.
(cherry picked from commit 63b4baa44e)
The previous implementation had at least a quadratic space
requirement in the number of host addresses and aliases.
(cherry picked from commit d8425e116c)
The IEEE 754 implementation of lgammal in sysdeps/ieee754/ldbl-128/ used
to be shared by IBM's implementation in sysdeps/ieee754/ldbl-128ibm/ (by
an inclusion of the source file). In order for the algorithm to work
for IBM's implementation, a check for LDBL_MANT_DIG was required. Since
the source file is no longer shared, the requirement for the check is
gone. This patch removes the conditionals.
Tested for powerpc64le and s390x.
* sysdeps/ieee754/ldbl-128/e_lgammal_r.c (__ieee754_lgammal_r):
Remove conditionals on LDBL_MANT_DIG.
* sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c
(__ieee754_lgammal_r): Likewise.
(cherry picked from commit 9ac3c68218)
The ldbl-128ibm implementation of j0l, j1l, lgammal_r, and cbrtl, as
well as the tables used by expl were copied from ldbl-128. However, the
original files used _Float128 for the type and L() for the literal
suffix. This patch uses the following sed command to rewrite _Float128
as long double and L(x) as xL (for e_expl.c, e_j0l.c, e_j1l.c,
e_lgammal_r.c, and t_expl.h):
sed -i <filename> \
-e "/^#define _Float128 long double/d" \
-e "/^#define L(x) x ## L/d" \
-e "/L(/s/)/L/" \
-e "/L(/s/L(//" \
-e "s/_Float128/long double/g"
For sysdeps/ieee754/ldbl-128ibm/s_cbrtl.c, this sed command incorrectly
replaces a few occurrences of L(), so the following command is used
instead:
sed -i sysdeps/ieee754/ldbl-128ibm/s_cbrtl.c \
-e "/^#define _Float128 long double/d" \
-e "/^#define L(x) x ## L/d" \
-e "s/L(0\.3\{40\})/0.3333333333333333333333333333333333333333L/" \
-e "s/L(3\.7568280825958912391243e-1)/3.7568280825958912391243e-1L/" \
-e "/L(/s/)/L/" \
-e "/L(/s/L(//" \
-e "s/_Float128/long double/g"
Tested for powerpc64le with patched [1] and unpatched gcc.
[1] https://gcc.gnu.org/ml/gcc-patches/2017-08/msg01028.html
* sysdeps/ieee754/ldbl-128ibm/e_expl.c: Remove definitions of
_Float128 and L().
* sysdeps/ieee754/ldbl-128ibm/e_j0l.c: Remove definitions of
_Float128 and L(). Replace _Float128 with long double and L(x)
with xL, throughout the file.
* sysdeps/ieee754/ldbl-128ibm/e_j1l.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_cbrtl.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/t_expl.h: Likewise.
(cherry picked from commit d2f0ed09f8)
Some files under sysdeps/ieee754/ldbl-128ibm/ are able to reuse the
implementation in sysdeps/ieee754/ldbl-128/ by defining _Float128 to
long double. This relied on compiler support for _Float128 being
disabled. On powerpc, such support was disabled by default, however, it
got enabled by default [1] in GCC 8.
This patch copies the implementations from ldbl-128 to ldbl-128ibm. The
uses of _Float128 and L() are kept intact in this patch and are replaced
with a script in a subsequent patch.
[1] https://gcc.gnu.org/ml/gcc-patches/2017-08/msg01028.html
Tested for powerpc64 and powerpc64le.
* sysdeps/ieee754/ldbl-128ibm/e_expl.c: Include tables from
sysdeps/ieee754/ldbl-128ibm.
* sysdeps/ieee754/ldbl-128ibm/e_j0l.c: Copy contents from the
equivalent implementation in sysdeps/ieee754/ldbl-128/ instead
of including it. Keep _Float128 and L() intact. These will be
reviewed by a separate patch.
* sysdeps/ieee754/ldbl-128ibm/e_j1l.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_cbrtl.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/t_expl.h: Likewise.
(cherry picked from commit c5c2e667bf)
On powerpc64le, compiler support for float128 is not enabled by default
on gcc. To enable it, the flag -mfloat128 must be passed as a command
line option to the compiler. This means that only the few files that
actively have -mfloat128 passed as an argument get compiler support for
float128, whereas all other files don't.
When -mfloat128 becomes enabled by default on powerpc [1], all the files
that do not currently have compiler support for float128 enabled during
their compilation, will start to have it. This will lead to build
errors in s_finite.c, s_isinf.c, and s_isnan.c.
The errors are due to the unintended macro expansion of __finitef128 to
__redirect_finitef128 in math/bits/mathcalls-helper-functions.h. In
that header, __MATHDECL_1 takes '__finite' and 'f128' as arguments and
concatenates them. However, since '__finite' has been redefined in
s_finite.c, the function declaration becomes __redirect_finitef128:
extern int __redirect___finitef128 (_Float128 __value) __attribute__ ((__nothrow__ )) __attribute__ ((__const__));
This declaration itself is OK. The problem arises when include/math.h
creates the hidden prototype ('hidden_proto (__finitef128)'), which
expands to:
extern __typeof (__finitef128) __finitef128 __attribute__ ((visibility ("hidden")));
Since __finitef128 is not declared, __typeof fails. This effect was
already true for the 'float' and 'long double' versions and is now true
for float128. Likewise for isinsff128 and isnanf128.
This patch defines __finitef128 as __redirect___finitef128 in
sysdeps/powerpc/powerpc64/fpu/multiarch/s_finite.c, similarly to what's
done for the float and long double versions of these functions, to get
rid of the build error. Likewise for isinff128 and isnanf128.
[1] https://gcc.gnu.org/ml/gcc-patches/2017-08/msg01028.html
Tested for powerpc64 and powerpc64le.
* sysdeps/powerpc/powerpc64/fpu/multiarch/s_finite.c
(__finitef128): Define to __redirect___finitef128.
* sysdeps/powerpc/powerpc64/fpu/multiarch/s_isinf.c
(__isinff128): Define to __redirect___isinff128.
* sysdeps/powerpc/powerpc64/fpu/multiarch/s_isnan.c
(__isnanf128): Define to __redirect___isnanf128.
(cherry picked from commit e010deb231)
On powerpc64le, not all files can have the flag -mfloat128 passed as an
option on the compile command, since that could conflict with other
flags, such as -mno-vsx. Each file that needs the flag, gets it through
a CFLAGS-filename variable on sysdeps/powerpc/powerpc64le/Makefile.
The test cases tst-strtod-nan-locale and tst-wcstod-nan-locale are
missing this flag.
Tested for powerpc64le.
* sysdeps/powerpc/powerpc64le/Makefile
(CFLAGS-tst-strtod-nan-locale.c): New variable.
(CFLAGS-tst-wcstod-nan-locale.c): New variable.
(cherry picked from commit ffa448041b)
This is an optimized memmove implementation for the Qualcomm Falkor
processor core. Due to the way the falkor memcpy needs to be written,
code cannot be easily shared between memmove and memcpy like in case
of other aarch64 memcpy implementations due to which this routine is
separate. The underlying principle is the same as that of memcpy
where it tries to use registers with the same lower 4 bits for
fetching the same stream, thus optimizing hardware prefetcher
performance.
The memcpy copy loop copies 64 bytes at a time using the same register
pair since that's the way to train the hardware prefetcher on the
falkor core. memmove cannot quite do that since it needs to avoid
overlaps, so it does the next best thing, i.e. has a 32 byte loop with
a 32 byte end (prefetch a loop ahead to account for overlapping
locations) with register pairs that alias so that they hit the same
prefetcher. Due to this difference in loop size, they have to
currently be separate implementations but efforts are on to try and
get memmove to fall back into memcpy whenever it can without simply
duplicating all of the code.
Performance:
The routine fares around 20-25% better than the generic memmove for
most medium to large sizes (i.e. > 128 bytes) for the new walking
memmove benchmark (memmove-walk) with an unexplained regression
between 1K and 2K. The minor regression is something worth looking
into for us, but the remaining gains are significant enough that we
would like this included upstream as we looking into the cause for the
regression. Here is a snippet of the numbers as generated from the
microbenchmark by the compare_strings script. Comparisons are against
__memmove_generic:
Function: memmove
Variant: walk
__memmove_thunderx __memmove_falkor __memmove_generic
========================================================================================================================
<snip>
length=16384: 12508800.00 ( 6.09%) 11486800.00 ( 13.76%) 13319600.00
length=16400: 13614200.00 ( -0.67%) 11585000.00 ( 14.33%) 13523600.00
length=16385: 13448400.00 ( 0.10%) 11732700.00 ( 12.84%) 13461200.00
length=16399: 13594100.00 ( -0.22%) 11859600.00 ( 12.57%) 13564400.00
length=16386: 13211600.00 ( 1.13%) 11503800.00 ( 13.91%) 13362400.00
length=16398: 13218600.00 ( 2.12%) 11573200.00 ( 14.30%) 13504700.00
length=16387: 13510900.00 ( -0.37%) 11744200.00 ( 12.76%) 13461300.00
length=16397: 13603700.00 ( -0.15%) 11878200.00 ( 12.55%) 13583200.00
length=16388: 13461700.00 ( -0.13%) 11558000.00 ( 14.03%) 13444100.00
length=16396: 13517500.00 ( -0.03%) 11561300.00 ( 14.45%) 13513900.00
length=16389: 13534100.00 ( 0.17%) 11756800.00 ( 13.28%) 13556900.00
length=16395: 13585600.00 ( 0.11%) 11791800.00 ( 13.30%) 13601200.00
length=16390: 13480100.00 ( -0.13%) 11685500.00 ( 13.20%) 13462100.00
length=16394: 13529900.00 ( -0.23%) 11549800.00 ( 14.43%) 13498200.00
length=16391: 13595400.00 ( -0.26%) 11768200.00 ( 13.22%) 13560600.00
length=16393: 13567000.00 ( 0.20%) 11779700.00 ( 13.35%) 13594700.00
length=32768: 71308800.00 ( -6.53%) 50220800.00 ( 24.98%) 66939200.00
length=32784: 72100800.00 (-11.55%) 50114100.00 ( 22.47%) 64636300.00
length=32769: 71767000.00 ( -7.10%) 51238400.00 ( 23.54%) 67010000.00
length=32783: 70113700.00 (-40.95%) 51129000.00 ( -2.78%) 49744400.00
length=32770: 71367600.00 ( -6.52%) 50244700.00 ( 25.01%) 67000900.00
length=32782: 64366700.00 ( 4.71%) 50101400.00 ( 25.83%) 67545600.00
length=32771: 71440100.00 ( -6.51%) 51263900.00 ( 23.57%) 67074900.00
length=32781: 66993000.00 ( 0.34%) 51108300.00 ( 23.97%) 67220300.00
length=32772: 71443900.00 (-60.50%) 50062100.00 (-12.47%) 44512600.00
length=32780: 71759100.00 ( -6.58%) 50263200.00 ( 25.35%) 67328600.00
length=32773: 71714900.00 (-33.21%) 51076600.00 ( 5.12%) 53835400.00
length=32779: 71756900.00 ( -6.56%) 51290800.00 ( 23.83%) 67337800.00
length=32774: 59689300.00 (-34.55%) 50068400.00 (-12.86%) 44363300.00
length=32778: 71847500.00 (-18.20%) 50084100.00 ( 17.61%) 60786500.00
length=32775: 71599300.00 ( -6.54%) 51278200.00 ( 23.70%) 67204800.00
length=32777: 71862900.00 (-60.85%) 51094000.00 (-14.36%) 44677900.00
length=65536: 282848000.00 ( -6.60%) 199187000.00 ( 24.93%) 265325000.00
length=65552: 243285000.00 (-41.61%) 198512000.00 (-15.54%) 171805000.00
length=65537: 255415000.00 (-23.47%) 202499000.00 ( 2.11%) 206858000.00
length=65551: 280122000.00 (-62.95%) 203349000.00 (-18.29%) 171911000.00
length=65538: 283676000.00 (-14.46%) 198368000.00 ( 19.96%) 247848000.00
length=65550: 275566000.00 (-51.76%) 198494000.00 ( -9.31%) 181581000.00
length=65539: 283699000.00 ( -6.58%) 203453000.00 ( 23.57%) 266195000.00
length=65549: 286572000.00 ( -6.65%) 202607000.00 ( 24.60%) 268712000.00
length=65540: 283710000.00 ( -6.59%) 199161000.00 ( 25.17%) 266160000.00
length=65548: 237573000.00 ( 11.48%) 198462000.00 ( 26.06%) 268395000.00
length=65541: 284150000.00 ( -6.58%) 203273000.00 ( 23.75%) 266600000.00
length=65547: 286250000.00 ( -6.70%) 202594000.00 ( 24.48%) 268263000.00
length=65542: 284167000.00 ( -6.60%) 199122000.00 ( 25.31%) 266584000.00
length=65546: 285656000.00 ( -6.59%) 198443000.00 ( 25.95%) 268002000.00
length=65543: 284600000.00 ( -6.58%) 203247000.00 ( 23.89%) 267030000.00
length=65545: 285665000.00 ( -6.40%) 202575000.00 ( 24.55%) 268472000.00
<snip>
* sysdeps/aarch64/multiarch/Makefile (sysdep_routines): Add
memmove_falkor.
* sysdeps/aarch64/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Likewise.
* sysdeps/aarch64/multiarch/memmove.c: Likewise.
* sysdeps/aarch64/multiarch/memmove_falkor.S: New file.
The malloc tcache added in 2.26 will leak all of the elements remaining
in the cache and the cache structure itself when a thread exits. The
defect is that we do not set tcache_shutting_down early enough, and the
thread simply recreates the tcache and places the elements back onto a
new tcache which is subsequently lost as the thread exits (unfreed
memory). The fix is relatively simple, move the setting of
tcache_shutting_down earlier in tcache_thread_freeres. We add a test
case which uses mallinfo and some heuristics to look for unaccounted for
memory usage between the start and end of a thread start/join loop. It
is very reliable at detecting that there is a leak given the number of
iterations. Without the fix the test will consume 122MiB of leaked
memory.
(cherry picked from commit 1e26d35193)
Fix GCC 7 compilation error:
test-math-iscanonical.cc: In function ‘void check_type()’:
test-math-iscanonical.cc:33:11: error: use of an operand of type ‘bool’ in ‘operator++’ is deprecated [-Werror=deprecated]
errors++;
^~
Since not all non-zero error counts are errors, return errors != 0
instead.
* math/test-math-iscanonical.cc (error): Replace bool with int.
(do_test): Return errors != 0.
(cherry picked from commit cdd4155d6c and
commit 758f1bfa2a)
All representations of floating-point numbers in types with IEC 60559
binary exchange format are canonical. On the other hand, types with IEC
60559 extended formats, such as those implemented under ldbl-96 and
ldbl-128ibm, contain representations that are not canonical.
TS 18661-1 introduced the type-generic macro iscanonical, which returns
whether a floating-point value is canonical or not. In Glibc, this
type-generic macro is implemented using the macro __MATH_TG, which, when
support for float128 is enabled, relies on __builtin_types_compatible_p
to select between floating-point types. However, this use of
iscanonical breaks C++ applications, because the builtin is only
available in C mode.
This patch provides a C++ implementation of iscanonical that relies on
function overloading, rather than builtins, to select between
floating-point types.
Unlike the C++ implementations for iszero and issignaling, this
implementation ignores __NO_LONG_DOUBLE_MATH. The double type always
matches IEC 60559 double format, which is always canonical. Thus, when
double and long double are the same (__NO_LONG_DOUBLE_MATH), iscanonical
always returns 1 and is not implemented with __MATH_TG.
Tested for powerpc64, powerpc64le and x86_64.
[BZ #22235]
* math/math.h: Trivial fix for unbalanced parentheses in comment.
* math/Makefile [CXX] (tests): Add test-math-iscanonical.cc.
(CFLAGS-test-math-iscanonical.cc): New variable.
* math/test-math-iscanonical.cc: New file.
* sysdeps/ieee754/ldbl-96/bits/iscanonical.h (iscanonical):
Provide a C++ implementation based on function overloading,
rather than using __MATH_TG, which uses C-only builtins.
* sysdeps/ieee754/ldbl-128ibm/bits/iscanonical.h (iscanonical):
Likewise.
* sysdeps/powerpc/powerpc64le/Makefile
(CFLAGS-test-math-iscanonical.cc): New variable.
(cherry picked from commit aa0235dfde)
My refactoring of long double information
commit 0acb8a2a85
Author: Joseph Myers <joseph@codesourcery.com>
Date: Wed Dec 14 18:27:56 2016 +0000
Refactor long double information into bits/long-double.h.
resulted in sparc32 configurations installing the ldbl-opt version of
bits/long-double.h instead of the intended
sysdeps/unix/sysv/linux/sparc version.
For sparc32 by itself, this is not a problem, since the ldbl-opt
version is correct for sparc32. However, both sparc32 and sparc64 are
supposed to install sets of headers that work for both of them, so
that a single sysroot, whichever order the libraries are built and
installed in, works for both. The effect of having the wrong version
installed is that you end up with a miscompiled sparc64 libstdc++
which fails glibc's configure tests for the C++ compiler.
This patch moves the header from sysdeps/unix/sysv/linux/sparc to
separate copies of the same file for sparc32 and sparc64, to ensure it
comes before ldbl-opt in the sysdeps directory ordering.
Tested with build-many-glibcs.py for sparc64-linux-gnu and
sparcv9-linux-gnu.
[BZ #21987]
* sysdeps/unix/sysv/linux/sparc/bits/long-double.h: Remove file
and copy to ...
* sysdeps/unix/sysv/linux/sparc/sparc32/bits/long-double.h:
... here.
* sysdeps/unix/sysv/linux/sparc/sparc64/bits/long-double.h:
... and here.
(cherry picked from commit 80f91666fe)
In <https://sourceware.org/ml/libc-alpha/2013-05/msg00722.html> I
remarked on the possibility of arithmetic in various nearbyint
implementations being scheduled before feholdexcept calls, resulting
in spurious "inexact" exceptions.
I'm now actually observing this occurring in glibc built for ARM with
GCC 7 (in fact, both copies of the same addition/subtraction sequence
being combined and moved out before the conditionals and
feholdexcept/fesetenv pairs), resulting in test failures.
This patch makes the nearbyint implementations with this particular
feholdexcept / arithmetic / fesetenv pattern consistently use
math_opt_barrier on the function argument when first used in
arithmetic, and also consistently use math_force_eval before fesetenv
(the latter was generally already done, but the dbl-64/wordsize-64
implementation used math_opt_barrier instead, and as
math_opt_barrier's intended effect is through its output value being
used, such a use that doesn't use the return value is suspect).
Tested for x86_64 (--disable-multi-arch so more of these
implementations get used), and for ARM in a configuration where I saw
the problem scheduling.
[BZ #22225]
* sysdeps/ieee754/dbl-64/s_nearbyint.c (__nearbyint): Use
math_opt_barrier on argument when doing arithmetic on it.
* sysdeps/ieee754/dbl-64/wordsize-64/s_nearbyint.c (__nearbyint):
Likewise. Use math_force_eval not math_opt_barrier after
arithmetic.
* sysdeps/ieee754/flt-32/s_nearbyintf.c (__nearbyintf): Use
math_opt_barrier on argument when doing arithmetic on it.
* sysdeps/ieee754/ldbl-128/s_nearbyintl.c (__nearbyintl):
Likewise.
(cherry picked from commit f124cb3811)
When optimization for size is on (-Os), fpclassify does not use the
type-generic __builtin_fpclassify builtin, instead it uses __MATH_TG.
However, when library support for float128 is available, __MATH_TG uses
__builtin_types_compatible_p, which is not available in C++ mode.
On the other hand, libstdc++ undefines (in cmath) many macros from
math.h, including fpclassify, so that it can provide its own functions.
However, during its configure tests, libstdc++ just tests for the
availability of the macros (it does not undefine them, nor does it
provide its own functions).
Finally, when libstdc++ is configured with optimization for size
enabled, its configure tests include math.h and get the definition of
fpclassify that uses __MATH_TG (and __builtin_types_compatible_p).
Since libstdc++ does not undefine the macros during its configure tests,
they fail.
This patch lets fpclassify use the builtin in C++ mode, even when
optimization for size is on. This allows the configure test in
libstdc++ to work.
Tested for powerpc64le and x86_64.
[BZ #22146]
math/math.h: Let fpclassify use the builtin in C++ mode, even
when optimazing for size.
(cherry picked from commit c5c4a62609)
Since Martin Sebor's commit
commit ee4e992ebe
Author: Martin Sebor <msebor@redhat.com>
Date: Tue Aug 22 09:35:23 2017 -0600
Declare ifunc resolver to return a pointer to the same type as the target
function to help GCC detect incompatibilities between the two when it's
enhanced to do so.
builds for powerpc64le fail in the declaration of some ifunc resolvers,
because the ifunc is declared with unmatching return types. One of the
declarations comes from the __ifunc_resolver macro, which was patched by
the aforementioned commit:
/* Helper / base macros for indirect function symbols. */
#define __ifunc_resolver(type_name, name, expr, arg, init, classifier) \
classifier inhibit_stack_protector \
__typeof (type_name) *name##_ifunc (arg) \
whereas the other comes from the unpatched __ifunc macro when
HAVE_GCC_IFUNC is not defined:
# define __ifunc(type_name, name, expr, arg, init) \
extern __typeof (type_name) name; \
void *name##_ifunc (arg) __asm__ (#name); \
This patch changes the return type of the ifunc resolver in the __ifunc
macro, so that it matches the return type of the target function,
similarly to what the aforementioned commit does.
Tested for powerpc64le and s390x with unpatched GCC.
* include/libc-symbols.h: [!defined HAVE_GCC_IFUNC] (__ifunc):
Change the return type of the ifunc resolver to match the return
type of the target function.
(cherry picked from commit b513da7e80)
This makes the __tls_get_addr_opt test run as a shared library, and so
actually test that DTPMOD64/DTPREL64 pairs are processed by ld.so to
support the __tls_get_adfr_opt call stub fast return. After a
2017-01-24 patch (binutils f0158f4416) ld.bfd no longer emitted
unnecessary dynamic relocations against local thread variables,
instead setting up the __tls_index GOT entries for the call stub fast
return. This meant tst-tlsopt-powerpc passed but did not check ld.so
relocation support. After a 2017-07-16 patch (binutils 676ee2b5fa)
ld.bfd no longer set up the __tls_index GOT entries for the call stub
fast return, and tst-tlsopt-powerpc failed.
Compiling mod-tlsopt-powerpc.c with -DSHARED exposed a bug in
powerpc64/tls-macros.h, which defines a __TLS_GET_ADDR macro that
clashes with one defined in dl-tls.h. The tls-macros.h version is
only used in that file, so delete it and expand.
* sysdeps/powerpc/mod-tlsopt-powerpc.c: Extract from
tst-tlsopt-powerpc.c with function name change and no test harness.
* sysdeps/powerpc/tst-tlsopt-powerpc.c: Remove body of test.
Call tls_get_addr_opt_test.
* sysdeps/powerpc/Makefile (LDFLAGS-tst-tlsopt-powerpc): Don't define.
(modules-names): Add mod-tlsopt-powerpc.
(mod-tlsopt-powerpc.so-no-z-defs): Define.
(tst-tlsopt-powerpc): Depend on .so.
* sysdeps/powerpc/powerpc64/tls-macros.h (__TLS_GET_ADDR): Don't
define. Expand use in TLS_GD and TLS_LD.
(cherry picked from commit e98c925fa4)
Fix GCC 7 errors when string/stratcliff.c is compiled with -O3:
stratcliff.c: In function ‘do_test’:
cc1: error: assuming signed overflow does not occur when assuming that (X - c) <= X is always true [-Werror=strict-overflow]
[BZ #21982]
* string/stratcliff.c (do_test): Declare size, nchars, inner,
middle and outer with size_t instead of int. Repleace %d and
%Zd with %zu in printf. Update "MAX (0, nchars - 128)" and
"MAX (outer, nchars - 64)" to support unsigned outer and
nchars. Also exit loop when outer == 0.
(cherry picked from commit 376b40a27a)
Since sofini.os terminates .eh_frame section, it should be placed last.
[BZ #22051]
* Makerules (build-module-helper-objlist): Filter out
$(elf-objpfx)sofini.os.
(build-shlib-objlist): Append $(elf-objpfx)sofini.os if it is
needed.
(cherry picked from commit ecd0747df3)
The old code uses errno as the primary indicator for success or
failure. This is wrong because errno is only set for specific
combinations of the status return value and the h_errno variable.
(cherry picked from commit f4a6be2582)
This simplifies the code because it is not necessary to propagate the
temporary h_errno value to the thread-local variable. It also increases
compatibility with NSS modules which update only one of the two places.
(cherry picked from commit 53250a21b8)
tests-unsupported has to be defined before the inclusion of Rules in a
subdirectory Makefile; otherwise it is ineffective. This patch fixes
the ordering in assert/Makefile, where a recent test addition put
tests-unsupported too late (resulting in build failures when the C++
compiler was missing or broken, and thereby showing up the unrelated
bug 21987).
Incidentally, I don't see why these tests depend on
$(have-cxx-thread_local) rather than just a working C++ compiler.
Tested in such a configuration (broken compiler/libstdc++) with
build-many-glibcs.py.
* assert/Makefile [$(have-cxx-thread_local)]: Move conditional
variable definitions above inclusion of ../Rules.
(cherry picked from commit 75dfe623df)
When signaling nans are enabled (with -fsignaling-nans), the C++ version
of iszero uses the fpclassify macro, which is defined with __MATH_TG.
However, when support for float128 is available, __MATH_TG uses the
builtin __builtin_types_compatible_p, which is only available in C mode.
This patch refactors the C++ version of iszero so that it uses function
overloading to select between the floating-point types, instead of
relying on fpclassify and __MATH_TG.
Tested for powerpc64le, s390x, x86_64, and with build-many-glibcs.py.
[BZ #21930]
* math/math.h [defined __cplusplus && defined __SUPPORT_SNAN__]
(iszero): New C++ implementation that does not use
fpclassify/__MATH_TG/__builtin_types_compatible_p, when
signaling nans are enabled, since __builtin_types_compatible_p
is a C-only feature.
* math/test-math-iszero.cc: When __HAVE_DISTINCT_FLOAT128 is
defined, include ieee754_float128.h for access to the union and
member ieee854_float128.ieee.
[__HAVE_DISTINCT_FLOAT128] (do_test): Call check_float128.
[__HAVE_DISTINCT_FLOAT128] (check_float128): New function.
* sysdeps/powerpc/powerpc64le/Makefile [subdir == math]
(CXXFLAGS-test-math-iszero.cc): Add -mfloat128 to the build
options of test-math-zero on powerpc64le.
(cherry picked from commit 42496114ec)
When __NO_LONG_DOUBLE_MATH is defined, __issignalingl is not available,
thus issignaling with long double argument should call __issignaling,
instead.
Tested for powerpc64le.
* math/math.h [defined __cplusplus] (issignaling): In the long
double case, call __issignalingl only if __NO_LONG_DOUBLE_MATH
is not defined. Call __issignaling, otherwise.
(cherry picked from commit 3d7b66f66c)
The macro __MATH_TG contains the logic to select between long double and
_Float128, when these types are ABI-distinct. This logic relies on
__builtin_types_compatible_p, which is not available in C++ mode.
On the other hand, C++ function overloading provides the means to
distinguish between the floating-point types. The overloading
resolution will match the correct parameter regardless of type
qualifiers, i.e.: const and volatile.
Tested for powerpc64le, s390x, and x86_64.
* math/math.h [defined __cplusplus] (issignaling): Provide a C++
definition for issignaling that does not rely on __MATH_TG,
since __MATH_TG uses __builtin_types_compatible_p, which is only
available in C mode.
(CFLAGS-test-math-issignaling.cc): New variable.
* math/Makefile [CXX] (tests): Add test-math-issignaling.
* math/test-math-issignaling.cc: New test for C++ implementation
of type-generic issignaling.
* sysdeps/powerpc/powerpc64le/Makefile [subdir == math]
(CXXFLAGS-test-math-issignaling.cc): Add -mfloat128 to the build
options of test-math-issignaling on powerpc64le.
(cherry picked from commit a16e8bc08e)
The hidden attribute was overridden by libc_hidden_proto on GNU/Linux.
It is incorrect because the function is used from nscd.
internal_function is not supposed to be used across DSO boundaries,
so this commit removes it (again, due to the use in nscd).
(cherry picked from commit f87cc2bfba)
The logic to protect the use of generic selection (_Generic) does not
check for C or C++ mode, however, generic selection is a C-only
feature.
Tested for powerpc64le.
* misc/sys/cdefs.h (__HAVE_GENERIC_SELECTION): Define to 0, if
in C++ mode.
(cherry picked from commit 6913ad65e0)
The logic to define isinf for float128 depends on the availability of
__builtin_types_compatible_p, which is only available in C mode,
however, the conditionals do not check for C or C++ mode. This lead to
an error in libstdc++ configure, as reported by bug 21930.
This patch adds a conditional for C mode in the definition of isinf for
float128. No definition is provided in C++ mode, since libstdc++
headers undefine isinf.
Tested for powerpc64le (glibc test suite and libstdc++-v3 configure).
[BZ #21930]
* math/math.h (isinf): Check if in C or C++ mode before using
__builtin_types_compatible_p, since this is a C mode feature.
(cherry picked from commit 47a67213a9)
POWER ISA 3.0 introduces the xssqrtqp instructions, which expects
operands to be in Vector Registers (Altivec/VMX), even though this
instruction belongs to the Vector-Scalar Instruction Set.
In GCC's Extended Assembly for POWER, the 'wq' register constraint is
provided for use with IEEE 754 128-bit floating-point values. However,
this constraint does not limit the register allocation to Vector
Registers (Altivec/VMX) and could assign a Vector-Scalar Register (VSX)
to the operands of the instruction.
This patch changes the register constraint used in sqrtf128 from 'wq' to
'v', in order to request a Vector Register (Altivec/VMX) for use with
the xssqrtqp instruction.
Tested for powerpc64le and --with-cpu=power9.
[BZ #21941]
* sysdeps/powerpc/fpu/math_private.h (__ieee754_sqrtf128): Since
xssqrtqp requires operands to be in Vector Registers
(Altivec/VMX), replace the register constraint 'wq' with 'v'.
* sysdeps/powerpc/powerpc64le/power9/fpu/e_sqrtf128.c
(__ieee754_sqrtf128): Likewise.
(cherry picked from commit 4d98ace9de)
Different than other architectures hppa-linux-gnu define different values
for ENOTSUP and EOPNOTSUPP, where the later is a Linux specific one.
This leads to tst-preadwritev{64}v2 tests failures:
$ ./testrun.sh misc/tst-preadvwritev2
error: tst-preadvwritev2-common.c:35: preadv2 failure did not set errno to ENOTSUP (223)
error: 1 test failures
The straightforward fix is to return the POSIX defined ENOTSUP on all
p{read,write}v{64}v2 implementations instead of Linux specific one.
Checked on x86_64-linux-gnu and the tst-preadwritev{64}v2 on
hppa-linux-gnu (although due the installed kernel on my testing system
the pwritev{64}v2 with an invalid flag still fails due a known kernel
issue [1]).
[BZ #21780]
* sysdeps/posix/preadv2.c (preadv2): Use ENOTSUP instead of
EOPNOTSUPP.
* sysdeps/posix/preadv64v2.c (preadv64v2): Likewise.
* sysdeps/posix/pwritev2.c (pwritev2): Likewise.
* sysdeps/posix/pwritev64v2.c (pwritev64v2): Likewise.
* sysdeps/unix/sysv/linux/preadv2.c (preadv2): Likewise.
* sysdeps/unix/sysv/linux/preadv64v2.c (preadv64v2): Likewise.
* sysdeps/unix/sysv/linux/pwritev2.c (pwritev2): Likewise.
* sysdeps/unix/sysv/linux/pwritev64v2.c (pwritev64v2): Likewise.
[1] https://sourceware.org/ml/libc-alpha/2017-06/msg00726.html
Cherry-pick of 852d631207
On AVX machines with XGETBV (ECX == 1) like Skylake processors,
(gdb) disass _dl_runtime_resolve_avx_opt
Dump of assembler code for function _dl_runtime_resolve_avx_opt:
0x0000000000015890 <+0>: push %rax
0x0000000000015891 <+1>: push %rcx
0x0000000000015892 <+2>: push %rdx
0x0000000000015893 <+3>: mov $0x1,%ecx
0x0000000000015898 <+8>: xgetbv
0x000000000001589b <+11>: mov %eax,%r11d
0x000000000001589e <+14>: pop %rdx
0x000000000001589f <+15>: pop %rcx
0x00000000000158a0 <+16>: pop %rax
0x00000000000158a1 <+17>: and $0x4,%r11d
0x00000000000158a5 <+21>: bnd je 0x16200 <_dl_runtime_resolve_sse_vex>
End of assembler dump.
is slower than:
(gdb) disass _dl_runtime_resolve_avx_slow
Dump of assembler code for function _dl_runtime_resolve_avx_slow:
0x0000000000015850 <+0>: vorpd %ymm0,%ymm1,%ymm8
0x0000000000015854 <+4>: vorpd %ymm2,%ymm3,%ymm9
0x0000000000015858 <+8>: vorpd %ymm4,%ymm5,%ymm10
0x000000000001585c <+12>: vorpd %ymm6,%ymm7,%ymm11
0x0000000000015860 <+16>: vorpd %ymm8,%ymm9,%ymm9
0x0000000000015865 <+21>: vorpd %ymm10,%ymm11,%ymm10
0x000000000001586a <+26>: vpcmpeqd %xmm8,%xmm8,%xmm8
0x000000000001586f <+31>: vorpd %ymm9,%ymm10,%ymm10
0x0000000000015874 <+36>: vptest %ymm10,%ymm8
0x0000000000015879 <+41>: bnd jae 0x158b0 <_dl_runtime_resolve_avx>
0x000000000001587c <+44>: vzeroupper
0x000000000001587f <+47>: bnd jmpq 0x16200 <_dl_runtime_resolve_sse_vex>
End of assembler dump.
(gdb)
since xgetbv takes much more cycles than single cycle operations like
vpord/vvpcmpeq/ptest. _dl_runtime_resolve_opt should be used only with
AVX512 where AVX512 instructions lead to lower CPU frequency on Skylake
server.
[BZ #21871]
* sysdeps/x86/cpu-features.c (init_cpu_features): Set
bit_arch_Use_dl_runtime_resolve_opt only with AVX512F.
(cherry picked from commit d2cf37c0a2)
The return type of the getentropy stub is wrongly defined as ssize_t,
while both the <sys/random.h> header and the Linux implementation
define it as int. This patch fixes that.
Changelog:
* stdlib/getentropy.c (getentropy): Change return type to int.
(cherry picked from commit 2b34e2716f)
This comes from running “make regen-ulps” on an AMD Opteron 2378 CPU.
Changelog:
* sysdeps/i386/i686/fpu/multiarch/libm-test-ulps: Regenerated.
(cherry picked from commit 144bdab050)
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Alex Butler
Szabolcs Nagy
Andreas Schwab
Florian Weimer
Marcin Kościelnicki
Dragan Mladjenovic
Wilco Dijkstra
Rajalakshmi Srinivasaraghavan
Wilco Dijkstra
Siddhesh Poyarekar
Adhemerval Zanella
Stefan Liebler
H.J. Lu
Gabriel F. T. Gomes
Tulio Magno Quites Machado Filho
Florian Weimer
Alexandra Hájková
Joseph Myers
Steve Ellcey
Paul Eggert
Martin Sebor
Ilya Yu. Malakhov
Martin Kuchta
Daniel Alvarez
Paul Pluzhnikov
Stefan Liebler
DJ Delorie
Andrew Senkevich
Jesse Hathaway
Arjun Shankar
Dmitry V. Levin
Aurelien Jarno
Luke Shumaker
James Clarke
Guido Trentalancia
Alexey Makhalov
Romain Naour
Carlos O'Donell
Gabriel F. T. Gomes
Alan Modra
Markus Trippelsdorf