According to the latest Unicode standard, a conversion from/to UTF-xx has
to report an error if the character value is in range of an utf16 surrogate
(0xd800..0xdfff). See https://sourceware.org/ml/libc-help/2015-12/msg00015.html.
Thus the cu41 instruction, which converts from utf32 to utf8, has to be
disabled because it does not report an error in case of a value in range of
a low surrogate (0xdc00..0xdfff). The etf3eh variant is removed and the c,
vector variant is adjusted to handle the value in range of an utf16 low
surrogate correctly.
ChangeLog:
* sysdeps/s390/utf8-utf32-z9.c: Disable cu41 instruction and report
an error in case of a value in range of an utf16 low surrogate.
This patch reworks the existing s390 64bit specific iconv modules in order
to use them on s390 31bit, too.
Thus the parts for subdirectory iconvdata in sysdeps/s390/s390-64/Makefile
were moved to sysdeps/s390/Makefile so that they apply on 31bit, too.
All those modules are moved from sysdeps/s390/s390-64 directory to sysdeps/s390.
The iso-8859-1 to/from cp037 module was adjusted, to use brct (branch relative
on count) instruction on 31bit s390 instead of brctg, because the brctg is a
zarch instruction and is not available on a 31bit kernel.
The utf modules are using zarch instructions, thus the directive machinemode
zarch_nohighgprs was added to the inline assemblies to omit the high-gprs flag
in the shared libraries. Otherwise they can't be loaded on a 31bit kernel.
The ifunc resolvers were adjusted in order to call the etf3eh or vector variants
only if zarch instructions are available (64bit kernel in 31bit compat-mode).
Furthermore some variable types were changed. E.g. unsigned long long would be
a register pair on s390 31bit, but we want only one single register.
For variables of type size_t the register contents have to be enlarged from a
32bit to a 64bit value on 31bit, because the inline assemblies uses 64bit values
in such cases.
ChangeLog:
* sysdeps/s390/s390-64/Makefile (iconvdata-subdirectory):
Move to ...
* sysdeps/s390/Makefile: ... here.
* sysdeps/s390/s390-64/iso-8859-1_cp037_z900.c: Move to ...
* sysdeps/s390/iso-8859-1_cp037_z900.c: ... here.
(BRANCH_ON_COUNT): New define.
(TR_LOOP): Use BRANCH_ON_COUNT instead of brctg.
* sysdeps/s390/s390-64/utf16-utf32-z9.c: Move to ...
* sysdeps/s390/utf16-utf32-z9.c: ... here and adjust to
run on s390-32, too.
* sysdeps/s390/s390-64/utf8-utf16-z9.c: Move to ...
* sysdeps/s390/utf8-utf16-z9.c: ... here and adjust to
run on s390-32, too.
* sysdeps/s390/s390-64/utf8-utf32-z9.c: Move to ...
* sysdeps/s390/utf8-utf32-z9.c: ... here and adjust to
run on s390-32, too.
This patch reworks the s390 specific module to convert between utf16 and utf32.
Now ifunc is used to choose either the c or etf3eh (with convert utf
instruction) variants at runtime.
Furthermore a new vector variant for z13 is introduced which will be build
and chosen if vector support is available at build / runtime.
In case of converting utf 32 to utf16, the vector variant optimizes input of
2byte utf16 characters. The convert utf instruction is used if an utf16
surrogate is found.
For the other direction utf16 to utf32, the cu24 instruction can't be re-
enabled, because it does not report an error, if the input-stream consists of
a single low surrogate utf16 char (e.g. 0xdc00). This applies to the newest z13,
too. Thus there is only the c or the new vector variant, which can handle utf16
surrogate characters.
This patch also fixes some whitespace errors. Furthermore, the etf3eh variant is
handling the "UTF-xx//IGNORE" case now. Before they ignored the ignore-case and
always stopped at an error.
ChangeLog:
* sysdeps/s390/s390-64/utf16-utf32-z9.c: Use ifunc to select c,
etf3eh or new vector loop-variant.
This patch reworks the s390 specific module to convert between utf8 and utf16.
Now ifunc is used to choose either the c or etf3eh (with convert utf instruction)
variants at runtime. Furthermore a new vector variant for z13 is introduced
which will be build and chosen if vector support is available at build / runtime.
In case of converting utf 8 to utf16, the vector variant optimizes input of
1byte utf8 characters. The convert utf instruction is used if a multibyte utf8
character is found.
For the other direction utf16 to utf8, the cu21 instruction can't be re-enabled,
because it does not report an error, if the input-stream consists of a single
low surrogate utf16 char (e.g. 0xdc00). This applies to the newest z13, too.
Thus there is only the c or the new vector variant, which can handle 1..4 byte
utf8 characters.
The c variant from utf16 to utf8 has beed fixed. If a high surrogate was at the
end of the input-buffer, then errno was set to EINVAL and the input-pointer
pointed just after the high surrogate. Now it points to the beginning of the
high surrogate.
This patch also fixes some whitespace errors. The c variant from utf8 to utf16
is now checking that tail-bytes starts with 0b10... and the value is not in
range of an utf16 surrogate.
Furthermore, the etf3eh variants are handling the "UTF-xx//IGNORE" case now.
Before they ignored the ignore-case and always stopped at an error.
ChangeLog:
* sysdeps/s390/s390-64/utf8-utf16-z9.c: Use ifunc to select c,
etf3eh or new vector loop-variant.
This patch reworks the s390 specific module to convert between utf8 and utf32.
Now ifunc is used to choose either the c or etf3eh (with convert utf
instruction) variants at runtime.
Furthermore a new vector variant for z13 is introduced which will be build
and chosen if vector support is available at build / runtime.
The vector variants optimize input of 1byte utf8 characters. The convert utf
instruction is used if a multibyte utf8 character is found.
This patch also fixes some whitespace errors. The c variants are rejecting
UTF-16 surrogates and values above 0x10ffff now.
Furthermore, the etf3eh variants are handling the "UTF-xx//IGNORE" case now.
Before they ignored the ignore-case and always stopped at an error.
ChangeLog:
* sysdeps/s390/s390-64/utf8-utf32-z9.c: Use ifunc to select c, etf3eh
or new vector loop-variant.
This patch reworks the s390 specific module which used the z900
translate one to one instruction. Now the g5 translate instruction is used,
because it outperforms the troo instruction.
ChangeLog:
* sysdeps/s390/s390-64/iso-8859-1_cp037_z900.c (TROO_LOOP):
Rename to TR_LOOP and usage of tr instead of troo instruction.
This patch introduces a s390 specific gconv_simple.c file which provides
optimized versions for z13 with vector instructions, which will be chosen at
runtime via ifunc.
The optimized conversions can convert between internal and ascii, ucs4, ucs4le,
ucs2, ucs2le.
If the build-environment lacks vector support, then iconv/gconv_simple.c
is used wihtout any change. Otherwise iconvdata/gconv_simple.c is used to create
conversion loop routines without vector instructions as fallback, if vector
instructions aren't available at runtime.
ChangeLog:
* sysdeps/s390/multiarch/gconv_simple.c: New File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add gconv_simple.
This patch introduces a s390 specific 8bit-generic.c file which provides an
optimized version for z13 with translate-/vector-instructions, which will be
chosen at runtime via ifunc.
If the build-environment lacks vector support, then iconvdata/8bit-generic.c
is used wihtout any change. Otherwise iconvdata/8bit-generic.c is used to create
conversion loop routines without vector instructions as fallback, if vector
instructions aren't available at runtime.
The vector routines can only be used with charsets where the maximum UCS4 value
fits in 1 byte size. Then the hardware translate-instruction is used
to translate between up to 256 generic characters and "1 byte UCS4"
characters at once. The vector instructions are used to convert between
the "1 byte UCS4" and UCS4.
The gen-8bit.sh script in sysdeps/s390/multiarch generates the conversion
table to_ucs1. Therefore in sysdeps/s390/multiarch/Makefile is added an
override define generate-8bit-table, which is originally defined in
iconvdata/Makefile. This version calls the gen-8bit.sh in iconvdata folder
and the s390 one.
ChangeLog:
* sysdeps/s390/multiarch/8bit-generic.c: New File.
* sysdeps/s390/multiarch/gen-8bit.sh: New File.
* sysdeps/s390/multiarch/Makefile (generate-8bit-table):
New override define.
* sysdeps/s390/multiarch/iconv/skeleton.c: Likewise.
The S390 specific test checks if the gcc has support for vector registers
by compiling an inline assembly which clobbers vector registers.
On success the macro HAVE_S390_VX_GCC_SUPPORT is defined.
This macro can be used to determine if e.g. clobbering vector registers
is allowed or not.
ChangeLog:
* config.h.in (HAVE_S390_VX_GCC_SUPPORT): New macro undefine.
* sysdeps/s390/configure.ac: Add test for S390 vector register
support in gcc.
* sysdeps/s390/configure: Regenerated.
This patch introduces a way to provide an architecture dependent gconv-modules
file. Before this patch, the gconv-modules file was normally installed from
src-dir/iconvdata/gconv-modules. The S390 Makefile had overridden the
installation recipe (with a make warning) in order to install the
gconv-module-s390 file from build-dir.
The iconvdata/Makefile provides another recipe, which copies the gconv-modules
file from src to build dir, which are used by the testcases.
Thus the testcases does not use the currently build s390-modules.
This patch uses build-dir/iconvdata/gconv-modules for installation, which
is generated by concatenating src-dir/iconvdata/gconv-modules and the
architecture specific one. The latter one can be specified by setting the variable
sysdeps-gconv-modules in sysdeps/.../Makefile.
The architecture specific gconv-modules file is emitted before the common one
because these modules aren't used in all possible conversions. E.g. the converting
from INTERNAL to UTF-16 used the common UTF-16.so module instead of UTF16_UTF32_Z9.so.
This way, the s390-Makefile does not need to override the recipe for gconv-modules
and no warning is emitted anymore.
Since we no longer support empty objpfx the conditional test in iconvdata/Makefile
is removed.
ChangeLog:
* iconvdata/Makefile ($(inst_gconvdir)/gconv-modules):
Install file from $(objpfx)gconv-modules.
($(objpfx)gconv-modules): Concatenate architecture specific file
in variable sysdeps-gconv-modules and gconv-modules in src dir.
* sysdeps/s390/gconv-modules: New file.
* sysdeps/s390/s390-64/Makefile: ($(inst_gconvdir)/gconv-modules):
Deleted.
($(objpfx)gconv-modules-s390): Deleted.
(sysdeps-gconv-modules): New variable.
Continuing fixes for ceil and floor functions not to raise the
"inexact" exception, this patch fixes the x86_64 SSE4.1 versions. The
roundss / roundsd instructions take an immediate operand that
determines the rounding mode and whether to raise "inexact"; this just
needs bit 3 set to disable "inexact", which this patch does.
Remark: we don't have an SSE4.1 version of trunc / truncf (using this
instruction with operand 11); I'd expect one to make sense, but of
course it should be benchmarked against the existing C code. I'll
file a bug in Bugzilla for the lack of such a version.
Tested for x86_64.
[BZ #15479]
* sysdeps/x86_64/fpu/multiarch/s_ceil.S (__ceil_sse41): Set bit 3
of immediate operand to rounding instruction.
* sysdeps/x86_64/fpu/multiarch/s_ceilf.S (__ceilf_sse41):
Likewise.
* sysdeps/x86_64/fpu/multiarch/s_floor.S (__floor_sse41):
Likewise.
* sysdeps/x86_64/fpu/multiarch/s_floorf.S (__floorf_sse41):
Likewise.
This file is partially generated. To make updates a little
simpler, I have moved the generated code into a partially
contained header to simplify regeneration.
gen-tst-strtod-round.c now takes two, mandatory arguments.
These arguments specify the input test data and the output
destination, respectively.
C99 and C11 allow but do not require ceil, floor, round and trunc to
raise the "inexact" exception for noninteger arguments. TS 18661-1
requires that this exception not be raised by these functions. This
aligns them with general IEEE semantics, where "inexact" is only
raised if the final step of rounding the infinite-precision result to
the result type is inexact; for these functions, the
infinite-precision integer result is always representable in the
result type, so "inexact" should never be raised.
The generic implementations of ceil, floor and round functions contain
code to force "inexact" to be raised. This patch removes it for round
functions to align them with TS 18661-1 in this regard. The tests
*are* updated by this patch; there are fewer architecture-specific
versions than for ceil and floor, and I fixed the powerpc ones some
time ago. If any others still have the issue, as shown by tests for
round failing with spurious exceptions, they can be fixed separately
by architecture maintainers or others.
Tested for x86_64, x86 and mips64.
[BZ #15479]
* sysdeps/ieee754/dbl-64/s_round.c (huge): Remove variable.
(__round): Do not force "inexact" exception.
* sysdeps/ieee754/dbl-64/wordsize-64/s_round.c (huge): Remove
variable.
(__round): Do not force "inexact" exception.
* sysdeps/ieee754/flt-32/s_roundf.c (huge): Remove variable.
(__roundf): Do not force "inexact" exception.
* sysdeps/ieee754/ldbl-128/s_roundl.c (huge): Remove variable.
(__roundl): Do not force "inexact" exception.
* sysdeps/ieee754/ldbl-96/s_roundl.c (huge): Remove variable.
(__roundl): Do not force "inexact" exception.
* math/libm-test.inc (round_test_data): Do not allow spurious
"inexact" exceptions.
C99 and C11 allow but do not require ceil, floor, round and trunc to
raise the "inexact" exception for noninteger arguments. TS 18661-1
requires that this exception not be raised by these functions. This
aligns them with general IEEE semantics, where "inexact" is only
raised if the final step of rounding the infinite-precision result to
the result type is inexact; for these functions, the
infinite-precision integer result is always representable in the
result type, so "inexact" should never be raised.
The generic implementations of ceil, floor and round functions contain
code to force "inexact" to be raised. This patch removes it for floor
functions to align them with TS 18661-1 in this regard. Note that
some architecture-specific versions may still raise "inexact", so the
tests are not updated and the bug is not yet fixed.
Tested for x86_64, x86 and mips64.
[BZ #15479]
* sysdeps/ieee754/dbl-64/s_floor.c: Do not mention "inexact"
exception in comment.
(huge): Remove variable.
(__floor): Do not force "inexact" exception.
* sysdeps/ieee754/dbl-64/wordsize-64/s_floor.c: Do not mention
"inexact" exception in comment.
(huge): Remove variable.
(__floor): Do not force "inexact" exception.
* sysdeps/ieee754/flt-32/s_floorf.c: Do not mention "inexact"
exception in comment.
(huge): Remove variable.
(__floorf): Do not force "inexact" exception.
* sysdeps/ieee754/ldbl-128/s_floorl.c: Do not mention "inexact"
exception in comment.
(huge): Remove variable.
(__floorl): Do not force "inexact" exception.
C99 and C11 allow but do not require ceil, floor, round and trunc to
raise the "inexact" exception for noninteger arguments. TS 18661-1
requires that this exception not be raised by these functions. This
aligns them with general IEEE semantics, where "inexact" is only
raised if the final step of rounding the infinite-precision result to
the result type is inexact; for these functions, the
infinite-precision integer result is always representable in the
result type, so "inexact" should never be raised.
The generic implementations of ceil, floor and round functions contain
code to force "inexact" to be raised. This patch removes it for ceil
functions to align them with TS 18661-1 in this regard. Note that
some architecture-specific versions may still raise "inexact", so the
tests are not updated and the bug is not yet fixed.
Tested for x86_64, x86 and mips64.
[BZ #15479]
* sysdeps/ieee754/dbl-64/s_ceil.c: Do not mention "inexact"
exception in comment.
(huge): Remove variable.
(__ceil): Do not force "inexact" exception.
* sysdeps/ieee754/dbl-64/wordsize-64/s_ceil.c: Do not mention
"inexact" exception in comment.
(huge): Remove variable.
(__ceil): Do not force "inexact" exception.
* sysdeps/ieee754/flt-32/s_ceilf.c (huge): Remove variable.
(__ceilf): Do not force "inexact" exception.
* sysdeps/ieee754/ldbl-128/s_ceill.c: Do not mention "inexact"
exception in comment.
(huge): Remove variable.
(__ceill): Do not force "inexact" exception.
When --enable-bind-now is used to configure glibc build, we can avoid
an extra branch to the PLT entry by using indirect branch via the GOT
slot instead, which is similar to the first instructuon in the PLT
entry. Changes in the shared library sizes in text sections:
Shared library Before (bytes) After (bytes)
libm.so 1060813 1060797
libmvec.so 160881 160805
libpthread.so 94992 94984
librt.so 25064 25048
* config.h.in (BIND_NOW): New.
* configure.ac (BIND_NOW): New. Defined for --enable-bind-now.
* configure: Regenerated.
* sysdeps/x86_64/sysdep.h (JUMPTARGET)[BIND_NOW]: Defined to
indirect branch via the GOT slot.
There exist optimized memcpy functions on s390, but no optimized mempcpy.
This patch adds mempcpy entry points in memcpy.S files, which
use the memcpy implementation. Now mempcpy itself is also an IFUNC function
as memcpy is and the variants are listed in ifunc-impl-list.c.
The s390 string.h does not define _HAVE_STRING_ARCH_mempcpy.
Instead mempcpy string/string.h inlines memcpy() + n.
If n is constant and small enough, GCC emits instructions like mvi or mvc
and avoids the function call to memcpy.
If n is not constant, then memcpy is called and n is added afterwards.
If _HAVE_STRING_ARCH_mempcpy would be defined, mempcpy would be called in
every case.
According to PR70140 "Inefficient expansion of __builtin_mempcpy"
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70140) GCC should handle a
call to mempcpy in the same way as memcpy. Then either the mempcpy macro
in string/string.h has to be removed or _HAVE_STRING_ARCH_mempcpy has to
be defined for S390.
ChangeLog:
[BZ #19765]
* sysdeps/s390/mempcpy.S: New File.
* sysdeps/s390/multiarch/mempcpy.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add mempcpy.
* sysdeps/s390/multiarch/ifunc-impl-list.c (__libc_ifunc_impl_list):
Add mempcpy variants.
* sysdeps/s390/s390-32/memcpy.S: Add mempcpy entry point.
(memcpy): Adjust to be usable from mempcpy entry point.
(__memcpy_mvcle): Likewise.
* sysdeps/s390/s390-64/memcpy.S: Likewise.
* sysdeps/s390/s390-32/multiarch/memcpy-s390.S: Add entry points
____mempcpy_z196, ____mempcpy_z10 and add __GI_ symbols for mempcpy.
(__memcpy_z196): Adjust to be usable from mempcpy entry point.
(__memcpy_z10): Likewise.
* sysdeps/s390/s390-64/multiarch/memcpy-s390x.S: Likewise.
On s390, the memcpy, memcmp, memset functions are IFUNC symbols,
which are created with s390_libc_ifunc-macro.
This macro creates a __GI_ symbol which is set to the
ifunced symbol. Thus calls within libc.so to e.g. memcpy
result in a call to *ABS*+0x954c0@plt stub and afterwards
to the resolved memcpy-ifunc-variant.
This patch sets the __GI_ symbol to the default-ifunc-variant
to avoid the plt call. The __GI_ symbols are now created at the
default variant of ifunced function.
ChangeLog:
* sysdeps/s390/multiarch/ifunc-resolve.h (s390_libc_ifunc):
Remove __GI_ symbol.
* sysdeps/s390/s390-32/multiarch/memcmp-s390.S: Add __GI_memcmp symbol.
* sysdeps/s390/s390-64/multiarch/memcmp-s390x.S: Likewise.
* sysdeps/s390/s390-32/multiarch/memcpy-s390.S: Add __GI_memcpy symbol.
* sysdeps/s390/s390-64/multiarch/memcpy-s390x.S: Likewise.
* sysdeps/s390/s390-32/multiarch/memset-s390.S: Add __GI_memset symbol.
* sysdeps/s390/s390-64/multiarch/memset-s390x.S: Likewise.
The __memcpy_default variant on s390 64bit calculates the number of
256byte blocks in a 64bit register and checks, if they exceed 1MB
to jump to mvcle. Otherwise a mvc-loop is used. The compare-instruction
only checks a 32bit value.
This patch uses a 64bit compare.
ChangeLog:
* sysdeps/s390/s390-64/memcpy.S (memcpy):
Use cghi instead of chi to compare 64bit value.
If more than 255 bytes should be copied, the algorithm jumps away.
Before this patch, it jumps to the mvc-loop (.L_G5_12).
Now it jumps first to the "> 1MB" check, which jumps away to
__memcpy_mvcle. Otherwise, the mvc-loop (.L_G5_12) copies the bytes.
ChangeLog:
* sysdeps/s390/s390-32/memcpy.S (memcpy):
Jump to 1MB check before executing mvc-loop.
After the heap rewriting added in commit
4cf6c72fd2 (malloc: Rewrite dumped heap
for compatibility in __malloc_set_state), we can change malloc alignment
for new allocations because the alignment of old allocations no longer
matters.
We need to increase the malloc state version number, so that binaries
containing dumped heaps of the new layout will not try to run on
previous versions of glibc, resulting in obscure crashes.
This commit addresses a failure of tst-malloc-thread-fail on the
affected architectures (32-bit ppc and mips) because the test checks
pointer alignment.
The call is technically in a loop, and under certain circumstances
(which are quite difficult to reproduce in a test case), alloca
can be invoked repeatedly during a single call to clntudp_call.
As a result, the available stack space can be exhausted (even
though individual alloca sizes are bounded implicitly by what
can fit into a UDP packet, as a side effect of the earlier
successful send operation).
This avoids aliasing issues with GCC 6 in -fno-strict-aliasing
mode. (With implicit padding, not all data is copied.)
This change makes it explicit that struct sockaddr_storage is
only 126 bytes large on m68k (unlike elsewhere, where we end up
with the requested 128 bytes). The new test case makes sure that
this does not happen on other architectures.
The conform/ test of limits.h namespace for XPG3 was failing because
of NL_* and NZERO defines. Those symbols are EX-shaded, not
UX-shaded, so it's correct for them to be defined for XPG3; this patch
corrects the expectations accordingly. (Actually it looks like they
should not be listed as optional for these standards, but that's
another matter.)
Tested for x86_64 and x86.
* conform/data/limits.h-data (CHARCLASS_NAME_MAX): Also expect for
[XPG3 || XPG4].
(NL_ARGMAX): Also expect for [XPG3].
(NL_LANGMAX): Likewise.
(NL_MSGMAX): Likewise.
(NL_NMAX): Likewise.
(NL_SETMAX): Likewise.
(NL_TEXTMAX): Likewise.
(NZERO): Likewise.
(TMP_MAX): Likewise.
* conform/Makefile (test-xfail-XPG3/limits.h/conform): Remove
variable.
This patch updates sysdeps/unix/sysv/linux/bits/socket.h for new
constants added in Linux 4.6. AF_KCM / PF_KCM are added. SOL_KCM is
new, and I added a lot of SOL_* values postdating the last one present
in the header, since I saw no apparent reason for the set in glibc to
stop at SOL_IRDA. MSG_BATCH is added; Linux also has
MSG_SENDPAGE_NOTLAST which is not in glibc, but given the comment
starts "sendpage() internal" I presume it's correct for it not to be
in glibc.
(Note that this is a case where the Linux kernel header with userspace
relevant values is *not* a uapi header but include/linux/socket.h - I
don't know why, but at least this header, as well as uapi headers,
needs reviewing for glibc-relevant changes each release.)
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
* sysdeps/unix/sysv/linux/bits/socket.h (PF_KCM): New macro.
(PF_MAX): Update value.
(AF_KCM): New macro.
(SOL_NETBEUI): Likewise.
(SOL_LLC): Likewise.
(SOL_DCCP): Likewise.
(SOL_NETLINK): Likewise.
(SOL_TIPC): Likewise.
(SOL_RXRPC): Likewise.
(SOL_PPPOL2TP): Likewise.
(SOL_BLUETOOTH): Likewise.
(SOL_PNPIPE): Likewise.
(SOL_RDS): Likewise.
(SOL_IUCV): Likewise.
(SOL_CAIF): Likewise.
(SOL_ALG): Likewise.
(SOL_NFC): Likewise.
(SOL_KCM): Likewise.
(MSG_BATCH): New enum value and macro.
L2 cache is shared by 2 cores on Knights Landing, which has 4 threads
per core:
https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing
So L2 cache is shared by 8 threads on Knights Landing as reported by
CPUID. We should remove special L2 cache case for Knights Landing.
[BZ #18185]
* sysdeps/x86/cacheinfo.c (init_cacheinfo): Don't limit threads
sharing L2 cache to 2 for Knights Landing.
This patch corrects various conformtest expectations in ftw.h for XPG3
and XPG4.
Tested for x86_64 and x86.
* conform/data/ftw.h-data (struct FTW): Do not expect for [XPG3].
(FTW_DP): Do not expect for [XPG3 || XPG4].
(FTW_SL): Do not expect for [XPG3].
(FTW_SLN): Likewise.
(FTW_PHYS): Likewise.
(FTW_MOUNT): Likewise.
(FTW_DEPTH): Likewise.
(FTW_CHDIR): Likewise.
(nftw): Likewise.
* conform/Makefile (test-xfail-XPG3/ftw.h/conform): Remove
variable.
ldbl-128ibm had an implementation of fmal that just did (x * y) + z in
most cases, with no attempt at actually being a fused operation.
This patch replaces it with a genuine fused operation. It is not
necessarily correctly rounding, but should produce a result at least
as accurate as the long double arithmetic operations in libgcc, which
I think is all that can reasonably be expected for such a non-IEEE
format where arithmetic is approximate rather than rounded according
to any particular rule for determining the exact result. Like the
libgcc arithmetic, it may produce spurious overflow and underflow
results, and it falls back to the libgcc multiplication in the case of
(finite, finite, zero).
This concludes the fixes for bug 13304; any subsequently found fma
issues should go in separate Bugzilla bugs. Various other pieces of
bug 13304 were fixed in past releases over the past several years.
Tested for powerpc.
[BZ #13304]
* sysdeps/ieee754/ldbl-128ibm/s_fmal.c: Include <fenv.h>,
<float.h>, <math_private.h> and <stdlib.h>.
(add_split): New function.
(mul_split): Likewise.
(ext_val): New typedef.
(store_ext_val): New function.
(mul_ext_val): New function.
(compare): New function.
(add_split_ext): New function.
(__fmal): After checking for Inf, NaN and zero, compute result as
an exact sum of scaled double values in round-to-nearest before
adding those up and adjusting for other rounding modes.
* math/auto-libm-test-in: Remove xfail-rounding:ldbl-128ibm from
tests of fma.
* math/auto-libm-test-out: Regenerated.
Intel CPUID with EAX == 11 returns:
ECX Bits 07 - 00: Level number. Same value in ECX input.
Bits 15 - 08: Level type.
^^^^^^^^^^^^^^^^^^^^^^^^ This is level type.
Bits 31 - 16: Reserved.
Intel processor level type mask should be 0xff00, not 0xff0.
[BZ #20119]
* sysdeps/x86/cacheinfo.c (init_cacheinfo): Correct Intel
processor level type mask for CPUID with EAX == 11.
Skip counting logical threads for Intel processors if the HTT bit is 0
which indicates there is only a single logical processor.
* sysdeps/x86/cacheinfo.c (init_cacheinfo): Skip counting
logical threads if the HTT bit is 0.
* sysdeps/x86/cpu-features.h (bit_cpu_HTT): New.
(index_cpu_HTT): Likewise.
(reg_HTT): Likewise.
X86-64 memset-vec-unaligned-erms.S aligns many jump targets, which
increases code sizes, but not necessarily improve performance. As
memset benchtest data of align vs no align on various Intel and AMD
processors
https://sourceware.org/bugzilla/attachment.cgi?id=9277
shows that aligning jump targets isn't necessary.
[BZ #20115]
* sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S (__memset):
Remove alignments on jump targets.
This patch corrects some conformtest expectations for pwd.h for XPG4.
Tested for x86_64 and x86.
* conform/data/pwd.h-data (endpwent): Do not expect for [XPG3].
(getpwent): Likewise.
(setpwent): Likewise.
* conform/Makefile (test-xfail-XPG3/pwd.h/conform): Remove
variable.
This patch corrects some conformtest expectations for search.h for
XPG3.
Tested for x86_64 and x86.
* conform/data/search.h-data (insque): Do not expect for [XPG3].
(remque): Likewise.
* conform/Makefile (test-xfail-XPG3/search.h/conform): Remove
variable.
libm-test.inc has a comment about signs of NaNs not being tested.
This is out of date since:
commit ce66581742
Author: Joseph Myers <joseph@codesourcery.com>
Date: Sat Nov 16 12:48:35 2013 +0000
Test signs of NaNs in libm-test.inc where appropriate.
This patch removes the inaccurate statement.
* math/libm-test.inc: Update comment not to refer to signs of NaNs
not being tested.
There is no need to call the internal funtion, _Unwind_Resume, which
is defined in unwind-forcedunwind.c, via PLT.
* sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S
(__condvar_cleanup2): Remove JUMPTARGET from _Unwind_Resume
call.
* sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S
(__condvar_cleanup1): Likewise.
Add PTHREAD_UNWIND to replace JUMPTARGET(__pthread_unwind) and define
it to __GI___pthread_unwind within libpthread.
* sysdeps/unix/sysv/linux/x86_64/cancellation.S (PTHREAD_UNWIND):
New
(__pthread_unwind): Renamed to ...
(PTHREAD_UNWIND): This.
(__pthread_enable_asynccancel): Replace
JUMPTARGET(__pthread_unwind) with PTHREAD_UNWIND.
This patch adds CLONE_NEWCGROUP, new in Linux 4.6, to
sysdeps/unix/sysv/linux/bits/sched.h.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
* sysdeps/unix/sysv/linux/bits/sched.h [__USE_GNU]
(CLONE_NEWCGROUP): New macro.
This patch adds Q_GETNEXTQUOTA, new in Linux 4.6, to
sysdeps/unix/sysv/linux/sys/quota.h.
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
* sysdeps/unix/sysv/linux/sys/quota.h [_LINUX_QUOTA_VERSION >= 2]
(Q_GETNEXTQUOTA): New macro.
Clear the destination buffer updated by the previous run in bench-memcpy.c
and test-memcpy.c to catch the error when the following implementations do
not copy anything.
[BZ #19907]
* benchtests/bench-memcpy.c (do_one_test): Clear the destination
buffer updated by the previous run.
* string/test-memcpy.c (do_one_test): Likewise.
* benchtests/bench-memmove.c (do_one_test): Add a comment.
* string/test-memmove.c (do_one_test): Likewise.
stdlib.h declares grantpt, ptsname, unlockpt for __USE_XOPEN. This
patch corrects the condition to __USE_XOPEN_EXTENDED (these functions
are new in XPG4).
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
[BZ #20094]
* stdlib/stdlib.h (grantpt): Declare if [__USE_XOPEN_EXTENDED],
not [__USE_XOPEN].
(unlockpt): Likewise.
(ptsname): Likewise.
* conform/Makefile (test-xfail-XPG3/stdlib.h/conform): Remove
variable.
sys/stat.h should define S_IFSOCK and S_ISSOCK for XPG4 (XNS), but
does not. This patch corrects the relevant header conditionals.
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
[BZ #20076]
* io/sys/stat.h (S_IFSOCK): Define for [__USE_XOPEN_EXTENDED]
instead of [__USE_UNIX98].
(S_ISSOCK): Likewise.
* conform/Makefile (test-xfail-XPG4/sys/stat.h/conform): Remove
variable.
stdlib.h declares rand_r if __USE_POSIX; i.e., POSIX.1:1990. But
rand_r was added along with threads, so the condition should be
__USE_POSIX199506. This patch corrects the condition.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20074]
* stdlib/stdlib.h (rand_r): Declare if [__USE_POSIX199506], not if
[__USE_POSIX].
* conform/Makefile (test-xfail-XPG4/stdlib.h/conform): Remove
variable.
UNIX98 and XPG4 have ttyslot in <stdlib.h>. glibc, however, has it in
<unistd.h>, for __USE_MISC || (__USE_XOPEN_EXTENDED && !__USE_UNIX98),
but no supported standard has it in <unistd.h>.
This patch adds a properly conditioned declaration to <stdlib.h> (only
enabled for the relevant standards, not for __USE_MISC or __USE_GNU).
The <unistd.h> declaration is restricted to __USE_MISC. Some relevant
XFAILs are removed.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20051]
* posix/unistd.h [__USE_XOPEN_EXTENDED && !__USE_UNIX98]
(ttyslot): Do not declare.
* stdlib/stdlib.h [__USE_XOPEN_EXTENDED && !__USE_XOPEN2K]
(ttyslot): New prototype.
* conform/Makefile (test-xfail-XPG4/unistd.h/conform): Remove
variable.
(test-xfail-UNIX98/stdlib.h/conform): Likewise.
The testcase tst-cancel[x]17 ends sometimes with a segmentation fault.
This happens in one of 10000 cases. Then the real testcase has already
exited with success and returned from do_test(). The segmentation fault
occurs after returning from main in _dl_fini().
In those cases, the aio_read(&a) was not canceled because the read
request was already in progress. In the meanwhile aio_write(ap) wrote
something to the pipe and the read request is able to read the
requested byte.
The read request hasn't finished before returning from do_test().
After it finishes, it writes the return value and error code from the
read syscall to the struct aiocb a, which lies on the stack of do_test.
The stack of the subsequent function call of _dl_fini or _dl_sort_fini,
which is inlined in _dl_fini is corrupted.
In case of S390, it reads a zero and decrements it by 1:
unsigned int k = nmaps - 1;
struct link_map **runp = maps[k]->l_initfini;
The load from unmapped memory leads to the segmentation fault.
The stack corruption also happens on other architectures.
I saw them e.g. on x86 and ppc, too.
This patch adds an aio_suspend call to ensure, that the read request
is finished before returning from do_test().
ChangeLog:
* nptl/tst-cancel17.c (do_test): Wait for finishing aio_read(&a).
The first SIGUSR1 signal could arrive when sigusr1_sender_pid
was still 0. As a result, kill would send SIGSTOP to the
entire process group. This would cause the test to hang before
printing any output.
This commit also adds a sched_yield to the signal source, so that
it does not flood the parent process with signals it has never a
chance to handle.
Even with these changes, tst-mallocfork2 still fails reliably
after the fix in commit commit 56290d6e76
(Increase fork signal safety for single-threaded processes) is
backed out.
This patch corrects some spurious conformtest stdlib.h expectations
for XPG3 (not based on a full review of the expectations for that
standard, so other issues may remain).
Tested for x86_64 and x86.
* conform/data/stdlib.h-data (a64l): Do not expect for [XPG3].
(ecvt): Likewise.
(fcvt): Likewise.
(gcvt): Likewise.
(getsubopt): Likewise.
(grantpt): Likewise.
(initstate): Likewise.
(l64a): Likewise.
(mktemp): Likewise.
(mkstemp): Likewise.
(ptsname): Likewise.
(random): Likewise.
(realpath): Likewise.
(setstate): Likewise.
(srandom): Likewise.
(ttyslot): Likewise.
(unlockpt): Likewise.
(valloc): Likewise.
This patch corrects a spurious conformtest expectation of strdup in
string.h for XPG3.
Tested for x86_64 and x86.
* conform/data/string.h-data (strdup): Do not expect for [XPG3].
* conform/Makefile (test-xfail-XPG3/string.h/conform): Remove
variable.
In static executable, since init_cpu_features is called early from
__libc_start_main, there is no need to call it again in dl_platform_init.
[BZ #20072]
* sysdeps/i386/dl-machine.h (dl_platform_init): Call
init_cpu_features only if SHARED is defined.
* sysdeps/x86_64/dl-machine.h (dl_platform_init): Likewise.
This will allow us to change many aspects of the malloc implementation
while preserving compatibility with existing Emacs binaries.
As a result, existing Emacs binaries will have a larger RSS, and Emacs
needs a few more milliseconds to start. This overhead is specific
to Emacs (and will go away once Emacs switches to its internal malloc).
The new checks to make free and realloc compatible with the dumped heap
are confined to the mmap paths, which are already quite slow due to the
munmap overhead.
This commit weakens some security checks, but only for heap pointers
in the dumped main arena. By default, this area is empty, so those
checks are as effective as before.
This patch corrects spurious conformtest expectations of getdate and
getdate_err for XPG3. (This is not based on a full review of the
expectations, so there may be other issues where the header and tests
agree but are both incorrect.)
Tested for x86_64 and x86.
* conform/data/time.h-data (getdate_err): Do not expect for
[XPG3].
(getdate): Likewise.
* conform/Makefile (test-xfail-XPG3/time.h/conform): Remove
variable.
The conformtest tests of unistd.h fail for XPG3 because of various
expectations that are incorrect for XPG3. This patch corrects those
bogus expectations, and one bogus expectation for XPG4. (This is not
based on a full review of the standards so there may well still be
other bugs in the expectations for this header.)
Tested for x86_64 and x86.
* conform/data/unistd.h-data (F_LOCK): Do not expect for [XPG3].
(F_ULOCK): Likewise.
(F_TEST): Likewise.
(F_TLOCK): Likewise.
(useconds_t): Likewise.
(intptr_t): Do not expect for [XPG3] or [XPG4].
(brk): Do not expect for [XPG3]
(fchown): Likewise.
(fchdir): Likewise.
(ftruncate): Likewise.
(getdtablesize): Likewise.
(gethostid): Likewise.
(getpagesize): Likewise.
(getpgid): Likewise.
(getsid): Likewise.
(getwd): Likewise.
(lchown): Likewise.
(lockf): Likewise.
(readlink): Likewise.
(sbrk): Likewise.
(setpgrp): Likewise.
(setregid): Likewise.
(setreuid): Likewise.
(symlink): Likewise.
(sync): Likewise.
(truncate): Likewise.
(ualarm): Likewise.
(usleep): Likewise.
(vfork): Likewise.
* conform/Makefile (test-xfail-XPG3/unistd.h/conform): Remove
variable.
unistd.h declares gethostname for __USE_UNIX98 || __USE_XOPEN2K. But
it's also in XPG4 (XNS volume - C438 - not the main definitions of
system interfaces and headers in C435). This patch corrects the
condition.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20054]
* posix/unistd.h (gethostname): Declare if [__USE_XOPEN_EXTENDED],
not [__USE_UNIX98].
* conform/data/unistd.h-data (gethostname): Do not expect for
[XPG3].
The conform/ tests for fcntl.h are failing for XPG3 and XPG4 because
of missing S_IFSOCK.
This is a case of a bogus test. The relevant wording requiring such
constants is, in current POSIX (and this requirement dates back as far
as XPG4), "The <fcntl.h> header shall define the symbolic constants
for file modes for use as values of mode_t as described in
<sys/stat.h>.". Note that this is *file modes* not *file types*.
That makes sense, since the point is presumably for use with functions
such as open that are declared in fcntl.h, where file modes are
relevant but file types aren't. So this patch removes all those
spurious S_IF* expectations for fcntl.h (the macros are generally
still *allowed* through the permission to make everything from
sys/stat.h visible).
Tested for x86_64 and x86.
* conform/data/fcntl.h-data [!POSIX] (S_IFMT): Do not expect.
[!POSIX] (S_IFBLK): Likewise.
[!POSIX] (S_IFCHR): Likewise.
[!POSIX] (S_IFIFO): Likewise.
[!POSIX] (S_IFREG): Likewise.
[!POSIX] (S_IFDIR): Likewise.
[!POSIX] (S_IFLNK): Likewise.
[!POSIX] (S_IFSOCK): Likewise.
* conform/Makefile (test-xfail-XPG3/fcntl.h/conform): Remove
variable.
(test-xfail-XPG4/fcntl.h/conform): Likewise.
small sets of up to 16 bytes, medium of 16..96 bytes which are fully unrolled.
Large memsets of more than 96 bytes align the destination and use an unrolled
loop processing 64 bytes per iteration. Memsets of zero of more than 256 use
the dc zva instruction, and there are faster versions for the common ZVA sizes
64 or 128. STP of Q registers is used to reduce codesize without loss of
performance.
The speedup on test-memset is 1% on Cortex-A57 and 8% on Cortex-A53.
* sysdeps/aarch64/memset.S (__memset):
Rewrite of optimized memset.
This provides a band-aid and addresses the scenario where fork is
called from a signal handler while the process is in the malloc
subsystem (or has acquired the libio list lock). It does not
address the general issue of async-signal-safety of fork;
multi-threaded processes are not covered, and some glibc
subsystems have fork handlers which are not async-signal-safe.
termios.h should declare tcgetsid for XPG4, but only does so for
__USE_UNIX98 || __USE_XOPEN2K8 at present. This patch fixes the
declaration conditions. A spurious conformtest expectation of this
declaration for XPG3 is removed, and two XFAILs that are fixed by
these changes are also removed.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20055]
* termios/termios.h (pid_t): Define for [__USE_XOPEN_EXTENDED]
instead of [__USE_UNIX98].
(tcgetsid): Declare for [__USE_XOPEN_EXTENDED] instead of
[__USE_UNIX98].
* conform/data/termios.h-data (tcgetsid): Do not expect for
[XPG3].
* conform/Makefile (test-xfail-XPG3/termios.h/conform): Remove
variable.
(test-xfail-XPG4/termios.h/conform): Likewise.
if glibc is build with -march=z900 | -march=z990,
the startup file gcrt1.o (used if you link with gcc -pg)
contains R_390_GOT12 | R_390_GOT20 relocations.
Thus, an entry in the GOT can be addressed relative to the GOT pointer
with a 12 | 20 bit displacement value.
The startup files should not contain R_390_GOT12,
R_390_GOT20 relocations, but R_390_GOTENT ones.
This patch removes the overrides of pic-ccflag and
the default pic-ccflag = -fPIC in Makeconfig
is used instead to get the R_390_GOTENT relocations in gcrt1.o.
ChangeLog:
* sysdeps/s390/s390-32/Makefile (pic-ccflag): Remove.
* sysdeps/s390/s390-64/Makefile: Likewise.
Merge x86 ifunc-defines.sym with x86 cpu-features-offsets.sym. Remove
x86 ifunc-defines.sym and rtld-global-offsets.sym. No code changes on
i686 and x86-64.
* sysdeps/i386/i686/multiarch/Makefile (gen-as-const-headers):
Remove ifunc-defines.sym.
* sysdeps/x86_64/multiarch/Makefile (gen-as-const-headers):
Likewise.
* sysdeps/i386/i686/multiarch/ifunc-defines.sym: Removed.
* sysdeps/x86/rtld-global-offsets.sym: Likewise.
* sysdeps/x86_64/multiarch/ifunc-defines.sym: Likewise.
* sysdeps/x86/Makefile (gen-as-const-headers): Remove
rtld-global-offsets.sym.
* sysdeps/x86_64/multiarch/ifunc-defines.sym: Merged with ...
* sysdeps/x86/cpu-features-offsets.sym: This.
* sysdeps/x86/cpu-features.h: Include <cpu-features-offsets.h>
instead of <ifunc-defines.h> and <rtld-global-offsets.h>.
sys/stat.h declares fchmod if __USE_POSIX (i.e. POSIX.1:1990). But it
was actually added in 1993 and also in XPG4. This patch changes the
conditions to the correct __USE_POSIX199309 || __USE_XOPEN_EXTENDED.
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
[BZ #20073]
* io/sys/stat.h (fchmod): Declare for
[__USE_POSIX199309 || __USE_XOPEN_EXTENDED], not [__USE_POSIX].
* conform/Makefile (test-xfail-XPG3/sys/stat.h/conform): Remove
variable.
This patch fixes various conformtest sys/stat.h expectations that were
incorrect for XPG3 (not based on a full review, so not necessarily an
exhaustive set of such corrections). Most of these corrections fix
spurious failures shown in testing (but that for fchmod introduces a
correct failure, as that function is wrongly declared for XPG3, so
this doesn't eliminate any XFAILs).
Tested for x86_64 and x86.
* conform/data/sys/stat.h-data [XPG3] (S_IFLNK): Do not expect.
[XPG3] (S_IFSOCK): Likewise.
[XPG3] (S_ISVTX): Likewise.
[XPG3] (S_ISLNK): Likewise.
[XPG3] (S_ISSOCK): Likewise.
[XPG3] (fchmod): Likewise.
[XPG3] (lstat): Likewise.
[XPG3] (mknod): Likewise.
The conformtest expectations expect the struct stat fields st_blksize
and st_blocks to be of types blksize_t and blkcnt_t. But XPG4 does
not have those types, using long instead, and XPG3 does not have these
fields at all. This patch adjusts the expectations for those
standards, XFAILing them for XPG4 to allow for systems where the
typedefs don't correspond to long.
Tested for x86_64 and x86.
* conform/data/sys/stat.h-data (st_blksize): Do not expect for
[XPG3]. Expect type long and XFAIL for [XPG4].
(st_blocks): Likewise.
For UNIX98 (only), unistd.h should declare pthread_atfork, but that
declaration is missing. This patch adds it.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20044]
* posix/unistd.h [__USE_UNIX98 && !__USE_XOPEN2K]
(pthread_atfork): New prototype.
* conform/Makefile (test-xfail-UNIX98/unistd.h/conform): Remove
variable.
For UNIX98 and older X/Open standards, unistd.h should have a
declaration of the legacy cuserid function, but such a declaration is
missing. This patch adds that missing declaration.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20043]
* posix/unistd.h [__USE_XOPEN && !__USE_XOPEN2K] (cuserid): New
prototype.
The conformtest tests for sys/types.h for XPG3 and XPG4 fail because
of missing blksize_t. This is a bug in the expectations; that type is
not part of those standards. This patch stops the tests from
expecting it, and some other types that also are not part of XPG3 and
XPG4.
Tested for x86_64 and x86.
* conform/data/sys/types.h-data (blkcnt_t): Do not expect for
[XPG3 || XPG4].
(blksize_t): Likewise.
(clockid_t): Likewise.
* conform/Makefile (test-xfail-XPG3/sys/types.h/conform): Remove
variable.
(test-xfail-XPG4/sys/types.h/conform): Likewise.
This patch adds support for symbol __kernel_getcpu in vDSO,
which is available with kernel 4.5.
Now sched_getcpu is using this symbol if available in mapped vDSO
by defining macro HAVE_GETCPU_VSYSCALL. If not available at runtime,
the former syscall is used.
Move sysdeps/x86_64/cacheinfo.c to sysdeps/x86. No code changes on x86
and x86_64.
* sysdeps/i386/cacheinfo.c: Include <sysdeps/x86/cacheinfo.c>
instead of <sysdeps/x86_64/cacheinfo.c>.
* sysdeps/x86_64/cacheinfo.c: Moved to ...
* sysdeps/x86/cacheinfo.c: Here.
* sysdeps/pthread/aio_misc.c (__aio_enqueue_request): Do not write
`running` field of `newp` when a thread was started to process it,
since that thread will not take `__aio_requests_mutex`, and the field
already has the proper value actually.
The AF_LOCAL and AF_INET/AF_INET6 non-numerci service conversion
did not return EAI_OVERFLOW if the supplied buffer was too small,
silently returning truncated data. In the AF_INET/AF_INET6
numeric cases, the snprintf return value checking was incorrect.
In the numeric AF_INET/AF_INET6 case, if inet_ntop fails
as the result of a short host buffer, we used to call strnlen
on the uninitialized host buffer.
The file sysdeps/powerpc/sysdeps.h defines aliases for condition register
operands. E.g.: 'cr7' means condition register 7. On the one hand, this
increases readability, as it makes it easier for readers to know whether the
operand is a condition register, a general purpose register or an immediate.
On the other hand, this permits that condition registers be written as if they
were general purpose, and vice-versa, thus reducing the readability of the
code.
This commit removes some of these unintentional misuses.
The changes have no effect on the final code. Checked with objdump.
Instead, we store the data we need from the return value of
readdir in an object of the new type struct readdir_result.
This type is independent of the layout of struct dirent.
For UNIX98 and older standards, sys/time.h should not define struct
timespec, but does so via the inclusion of sys/select.h (which is a
new header in the 2001 edition of POSIX, and defines struct timespec
because of the declaration of pselect, a new function in the 2001
edition of POSIX). In turn, this affects some other headers that
themselves include sys/time.h.
This patch fixes this by conditioning the __need_timespec definition
in sys/select.h on __USE_XOPEN2K, the same condition used there for
the declaration of pselect (this has no effect on direct uses of
sys/select.h with feature test macros for any standard that includes
that header, since such standards result in __USE_XOPEN2K being
defined).
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20041]
* misc/sys/select.h (__need_timespec): Only define if
[__USE_XOPEN2K].
* conform/Makefile (test-xfail-XPG4/sys/time.h/conform): Remove
variable.
(test-xfail-XPG4/utmpx.h/conform): Likewise.
(test-xfail-UNIX98/sys/time.h/conform): Likewise.
(test-xfail-UNIX98/utmpx.h/conform): Likewise.
fcntl.h defines struct timespec if __USE_XOPEN || __USE_XOPEN2K8. But
(a) the subsequent bits/stat.h include only needs it if __USE_XOPEN2K8
and (b) older standards did not allow struct timespec here. (It's
allowed for newer standards by virtue of the permission to include
symbols from sys/stat.h. But sys/stat.h is only required to provide
struct timespec from the 2008 edition of POSIX onwards, and permitted
by the 2004 TC to the 2001 edition in anticipation of the addition of
nanosecond timestamp support to struct stat in the 2008 edition.)
This patch limits the timespec definition to the __USE_XOPEN2K8 case,
that being the only case where it is actually needed for the
<bits/stat.h> include.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20023]
* io/fcntl.h [__USE_XOPEN && !__USE_XOPEN2K8]: Do not include
<time.h>.
* conform/Makefile (test-xfail-UNIX98/fcntl.h/conform): Remove
variable.
(test-xfail-XOPEN2K/fcntl.h/conform): Likewise.
This patch fixes the clone CLONE_VM change from 0cb313f (BZ#19957)
where the commit changed the register that contains the save flags
argument to compare with (from r28 to r29). This patch changes
back to correct register.
Tested on powerpc32 (thanks to Tulio Magno Quites Machado Filho).
* sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S (__clone): Fix
flags CLONE_VM compare.
The Hesiod implementation imported into glibc was enhanced
to support caller-supplied resolver states. But its only
consumer is nss_hesiod, and it supplies the thread-local
resolver state. Therefore, this commit changes the Hesiod
implementation to use the thread-local resolver state (_res)
directly. This fixes bug 19573 because the Hesiod
implementation no longer has to initialize and free any
resolver state.
To avoid any risk of interposition of ABI-incompatible Hesiod
function implementations, this commit marks the Hesiod functions
as hidden. (They were already hidden using a linker version
script.)
https://sourceware.org/glibc/wiki/Proposals/GroupMerging
== Justification ==
It is common today for users to rely on centrally-managed user stores for
handling their user accounts. However, much software existing today does
not have an innate understanding of such accounts. Instead, they commonly
rely on membership in known groups for managing access-control (for
example the "wheel" group on Fedora and RHEL systems or the "adm" group
on Debian-derived systems). In the present incarnation of nsswitch, the
only way to have such groups managed by a remote user store such as
FreeIPA or Active Directory would be to manually remove the groups from
/etc/group on the clients so that nsswitch would then move past nss_files
and into the SSSD, nss-ldap or other remote user database.
== Solution ==
With this patch, a new action is introduced for nsswitch:
NSS_ACTION_MERGE. To take advantage of it, one will add [SUCCESS=merge]
between two database entries in the nsswitch.conf file. When a group is
located in the first of the two group entries, processing will continue
on to the next one. If the group is also found in the next entry (and the
group name and GID are an exact match), the member list of the second
entry will be added to the group object to be returned.
== Implementation ==
After each DL_LOOKUP_FN() returns, the next action is checked. If the
function returned NSS_STATUS_SUCCESS and the next action is
NSS_ACTION_MERGE, a copy of the result buffer is saved for the next pass
through the loop. If on this next pass through the loop the database
returns another instance of a group matching both the group name and GID,
the member list is added to the previous list and it is returned as a
single object. If the following database does not contain the same group,
then the original is copied back into the destination buffer.
This patch implements merge functionality only for the group database.
For other databases, there is a default implementation that will return
the EINVAL errno if a merge is requested. The merge functionality can be
implemented for other databases at a later time if such is needed. Each
database must provide a unique implementation of the deep-copy and merge
functions.
If [SUCCESS=merge] is present in nsswitch.conf for a glibc version that
does not support it, glibc will process results up until that operation,
at which time it will return results if it has found them or else will
simply return an error. In practical terms, this ends up behaving like
the remainder of the nsswitch.conf line does not exist.
== Iterators ==
This feature does not modify the iterator functionality from its current
behavior. If getgrnam() or getgrgid() is called, glibc will iterate
through all entries in the `group` line in nsswitch.conf and display the
list of members without attempting to merge them. This is consistent with
the behavior of nss_files where if two separate lines are specified for
the same group in /etc/groups, getgrnam()/getgrgid() will display both.
Clients are already expected to handle this gracefully.
== No Premature Optimizations ==
The following is a list of places that might be eligible for
optimization, but were not overengineered for this initial contribution:
* Any situation where a merge may occur will result in one malloc() of
the same size as the input buffer.
* Any situation where a merge does occur will result in a second
malloc() to hold the list of pointers to member name strings.
* The list of members is simply concatenated together and is not tested
for uniqueness (which is identical to the behavior for nss_files,
which will simply return identical values if they both exist on the
line in the file. This could potentially be optimized to reduce space
usage in the buffer, but it is both complex and computationally
expensive to do so.
== Testing ==
I performed testing by running the getent utility against my newly-built
glibc and configuring /etc/nsswitch.conf with the following entry:
group: group: files [SUCCESS=merge] sss
In /etc/group I included the line:
wheel❌10:sgallagh
I then configured my local SSSD using the id_provider=local to respond
with:
wheel:*:10:localuser,localuser2
I then ran `getent group wheel` against the newly-built glibc in
multiple situations and received the expected output as described
above:
* When SSSD was running.
* When SSSD was configured in nsswitch.conf but the daemon was not
running.
* When SSSD was configured in nsswitch.conf but nss_sss.so.2 was not
installed on the system.
* When the order of 'sss' and 'files' was reversed.
* All of the above with the [SUCCESS=merge] removed (to ensure no
regressions).
* All of the above with `getent group 10`.
* All of the above with `getent group` with and without
`enumerate=true` set in SSSD.
* All of the above with and without nscd enabled on the system.
The fmemopen implementation does not account the file position correctly in
append mode. The following example shows the failure:
===
int main ()
{
char buf[10] = "test";
FILE *fp = fmemopen (buf, 10, "a+");
fseek (fp, 0, SEEK_SET);
int gr;
if ((gr = getc (fp)) != 't' ||
(gr = getc (fp)) != 'e' ||
(gr = getc (fp)) != 's' ||
(gr = getc (fp)) != 't' ||
(gr = getc (fp)) != EOF)
{
printf ("%s: getc failed returned %i\n", __FUNCTION__, gr);
return 1;
}
return 0;
}
===
This is due both how read and write operation update the buffer position,
taking in consideration buffer lenght instead of maximum position defined
by the open mode. This patch fixes it and also fixes fseek not returning
EINVAL for invalid whence modes.
Tested on x86_64 and i686.
[BZ #20012]
* libio/fmemopen.c (fmemopen_read): Use buffer maximum position, not
length to calculate the buffer to read.
(fmemopen_write): Set the buffer position based on bytes written.
(fmemopen_seek): Return EINVAL for invalid whence modes.
As discussed in libc-alpha [1] current clone with CLONE_VM (without
CLONE_THREAD set) will reset the pthread pid/tid fields to -1. The
issue is since memory is shared between the parent and child it will
clobber parent's cached pid/tid leading to internal inconsistencies
if the value is not restored.
And even it is restored it may lead to racy conditions when between
set/restore a thread might invoke pthread function that validate the
pthread with INVALID_TD_P/INVALID_NOT_TERMINATED_TD_P and thus get
wrong results.
As stated in BZ19957, previously reports of this behaviour was close
with EWONTFIX due the fact usage of clone outside glibc is tricky
since glibc requires consistent internal pthread, while using clone
directly may not provide it. However since now posix_spawn uses
clone (CLONE_VM) to fixes various issues related to previous vfork
usage this issue requires fixing.
The vfork implementation also does something similar, but instead
it negates and restores only the *pid* field and functions that
might access its value know to handle such case (getpid, raise
and pthread ones that uses INVALID_TD_P/INVALID_NOT_TERMINATED_TD_P
macros that check only *tid* field). Also vfork does not call
__clone directly, instead calling either __NR_vfork or __NR_clone
directly.
So this patch removes this clone behavior by avoiding setting
the pthread pid/tid field for CLONE_VM. There is no need to
check for CLONE_THREAD, since the minimum supported kernel in all
architecture implies that CLONE_VM must be used with CLONE_THREAD,
otherwise clone returns EINVAL.
Instead of current approach of:
int clone(int (*fn)(void *), void *child_stack, int flags, ...)
[...]
if (flags & CLONE_THREAD)
goto do_syscall;
pid_t new_value;
if (flags & CLONE_VM)
new_value = -1;
else
new_value = getpid ();
THREAD_SETMEM (THREAD_SELF, pid, new_value);
THREAD_SETMEM (THREAD_SELF, tid, new_value);
do_syscall:
[...]
The new approach uses:
int clone(int (*fn)(void *), void *child_stack, int flags, ...)
[...]
if (flags & CLONE_VM)
goto do_syscall;
pid_t new_value = getpid ();
THREAD_SETMEM (THREAD_SELF, pid, new_value);
THREAD_SETMEM (THREAD_SELF, tid, new_value);
do_syscall:
[...]
It also removes the linux tst-getpid2.c test which expects the previous
behavior and instead add another clone test.
Tested on x86_64, i686, x32, powerpc64le, aarch64, armhf, s390, and
s390x. I also did limited check on mips32 and sparc64 (using the new
added test).
I also got reviews from both m68k, hppa, and tile. So I presume for
these architecture the patch works.
The fixes for alpha, microblaze, sh, ia64, and nio2 have not been
tested.
[1] https://sourceware.org/ml/libc-alpha/2016-04/msg00307.html
* sysdeps/unix/sysv/linux/Makefile [$(subdir) == nptl] (test): Remove
tst-getpid2.
(test): Add tst-clone2.
* sysdeps/unix/sysv/linux/tst-clone2.c: New file.
* sysdeps/unix/sysv/linux/aarch64/clone.S (__clone): Do not change
pid/tid fields for CLONE_VM.
* sysdeps/unix/sysv/linux/arm/clone.S: Likewise.
* sysdeps/unix/sysv/linux/i386/clone.S: Likewise.
* sysdeps/unix/sysv/linux/mips/clone.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/clone.S: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/clone.S: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/x86_64/clone.S: Likewise.
* sysdeps/unix/sysv/linux/tst-getpid2.c: Remove file.
Stefan Liebler
Joseph Myers
Paul E. Murphy
H.J. Lu
Florian Weimer
ricaljasan@pacific.net
Wilco Dijkstra
Samuel Thibault
Gabriel F. T. Gomes
Adhemerval Zanella
Arnas Udovičius
Andreas Schwab
Stephen Gallagher