Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
#!/usr/bin/python3
|
|
|
|
# Produce headers of assembly constants from C expressions.
|
|
|
|
# Copyright (C) 2018 Free Software Foundation, Inc.
|
|
|
|
# This file is part of the GNU C Library.
|
|
|
|
#
|
|
|
|
# The GNU C Library is free software; you can redistribute it and/or
|
|
|
|
# modify it under the terms of the GNU Lesser General Public
|
|
|
|
# License as published by the Free Software Foundation; either
|
|
|
|
# version 2.1 of the License, or (at your option) any later version.
|
|
|
|
#
|
|
|
|
# The GNU C Library is distributed in the hope that it will be useful,
|
|
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
# Lesser General Public License for more details.
|
|
|
|
#
|
|
|
|
# You should have received a copy of the GNU Lesser General Public
|
|
|
|
# License along with the GNU C Library; if not, see
|
|
|
|
# <http://www.gnu.org/licenses/>.
|
|
|
|
|
|
|
|
# The input to this script looks like:
|
|
|
|
# #cpp-directive ...
|
|
|
|
# NAME1
|
|
|
|
# NAME2 expression ...
|
|
|
|
# A line giving just a name implies an expression consisting of just that name.
|
|
|
|
|
|
|
|
import argparse
|
|
|
|
|
Move tst-signal-numbers to Python.
This patch converts the tst-signal-numbers test from shell + awk to
Python.
As with gen-as-const, the point is not so much that shell and awk are
problematic for this code, as that it's useful to build up general
infrastructure in Python for use of a range of code involving
extracting values from C headers. This patch moves some code from
gen-as-const.py to a new glibcextract.py, which also gains functions
relating to listing macros, and comparing the values of a set of
macros from compiling two different pieces of code.
It's not just signal numbers that should have such tests; pretty much
any case where glibc copies constants from Linux kernel headers should
have such tests that the values and sets of constants agree except
where differences are known to be OK. Much the same also applies to
structure layouts (although testing those without hardcoding lists of
fields to test will be more complicated).
Given this patch, another test for a set of macros would essentially
be just a call to glibcextract.compare_macro_consts (plus boilerplate
code - and we could move to having separate text files defining such
tests, like the .sym inputs to gen-as-const, so that only a single
Python script is needed for most such tests). Some such tests would
of course need new features, e.g. where the set of macros changes in
new kernel versions (so you need to allow new macro names on the
kernel side if the kernel headers are newer than the version known to
glibc, and extra macros on the glibc side if the kernel headers are
older). tst-syscall-list.sh could become a Python script that uses
common code to generate lists of macros but does other things with its
own custom logic.
There are a few differences from the existing shell + awk test.
Because the new test evaluates constants using the compiler, no
special handling is needed any more for one signal name being defined
to another. Because asm/signal.h now needs to pass through the
compiler, not just the preprocessor, stddef.h is included as well
(given the asm/signal.h issue that it requires an externally provided
definition of size_t). The previous code defined __ASSEMBLER__ with
asm/signal.h; this is removed (__ASSEMBLY__, a different macro,
eliminates the requirement for stddef.h on some but not all
architectures).
Tested for x86_64, and with build-many-glibcs.py.
* scripts/glibcextract.py: New file.
* scripts/gen-as-const.py: Do not import os.path, re, subprocess
or tempfile. Import glibcexctract.
(compute_c_consts): Remove. Moved to glibcextract.py.
(gen_test): Update reference to compute_c_consts.
(main): Likewise.
* sysdeps/unix/sysv/linux/tst-signal-numbers.py: New file.
* sysdeps/unix/sysv/linux/tst-signal-numbers.sh: Remove.
* sysdeps/unix/sysv/linux/Makefile
($(objpfx)tst-signal-numbers.out): Use tst-signal-numbers.py.
Redirect stderr as well as stdout.
2018-12-10 22:27:13 +00:00
|
|
|
import glibcextract
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
|
|
|
|
|
|
|
|
def gen_test(sym_data):
|
|
|
|
"""Generate a test for the values of some C constants.
|
|
|
|
|
Move tst-signal-numbers to Python.
This patch converts the tst-signal-numbers test from shell + awk to
Python.
As with gen-as-const, the point is not so much that shell and awk are
problematic for this code, as that it's useful to build up general
infrastructure in Python for use of a range of code involving
extracting values from C headers. This patch moves some code from
gen-as-const.py to a new glibcextract.py, which also gains functions
relating to listing macros, and comparing the values of a set of
macros from compiling two different pieces of code.
It's not just signal numbers that should have such tests; pretty much
any case where glibc copies constants from Linux kernel headers should
have such tests that the values and sets of constants agree except
where differences are known to be OK. Much the same also applies to
structure layouts (although testing those without hardcoding lists of
fields to test will be more complicated).
Given this patch, another test for a set of macros would essentially
be just a call to glibcextract.compare_macro_consts (plus boilerplate
code - and we could move to having separate text files defining such
tests, like the .sym inputs to gen-as-const, so that only a single
Python script is needed for most such tests). Some such tests would
of course need new features, e.g. where the set of macros changes in
new kernel versions (so you need to allow new macro names on the
kernel side if the kernel headers are newer than the version known to
glibc, and extra macros on the glibc side if the kernel headers are
older). tst-syscall-list.sh could become a Python script that uses
common code to generate lists of macros but does other things with its
own custom logic.
There are a few differences from the existing shell + awk test.
Because the new test evaluates constants using the compiler, no
special handling is needed any more for one signal name being defined
to another. Because asm/signal.h now needs to pass through the
compiler, not just the preprocessor, stddef.h is included as well
(given the asm/signal.h issue that it requires an externally provided
definition of size_t). The previous code defined __ASSEMBLER__ with
asm/signal.h; this is removed (__ASSEMBLY__, a different macro,
eliminates the requirement for stddef.h on some but not all
architectures).
Tested for x86_64, and with build-many-glibcs.py.
* scripts/glibcextract.py: New file.
* scripts/gen-as-const.py: Do not import os.path, re, subprocess
or tempfile. Import glibcexctract.
(compute_c_consts): Remove. Moved to glibcextract.py.
(gen_test): Update reference to compute_c_consts.
(main): Likewise.
* sysdeps/unix/sysv/linux/tst-signal-numbers.py: New file.
* sysdeps/unix/sysv/linux/tst-signal-numbers.sh: Remove.
* sysdeps/unix/sysv/linux/Makefile
($(objpfx)tst-signal-numbers.out): Use tst-signal-numbers.py.
Redirect stderr as well as stdout.
2018-12-10 22:27:13 +00:00
|
|
|
The first argument is as for glibcextract.compute_c_consts.
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
|
|
|
|
"""
|
|
|
|
out_lines = []
|
|
|
|
for arg in sym_data:
|
|
|
|
if isinstance(arg, str):
|
2018-12-03 22:08:50 +00:00
|
|
|
if arg == 'START':
|
|
|
|
out_lines.append('#include <stdint.h>\n'
|
|
|
|
'#include <stdio.h>\n'
|
|
|
|
'#include <bits/wordsize.h>\n'
|
|
|
|
'#if __WORDSIZE == 64\n'
|
|
|
|
'typedef uint64_t c_t;\n'
|
|
|
|
'# define U(n) UINT64_C (n)\n'
|
|
|
|
'#else\n'
|
|
|
|
'typedef uint32_t c_t;\n'
|
|
|
|
'# define U(n) UINT32_C (n)\n'
|
|
|
|
'#endif\n'
|
|
|
|
'static int\n'
|
|
|
|
'do_test (void)\n'
|
|
|
|
'{\n'
|
|
|
|
# Compilation test only, using static
|
|
|
|
# assertions.
|
|
|
|
' return 0;\n'
|
|
|
|
'}\n'
|
|
|
|
'#include <support/test-driver.c>')
|
|
|
|
else:
|
|
|
|
out_lines.append(arg)
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
continue
|
|
|
|
name = arg[0]
|
|
|
|
value = arg[1]
|
|
|
|
out_lines.append('_Static_assert (U (asconst_%s) == (c_t) (%s), '
|
|
|
|
'"value of %s");'
|
|
|
|
% (name, value, name))
|
|
|
|
return '\n'.join(out_lines)
|
|
|
|
|
|
|
|
|
|
|
|
def main():
|
|
|
|
"""The main entry point."""
|
|
|
|
parser = argparse.ArgumentParser(
|
|
|
|
description='Produce headers of assembly constants.')
|
|
|
|
parser.add_argument('--cc', metavar='CC',
|
|
|
|
help='C compiler (including options) to use')
|
|
|
|
parser.add_argument('--test', action='store_true',
|
|
|
|
help='Generate test case instead of header')
|
|
|
|
parser.add_argument('sym_file',
|
|
|
|
help='.sym file to process')
|
|
|
|
args = parser.parse_args()
|
|
|
|
sym_data = []
|
|
|
|
with open(args.sym_file, 'r') as sym_file:
|
2018-12-03 22:08:50 +00:00
|
|
|
started = False
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
for line in sym_file:
|
|
|
|
line = line.strip()
|
|
|
|
if line == '':
|
|
|
|
continue
|
|
|
|
# Pass preprocessor directives through.
|
|
|
|
if line.startswith('#'):
|
|
|
|
sym_data.append(line)
|
|
|
|
continue
|
|
|
|
words = line.split(maxsplit=1)
|
2018-12-03 22:08:50 +00:00
|
|
|
if not started:
|
|
|
|
sym_data.append('START')
|
|
|
|
started = True
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
# Separator.
|
|
|
|
if words[0] == '--':
|
|
|
|
continue
|
|
|
|
name = words[0]
|
|
|
|
value = words[1] if len(words) > 1 else words[0]
|
|
|
|
sym_data.append((name, value))
|
2018-12-03 22:08:50 +00:00
|
|
|
if not started:
|
|
|
|
sym_data.append('START')
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
if args.test:
|
|
|
|
print(gen_test(sym_data))
|
|
|
|
else:
|
Move tst-signal-numbers to Python.
This patch converts the tst-signal-numbers test from shell + awk to
Python.
As with gen-as-const, the point is not so much that shell and awk are
problematic for this code, as that it's useful to build up general
infrastructure in Python for use of a range of code involving
extracting values from C headers. This patch moves some code from
gen-as-const.py to a new glibcextract.py, which also gains functions
relating to listing macros, and comparing the values of a set of
macros from compiling two different pieces of code.
It's not just signal numbers that should have such tests; pretty much
any case where glibc copies constants from Linux kernel headers should
have such tests that the values and sets of constants agree except
where differences are known to be OK. Much the same also applies to
structure layouts (although testing those without hardcoding lists of
fields to test will be more complicated).
Given this patch, another test for a set of macros would essentially
be just a call to glibcextract.compare_macro_consts (plus boilerplate
code - and we could move to having separate text files defining such
tests, like the .sym inputs to gen-as-const, so that only a single
Python script is needed for most such tests). Some such tests would
of course need new features, e.g. where the set of macros changes in
new kernel versions (so you need to allow new macro names on the
kernel side if the kernel headers are newer than the version known to
glibc, and extra macros on the glibc side if the kernel headers are
older). tst-syscall-list.sh could become a Python script that uses
common code to generate lists of macros but does other things with its
own custom logic.
There are a few differences from the existing shell + awk test.
Because the new test evaluates constants using the compiler, no
special handling is needed any more for one signal name being defined
to another. Because asm/signal.h now needs to pass through the
compiler, not just the preprocessor, stddef.h is included as well
(given the asm/signal.h issue that it requires an externally provided
definition of size_t). The previous code defined __ASSEMBLER__ with
asm/signal.h; this is removed (__ASSEMBLY__, a different macro,
eliminates the requirement for stddef.h on some but not all
architectures).
Tested for x86_64, and with build-many-glibcs.py.
* scripts/glibcextract.py: New file.
* scripts/gen-as-const.py: Do not import os.path, re, subprocess
or tempfile. Import glibcexctract.
(compute_c_consts): Remove. Moved to glibcextract.py.
(gen_test): Update reference to compute_c_consts.
(main): Likewise.
* sysdeps/unix/sysv/linux/tst-signal-numbers.py: New file.
* sysdeps/unix/sysv/linux/tst-signal-numbers.sh: Remove.
* sysdeps/unix/sysv/linux/Makefile
($(objpfx)tst-signal-numbers.out): Use tst-signal-numbers.py.
Redirect stderr as well as stdout.
2018-12-10 22:27:13 +00:00
|
|
|
consts = glibcextract.compute_c_consts(sym_data, args.cc)
|
2018-12-03 13:40:48 +00:00
|
|
|
print(''.join('#define %s %s\n' % c for c in sorted(consts.items())), end='')
|
Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
2018-11-30 15:20:41 +00:00
|
|
|
|
|
|
|
if __name__ == '__main__':
|
|
|
|
main()
|