qtbase/cmake/QtToolHelpers.cmake

866 lines
34 KiB
CMake
Raw Normal View History

# Copyright (C) 2022 The Qt Company Ltd.
# SPDX-License-Identifier: BSD-3-Clause
# This function is used to define a "Qt tool", such as moc, uic or rcc.
#
Generate information about user-facing applications in build dir When packaging different Qt versions for Linux distributions (or any distribution with a common bin dir), Qt tools cannot be installed to /usr/bin, because the executable names of the different Qt versions clash. To solve this conflict, our recommendation is to install Qt's tools to /usr/lib/qt6/bin and to create versioned symlinks to user-facing tools in /usr/bin. User-facing tools are tools that are supposed to be started manually by the user. They are marked in Qt's build system. Distro package maintainers can now configure with -DCMAKE_INSTALL_PREFIX=/usr -DINSTALL_BINDIR=/usr/lib/qt6/bin -DINSTALL_PUBLICBINDIR=/usr/bin and will find a file called user_facing_tool_links.txt in the build directory after the cmake run. Nothing will be installed to INSTALL_PUBLICBINDIR. Each line of user_facing_tool_links.txt consists of the installation path of a user-facing application followed by a space and the versioned link name in INSTALL_PUBLICBINDIR. Example content: /usr/lib/qt6/bin/qmake /usr/bin/qmake6 To actually create the versioned symlinks, the content of this file can be fed to ln like this: xargs ln -s < build-dir/user_facing_tool_links.txt Or the package maintainer may decide to do something completely different as suits their needs. This patch adds the USER_FACING argument to qt_internal_add_tool to mark tools as user-facing. In addition, every Qt created by qt_internal_add_app is treated as user-facing. The only tool this patch marks as user-facing in qtbase is qmake. Pick-to: 6.1 Fixes: QTBUG-89170 Change-Id: I52673b1c8d40f40f56a74203065553115e2c4de5 Reviewed-by: Kai Koehne <kai.koehne@qt.io> Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
2021-02-11 13:01:58 +00:00
# USER_FACING can be passed to mark the tool as a program that is supposed to be
# started directly by users.
#
# We must pass this function a target name obtained from
# qt_get_tool_target_name like this:
# qt_get_tool_target_name(target_name my_tool)
# qt_internal_add_tool(${target_name})
#
# Option Arguments:
# INSTALL_VERSIONED_LINK
# Prefix build only. On installation, create a versioned hard-link of the installed file.
# E.g. create a link of "bin/qmake6" to "bin/qmake".
# TRY_RUN
# On Windows, it creates a helper batch script that tests whether the tool can be executed
# successfully or not. If not, build halts and an error will be show, with tips on what
# might be cause, and how to fix it. TRY_RUN is disabled when cross-compiling.
# TRY_RUN_FLAGS
# Command line flags that are going to be passed to the tool for testing its correctness.
# If no flags were given, we default to `-v`.
# REQUIRED_FOR_DOCS
# Specifies that the built tool is required to generate documentation. Examples are qdoc,
# and qvkgen (because they participate in header file generation, which are needed for
# documentation generation).
#
# One-value Arguments:
# EXTRA_CMAKE_FILES
# List of additional CMake files that will be installed alongside the tool's exported CMake
# files.
# EXTRA_CMAKE_INCLUDES
# List of files that will be included in the Qt6${module}Tools.cmake file.
# Also see TOOLS_TARGET.
# INSTALL_DIR
# Takes a path, relative to the install prefix, like INSTALL_LIBEXECDIR.
# If this argument is omitted, the default is INSTALL_BINDIR.
# TOOLS_TARGET
# Specifies the module this tool belongs to. The module's Qt6${module}Tools.cmake file
# will then contain targets for this tool.
# CORE_LIBRARY
# The argument accepts 'Bootstrap' or 'None' values. If the argument value is set to
# 'Bootstrap' the Qt::Bootstrap library is linked to the executable instead of Qt::Core.
# The 'None' value points that core library is not necessary and avoids linking neither
# Qt::Core or Qt::Bootstrap libraries. Otherwise the Qt::Core library will be publicly
# linked to the executable target by default.
function(qt_internal_add_tool target_name)
qt_tool_target_to_name(name ${target_name})
set(option_keywords
NO_INSTALL
USER_FACING
INSTALL_VERSIONED_LINK
EXCEPTIONS
NO_UNITY_BUILD
TRY_RUN
REQUIRED_FOR_DOCS
${__qt_internal_sbom_optional_args}
)
set(one_value_keywords
TOOLS_TARGET
INSTALL_DIR
CORE_LIBRARY
TRY_RUN_FLAGS
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
${__default_target_info_args}
${__qt_internal_sbom_single_args}
)
set(multi_value_keywords
EXTRA_CMAKE_FILES
EXTRA_CMAKE_INCLUDES
PUBLIC_LIBRARIES
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
${__default_private_args}
${__qt_internal_sbom_multi_args}
)
cmake_parse_arguments(PARSE_ARGV 1 arg
"${option_keywords}"
"${one_value_keywords}"
"${multi_value_keywords}")
_qt_internal_validate_all_args_are_parsed(arg)
qt_internal_find_tool(will_build_tools ${target_name} "${arg_TOOLS_TARGET}")
if(NOT will_build_tools)
return()
endif()
set(disable_autogen_tools "${arg_DISABLE_AUTOGEN_TOOLS}")
set(corelib "")
if(arg_CORE_LIBRARY STREQUAL "Bootstrap" OR arg_CORE_LIBRARY STREQUAL "None")
set(corelib CORE_LIBRARY ${arg_CORE_LIBRARY})
list(APPEND disable_autogen_tools "uic" "moc" "rcc")
endif()
set(exceptions "")
if(arg_EXCEPTIONS)
set(exceptions EXCEPTIONS)
endif()
set(install_dir "${INSTALL_BINDIR}")
if(arg_INSTALL_DIR)
set(install_dir "${arg_INSTALL_DIR}")
endif()
set(output_dir "${QT_BUILD_DIR}/${install_dir}")
if(arg_PUBLIC_LIBRARIES)
message(WARNING
"qt_internal_add_tool's PUBLIC_LIBRARIES option is deprecated, and will be "
"removed in a future Qt version. Use the LIBRARIES option instead.")
endif()
if(arg_NO_UNITY_BUILD)
set(arg_NO_UNITY_BUILD "NO_UNITY_BUILD")
else()
set(arg_NO_UNITY_BUILD "")
endif()
_qt_internal_forward_function_args(
FORWARD_PREFIX arg
FORWARD_OUT_VAR add_executable_args
FORWARD_SINGLE
TARGET_COMPANY
TARGET_COPYRIGHT
TARGET_DESCRIPTION
TARGET_PRODUCT
TARGET_VERSION
)
qt_internal_add_executable("${target_name}"
OUTPUT_DIRECTORY "${output_dir}"
${exceptions}
NO_INSTALL
${arg_NO_UNITY_BUILD}
SOURCES ${arg_SOURCES}
NO_PCH_SOURCES ${arg_NO_PCH_SOURCES}
NO_UNITY_BUILD_SOURCES ${arg_NO_UNITY_BUILD_SOURCES}
INCLUDE_DIRECTORIES
${arg_INCLUDE_DIRECTORIES}
DEFINES
${arg_DEFINES}
${corelib}
LIBRARIES
${arg_LIBRARIES}
${arg_PUBLIC_LIBRARIES}
Qt::PlatformToolInternal
COMPILE_OPTIONS ${arg_COMPILE_OPTIONS}
LINK_OPTIONS ${arg_LINK_OPTIONS}
MOC_OPTIONS ${arg_MOC_OPTIONS}
DISABLE_AUTOGEN_TOOLS ${disable_autogen_tools}
${add_executable_args}
# If you are putting anything after these, make sure that
# qt_set_target_info_properties knows how to process them
)
qt_internal_add_target_aliases("${target_name}")
qt_internal_adjust_main_config_runtime_output_dir("${target_name}" "${output_dir}")
if (WIN32)
_qt_internal_generate_longpath_win32_rc_file_and_manifest("${target_name}")
endif()
CMake: Record used package version for each target dependency When recording which package version to look for in QtFooModuleDependencies.cmake and other files like it, instead of using PROJECT_VERSION, use the version of the package that contains the dependency. For example if we're hypothetically building the qtdeclarative repo from the 6.4 branch, against an installed 6.2 qtbase, then the Qt6QmlModuleDependencies.cmake file will have a find_package(Qt6Core 6.2) call because qtdeclarative's find_package(Qt6Core) call found a 6.2 Core when it was configured. This allows switching the versioning scheme of specific Qt modules that might not want to follow the general Qt versioning scheme. The first candidate would be QtWebEngine which might want to follow the Chromium versioning scheme, something like Qt 6.94.0 where 94 is the Chromium major version. Implementation notes. We now record the package version of a target in a property called _qt_package_version. We do it for qt modules, plugins, 3rd party libraries, tools and the Platform target. When we try to look up which version to write into the QtFooModuleDependencies.cmake file (or the equivalent Plugins and Tools file), we try to find the version from a few sources: the property mentioned above, then the Qt6{target}_VERSION variable, and finally PROJECT_VERSION. In the latter case, we issue a warning because technically that should never have to happen, and it's a bug or an unforeseen case if it does. A few more places also need adjustments: - package versions to look for when configuring standalone tests and generating standalone tests Config files - handling of tools packages - The main Qt6 package lookup in each Dependencies.cmake files Note that there are some requirements and consequences in case a module wants to use a different versioning scheme like 6.94.0. Requirements. - The root CMakeLists.txt file needs to call find_package with a version different from the usual PROJECT_VERSION. Ideally it should look for a few different Qt versions which are known to be compatible, for example the last stable and LTS versions, or just the lowest supported Qt version, e.g. 6.2.6 or whenever this change would land in the 6.2 branch. - If the repository has multiple modules, some of which need to follow the Qt versioning scheme and some not, project(VERSION x.y.z) calls need to be carefully placed in subdirectory scopes with appropriate version numbers, so that qt_internal_add_module / _tool / _plugin pick up the correct version. Consequences. - The .so / .dylib names will contain the new version, e.g. .so.6.94 - Linux ELF symbols will contain the new versions - syncqt private headers will now exist under a include/QtFoo/6.94.0/QtFoo/private folder - pri and prl files will also contain the new version numbers - pkg-config .pc files contain the new version numbers - It won't be possible to write find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code. One would have to write find_package(Qt6WebEngineWidgets 6.94) otherwise CMake will try to look for Qt6Config 6.94 which won't exist. - Similarly, a find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call would always find any kind of WebEngine package that is higher than 6.4, which might be 6.94, 6.95, etc. - In the future, if we fix Qt6Config to pass EXACT to its subcomponent find_package calls, a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets) would fail to find WebEngineWidgets, because its 6.94.0 version will not be equal to 6.5.0. Currently we don't pass through EXACT, so it's not an issue. Augments 5ffc744b791a114a3180a425dd26e298f7399955 Task-number: QTBUG-103500 Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab Reviewed-by: Michal Klocek <michal.klocek@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-17 06:44:43 +00:00
set_target_properties(${target_name} PROPERTIES
_qt_package_version "${PROJECT_VERSION}"
)
set_property(TARGET ${target_name}
APPEND PROPERTY
EXPORT_PROPERTIES "_qt_package_version")
get_cmake_property(is_multi_config GENERATOR_IS_MULTI_CONFIG)
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.19.0"
AND QT_FEATURE_debug_and_release
AND is_multi_config)
set_property(TARGET "${target_name}"
PROPERTY EXCLUDE_FROM_ALL "$<NOT:$<CONFIG:${QT_MULTI_CONFIG_FIRST_CONFIG}>>")
endif()
if (NOT target_name STREQUAL name)
set_target_properties(${target_name} PROPERTIES
OUTPUT_NAME ${name}
EXPORT_NAME ${name}
)
endif()
if(TARGET host_tools)
add_dependencies(host_tools "${target_name}")
if(arg_REQUIRED_FOR_DOCS)
add_dependencies(doc_tools "${target_name}")
endif()
if(arg_CORE_LIBRARY STREQUAL "Bootstrap")
add_dependencies(bootstrap_tools "${target_name}")
endif()
endif()
if(arg_EXTRA_CMAKE_FILES)
set_target_properties(${target_name} PROPERTIES
EXTRA_CMAKE_FILES "${arg_EXTRA_CMAKE_FILES}"
)
endif()
if(arg_EXTRA_CMAKE_INCLUDES)
set_target_properties(${target_name} PROPERTIES
EXTRA_CMAKE_INCLUDES "${arg_EXTRA_CMAKE_INCLUDES}"
)
endif()
Generate information about user-facing applications in build dir When packaging different Qt versions for Linux distributions (or any distribution with a common bin dir), Qt tools cannot be installed to /usr/bin, because the executable names of the different Qt versions clash. To solve this conflict, our recommendation is to install Qt's tools to /usr/lib/qt6/bin and to create versioned symlinks to user-facing tools in /usr/bin. User-facing tools are tools that are supposed to be started manually by the user. They are marked in Qt's build system. Distro package maintainers can now configure with -DCMAKE_INSTALL_PREFIX=/usr -DINSTALL_BINDIR=/usr/lib/qt6/bin -DINSTALL_PUBLICBINDIR=/usr/bin and will find a file called user_facing_tool_links.txt in the build directory after the cmake run. Nothing will be installed to INSTALL_PUBLICBINDIR. Each line of user_facing_tool_links.txt consists of the installation path of a user-facing application followed by a space and the versioned link name in INSTALL_PUBLICBINDIR. Example content: /usr/lib/qt6/bin/qmake /usr/bin/qmake6 To actually create the versioned symlinks, the content of this file can be fed to ln like this: xargs ln -s < build-dir/user_facing_tool_links.txt Or the package maintainer may decide to do something completely different as suits their needs. This patch adds the USER_FACING argument to qt_internal_add_tool to mark tools as user-facing. In addition, every Qt created by qt_internal_add_app is treated as user-facing. The only tool this patch marks as user-facing in qtbase is qmake. Pick-to: 6.1 Fixes: QTBUG-89170 Change-Id: I52673b1c8d40f40f56a74203065553115e2c4de5 Reviewed-by: Kai Koehne <kai.koehne@qt.io> Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
2021-02-11 13:01:58 +00:00
if(arg_USER_FACING)
set_property(GLOBAL APPEND PROPERTY QT_USER_FACING_TOOL_TARGETS ${target_name})
endif()
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
if(QT_GENERATE_SBOM)
set(sbom_args "")
list(APPEND sbom_args DEFAULT_SBOM_ENTITY_TYPE QT_TOOL)
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
endif()
if(NOT arg_NO_INSTALL AND arg_TOOLS_TARGET)
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
set(will_install TRUE)
else()
set(will_install FALSE)
if(QT_GENERATE_SBOM)
list(APPEND sbom_args NO_INSTALL)
endif()
endif()
if(will_install)
# Assign a tool to an export set, and mark the module to which the tool belongs.
qt_internal_append_known_modules_with_tools("${arg_TOOLS_TARGET}")
# Also append the tool to the module list.
qt_internal_append_known_module_tool("${arg_TOOLS_TARGET}" "${target_name}")
qt_get_cmake_configurations(cmake_configs)
set(install_initial_call_args
EXPORT "${INSTALL_CMAKE_NAMESPACE}${arg_TOOLS_TARGET}ToolsTargets"
COMPONENT host_tools
)
foreach(cmake_config ${cmake_configs})
qt_get_install_target_default_args(
OUT_VAR install_targets_default_args
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
OUT_VAR_RUNTIME runtime_install_destination
RUNTIME "${install_dir}"
CMAKE_CONFIG "${cmake_config}"
ALL_CMAKE_CONFIGS ${cmake_configs})
# Make installation optional for targets that are not built by default in this config
if(QT_FEATURE_debug_and_release
AND NOT (cmake_config STREQUAL QT_MULTI_CONFIG_FIRST_CONFIG))
set(install_optional_arg OPTIONAL)
else()
unset(install_optional_arg)
endif()
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
if(QT_GENERATE_SBOM)
_qt_internal_sbom_append_multi_config_aware_single_arg_option(
RUNTIME_PATH
"${runtime_install_destination}"
"${cmake_config}"
sbom_args
)
endif()
qt_install(TARGETS "${target_name}"
${install_initial_call_args}
${install_optional_arg}
CONFIGURATIONS ${cmake_config}
${install_targets_default_args})
unset(install_initial_call_args)
endforeach()
if(arg_INSTALL_VERSIONED_LINK)
qt_internal_install_versioned_link(WORKING_DIRECTORY "${install_dir}"
TARGETS "${target_name}")
endif()
qt_apply_rpaths(TARGET "${target_name}" INSTALL_PATH "${install_dir}" RELATIVE_RPATH)
CMake: Work around build rpath issue when CMAKE_STAGING_PREFIX is set CMake has logic to rewrite build rpaths that contain CMAKE_STAGING_PREFIX to instead point to CMAKE_INSTALL_PREFIX. This breaks running executables from the build directory, because their build rpath will point to a location where the libraries might not exist yet (we didn't install Qt yet). Work around this by setting CMAKE_STAGING_PREFIX to a fake path, so that CMake does not do the rewriting anymore. CMAKE_STAGING_PREFIX needs to be set at subdirectory scope, not function scope, which is why qt_internal_apply_staging_prefix_build_rpath_workaround() is a macro that is called from within each Qt internal function that creates a target. The workaround can be disabled by configuring with -DQT_NO_STAGING_PREFIX_BUILD_RPATH_WORKAROUND=ON The downside of this workaround is that it breaks per-subdirectory install rules like 'ninja src/gui/install'. Regular global installation like 'ninja install' works fine. This is similar to what we do for tests in qt_set_up_fake_standalone_tests_install_prefix() introduced by 20292250d44e08437306096e9096fc655cc9fb8b The reason it's not as good for other target types is because in contrast to tests, we do want to install them. In case if someone does call `ninja src/gui/install' they will most likely get a permission error, telling them it's not possible to install into /qt_fake_staging_prefix/ check_qt_internal_apply_staging_prefix_build_rpath_workaround Fixes: QTBUG-102592 Change-Id: I6ce78dde1924a8d830ef5c62808ff674c9639d65 Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-04-14 16:04:41 +00:00
qt_internal_apply_staging_prefix_build_rpath_workaround()
endif()
if(arg_TRY_RUN AND WIN32 AND NOT CMAKE_CROSSCOMPILING)
if(NOT arg_TRY_RUN_FLAGS)
set(arg_TRY_RUN_FLAGS "-v")
endif()
_qt_internal_add_try_run_post_build("${target_name}" "${arg_TRY_RUN_FLAGS}")
endif()
qt_internal_defer_separate_debug_info("${target_name}"
SEPARATE_DEBUG_INFO_ARGS
"${install_dir}"
QT_EXECUTABLE
)
qt_internal_install_pdb_files(${target_name} "${install_dir}")
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
if(QT_GENERATE_SBOM)
_qt_internal_forward_function_args(
FORWARD_APPEND
FORWARD_PREFIX arg
FORWARD_OUT_VAR sbom_args
FORWARD_OPTIONS
${__qt_internal_sbom_optional_args}
FORWARD_SINGLE
${__qt_internal_sbom_single_args}
FORWARD_MULTI
${__qt_internal_sbom_multi_args}
)
qt_internal_extend_qt_entity_sbom(${target_name} ${sbom_args})
CMake: Generate an SPDX v2.3 SBOM file for each built repository This change adds a new -sbom configure option to allow generating and installing an SPDX v2.3 SBOM file when building a qt repo. The -sbom-dir option can be used to configure the location where each repo sbom file will be installed. By default it is installed into $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx which is basically ~/Qt/sbom/qtbase-6.8.0.spdx The file is installed as part of the default installation rules, but it can also be installed manually using the "sbom" installation component, or "sbom_$lower_project_name" in a top-level build. For example: cmake install . --component sbom_qtbase CMake 3.19+ is needed to read the qt_attribution.json files for copyrights, license info, etc. When using an older cmake version, configuration will error out. It is possible to opt into using an older cmake version, but the generated sbom will lack all the attribution file information. Using an older cmake version is untested and not officially supported. Implementation notes. The bulk of the implementation is split into 4 new files: - QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing and dispatching the generation of various pieces of the SBOM document e.g. a SDPX package associated with a target like Core, a SDPX file entry for each target binary file (per-config shared library, archive, executable, etc) - QtPublicSbomGenerationHelpers.cmake - for non-Qt specific implementation of SPDX generation. This also has some code that was taken from the cmake-sbom 3rd party project, so it is dual licensed under the usual Qt build system BSD license, as well as the MIT license of the 3rd party project - QtPublicGitHelpers.cmake - for git related features, mainly to embed queried hashes or tags into version strings, is dual-licensed for the same reasons as QtPublicSbomGenerationHelpers.cmake - QtSbomHelpers.cmake - Qt-specific functions that just forward arguments to the public functions. These are meant to be used in our Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones for naming consistency. These function would mostly be used to annotate 3rd party libraries with sbom info and to add sbom info for unusual target setups (like the Bootstrap library), because most of the handling is already done automatically via qt_internal_add_module/plugin/etc. The files are put into Public cmake files, with the future hope of making this available to user projects in some capacity. The distinction of Qt-specific and non-Qt specific code might blur a bit, and thus the separation across files might not always be consistent, but it was best effort. The main purpose of the code is to collect various information about targets and their relationships and generate equivalent SPDX info. Collection is currently done for the following targets: Qt modules, plugins, apps, tools, system libraries, bundled 3rd party libraries and partial 3rd party sources compiled directly as part of Qt targets. Each target has an equivalent SPDX package generated with information like version, license, copyright, CPE (common vulnerability identifier), files that belong to the package, and relationships on other SPDX packages (associated cmake targets), mostly gathered from direct linking dependencies. Each package might also contain files, e.g. libQt6Core.so for the Core target. Each file also has info like license id, copyrights, but also the list of source files that were used to generate the file and a sha1 checksum. SPDX documents can also refer to packages in other SPDX documents, and those are referred to via external document references. This is the case when building qtdeclarative and we refer to Core. For qt provided targets, we have complete information regarding licenses, and copyrights. For bundled 3rd party libraries, we should also have most information, which is usually parsed from the src/3rdparty/libfoo/qt_attribution.json files. If there are multiple attribution files, or if the files have multiple entries, we create a separate SBOM package for each of those entries, because each might have a separate copyright or version, and an sbom package can have only one version (although many copyrights). For system libraries we usually lack the information because we don't have attribution files for Find scripts. So the info needs to be manually annotated via arguments to the sbom function calls, or the FindFoo.cmake scripts expose that information in some form and we can query it. There are also corner cases like 3rdparty sources being directly included in a Qt library, like the m4dc files for Gui, or PCRE2 for Bootstrap. Or QtWebEngine libraries (either Qt bundled or Chromium bundled or system libraries) which get linked in by GN instead of CMake, so there are no direct targets for them. The information for these need to be annotated manually as well. There is also a distinction to be made for static Qt builds (or any static Qt library in a shared build), where the system libraries found during the Qt build might not be the same that are linked into the final user application or library. The actual generation of the SBOM is done by file(GENERATE)-ing one .cmake file for each target, file, external ref, etc, which will be included in a top-level cmake script. The top-level cmake script will run through each included file, to append to a "staging" spdx file, which will then be used in a configure_file() call to replace some final variables, like embedding a file checksum. There are install rules to generate a complete SBOM during installation, and an optional 'sbom' custom target that allows building an incomplete SBOM during the build step. The build target is just for convenience and faster development iteration time. It is incomplete because it is missing the installed file SHA1 checksums and the document verification code (the sha1 of all sha1s). We can't compute those during the build before the files are actually installed. A complete SBOM can only be achieved at installation time. The install script will include all the generated helper files, but also set some additional variables to ensure checksumming happens, and also handle multi-config installation, among other small things. For multi-config builds, CMake doesn't offer a way to run code after all configs are installed, because they might not always be installed, someone might choose to install just Release. To handle that, we rely on ninja installing each config sequentially (because ninja places the install rules into the 'console' pool which runs one task at a time). For each installed config we create a config-specific marker file. Once all marker files are present, whichever config ends up being installed as the last one, we run the sbom generation once, and then delete all marker files. There are a few internal variables that can be set during configuration to enable various checks (and other features) on the generated spdx files: - QT_INTERNAL_SBOM_VERIFY - QT_INTERNAL_SBOM_AUDIT - QT_INTERNAL_SBOM_AUDIT_NO_ERROR - QT_INTERNAL_SBOM_GENERATE_JSON - QT_INTERNAL_SBOM_SHOW_TABLE - QT_INTERNAL_SBOM_DEFAULT_CHECKS These use 3rd party python tools, so they are not enabled by default. If enabled, they run at installation time after the sbom is installed. We will hopefully enable them in CI. Overall, the code is still a bit messy in a few places, due to time constraints, but can be improved later. Some possible TODOs for the future: - Do we need to handle 3rd party libs linked into a Qt static library in a Qt shared build, where the Qt static lib is not installed, but linked into a Qt shared library, somehow specially? We can record a package for it, but we can't create a spdx file record for it (and associated source relationships) because we don't install the file, and spdx requires the file to be installed and checksummed. Perhaps we can consider adding some free-form text snippet to the package itself? - Do we want to add parsing of .cpp source files for Copyrights, to embed them into the packages? This will likely slow down configuration quite a bit. - Currently sbom info attached to WrapFoo packages in one repo is not exported / available in other repos. E.g. If we annotate WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be available when looking up WrapZLIB in qtimageformats. This is because they are IMPORTED libraries, and are not exported. We might want to record this info in the future. [ChangeLog][Build System] A new -sbom configure option can be used to generate and install a SPDX SBOM (Software Bill of Materials) file for each built Qt repository. Pick-to: 6.8 Task-number: QTBUG-122899 Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-03-07 17:02:56 +00:00
endif()
qt_add_list_file_finalizer(qt_internal_finalize_tool ${target_name})
endfunction()
function(qt_internal_finalize_tool target)
_qt_internal_finalize_sbom(${target})
endfunction()
function(_qt_internal_add_try_run_post_build target try_run_flags)
qt_internal_get_upper_case_main_cmake_configuration(main_cmake_configuration)
get_target_property(target_out_dir ${target}
RUNTIME_OUTPUT_DIRECTORY_${main_cmake_configuration})
get_target_property(target_bin_dir ${target}
BINARY_DIR)
set(try_run_scripts_path "${target_bin_dir}/${target}_try_run.bat")
# The only reason -h is passed is because some of the tools, e.g., moc
# wait for an input without any arguments.
qt_configure_file(OUTPUT "${try_run_scripts_path}"
CONTENT "@echo off
${target_out_dir}/${target}.exe ${try_run_flags} > nul 2>&1
if \"%errorlevel%\" == \"-1073741515\" (
echo
echo '${target}' is built successfully, but some of the libraries
echo necessary for running it are missing. If you are building Qt with
echo 3rdparty libraries, make sure that you add their directory to the
echo PATH environment variable.
echo
exit /b %errorlevel%
)
echo. > ${target_bin_dir}/${target}_try_run_passed"
)
add_custom_command(
OUTPUT
${target_bin_dir}/${target}_try_run_passed
DEPENDS
${target}
COMMAND
${CMAKE_COMMAND} -E env QT_COMMAND_LINE_PARSER_NO_GUI_MESSAGE_BOXES=1
${try_run_scripts_path}
COMMENT
"Testing ${target} by trying to run it."
VERBATIM
)
add_custom_target(${target}_try_run ALL
DEPENDS ${target_bin_dir}/${target}_try_run_passed)
endfunction()
function(qt_export_tools module_name)
# Bail out when not building tools.
if(NOT QT_WILL_BUILD_TOOLS)
return()
endif()
# If no tools were defined belonging to this module, don't create a config and targets file.
if(NOT "${module_name}" IN_LIST QT_KNOWN_MODULES_WITH_TOOLS)
return()
endif()
# The tools target name. For example: CoreTools
set(target "${module_name}Tools")
set(path_suffix "${INSTALL_CMAKE_NAMESPACE}${target}")
qt_path_join(config_build_dir ${QT_CONFIG_BUILD_DIR} ${path_suffix})
qt_path_join(config_install_dir ${QT_CONFIG_INSTALL_DIR} ${path_suffix})
# Add the extra cmake statements to make the tool targets global, so it doesn't matter where
# find_package is called.
# Also assemble a list of tool targets to expose in the config file for informational purposes.
set(extra_cmake_statements "")
set(tool_targets "")
set(tool_targets_non_prefixed "")
# List of package dependencies that need be find_package'd when using the Tools package.
set(package_deps "")
set(third_party_deps "")
# Additional cmake files to install
set(extra_cmake_files "")
set(extra_cmake_includes "")
CMake: Record used package version for each target dependency When recording which package version to look for in QtFooModuleDependencies.cmake and other files like it, instead of using PROJECT_VERSION, use the version of the package that contains the dependency. For example if we're hypothetically building the qtdeclarative repo from the 6.4 branch, against an installed 6.2 qtbase, then the Qt6QmlModuleDependencies.cmake file will have a find_package(Qt6Core 6.2) call because qtdeclarative's find_package(Qt6Core) call found a 6.2 Core when it was configured. This allows switching the versioning scheme of specific Qt modules that might not want to follow the general Qt versioning scheme. The first candidate would be QtWebEngine which might want to follow the Chromium versioning scheme, something like Qt 6.94.0 where 94 is the Chromium major version. Implementation notes. We now record the package version of a target in a property called _qt_package_version. We do it for qt modules, plugins, 3rd party libraries, tools and the Platform target. When we try to look up which version to write into the QtFooModuleDependencies.cmake file (or the equivalent Plugins and Tools file), we try to find the version from a few sources: the property mentioned above, then the Qt6{target}_VERSION variable, and finally PROJECT_VERSION. In the latter case, we issue a warning because technically that should never have to happen, and it's a bug or an unforeseen case if it does. A few more places also need adjustments: - package versions to look for when configuring standalone tests and generating standalone tests Config files - handling of tools packages - The main Qt6 package lookup in each Dependencies.cmake files Note that there are some requirements and consequences in case a module wants to use a different versioning scheme like 6.94.0. Requirements. - The root CMakeLists.txt file needs to call find_package with a version different from the usual PROJECT_VERSION. Ideally it should look for a few different Qt versions which are known to be compatible, for example the last stable and LTS versions, or just the lowest supported Qt version, e.g. 6.2.6 or whenever this change would land in the 6.2 branch. - If the repository has multiple modules, some of which need to follow the Qt versioning scheme and some not, project(VERSION x.y.z) calls need to be carefully placed in subdirectory scopes with appropriate version numbers, so that qt_internal_add_module / _tool / _plugin pick up the correct version. Consequences. - The .so / .dylib names will contain the new version, e.g. .so.6.94 - Linux ELF symbols will contain the new versions - syncqt private headers will now exist under a include/QtFoo/6.94.0/QtFoo/private folder - pri and prl files will also contain the new version numbers - pkg-config .pc files contain the new version numbers - It won't be possible to write find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code. One would have to write find_package(Qt6WebEngineWidgets 6.94) otherwise CMake will try to look for Qt6Config 6.94 which won't exist. - Similarly, a find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call would always find any kind of WebEngine package that is higher than 6.4, which might be 6.94, 6.95, etc. - In the future, if we fix Qt6Config to pass EXACT to its subcomponent find_package calls, a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets) would fail to find WebEngineWidgets, because its 6.94.0 version will not be equal to 6.5.0. Currently we don't pass through EXACT, so it's not an issue. Augments 5ffc744b791a114a3180a425dd26e298f7399955 Task-number: QTBUG-103500 Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab Reviewed-by: Michal Klocek <michal.klocek@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-17 06:44:43 +00:00
set(first_tool_package_version "")
set(known_tools ${QT_KNOWN_MODULE_${module_name}_TOOLS})
foreach(tool_name IN LISTS known_tools)
# Specific tools can have package dependencies.
# e.g. qtwaylandscanner depends on WaylandScanner (non-qt package).
get_target_property(extra_packages "${tool_name}" QT_EXTRA_PACKAGE_DEPENDENCIES)
if(extra_packages)
foreach(third_party_dep IN LISTS extra_packages)
list(GET third_party_dep 0 third_party_dep_name)
list(GET third_party_dep 1 third_party_dep_version)
# Assume that all tool thirdparty deps are mandatory.
# TODO: Components are not supported
list(APPEND third_party_deps
"${third_party_dep_name}\\\;FALSE\\\;${third_party_dep_version}\\\;\\\;")
endforeach()
endif()
get_target_property(_extra_cmake_files "${tool_name}" EXTRA_CMAKE_FILES)
if (_extra_cmake_files)
foreach(cmake_file ${_extra_cmake_files})
file(COPY "${cmake_file}" DESTINATION "${config_build_dir}")
list(APPEND extra_cmake_files "${cmake_file}")
endforeach()
endif()
get_target_property(_extra_cmake_includes "${tool_name}" EXTRA_CMAKE_INCLUDES)
if(_extra_cmake_includes)
list(APPEND extra_cmake_includes "${_extra_cmake_includes}")
endif()
if (QT_WILL_RENAME_TOOL_TARGETS)
string(REGEX REPLACE "_native$" "" tool_name ${tool_name})
endif()
# `__qt_${target}_targets_file_included` is defined in the QtModuleToolsConfig.cmake.in
set(extra_cmake_statements "${extra_cmake_statements}
if(__qt_${target}_targets_file_included AND ${INSTALL_CMAKE_NAMESPACE}${target}_FOUND)
CMake: Allow promoting the Qt libraries to be global targets User projects can set the QT_PROMOTE_TO_GLOBAL_TARGETS variable to true so that the various imported targets created by find_package(Qt6) are promoted to global targets. This would allow a project to find Qt packages in a subdirectory scope while using those Qt targets from a different scope. E.g. it fixes errors like CMake Error at CMakeLists.txt:5 (target_link_libraries): Error evaluating generator expression: $<TARGET_OBJECTS:Qt6::Widgets_resources_1> Objects of target "Qt6::Widgets_resources_1" referenced but no such target exists. when trying to use a static Qt from a sibling scope. Various 3rd party dependency targets (like Atomic or ZLIB) are not made global due to limitations in CMake, but as long as those targets are not mentioned directly, it shouldn't cause issues. The targets are made global in the generated QtFooAdditionalTargetInfo.cmake file. To ensure that resource object libraries promoted, the generation of the file has to be done at the end of the defining scope where qt_internal_export_additional_targets_file is called, which is achieved with a deferred finalizer. Replaced all occurrences of target promotion with a helper function which allows tracing of all promoted targets by specifying --log-level=debug to CMake. Pick-to: 6.2 Fixes: QTBUG-92878 Change-Id: Ic4ec03b0bc383d7e591a58c520c3974fbea746d2 Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-05-20 11:38:30 +00:00
__qt_internal_promote_target_to_global(${INSTALL_CMAKE_NAMESPACE}::${tool_name})
endif()
")
list(APPEND tool_targets "${QT_CMAKE_EXPORT_NAMESPACE}::${tool_name}")
list(APPEND tool_targets_non_prefixed "${tool_name}")
CMake: Record used package version for each target dependency When recording which package version to look for in QtFooModuleDependencies.cmake and other files like it, instead of using PROJECT_VERSION, use the version of the package that contains the dependency. For example if we're hypothetically building the qtdeclarative repo from the 6.4 branch, against an installed 6.2 qtbase, then the Qt6QmlModuleDependencies.cmake file will have a find_package(Qt6Core 6.2) call because qtdeclarative's find_package(Qt6Core) call found a 6.2 Core when it was configured. This allows switching the versioning scheme of specific Qt modules that might not want to follow the general Qt versioning scheme. The first candidate would be QtWebEngine which might want to follow the Chromium versioning scheme, something like Qt 6.94.0 where 94 is the Chromium major version. Implementation notes. We now record the package version of a target in a property called _qt_package_version. We do it for qt modules, plugins, 3rd party libraries, tools and the Platform target. When we try to look up which version to write into the QtFooModuleDependencies.cmake file (or the equivalent Plugins and Tools file), we try to find the version from a few sources: the property mentioned above, then the Qt6{target}_VERSION variable, and finally PROJECT_VERSION. In the latter case, we issue a warning because technically that should never have to happen, and it's a bug or an unforeseen case if it does. A few more places also need adjustments: - package versions to look for when configuring standalone tests and generating standalone tests Config files - handling of tools packages - The main Qt6 package lookup in each Dependencies.cmake files Note that there are some requirements and consequences in case a module wants to use a different versioning scheme like 6.94.0. Requirements. - The root CMakeLists.txt file needs to call find_package with a version different from the usual PROJECT_VERSION. Ideally it should look for a few different Qt versions which are known to be compatible, for example the last stable and LTS versions, or just the lowest supported Qt version, e.g. 6.2.6 or whenever this change would land in the 6.2 branch. - If the repository has multiple modules, some of which need to follow the Qt versioning scheme and some not, project(VERSION x.y.z) calls need to be carefully placed in subdirectory scopes with appropriate version numbers, so that qt_internal_add_module / _tool / _plugin pick up the correct version. Consequences. - The .so / .dylib names will contain the new version, e.g. .so.6.94 - Linux ELF symbols will contain the new versions - syncqt private headers will now exist under a include/QtFoo/6.94.0/QtFoo/private folder - pri and prl files will also contain the new version numbers - pkg-config .pc files contain the new version numbers - It won't be possible to write find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code. One would have to write find_package(Qt6WebEngineWidgets 6.94) otherwise CMake will try to look for Qt6Config 6.94 which won't exist. - Similarly, a find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call would always find any kind of WebEngine package that is higher than 6.4, which might be 6.94, 6.95, etc. - In the future, if we fix Qt6Config to pass EXACT to its subcomponent find_package calls, a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets) would fail to find WebEngineWidgets, because its 6.94.0 version will not be equal to 6.5.0. Currently we don't pass through EXACT, so it's not an issue. Augments 5ffc744b791a114a3180a425dd26e298f7399955 Task-number: QTBUG-103500 Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab Reviewed-by: Michal Klocek <michal.klocek@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-17 06:44:43 +00:00
if(NOT first_tool_package_version)
qt_internal_get_package_version_of_target("${tool_name}" tool_package_version)
if(tool_package_version)
set(first_tool_package_version "${tool_package_version}")
endif()
endif()
endforeach()
string(APPEND extra_cmake_statements
"set(${QT_CMAKE_EXPORT_NAMESPACE}${module_name}Tools_TARGETS \"${tool_targets}\")")
# Extract package dependencies that were determined in QtPostProcess, but only if ${module_name}
# is an actual target.
# module_name can be a non-existent target, if the tool doesn't have an existing associated
# module, e.g. qtwaylandscanner.
if(TARGET "${module_name}")
get_target_property(module_package_deps "${module_name}" _qt_tools_package_deps)
if(module_package_deps)
list(APPEND package_deps "${module_package_deps}")
endif()
endif()
# Configure and install dependencies file for the ${module_name}Tools package.
configure_file(
"${QT_CMAKE_DIR}/QtModuleToolsDependencies.cmake.in"
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}Dependencies.cmake"
@ONLY
)
qt_install(FILES
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}Dependencies.cmake"
DESTINATION "${config_install_dir}"
COMPONENT Devel
)
if(extra_cmake_files)
qt_install(FILES
${extra_cmake_files}
DESTINATION "${config_install_dir}"
COMPONENT Devel
)
endif()
# Configure and install the ${module_name}Tools package Config file.
CMake: Enable NEW policies by CMake version with a global default When a CMake release introduces a new policy that affects most Qt modules, it may be appropriate to make each module aware of that newer CMake version and use the NEW policy without raising the minimum CMake version requirement. To reduce the churn associated with making that change across all Qt modules individually, this change allows it to be updated in a central place (qtbase), but in a way that allows a Qt module to override it in its own .cmake.conf file if required (e.g. to address the issues identified by policy warnings at a later time). The policies are modified at the start of the call to qt_build_repo_begin(). For commands defined by the qtbase module, qtbase needs to be in control of the policy settings at the point where those commands are defined. The above mechanism should not affect the policy settings for these commands, so the various *Config.cmake.in files must not specify policy ranges in a way that a Qt module's .cmake.conf file could influence. Starting with CMake 3.12, policies can be specified as a version range with the cmake_minimum_required() and cmake_policy() commands. All policies introduced in CMake versions up to the upper limit of that range will be set to NEW. The actual version of CMake being used only has to be at least the lower limit of the specified version range. This change uses cmake_minimum_required() rather than cmake_policy() due to the latter not halting further processing upon failure. See the following: https://gitlab.kitware.com/cmake/cmake/-/issues/21557 Task-number: QTBUG-88700 Pick-to: 6.0 Change-Id: I0a1f2611dd629f847a18186394f500d7f52753bc Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-11-30 07:46:49 +00:00
qt_internal_get_min_new_policy_cmake_version(min_new_policy_version)
qt_internal_get_max_new_policy_cmake_version(max_new_policy_version)
configure_package_config_file(
"${QT_CMAKE_DIR}/QtModuleToolsConfig.cmake.in"
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}Config.cmake"
INSTALL_DESTINATION "${config_install_dir}"
)
CMake: Record used package version for each target dependency When recording which package version to look for in QtFooModuleDependencies.cmake and other files like it, instead of using PROJECT_VERSION, use the version of the package that contains the dependency. For example if we're hypothetically building the qtdeclarative repo from the 6.4 branch, against an installed 6.2 qtbase, then the Qt6QmlModuleDependencies.cmake file will have a find_package(Qt6Core 6.2) call because qtdeclarative's find_package(Qt6Core) call found a 6.2 Core when it was configured. This allows switching the versioning scheme of specific Qt modules that might not want to follow the general Qt versioning scheme. The first candidate would be QtWebEngine which might want to follow the Chromium versioning scheme, something like Qt 6.94.0 where 94 is the Chromium major version. Implementation notes. We now record the package version of a target in a property called _qt_package_version. We do it for qt modules, plugins, 3rd party libraries, tools and the Platform target. When we try to look up which version to write into the QtFooModuleDependencies.cmake file (or the equivalent Plugins and Tools file), we try to find the version from a few sources: the property mentioned above, then the Qt6{target}_VERSION variable, and finally PROJECT_VERSION. In the latter case, we issue a warning because technically that should never have to happen, and it's a bug or an unforeseen case if it does. A few more places also need adjustments: - package versions to look for when configuring standalone tests and generating standalone tests Config files - handling of tools packages - The main Qt6 package lookup in each Dependencies.cmake files Note that there are some requirements and consequences in case a module wants to use a different versioning scheme like 6.94.0. Requirements. - The root CMakeLists.txt file needs to call find_package with a version different from the usual PROJECT_VERSION. Ideally it should look for a few different Qt versions which are known to be compatible, for example the last stable and LTS versions, or just the lowest supported Qt version, e.g. 6.2.6 or whenever this change would land in the 6.2 branch. - If the repository has multiple modules, some of which need to follow the Qt versioning scheme and some not, project(VERSION x.y.z) calls need to be carefully placed in subdirectory scopes with appropriate version numbers, so that qt_internal_add_module / _tool / _plugin pick up the correct version. Consequences. - The .so / .dylib names will contain the new version, e.g. .so.6.94 - Linux ELF symbols will contain the new versions - syncqt private headers will now exist under a include/QtFoo/6.94.0/QtFoo/private folder - pri and prl files will also contain the new version numbers - pkg-config .pc files contain the new version numbers - It won't be possible to write find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code. One would have to write find_package(Qt6WebEngineWidgets 6.94) otherwise CMake will try to look for Qt6Config 6.94 which won't exist. - Similarly, a find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call would always find any kind of WebEngine package that is higher than 6.4, which might be 6.94, 6.95, etc. - In the future, if we fix Qt6Config to pass EXACT to its subcomponent find_package calls, a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets) would fail to find WebEngineWidgets, because its 6.94.0 version will not be equal to 6.5.0. Currently we don't pass through EXACT, so it's not an issue. Augments 5ffc744b791a114a3180a425dd26e298f7399955 Task-number: QTBUG-103500 Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab Reviewed-by: Michal Klocek <michal.klocek@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-17 06:44:43 +00:00
qt_configure_file(
OUTPUT "${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}TargetsPrecheck.cmake"
CONTENT
"_qt_internal_should_include_targets(
TARGETS ${known_tools}
NAMESPACE ${INSTALL_CMAKE_NAMESPACE}::
OUT_VAR_SHOULD_SKIP __qt_${target}_skip_include_targets_file
)
")
CMake: Record used package version for each target dependency When recording which package version to look for in QtFooModuleDependencies.cmake and other files like it, instead of using PROJECT_VERSION, use the version of the package that contains the dependency. For example if we're hypothetically building the qtdeclarative repo from the 6.4 branch, against an installed 6.2 qtbase, then the Qt6QmlModuleDependencies.cmake file will have a find_package(Qt6Core 6.2) call because qtdeclarative's find_package(Qt6Core) call found a 6.2 Core when it was configured. This allows switching the versioning scheme of specific Qt modules that might not want to follow the general Qt versioning scheme. The first candidate would be QtWebEngine which might want to follow the Chromium versioning scheme, something like Qt 6.94.0 where 94 is the Chromium major version. Implementation notes. We now record the package version of a target in a property called _qt_package_version. We do it for qt modules, plugins, 3rd party libraries, tools and the Platform target. When we try to look up which version to write into the QtFooModuleDependencies.cmake file (or the equivalent Plugins and Tools file), we try to find the version from a few sources: the property mentioned above, then the Qt6{target}_VERSION variable, and finally PROJECT_VERSION. In the latter case, we issue a warning because technically that should never have to happen, and it's a bug or an unforeseen case if it does. A few more places also need adjustments: - package versions to look for when configuring standalone tests and generating standalone tests Config files - handling of tools packages - The main Qt6 package lookup in each Dependencies.cmake files Note that there are some requirements and consequences in case a module wants to use a different versioning scheme like 6.94.0. Requirements. - The root CMakeLists.txt file needs to call find_package with a version different from the usual PROJECT_VERSION. Ideally it should look for a few different Qt versions which are known to be compatible, for example the last stable and LTS versions, or just the lowest supported Qt version, e.g. 6.2.6 or whenever this change would land in the 6.2 branch. - If the repository has multiple modules, some of which need to follow the Qt versioning scheme and some not, project(VERSION x.y.z) calls need to be carefully placed in subdirectory scopes with appropriate version numbers, so that qt_internal_add_module / _tool / _plugin pick up the correct version. Consequences. - The .so / .dylib names will contain the new version, e.g. .so.6.94 - Linux ELF symbols will contain the new versions - syncqt private headers will now exist under a include/QtFoo/6.94.0/QtFoo/private folder - pri and prl files will also contain the new version numbers - pkg-config .pc files contain the new version numbers - It won't be possible to write find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code. One would have to write find_package(Qt6WebEngineWidgets 6.94) otherwise CMake will try to look for Qt6Config 6.94 which won't exist. - Similarly, a find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call would always find any kind of WebEngine package that is higher than 6.4, which might be 6.94, 6.95, etc. - In the future, if we fix Qt6Config to pass EXACT to its subcomponent find_package calls, a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets) would fail to find WebEngineWidgets, because its 6.94.0 version will not be equal to 6.5.0. Currently we don't pass through EXACT, so it's not an issue. Augments 5ffc744b791a114a3180a425dd26e298f7399955 Task-number: QTBUG-103500 Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab Reviewed-by: Michal Klocek <michal.klocek@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-17 06:44:43 +00:00
# There might be Tools packages which don't have a corresponding real module_name target, like
# WaylandScannerTools.
# In that case we'll use the package version of the first tool that belongs to that package.
if(TARGET "${module_name}")
qt_internal_get_package_version_of_target("${module_name}" tools_package_version)
elseif(first_tool_package_version)
set(tools_package_version "${first_tool_package_version}")
else()
# This should never happen, because tools_package_version should always have at least some
# value. Issue an assertion message just in case the pre-condition ever changes.
set(tools_package_version "${PROJECT_VERSION}")
if(FEATURE_developer_build)
message(WARNING
"Could not determine package version of tools package ${module_name}. "
"Defaulting to project version ${PROJECT_VERSION}.")
endif()
endif()
message(TRACE
"Exporting tools package ${module_name}Tools with package version ${tools_package_version}"
"\n included targets: ${tool_targets_non_prefixed}")
write_basic_package_version_file(
CMake: Allow disabling package version check When building Qt repos, all find_package(Qt6) calls request a PROJECT_VERSION version which is set in .cmake.conf via QT_REPO_MODULE_VERSION. This means trying to configure qtsvg from a 6.3 branch using a 6.2 qtbase won't work, because qtsvg will call find_package(Qt6 6.3) and no such Qt6 package version exists. There are certain scenarios where it might be useful to try to do that though. One of them is doing Qt development while locally mixing branches. Another is building a 6.4 QtWebEngine against a 6.2 Qt. Allow to opt out of the version check by configuring each Qt repo with -DQT_NO_PACKAGE_VERSION_CHECK=TRUE. This setting is not recorded and will have to be set again when configuring another repo. The version check will also be disabled by default when configuring with the -developer-build feature. This will be recorded and embedded into each ConfigVersion file. If the version check is disabled, a warning will be shown mentioning the incompatible version of a package that was found but that package will still be accepted. The warning will show both when building Qt or using Qt in a user project. The warnings can be disabled by passing -DQT_NO_PACKAGE_VERSION_INCOMPATIBLE_WARNING=TRUE Furthermore when building a Qt repo, another warning will show when an incompatible package version is detected, to suggest to the Qt builder whether they want to use the incompatible version by disabling the version check. Note that there are no compatibility promises when using mixed non-matching versions. Things might not work. These options are only provided for convenience and their users know what they are doing. Pick-to: 6.2 Fixes: QTBUG-96458 Change-Id: I1a42e0b2a00b73513d776d89a76102ffd9136422 Reviewed-by: Craig Scott <craig.scott@qt.io>
2021-10-22 11:38:00 +00:00
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}ConfigVersionImpl.cmake"
CMake: Record used package version for each target dependency When recording which package version to look for in QtFooModuleDependencies.cmake and other files like it, instead of using PROJECT_VERSION, use the version of the package that contains the dependency. For example if we're hypothetically building the qtdeclarative repo from the 6.4 branch, against an installed 6.2 qtbase, then the Qt6QmlModuleDependencies.cmake file will have a find_package(Qt6Core 6.2) call because qtdeclarative's find_package(Qt6Core) call found a 6.2 Core when it was configured. This allows switching the versioning scheme of specific Qt modules that might not want to follow the general Qt versioning scheme. The first candidate would be QtWebEngine which might want to follow the Chromium versioning scheme, something like Qt 6.94.0 where 94 is the Chromium major version. Implementation notes. We now record the package version of a target in a property called _qt_package_version. We do it for qt modules, plugins, 3rd party libraries, tools and the Platform target. When we try to look up which version to write into the QtFooModuleDependencies.cmake file (or the equivalent Plugins and Tools file), we try to find the version from a few sources: the property mentioned above, then the Qt6{target}_VERSION variable, and finally PROJECT_VERSION. In the latter case, we issue a warning because technically that should never have to happen, and it's a bug or an unforeseen case if it does. A few more places also need adjustments: - package versions to look for when configuring standalone tests and generating standalone tests Config files - handling of tools packages - The main Qt6 package lookup in each Dependencies.cmake files Note that there are some requirements and consequences in case a module wants to use a different versioning scheme like 6.94.0. Requirements. - The root CMakeLists.txt file needs to call find_package with a version different from the usual PROJECT_VERSION. Ideally it should look for a few different Qt versions which are known to be compatible, for example the last stable and LTS versions, or just the lowest supported Qt version, e.g. 6.2.6 or whenever this change would land in the 6.2 branch. - If the repository has multiple modules, some of which need to follow the Qt versioning scheme and some not, project(VERSION x.y.z) calls need to be carefully placed in subdirectory scopes with appropriate version numbers, so that qt_internal_add_module / _tool / _plugin pick up the correct version. Consequences. - The .so / .dylib names will contain the new version, e.g. .so.6.94 - Linux ELF symbols will contain the new versions - syncqt private headers will now exist under a include/QtFoo/6.94.0/QtFoo/private folder - pri and prl files will also contain the new version numbers - pkg-config .pc files contain the new version numbers - It won't be possible to write find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code. One would have to write find_package(Qt6WebEngineWidgets 6.94) otherwise CMake will try to look for Qt6Config 6.94 which won't exist. - Similarly, a find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call would always find any kind of WebEngine package that is higher than 6.4, which might be 6.94, 6.95, etc. - In the future, if we fix Qt6Config to pass EXACT to its subcomponent find_package calls, a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets) would fail to find WebEngineWidgets, because its 6.94.0 version will not be equal to 6.5.0. Currently we don't pass through EXACT, so it's not an issue. Augments 5ffc744b791a114a3180a425dd26e298f7399955 Task-number: QTBUG-103500 Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab Reviewed-by: Michal Klocek <michal.klocek@qt.io> Reviewed-by: Alexey Edelev <alexey.edelev@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-17 06:44:43 +00:00
VERSION "${tools_package_version}"
COMPATIBILITY AnyNewerVersion
ARCH_INDEPENDENT
)
CMake: Allow disabling package version check When building Qt repos, all find_package(Qt6) calls request a PROJECT_VERSION version which is set in .cmake.conf via QT_REPO_MODULE_VERSION. This means trying to configure qtsvg from a 6.3 branch using a 6.2 qtbase won't work, because qtsvg will call find_package(Qt6 6.3) and no such Qt6 package version exists. There are certain scenarios where it might be useful to try to do that though. One of them is doing Qt development while locally mixing branches. Another is building a 6.4 QtWebEngine against a 6.2 Qt. Allow to opt out of the version check by configuring each Qt repo with -DQT_NO_PACKAGE_VERSION_CHECK=TRUE. This setting is not recorded and will have to be set again when configuring another repo. The version check will also be disabled by default when configuring with the -developer-build feature. This will be recorded and embedded into each ConfigVersion file. If the version check is disabled, a warning will be shown mentioning the incompatible version of a package that was found but that package will still be accepted. The warning will show both when building Qt or using Qt in a user project. The warnings can be disabled by passing -DQT_NO_PACKAGE_VERSION_INCOMPATIBLE_WARNING=TRUE Furthermore when building a Qt repo, another warning will show when an incompatible package version is detected, to suggest to the Qt builder whether they want to use the incompatible version by disabling the version check. Note that there are no compatibility promises when using mixed non-matching versions. Things might not work. These options are only provided for convenience and their users know what they are doing. Pick-to: 6.2 Fixes: QTBUG-96458 Change-Id: I1a42e0b2a00b73513d776d89a76102ffd9136422 Reviewed-by: Craig Scott <craig.scott@qt.io>
2021-10-22 11:38:00 +00:00
qt_internal_write_qt_package_version_file(
"${INSTALL_CMAKE_NAMESPACE}${target}"
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}ConfigVersion.cmake"
)
qt_install(FILES
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}Config.cmake"
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}ConfigVersion.cmake"
CMake: Allow disabling package version check When building Qt repos, all find_package(Qt6) calls request a PROJECT_VERSION version which is set in .cmake.conf via QT_REPO_MODULE_VERSION. This means trying to configure qtsvg from a 6.3 branch using a 6.2 qtbase won't work, because qtsvg will call find_package(Qt6 6.3) and no such Qt6 package version exists. There are certain scenarios where it might be useful to try to do that though. One of them is doing Qt development while locally mixing branches. Another is building a 6.4 QtWebEngine against a 6.2 Qt. Allow to opt out of the version check by configuring each Qt repo with -DQT_NO_PACKAGE_VERSION_CHECK=TRUE. This setting is not recorded and will have to be set again when configuring another repo. The version check will also be disabled by default when configuring with the -developer-build feature. This will be recorded and embedded into each ConfigVersion file. If the version check is disabled, a warning will be shown mentioning the incompatible version of a package that was found but that package will still be accepted. The warning will show both when building Qt or using Qt in a user project. The warnings can be disabled by passing -DQT_NO_PACKAGE_VERSION_INCOMPATIBLE_WARNING=TRUE Furthermore when building a Qt repo, another warning will show when an incompatible package version is detected, to suggest to the Qt builder whether they want to use the incompatible version by disabling the version check. Note that there are no compatibility promises when using mixed non-matching versions. Things might not work. These options are only provided for convenience and their users know what they are doing. Pick-to: 6.2 Fixes: QTBUG-96458 Change-Id: I1a42e0b2a00b73513d776d89a76102ffd9136422 Reviewed-by: Craig Scott <craig.scott@qt.io>
2021-10-22 11:38:00 +00:00
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}ConfigVersionImpl.cmake"
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}TargetsPrecheck.cmake"
DESTINATION "${config_install_dir}"
COMPONENT Devel
)
set(export_name "${INSTALL_CMAKE_NAMESPACE}${target}Targets")
qt_install(EXPORT "${export_name}"
NAMESPACE "${QT_CMAKE_EXPORT_NAMESPACE}::"
DESTINATION "${config_install_dir}")
qt_internal_export_additional_targets_file(
TARGETS ${QT_KNOWN_MODULE_${module_name}_TOOLS}
TARGET_EXPORT_NAMES ${tool_targets}
EXPORT_NAME_PREFIX ${INSTALL_CMAKE_NAMESPACE}${target}
CONFIG_INSTALL_DIR "${config_install_dir}")
# Create versionless targets file.
configure_file(
"${QT_CMAKE_DIR}/QtModuleToolsVersionlessTargets.cmake.in"
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}VersionlessTargets.cmake"
@ONLY
)
qt_install(FILES
"${config_build_dir}/${INSTALL_CMAKE_NAMESPACE}${target}VersionlessTargets.cmake"
DESTINATION "${config_install_dir}"
COMPONENT Devel
)
endfunction()
# Returns the target name for the tool with the given name.
#
# In most cases, the target name is the same as the tool name.
# If the user specifies to build tools when cross-compiling, then the
# suffix "_native" is appended.
function(qt_get_tool_target_name out_var name)
if (QT_WILL_RENAME_TOOL_TARGETS)
set(${out_var} ${name}_native PARENT_SCOPE)
else()
set(${out_var} ${name} PARENT_SCOPE)
endif()
endfunction()
# Returns the tool name for a given tool target.
# This is the inverse of qt_get_tool_target_name.
function(qt_tool_target_to_name out_var target)
set(name ${target})
if (QT_WILL_RENAME_TOOL_TARGETS)
string(REGEX REPLACE "_native$" "" name ${target})
endif()
set(${out_var} ${name} PARENT_SCOPE)
endfunction()
# Sets QT_WILL_BUILD_TOOLS if tools will be built and QT_WILL_RENAME_TOOL_TARGETS
# if those tools have replaced naming.
function(qt_check_if_tools_will_be_built)
# By default, we build our own tools unless we're cross-building or QT_HOST_PATH is set.
set(need_target_rename FALSE)
set(require_find_tools FALSE)
if(CMAKE_CROSSCOMPILING)
set(will_build_tools FALSE)
if(QT_FORCE_BUILD_TOOLS)
set(will_build_tools TRUE)
set(need_target_rename TRUE)
endif()
set(require_find_tools TRUE)
else()
if(QT_HOST_PATH)
set(will_build_tools FALSE)
else()
set(will_build_tools TRUE)
endif()
if(QT_FORCE_FIND_TOOLS)
set(will_build_tools FALSE)
set(require_find_tools TRUE)
endif()
if(QT_FORCE_BUILD_TOOLS)
set(will_build_tools TRUE)
set(need_target_rename TRUE)
endif()
endif()
set_property(GLOBAL PROPERTY qt_require_find_tools "${require_find_tools}")
set(QT_WILL_BUILD_TOOLS ${will_build_tools} CACHE INTERNAL "Are tools going to be built" FORCE)
set(QT_WILL_RENAME_TOOL_TARGETS ${need_target_rename} CACHE INTERNAL
"Do tool targets need to be renamed" FORCE)
endfunction()
# Use this macro to exit a file or function scope unless we're building tools. This is supposed to
# be called after qt_internal_add_tools() to avoid special-casing operations on imported targets.
macro(qt_internal_return_unless_building_tools)
if(NOT QT_WILL_BUILD_TOOLS)
return()
endif()
endmacro()
# Equivalent of qmake's qtNomakeTools(directory1 directory2).
# If QT_BUILD_TOOLS_BY_DEFAULT is true, then targets within the given directories will be excluded
# from the default 'all' target, as well as from install phase. The private variable is checked by
# qt_internal_add_executable.
function(qt_exclude_tool_directories_from_default_target)
if(NOT QT_BUILD_TOOLS_BY_DEFAULT)
set(absolute_path_directories "")
foreach(directory ${ARGV})
list(APPEND absolute_path_directories "${CMAKE_CURRENT_SOURCE_DIR}/${directory}")
endforeach()
set(__qt_exclude_tool_directories "${absolute_path_directories}" PARENT_SCOPE)
endif()
endfunction()
function(qt_internal_find_tool out_var target_name tools_target)
qt_tool_target_to_name(name ${target_name})
# Handle case when a tool does not belong to a module and it can't be built either (like
# during a cross-compile).
if(NOT tools_target AND NOT QT_WILL_BUILD_TOOLS)
message(FATAL_ERROR "The tool \"${name}\" has not been assigned to a module via"
" TOOLS_TARGET (so it can't be found) and it can't be built"
" (QT_WILL_BUILD_TOOLS is ${QT_WILL_BUILD_TOOLS}).")
endif()
if(NOT CMAKE_CROSSCOMPILING)
if(QT_INTERNAL_FORCE_FIND_HOST_TOOLS_MODULE_LIST AND
NOT "${tools_target}" IN_LIST QT_INTERNAL_FORCE_FIND_HOST_TOOLS_MODULE_LIST)
message(STATUS "Tool '${full_name}' will be built from source.")
set(${out_var} "TRUE" PARENT_SCOPE)
return()
endif()
endif()
if(QT_WILL_RENAME_TOOL_TARGETS AND (name STREQUAL target_name))
message(FATAL_ERROR
"qt_internal_add_tool must be passed a target obtained from qt_get_tool_target_name.")
endif()
set(full_name "${QT_CMAKE_EXPORT_NAMESPACE}::${name}")
set(imported_tool_target_already_found FALSE)
# This condition can only be TRUE if a previous find_package(Qt6${tools_target}Tools)
# was already done. That can happen if QT_FORCE_FIND_TOOLS was ON or we're cross-compiling.
# In such a case, we need to exit early if we're not going to also build the tools.
if(TARGET ${full_name})
get_property(path TARGET ${full_name} PROPERTY LOCATION)
message(STATUS "Tool '${full_name}' was found at ${path}.")
set(imported_tool_target_already_found TRUE)
if(NOT QT_WILL_BUILD_TOOLS)
set(${out_var} "FALSE" PARENT_SCOPE)
return()
endif()
endif()
# We need to search for the host Tools package when doing a cross-build
# or when QT_FORCE_FIND_TOOLS is ON.
# As an optimiziation, we don't search for the package one more time if the target
# was already brought into scope from a previous find_package.
set(search_for_host_package FALSE)
if(NOT QT_WILL_BUILD_TOOLS OR QT_WILL_RENAME_TOOL_TARGETS)
set(search_for_host_package TRUE)
endif()
if(search_for_host_package AND NOT imported_tool_target_already_found)
set(tools_package_name "${INSTALL_CMAKE_NAMESPACE}${tools_target}Tools")
message(STATUS "Searching for tool '${full_name}' in package ${tools_package_name}.")
# TODO: Remove these once developers have reconfigured their project.
# Create the tool targets, even if QT_NO_CREATE_TARGETS is set.
# Otherwise targets like Qt6::moc are not available in a top-level cross-build.
set(BACKUP_QT_NO_CREATE_TARGETS ${QT_NO_CREATE_TARGETS})
set(QT_NO_CREATE_TARGETS OFF)
# When cross-compiling, we want to search for Tools packages in QT_HOST_PATH.
# To do that, we override CMAKE_PREFIX_PATH and CMAKE_FIND_ROOT_PATH.
#
# We don't use find_package + PATHS option because any recursive find_dependency call
# inside a Tools package would not inherit the initial PATHS value given.
# TODO: Potentially we could set a global __qt_cmake_host_dir var like we currently
# do with _qt_cmake_dir in Qt6Config and change all our host tool find_package calls
# everywhere to specify that var in PATHS.
#
# Note though that due to path rerooting issue in
# https://gitlab.kitware.com/cmake/cmake/-/issues/21937
# we have to append a lib/cmake suffix to CMAKE_PREFIX_PATH so the value does not get
# rerooted on top of CMAKE_FIND_ROOT_PATH.
# Use QT_HOST_PATH_CMAKE_DIR for the suffix when available (it would be set by
# the qt.toolchain.cmake file when building other repos or given by the user when
# configuring qtbase) or derive it from from the Qt6HostInfo package which is
# found in QtSetup.
set(${tools_package_name}_BACKUP_CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH})
set(${tools_package_name}_BACKUP_CMAKE_FIND_ROOT_PATH "${CMAKE_FIND_ROOT_PATH}")
if(QT_HOST_PATH_CMAKE_DIR)
set(qt_host_path_cmake_dir_absolute "${QT_HOST_PATH_CMAKE_DIR}")
elseif(${INSTALL_CMAKE_NAMESPACE}HostInfo_DIR)
get_filename_component(qt_host_path_cmake_dir_absolute
"${${INSTALL_CMAKE_NAMESPACE}HostInfo_DIR}/.." ABSOLUTE)
else()
# This should never happen, serves as an assert.
message(FATAL_ERROR
"Neither QT_HOST_PATH_CMAKE_DIR nor "
"${INSTALL_CMAKE_NAMESPACE}HostInfo_DIR available.")
endif()
set(CMAKE_PREFIX_PATH "${qt_host_path_cmake_dir_absolute}")
# Look for tools in additional host Qt installations. This is done for conan support where
# we have separate installation prefixes per package. For simplicity, we assume here that
# all host Qt installations use the same value of INSTALL_LIBDIR.
if(DEFINED QT_ADDITIONAL_HOST_PACKAGES_PREFIX_PATH)
file(RELATIVE_PATH rel_host_cmake_dir "${QT_HOST_PATH}"
"${qt_host_path_cmake_dir_absolute}")
foreach(host_path IN LISTS QT_ADDITIONAL_HOST_PACKAGES_PREFIX_PATH)
set(host_cmake_dir "${host_path}/${rel_host_cmake_dir}")
list(PREPEND CMAKE_PREFIX_PATH "${host_cmake_dir}")
endforeach()
list(PREPEND CMAKE_FIND_ROOT_PATH "${QT_ADDITIONAL_HOST_PACKAGES_PREFIX_PATH}")
endif()
list(PREPEND CMAKE_FIND_ROOT_PATH "${QT_HOST_PATH}")
find_package(
${tools_package_name}
${PROJECT_VERSION}
NO_PACKAGE_ROOT_PATH
NO_CMAKE_ENVIRONMENT_PATH
NO_SYSTEM_ENVIRONMENT_PATH
NO_CMAKE_PACKAGE_REGISTRY
NO_CMAKE_SYSTEM_PATH
NO_CMAKE_SYSTEM_PACKAGE_REGISTRY)
# Restore backups.
set(CMAKE_FIND_ROOT_PATH "${${tools_package_name}_BACKUP_CMAKE_FIND_ROOT_PATH}")
set(CMAKE_PREFIX_PATH "${${tools_package_name}_BACKUP_CMAKE_PREFIX_PATH}")
# TODO: Remove these once developers have reconfigured their project.
set(QT_NO_CREATE_TARGETS ${BACKUP_QT_NO_CREATE_TARGETS})
if(${${tools_package_name}_FOUND} AND TARGET ${full_name})
# Even if the tool is already visible, make sure that our modules remain associated
# with the tools.
qt_internal_append_known_modules_with_tools("${tools_target}")
get_property(path TARGET ${full_name} PROPERTY LOCATION)
message(STATUS "${full_name} was found at ${path} using package ${tools_package_name}.")
if (NOT QT_FORCE_BUILD_TOOLS)
set(${out_var} "FALSE" PARENT_SCOPE)
return()
endif()
endif()
endif()
get_property(require_find_tools GLOBAL PROPERTY qt_require_find_tools)
if(require_find_tools AND NOT TARGET ${full_name})
if(${${tools_package_name}_FOUND})
set(pkg_found_msg "")
string(APPEND pkg_found_msg
"the ${tools_package_name} package, but the package did not contain the tool. "
"Make sure that the host module ${tools_target} was built with all features "
"enabled (no explicitly disabled tools).")
else()
set(pkg_found_msg "")
string(APPEND pkg_found_msg
"the ${tools_package_name} package, but the package could not be found. "
"Make sure you have built and installed the host ${tools_target} module, "
"which will ensure the creation of the ${tools_package_name} package.")
endif()
message(FATAL_ERROR
"Failed to find the host tool \"${full_name}\". It is part of "
${pkg_found_msg})
endif()
if(QT_WILL_BUILD_TOOLS)
message(STATUS "Tool '${full_name}' will be built from source.")
endif()
set(${out_var} "TRUE" PARENT_SCOPE)
endfunction()
# This function adds an internal tool that should be compiled at configure time.
# TOOLS_TARGET
# Specifies the module this tool belongs to. The Qt6${TOOLS_TARGET}Tools module
# will then expose targets for this tool. Ignored if NO_INSTALL is set.
function(qt_internal_add_configure_time_tool target_name)
set(one_value_args INSTALL_DIRECTORY TOOLS_TARGET CONFIG)
set(multi_value_args)
set(option_args NO_INSTALL)
cmake_parse_arguments(PARSE_ARGV 1 arg
"${option_args}" "${one_value_args}" "${multi_value_args}")
qt_internal_find_tool(will_build_tools ${target_name} "${arg_TOOLS_TARGET}")
if(NOT will_build_tools)
return()
endif()
qt_tool_target_to_name(name ${target_name})
set(extra_args "")
if(arg_NO_INSTALL OR NOT arg_TOOLS_TARGET)
list(APPEND extra_args "NO_INSTALL")
else()
set(install_dir "${INSTALL_BINDIR}")
if(arg_INSTALL_DIRECTORY)
set(install_dir "${arg_INSTALL_DIRECTORY}")
endif()
set(extra_args "INSTALL_DIRECTORY" "${install_dir}")
endif()
if(arg_CONFIG)
set(tool_config "${arg_CONFIG}")
elseif(QT_MULTI_CONFIG_FIRST_CONFIG)
set(tool_config "${arg_QT_MULTI_CONFIG_FIRST_CONFIG}")
else()
set(tool_config "${CMAKE_BUILD_TYPE}")
endif()
Replace the syncqt.pl script with syncqt tool syncqt.pl adds an extra dependency on perl when building Qt. Modern C++ provides the convenient cross-platform way to access a filesystem and to use regular expressions, so we may replace the perl script with C++ application. The syncqt executable is built at configure time and installed as QtCore tool. It's running at configure time to deliver the required header files for IDE to build a consistent code model and at the build time to keep tracking changes in header files and generate the missing aliases without reconfiguring. 'syncqt' only parses header files from a CMake build tree, so the resulting Qt installation only contains interfacing headers that belong to the platform that Qt is built for. 'sync.profile' files are not used as the 'source of truth' for sync qt procedure anymore, all the necessary information is taken from either CMake files at configure time or from the module header files while parsing them. syncqt.pl is still in place since it's required as fallback solution for a smooth transition to the new syncqt implementation for all qt repositories. This patchset only enables the C++ based syncqt for 'qtbase' repository. From the performance perspective C++ version works faster then perl script, also the configure time is reduced significally on subsequent reconfigurations - up x2 times faster when re-configuring repository, but it also takes time to compile the tool itself the first time. Numbers for qtbase: syncqt.pl syncqt.cpp initial: 0m16,035s 0m20,413s reconfig: 0m6,819s 0m3,725s The syncing procedure can be run separately for each module using <ModuleName>_sync_headers targets. The 'sync_headers' target can be used to sync all the modules at once. Task-number: QTBUG-87480 Task-number: QTBUG-103196 Change-Id: I8c938bcaf88a8713b39bbfd66d9e7ef12b2c3523 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-08-15 16:29:41 +00:00
string(REPLACE "\\\;" "\\\\\\\;" unparsed_arguments "${arg_UNPARSED_ARGUMENTS}")
qt_internal_add_configure_time_executable(${target_name}
OUTPUT_NAME ${name}
CONFIG ${tool_config}
${extra_args}
Replace the syncqt.pl script with syncqt tool syncqt.pl adds an extra dependency on perl when building Qt. Modern C++ provides the convenient cross-platform way to access a filesystem and to use regular expressions, so we may replace the perl script with C++ application. The syncqt executable is built at configure time and installed as QtCore tool. It's running at configure time to deliver the required header files for IDE to build a consistent code model and at the build time to keep tracking changes in header files and generate the missing aliases without reconfiguring. 'syncqt' only parses header files from a CMake build tree, so the resulting Qt installation only contains interfacing headers that belong to the platform that Qt is built for. 'sync.profile' files are not used as the 'source of truth' for sync qt procedure anymore, all the necessary information is taken from either CMake files at configure time or from the module header files while parsing them. syncqt.pl is still in place since it's required as fallback solution for a smooth transition to the new syncqt implementation for all qt repositories. This patchset only enables the C++ based syncqt for 'qtbase' repository. From the performance perspective C++ version works faster then perl script, also the configure time is reduced significally on subsequent reconfigurations - up x2 times faster when re-configuring repository, but it also takes time to compile the tool itself the first time. Numbers for qtbase: syncqt.pl syncqt.cpp initial: 0m16,035s 0m20,413s reconfig: 0m6,819s 0m3,725s The syncing procedure can be run separately for each module using <ModuleName>_sync_headers targets. The 'sync_headers' target can be used to sync all the modules at once. Task-number: QTBUG-87480 Task-number: QTBUG-103196 Change-Id: I8c938bcaf88a8713b39bbfd66d9e7ef12b2c3523 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-08-15 16:29:41 +00:00
${unparsed_arguments}
)
if(TARGET host_tools)
add_dependencies(host_tools "${target_name}_build")
endif()
if(NOT arg_NO_INSTALL AND arg_TOOLS_TARGET)
qt_internal_add_targets_to_additional_targets_export_file(
TARGETS ${target_name}
TARGET_EXPORT_NAMES ${QT_CMAKE_EXPORT_NAMESPACE}::${name}
EXPORT_NAME_PREFIX ${INSTALL_CMAKE_NAMESPACE}${arg_TOOLS_TARGET}Tools
)
endif()
endfunction()