Add a RunCMake test that exercises some of the basic SBOM API
as it would be used in a user project.
The 'minimal' case specifies the minimum arguments required to create
an SBOM project, whereas the 'full' case specifies as many options as
possible to _qt_internal_sbom_begin_project().
Add various target types and custom files.
Pick-to: 6.8 6.9 6.10
Change-Id: Idcf9a3c7a06ae7a632bce256f009f9c7e217ed30
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
These tell Coin the repos that the commands will require. Otherwise
there is possibility that those repositories are out of date.
Pick-to: 6.10 6.9 6.8
Change-Id: Ie61b5faa6f321b666f3f6a829ba4d333cb3eb6f1
Reviewed-by: Jarek Kobus <jaroslaw.kobus@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The feature allows controlling which toolchain should be used when
building tests. Since WoA supposed to build tests using the
crosscompiled Qt in native host environment, this option allows
preparing the correct build environment and use the crosscompiled
'target' Qt.
Drive-by change the way prefix.bat is called for native WoA configuration.
Pick-to: 6.10
Change-Id: Ifec7593b4216ee2e2ffe185ae0dc0559d501762b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
make and mingw32-make are not parallel by default, which can lead to
long build times. Make sure to call them with the -j option with the
amount of cores the VM has available.
Pick-to: 6.8 6.9 6.10
Task-number: QTBUG-139139
Change-Id: I654b91f117745af2db8aa250a08bfc441413b7b2
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Test a VMs network before building.
Don't build if the network is down.
Task-number: QTQAINFRA-6796
Change-Id: Ic39fc135e6a167ffc83041a2e30959f4eb858871
Reviewed-by: Toni Saario <toni.saario@qt.io>
Test a VMs network before building.
Don't build if the network is down.
Task-number: QTQAINFRA-6796
Change-Id: Ie6b35459f7c5596f0a9421d7023aea146c425b4c
Reviewed-by: Toni Saario <toni.saario@qt.io>
Since tests now use vcpkg for dependency lookup, we need to highjack
the CMAKE_MAKE_PROGRAM the same way we do this for Qt module builds.
Amends 3a629595eb
Change-Id: I4427257e9c44bf2abd971f662bf136673364081e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Set vcpkg-related variables when cross-compiling tests.
Amends 8ffa60e171
Change-Id: I557d7cbe9883977e51390b7ef11e75fd381f5bc7
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This works around the issue that we have when using vcpkg. It's
internal logic adds system root to CMAKE_PREFIX_PATH when
crosscompiling. As result ninja executable that is located by PATH
is ignored and the system one is used. There is no way to disable
this behavior from vcpkg side, so we need to set the CMAKE_MAKE_PROGRAM
variable explicitly.
Change-Id: I18fb41a525d6ddc70b0e8df12426146f2051e904
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If the USE_VCPKG environment variable is set to ON when evaluating
module build template it enabled the use of VCPKG for this module.
Change-Id: If418bb483f8366cf18aca4870497fa7840656c58
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reverting because it is being flaky on Windows CI runners and needs more
investigation
This reverts commit 30cbc27c37.
Change-Id: I57bd08c659c78367389785e0d1181ac7656c44aa
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
In principle it has the same value as `INSTALL_DIR_FOR_CMAKE_TESTS`, but
this is defined much earlier and guaranteed to be present next to
`HOST_INSTALL_DIR`.
Change-Id: I0c6d2109735872fc6013d91163352698d7b7bc08
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
tst_qxmlstream::initTestCase copies many folders, this takes time
for Intel it takes more than 600s
Task-number: QTBUG-137562
Change-Id: I1ab8dbbaa344a579e8bf4393eb9f71d2a02e079d
Reviewed-by: Toni Saario <toni.saario@qt.io>
Build static doc tools first, because after aftifacts builds
dependency builds start and qtbase continues building its tests
and some modules already needs tools before qtbase has built its
test artifacts, which causes them to fail.
Task-number: COIN-1247
Pick-to: 6.10 6.9 6.8
Change-Id: If7685e94033371a80422f99e5f3aaa32df63a3ec
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Jukka Jokiniva <jukka.jokiniva@qt.io>
Note this change has no impact on it own on current integrations,
as it barely adds tests compilation.
It is required for qtwebengine on WoA story.
Do not run some tests:
(required for qt5 patch introducing woa test node)
* cmake test as host tools are provided by mingw,
but tests are run natively by WoA platform.
* dbus as this does not run natively on Woa anyway
due to missing dbus service.
Change-Id: Ifdb3e1b76a7feb906a9031815984c716618e3882
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Running many tests on the VxWorks in the emulator without
restarting it between can cause exhaustion of some
resources and cause an RTP spawn fail in some tests.
This change will restart the emulator in such cases, which
allows the test to try again on fresh boot.
Pick-to: 6.9
Change-Id: I5ad715f243867d43b055271d6bc61ca4ccb3089b
Reviewed-by: Simo Fält <simo.falt@qt.io>
Reviewed-by: Michał Łoś <michal.los@siili.com>
Cloning the git sources will:
- allow embedding the git sha1 into the SBOM generated by the build
system
- ease development process in CI VMs when investigating issues.
This will slightly slow down the CI build phase by a minute or two.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTQAINFRA-6520
Change-Id: If554ed0fedca26ff77d04752288ee8497bc4b9ce
Reviewed-by: Toni Saario <toni.saario@qt.io>
This includes:
- turning VERIFY_SOURCE_SBOM ON
- adding exception to the licenseRule.json files
- correcting the licensing given via REUSE.toml files
- renaming license files not located in LICENSES folder.
They need to be named LICENSE. to be ignored by reuse and
excluded from the source SBOM. The name are updated in the
corresponding qt_attribution.json
A lot of files are skipped during the license test,
but all are present in the source SBOM.
This is why correction are needed before turning the
source SBOM check on.
[ChangeLog][Third-Party Code] Renaming the license files with prefix
LICENSE. to have them ignored by reuse tool.
Task-number: QTBUG-131434
Pick-to: 6.9
Change-Id: Iab517215bb10a17357d2d2436bba8d3af76e5cd1
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The license check is only called if VERIFY_SOURCE_SBOM is set to ON
in the module_config.yaml. This way, a module can turn on the check
only when ready.
The license check is done via a perl script. Since no perl is
available for Windows on ARM64, VERIFY_SOURCE_SBOM is set to OFF
in this case.
Task-number: QTBUG-131434
Pick-to: 6.9
Change-Id: I183b7726928bbc925940330d39779a4ac2307254
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Axivion analysis for qt5 is currently ran only for linux and
qtactiveqt started to fail because it now uploads aftifacts also
for linux and is not skipped anymore. Adding condition to prevent
running Axvion analysis for qtactiveqt.
Task-number: QTQAINFRA-6752
Change-Id: I2ce141a1b7c6de387493651c4429edac65e49506
Reviewed-by: Toni Saario <toni.saario@qt.io>
Unless a module overrides it by setting the environment variable
COIN_CTEST_FORCE_IGNORE_EXIT_CODE.
As this change trickles through submodule updates we'll mark modules
as COIN_CTEST_FORCE_IGNORE_EXIT_CODE if needed.
Change-Id: Ia37d6e4c4dee41a3dd7446fa23ab96593dab9402
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If COIN_CTEST_IGNORE_EXIT_CODE is in effect and ctest aborts for some
reason (for example due to the workitem being cancelled) the ctest exit
code will be ignored, as expected, but we will not have a LastTest.log
to copy, which will fail the workitem unexpectedly.
We now match the ignoreExitCode of the LastTest.log copy with the ctest
run.
Pick-to: 6.9
Change-Id: I9df0f863a42dd4cf25cee1694e85cb32058a4e5b
Reviewed-by: Toni Saario <toni.saario@qt.io>
Add crash handling to avoid waiting until 10min timeout on each crash.
Force restart on each crash by killing emulator.
Make health check more robust to allow it to work even when emulator is
down. This is done via wait in separate process which cannot be done
with normal timeout command, as input to pipe that is not being read
by emulator will block forever.
Pick-to: 6.9
Change-Id: I86c3c86f936cc96d57b38983da48d1d73162399d
Reviewed-by: Simo Fält <simo.falt@qt.io>
Increases to around 16MB with the format change.
Required by some tests.
Pick-to: 6.9
Change-Id: Ia41436b4269220f84271b614ea6f2b96ca605c32
Reviewed-by: Simo Fält <simo.falt@qt.io>
Startup the correct emulator for each arch.
Add a health check for emulator during test start. If emulator
does not respond, restart it. Occurs in declarative tests where
tests hang and crash. After the crashes the pipes also need to be
reset, only emulator reset is not sufficient.
Read variables from WIND_CC_SYSROOT which is the VSB path, so
correct VSB is used.
Change-Id: If918ee45d4fa3a452c484a48fe9507f12f788e86
Reviewed-by: Simo Fält <simo.falt@qt.io>
This reverts commit 8d5e7d50d8.
Reason for revert: Arm packages should be sign-able now.
Pick-to: 6.8
Change-Id: I6a896bfd84927da9c15647d207a6ad5024a27949
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
Backstory.
The current approach for building documentation in the CI is that we
build a static qdoc in a Jenkins job, manually provision it in
qt5.git, and then build qt documentation using that somewhat
old qdoc version.
Updating the provisioned qdoc is a very slow process, because changes
first need to be merged into qttools, then a Jenkins job needs to be
manually run, then provisioning needs to be updated, and sometimes it
also depends on a full qt5.git submodule update or has to wait for
other unrelated changes to qt5.git.
The new approach.
This change adds CI instructions to build a static qdoc (and
other relevant tools necessary for documentation generation)
using the latest qttools/dev/HEAD sha1 in a specific host platform's
qtbase build instructions, and then uses those tools for documentation
generation and doc warning checks in each repo's TestDocs CI test
phase.
To enable these new instructions, the host platform and the doc
checking platform need to be tagged with the 'DocsGenerationV2'
feature.
The built tools are cached in a separate archive using a new Coin
feature, which is then extracted during each repo's TestDocs phase.
Pros and cons.
The benefit of this new approach is that we will always use a mostly
up-to-date qdoc from qttools/dev/HEAD when generating documentation,
making the documentation team's life easier.
Specifically, once the doc tools are built in qtbase with the latest
qttools/dev/HEAD sha1 that was available during that integration, the
resulting built tools are then used for doc checks in dependent repos.
A newer qttools/dev/HEAD sha1 will only be used for a particular
repo once the qtbase dependencies.yaml sha1 is updated, or when doing
qt5.git changes that cause a rebuild of qtbase.
Note that while new tools are built due to dependency updates, the
used qttools sha1 is not recorded in any way in any of the
dependencies.yaml files.
qtbase doc checks will always use the latest qttools/dev/HEAD sha1 at
the integration time, because the tools will always be built from
scratch.
The downside of the new approach is that, by default, the doc
generation will not use a pinned version of qdoc anymore,
but rather follow the flow described above. This has a chance of
introducing failures which are unrelated to integrating changes.
This should happen rarely because the doc team usually tries to fix
doc warnings before changing qdoc's code.
If temporary pinning of qttools or qt5.git is required when testing the
docs of a specific repo, it can be achieved as follows:
- set the QT_CI_BUILD_REPO_DOC_TOOLS env var to "1" in the repo
coin/module_config.yaml file build instructions, to ensure the doc
tools are built in that repo, rather than reuse the ones built in
qtbase
- set the QT_CI_DOC_TOOLS_PIN_GIT_REF env var to the desired qttools
sha1 or other git ref, in the same build instructions phase
- optionally set the QT_CI_DOC_TOOLS_TOP_LEVEL_PIN_GIT_REF env var to
the desired qt5.git sha1 or other git ref, in the same build
instructions phase, if a different version of qt5.git is required
- optionally set the QT_CI_DOC_TOOLS_USE_CI_TOP_LEVEL_BRANCH env var
to ON, in case if the module branch that the CI job computes should
be used as the qt5.git branch
- set the QT_CI_FETCH_REPO_DOC_TOOLS env var to "1" in the repo test
instructions, to ensure the just built doc tools are fetched instead
of the qtbase-built ones
Test running this new approach will allow us to collect some feedback
on how often breakages happen, and how much easier it makes the
documentation team's work, or how much harder it makes the life of
regular integrations.
The abundant amount of optional pinning options should be enough to
avoid any permanent integration dead locks.
Pick-to: 6.8
Task-number: QTBUG-128730
Change-Id: I8606cb3076036a4a0ec652d0fa74d270e8f5dfdf
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Add a new coin_test_docs_common.yaml file that contains the common
instructions for configuring a documentation warning build.
Pick-to: 6.8
Task-number: QTBUG-128730
Change-Id: Ifc86570f3c98e64364f3c2c2bb71f45eaaee0c19
Reviewed-by: Toni Saario <toni.saario@qt.io>
This makes tqtc/ branches to use the public branches.
Amends 2a0e89981a
Pick-to: 6.8
Task-number: QTBUG-124453
Task-number: QTBUG-125211
Task-number: QTBUG-125569
Task-number: QTQAINFRA-3935
Change-Id: I1098165c3122befabc503d38c46d476920d835a2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
Coin can now dynamically refer to the current branch of the repo being
integrated in the module instructions.
Change the license checking instructions to use the current branch
of the qtqa repo, rather than the dev branch.
This makes the behavior consistent with what everyone expects, instead
of using a hardcoded qtqa branch for all stable branches.
Amends 2a0e89981a
Pick-to: 6.8
Task-number: COIN-573
Task-number: QTQAINFRA-3935
Task-number: QTQAINFRA-3878
Task-number: QTQAINFRA-4975
Change-Id: I6575c2868f3a7170570803f620feb5723fbbe6df
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The Qt default has changed to always generate an SBOM for all
platforms in the CI, so we always want to pass the location of the
python interpreter meant for SBOM processing to configure.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I1042c97b9133ca37e42eb3efd1689ec1f9f9ab31
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
It was removed in a previous change and does nothing now.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I7a84c243c08a65e445cbd657eb281b37e68f1893
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
If the tests fail we still want to upload the full CTest log.
Change-Id: Ib1c95d2b0b3498b05e335aeeb487190a747d162e
Reviewed-by: Toni Saario <toni.saario@qt.io>
This environment variable might be set by some party (not quite sure
what, maybe the host OS), and when androidtestrunner passes QT_* vars
it might interfere with the input tests.
Pick-to: 6.8
Change-Id: I2a88c78535d6e3a809e8fcee8f8d752f91044a8c
Reviewed-by: Toni Saario <toni.saario@qt.io>
Now that the ctest is not run as verbose the logging is significantly
decreased and VxWorks will hit timeout on some tests.
There are plenty of other timeouts that will be hit instead
before the total timeout specified for the command e.g.
- Ctest timeout 25min default per test case.
- Testlib timeout per test function 10min default.
Change-Id: Ic02e812d4b059609553d2ac8d44ec91f358d76c5
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
This stops confusing the 'reuse' tool when trying to generate a source
SBOM. The tool considers all files in the source dir as source files,
and because we placed the build dir inside the source dir, all build
files were considered source files. This caused errors saying the
files don't have copyright headers.
Change the build directory for cross-compilation jobs to be outside of
the source directory, similar how we do it for the native builds.
Augments ae62d908a5
Pick-to: 6.8
Task-number: QTBUG-82820
Task-number: QTBUG-96513
Task-number: QTBUG-122899
Task-number: QTBUG-124453
Task-number: QTBUG-125211
Change-Id: Ie82a6c55aa5326f7e9cc5c0e09342f3405e78d17
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
We don't build a host Qt during a cross-compilation build for a few
years now. Instead we reuse the prebuilt host from the platform
dependencies. This was left over to allow gradual porting to the new
way of using platform dependencies, which is now long done.
Remove the code to ease maintenance.
Amends 684fea1314
Pick-to: 6.8
Change-Id: I9988da6f33f9d9b98d304316317d39c3dfb7c633
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
It's mostly redundant, as ctest already prints timing information
for each individual test:
1/1 Test #572: tst_qwindowcontainer ............. Passed 0.71 sec
And we track test timing though the JUnit and Qt XML files we produce
and ship to our Grafana dashboard.
Change-Id: I374d0b2201efabccec18da6358e3859b02702c27
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
The test output of passing tests is most of the time unnecessary
noise when investigating a test failure in a failing test. With
the size of the qtbase log being 25MB uncompressed, working with
these logs locally or in a (mobile) browser is also painful.
We now disable verbose test output in ctest, and only output test
output if the test fails. We still run our tests with -o foo.txt,txt
and these files are uploaded to coin's artifact storage, so the
full logs are available if closer inspections are needed. We
also store CMake's LastTest.log, which include verbose output
of all non-testlib tests.
The verboseness of the coin config can also be enabled via the
VerboseCTestOutput feature, either for specific configs, or
on demand when scheduling a check via coin.
Fixes: QTQAINFRA-6592
Fixes: QTBUG-109441
Change-Id: I5cdfb8839190b1598f12b7f9ea1d9a242a32a790
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Daniel Smith <daniel.smith@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
The ASAN configuration runs on OpenSUSE and has the variable
CMAKE_MIN_SUPPORTED_BIN_PATH=/opt/cmake-3.16.8/bin
because in the non-ASAN configuration it prepends this to PATH to test
functionality with old CMake.
In the ASAN configuration the above variable is still set, but it's not
prepended to PATH, so the latest CMake is used. We now detect that
properly and enable junit XML summary files.
Change-Id: I75f7308b9332434853cbe2a9878f7b4c192710e7
Reviewed-by: Toni Saario <toni.saario@qt.io>
Match to Windows target OS and different target arch, which tells that
it is a cross-compilation.
Pick-to: 6.8
Change-Id: Iba2d087cda501cd2622a1bd8a8732100b920956b
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
So that we don't rely on building/running Qt auto tests just to
build the docs.
Change-Id: If3939717c57e8d4074afe0d611a1d04e65da7d36
Reviewed-by: Toni Saario <toni.saario@qt.io>
Alexandru Croitor
Toni Saario
Alexey Edelev
Axel Spoerl
Matti Paaso
Cristian Le
Cristian Le
Jacek Poplawski
Marianne Yrjänä
Lucie Gérard
Michal Klocek
Tor Arne Vestbø
Simo Fält
Assam Boudjelthia
Dimitrios Apostolou