CMake: Rewrite init-repository using CMake and .sh / .bat scripts
init-repository is now implemented using CMake + .sh / .bat scripts.
The intent behind the change is not to require Perl to checkout and
build Qt, because it can be troublesome to acquire on Windows and it
can also lead to issues during builds due to CMake picking up a Perl
distribution-shipped compiler.
All previous options were ported over like
- module-subset
- alternates
- etc.
A few new options were added:
- --resolve-deps / --no-resolve-deps
- --optional-deps / --no-optional-deps
- --verbose
and some other internal ones for testing reasons.
The new script does automatic resolving of dependencies
based on the depends / recommends keys in .gitmodules unless
--no-resolve-deps is passed.
So if you configure with --module-subset=qtsvg, the script will also
initialize qtbase.
If --no-optional-deps is passed, only required dependencies ('depends'
ky) will be included and optional dependencies ('recommends' key) will
be excluded.
The new script now has a new default behavior when calling
init-repository a second time with --force, without specifying a
--module-subset option. Instead of initializing all submodules, it
will just update the existing / previously initialized submodules.
It also understands a new module-subset keyword "existing", which
expands to the previously initialized submodules, so someone can
initialize an additional submodule by calling
init-repository -f --module-subset=existing,qtsvg
Implementation notes:
The overall code flow is init-repository -> cmake/QtIRScript.cmake
-> qt_ir_run_main_script -> qt_ir_run_after_args_parsed ->
qt_ir_handle_init_submodules (recursive) -> qt_ir_clone_one_submodule
with some bells and whistles on the side.
The command line parsing is an adapted copy of the functions
in qtbase/cmake/QtProcessConfigureArgs.cmake. We can't use those exact
functions because qtbase is not available when init-repository is
initially called, and force cloning qtbase was deemed undesirable.
We also have a new mechanism to detect whether init-repository was
previously called. The perl script used the existence of the qtbase
submodule as the check. In the cmake script, we instead set a custom
marker into the local repo config file.
Otherwise the code logic should be a faithful reimplementation of
init-repository.pl aside from some small things like logging and
progress reporting.
The pre-existing git cloning logic in QtTopLevelHelpers was not used
because it would not be compatible with the alternates option and I
didn't want to accidentally break the pre-existing code. Plus
init-repository is a bit opinionated about how it clones and checks
out repos.
The dependency collection and sorting logic uses the pre-existing code
though.
See follow up commit about implicitly calling init-repository when
qt5/configure is called and the repo was not initialized before.
[ChangeLog][General] init-repository was rewritten using CMake. Perl
is no longer required to initialize the qt5.git super repo.
Task-number: QTBUG-120030
Task-number: QTBUG-122622
Change-Id: Ibc38ab79d3fdedd62111ebbec496eabd64c20d2b
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2023-12-15 10:22:55 +00:00
|
|
|
Usage:
|
|
|
|
./init-repository [options]
|
|
|
|
|
|
|
|
This script may be run after an initial `git clone' of the Qt supermodule
|
|
|
|
in order to check out all submodules. It fetches them from canonical URLs
|
|
|
|
inferred from the clone's origin.
|
|
|
|
|
|
|
|
Options:
|
|
|
|
Global options:
|
|
|
|
|
|
|
|
--force, -f
|
|
|
|
Force initialization (even if the submodules are already checked
|
|
|
|
out).
|
|
|
|
|
|
|
|
--force-hooks
|
|
|
|
Force initialization of hooks (even if there are already hooks in
|
|
|
|
checked out submodules).
|
|
|
|
|
|
|
|
--quiet, -q
|
|
|
|
Be quiet. Will exit cleanly if the repository is already
|
|
|
|
initialized.
|
|
|
|
|
|
|
|
--verbose
|
|
|
|
Adds a bit more output when executing processes
|
|
|
|
|
|
|
|
--no-resolve-deps
|
|
|
|
By default, each submodule specified via the module-subset option
|
|
|
|
will have its required and optional dependencies also initialized.
|
|
|
|
This option can be passed to disable automatic initialization of
|
|
|
|
dependencies, so that the exact list passed to module-subset is
|
|
|
|
initialized.
|
|
|
|
|
|
|
|
--no-optional-deps
|
|
|
|
By default, each submodule specified via the module-subset option
|
|
|
|
will have its optional dependencies also initialized.
|
|
|
|
This option can be passed to initialize only required dependencies of
|
|
|
|
the given module-subset.
|
|
|
|
|
|
|
|
Module options:
|
|
|
|
|
CMake: Integrate init-repository with the configure script
Calling configure will now implicitly run init-repository when
appropriate. See further down below for what "appropriate" means.
All supported init-repository options can be passed to configure as
except for -mirror, -oslo, -berlin.
This includes useful options like -submodules, -no-resolve-deps and
-no-optional-deps.
When running configure on a qt5.git clone without any submodules
cloned, configure will exit with a helpful error message suggesting to
pass -init-submodules, so it automatically clones missing repositories.
This means cloning is opt-in, so that internet access is not done
implicitly.
The error message also suggests passing the -submodules option.
This will affect which submodules will be cloned by init-repository
and which submodules will be configured by configure.
In this case -submodules is effectively an alias of
init-repository's -module-subset for cloning purposes.
When calling configure a second time, without -init-submodules, on an
already configured repo, init-repository behavior is entirely skipped.
-submodules now accepts init-repository-style special values like
"essential", "addon", "all", "existing", "-deprecated" for the purpose
of cloning submodules. The values are then translated into actual repos
that should also be configured or skipped by configure.
The default subset of cloned submodules is currently the same one as
init-repository, "default", which clones 44 actively maintained
repositories as well as deprecated submodules.
If configure is called a second time WITH -init-submodules, it's the
same as calling init-repository --force to re-initialize submodules.
In this case passing something like
--submodules existing,<additional-submodules>
might make sense to add or remove submodules.
As a drive-by this also fixes the bug where you couldn't pass a
configure -- -DFOO=0
parameter to configure, because it got treated as '0>', redirecting
from a different stream than stdout, leading to empty content in the
file.
[ChangeLog][General][Build System] The configure script now implicitly
calls init-repository when appropriate and accepts init-repository
command line options.
Fixes: QTBUG-120030
Task-number: QTBUG-122622
Change-Id: Iedbfcbf0a87c8ee89e40d00b6377b68296a65a62
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2024-01-29 17:10:42 +00:00
|
|
|
--module-subset=<module1>,<module2>... / -submodules <module1>,<module2>...
|
CMake: Rewrite init-repository using CMake and .sh / .bat scripts
init-repository is now implemented using CMake + .sh / .bat scripts.
The intent behind the change is not to require Perl to checkout and
build Qt, because it can be troublesome to acquire on Windows and it
can also lead to issues during builds due to CMake picking up a Perl
distribution-shipped compiler.
All previous options were ported over like
- module-subset
- alternates
- etc.
A few new options were added:
- --resolve-deps / --no-resolve-deps
- --optional-deps / --no-optional-deps
- --verbose
and some other internal ones for testing reasons.
The new script does automatic resolving of dependencies
based on the depends / recommends keys in .gitmodules unless
--no-resolve-deps is passed.
So if you configure with --module-subset=qtsvg, the script will also
initialize qtbase.
If --no-optional-deps is passed, only required dependencies ('depends'
ky) will be included and optional dependencies ('recommends' key) will
be excluded.
The new script now has a new default behavior when calling
init-repository a second time with --force, without specifying a
--module-subset option. Instead of initializing all submodules, it
will just update the existing / previously initialized submodules.
It also understands a new module-subset keyword "existing", which
expands to the previously initialized submodules, so someone can
initialize an additional submodule by calling
init-repository -f --module-subset=existing,qtsvg
Implementation notes:
The overall code flow is init-repository -> cmake/QtIRScript.cmake
-> qt_ir_run_main_script -> qt_ir_run_after_args_parsed ->
qt_ir_handle_init_submodules (recursive) -> qt_ir_clone_one_submodule
with some bells and whistles on the side.
The command line parsing is an adapted copy of the functions
in qtbase/cmake/QtProcessConfigureArgs.cmake. We can't use those exact
functions because qtbase is not available when init-repository is
initially called, and force cloning qtbase was deemed undesirable.
We also have a new mechanism to detect whether init-repository was
previously called. The perl script used the existence of the qtbase
submodule as the check. In the cmake script, we instead set a custom
marker into the local repo config file.
Otherwise the code logic should be a faithful reimplementation of
init-repository.pl aside from some small things like logging and
progress reporting.
The pre-existing git cloning logic in QtTopLevelHelpers was not used
because it would not be compatible with the alternates option and I
didn't want to accidentally break the pre-existing code. Plus
init-repository is a bit opinionated about how it clones and checks
out repos.
The dependency collection and sorting logic uses the pre-existing code
though.
See follow up commit about implicitly calling init-repository when
qt5/configure is called and the repo was not initialized before.
[ChangeLog][General] init-repository was rewritten using CMake. Perl
is no longer required to initialize the qt5.git super repo.
Task-number: QTBUG-120030
Task-number: QTBUG-122622
Change-Id: Ibc38ab79d3fdedd62111ebbec496eabd64c20d2b
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2023-12-15 10:22:55 +00:00
|
|
|
Only initialize the specified subset of modules given as the
|
|
|
|
argument. Specified modules must already exist in .gitmodules. The
|
|
|
|
string "all" results in cloning all known modules. The strings
|
|
|
|
"essential", "addon", "preview", "deprecated", "obsolete",
|
|
|
|
"additionalLibrary", and "ignore" refer to classes of modules
|
|
|
|
identified by "status=" lines in the .gitmodules file.
|
|
|
|
You can use "existing" to to reference already initialized submodules.
|
|
|
|
Additionally, "qtrepotools" is implicitly always added to ensure
|
|
|
|
relevant git commit hooks are available. It can be excluded as described
|
|
|
|
below.
|
|
|
|
You can use "default" in the subset as a short-hand for
|
|
|
|
"essential,addon,preview,deprecated", which corresponds to the set of
|
|
|
|
maintained modules included in standard Qt releases; this is also the
|
|
|
|
default module subset when this option is not given when first running
|
|
|
|
init-repositoy. If init-repository is rerun a second time (with --force)
|
|
|
|
the default is to initialize the "existing" submodules, rather than the
|
|
|
|
default subset. Entries may be prefixed with a dash to exclude them
|
|
|
|
from a bigger set, e.g. "all,-ignore" or "existing,-qttools".
|
CMake: Integrate init-repository with the configure script
Calling configure will now implicitly run init-repository when
appropriate. See further down below for what "appropriate" means.
All supported init-repository options can be passed to configure as
except for -mirror, -oslo, -berlin.
This includes useful options like -submodules, -no-resolve-deps and
-no-optional-deps.
When running configure on a qt5.git clone without any submodules
cloned, configure will exit with a helpful error message suggesting to
pass -init-submodules, so it automatically clones missing repositories.
This means cloning is opt-in, so that internet access is not done
implicitly.
The error message also suggests passing the -submodules option.
This will affect which submodules will be cloned by init-repository
and which submodules will be configured by configure.
In this case -submodules is effectively an alias of
init-repository's -module-subset for cloning purposes.
When calling configure a second time, without -init-submodules, on an
already configured repo, init-repository behavior is entirely skipped.
-submodules now accepts init-repository-style special values like
"essential", "addon", "all", "existing", "-deprecated" for the purpose
of cloning submodules. The values are then translated into actual repos
that should also be configured or skipped by configure.
The default subset of cloned submodules is currently the same one as
init-repository, "default", which clones 44 actively maintained
repositories as well as deprecated submodules.
If configure is called a second time WITH -init-submodules, it's the
same as calling init-repository --force to re-initialize submodules.
In this case passing something like
--submodules existing,<additional-submodules>
might make sense to add or remove submodules.
As a drive-by this also fixes the bug where you couldn't pass a
configure -- -DFOO=0
parameter to configure, because it got treated as '0>', redirecting
from a different stream than stdout, leading to empty content in the
file.
[ChangeLog][General][Build System] The configure script now implicitly
calls init-repository when appropriate and accepts init-repository
command line options.
Fixes: QTBUG-120030
Task-number: QTBUG-122622
Change-Id: Iedbfcbf0a87c8ee89e40d00b6377b68296a65a62
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2024-01-29 17:10:42 +00:00
|
|
|
For compatibility with qt's configure script, -submodules is an alias
|
|
|
|
of --module-subset. Note the difference in dashes and the equal sign.
|
CMake: Rewrite init-repository using CMake and .sh / .bat scripts
init-repository is now implemented using CMake + .sh / .bat scripts.
The intent behind the change is not to require Perl to checkout and
build Qt, because it can be troublesome to acquire on Windows and it
can also lead to issues during builds due to CMake picking up a Perl
distribution-shipped compiler.
All previous options were ported over like
- module-subset
- alternates
- etc.
A few new options were added:
- --resolve-deps / --no-resolve-deps
- --optional-deps / --no-optional-deps
- --verbose
and some other internal ones for testing reasons.
The new script does automatic resolving of dependencies
based on the depends / recommends keys in .gitmodules unless
--no-resolve-deps is passed.
So if you configure with --module-subset=qtsvg, the script will also
initialize qtbase.
If --no-optional-deps is passed, only required dependencies ('depends'
ky) will be included and optional dependencies ('recommends' key) will
be excluded.
The new script now has a new default behavior when calling
init-repository a second time with --force, without specifying a
--module-subset option. Instead of initializing all submodules, it
will just update the existing / previously initialized submodules.
It also understands a new module-subset keyword "existing", which
expands to the previously initialized submodules, so someone can
initialize an additional submodule by calling
init-repository -f --module-subset=existing,qtsvg
Implementation notes:
The overall code flow is init-repository -> cmake/QtIRScript.cmake
-> qt_ir_run_main_script -> qt_ir_run_after_args_parsed ->
qt_ir_handle_init_submodules (recursive) -> qt_ir_clone_one_submodule
with some bells and whistles on the side.
The command line parsing is an adapted copy of the functions
in qtbase/cmake/QtProcessConfigureArgs.cmake. We can't use those exact
functions because qtbase is not available when init-repository is
initially called, and force cloning qtbase was deemed undesirable.
We also have a new mechanism to detect whether init-repository was
previously called. The perl script used the existence of the qtbase
submodule as the check. In the cmake script, we instead set a custom
marker into the local repo config file.
Otherwise the code logic should be a faithful reimplementation of
init-repository.pl aside from some small things like logging and
progress reporting.
The pre-existing git cloning logic in QtTopLevelHelpers was not used
because it would not be compatible with the alternates option and I
didn't want to accidentally break the pre-existing code. Plus
init-repository is a bit opinionated about how it clones and checks
out repos.
The dependency collection and sorting logic uses the pre-existing code
though.
See follow up commit about implicitly calling init-repository when
qt5/configure is called and the repo was not initialized before.
[ChangeLog][General] init-repository was rewritten using CMake. Perl
is no longer required to initialize the qt5.git super repo.
Task-number: QTBUG-120030
Task-number: QTBUG-122622
Change-Id: Ibc38ab79d3fdedd62111ebbec496eabd64c20d2b
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2023-12-15 10:22:55 +00:00
|
|
|
|
|
|
|
--no-update
|
|
|
|
Skip the `git submodule update' command.
|
|
|
|
|
|
|
|
--no-fetch
|
|
|
|
Skip the `git fetch' commands. Implied by --no-update.
|
|
|
|
|
|
|
|
--branch
|
|
|
|
Instead of checking out specific SHA1s, check out the submodule
|
|
|
|
branches that correspond with the current supermodule commit. By
|
|
|
|
default, this option will cause local commits in the submodules to
|
|
|
|
be rebased. With --no-update, the branches will be checked out, but
|
|
|
|
their heads will not move.
|
|
|
|
|
|
|
|
--ignore-submodules
|
|
|
|
Set git config to ignore submodules by default when doing operations
|
|
|
|
on the qt5 repo, such as `pull', `fetch', `diff' etc.
|
|
|
|
|
|
|
|
After using this option, pass `--ignore-submodules=none' to git to
|
|
|
|
override it as needed.
|
|
|
|
|
|
|
|
Repository options:
|
|
|
|
|
|
|
|
--berlin
|
|
|
|
Switch to internal URLs and make use of the Berlin git mirrors.
|
|
|
|
(Implies `--mirror').
|
|
|
|
|
|
|
|
--oslo
|
|
|
|
Switch to internal URLs and make use of the Oslo git mirrors.
|
|
|
|
(Implies `--mirror').
|
|
|
|
|
|
|
|
--codereview-username <Gerrit/JIRA username>
|
|
|
|
Specify the user name for the (potentially) writable `gerrit' remote
|
|
|
|
for each module, for use with the Gerrit code review tool.
|
|
|
|
|
|
|
|
If this option is omitted, the gerrit remote is created without a
|
|
|
|
username and port number, and thus relies on a correct SSH
|
|
|
|
configuration.
|
|
|
|
|
|
|
|
--alternates <path to other Qt5 repo>
|
|
|
|
Adds alternates for each submodule to another full qt5 checkout.
|
|
|
|
This makes this qt5 checkout very small, as it will use the object
|
|
|
|
store of the alternates before unique objects are stored in its own
|
|
|
|
object store.
|
|
|
|
|
|
|
|
This option has no effect when using `--no-update'.
|
|
|
|
|
|
|
|
NOTE: This will make this repo dependent on the alternate, which is
|
|
|
|
potentially dangerous! The dependency can be broken by also using
|
|
|
|
the `--copy-objects' option, or by running "git repack -a" in each
|
|
|
|
submodule, where required. Please read the note about the `--shared'
|
|
|
|
option in the documentation of `git clone' for more information.
|
|
|
|
|
|
|
|
--copy-objects
|
|
|
|
When `--alternates' is used, automatically do a "git repack -a" in
|
|
|
|
each submodule after cloning, to ensure that the repositories are
|
|
|
|
independent from the source used as a reference for cloning.
|
|
|
|
|
|
|
|
Note that this negates the disk usage benefits gained from the use
|
|
|
|
of `--alternates'.
|
|
|
|
--mirror <url-base>
|
|
|
|
Uses <url-base> as the base URL for submodule git mirrors.
|
|
|
|
|
|
|
|
For example:
|
|
|
|
|
|
|
|
--mirror user\@machine:/foo/bar/qt/
|
|
|
|
|
|
|
|
...will use the following as a mirror for qtbase:
|
|
|
|
|
|
|
|
user\@machine:/foo/bar/qt/qtbase.git
|
|
|
|
|
|
|
|
The mirror is permitted to contain a subset of the submodules; any
|
|
|
|
missing modules will fall back to the canonical URLs.
|