qiurui
/
armbian-build
mirror of https://github.com/armbian/build.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
armbian-build/lib/functions/cli/commands.sh

162 lines
8.4 KiB
Bash
Raw Normal View History

Add / modify (c) in bash scripts (#4922) * Add / modify (c) in bash scripts Signed-off-by: Igor <igor@armbian.com> * Add (c) to the source config files --------- Signed-off-by: Igor <igor@armbian.com>
2023-03-09 17:30:40 +00:00
#!/usr/bin/env bash
#
# SPDX-License-Identifier: GPL-2.0
#
# Copyright (c) 2013-2023 Igor Pecovnik, igor@armbian.com
#
# This file is a part of the Armbian Build Framework
# https://github.com/armbian/build/
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
function armbian_register_commands() {
# More than one command can map to the same handler. In that case, use ARMBIAN_COMMANDS_TO_VARS_DICT for specific vars.
declare -g -A ARMBIAN_COMMANDS_TO_HANDLERS_DICT=(
armbian-next: docker: implement `docker-purge` and `docker-shell` cli commands; deprecated mountpoints
2022-11-30 10:16:17 +00:00
["docker"]="docker" # thus requires cli_docker_pre_run and cli_docker_run
["docker-purge"]="docker"
["dockerpurge"]="docker"
["docker-shell"]="docker"
["dockershell"]="docker"
["generate-dockerfile"]="docker"
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
armbian-next: `json-info` command, wrapping up the Python tooling; running in Docker; fix config dump
2022-11-03 18:24:00 +00:00
["requirements"]="requirements" # implemented in cli_requirements_pre_run and cli_requirements_run
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
`configdump`/`json-info-boards`: revamp, all-JSON now; use_board=yes skip_kernel=no for config; refactor & use new Python bash-declare-to-JSON utility - use new capture'd vars scheme - so `./compile.sh BOARD=xxx BRANCH=yyyy config-dump-json | jq .` now works and is consistent/newline tolerant - introduce internal `skip_host_config=yes` for `prep_conf_main_minimal_ni()` to skip calling `check_basic_host()`
2023-03-26 15:59:22 +00:00
# Given a board/config/exts, dump out the (non-userspace) JSON of configuration
configdump: alias `config-dump`, `config-dump-json` and new `config-dump-no-json` (bash declare format)
2023-05-04 11:21:06 +00:00
["configdump"]="config_dump_json" # implemented in cli_config_dump_json_pre_run and cli_config_dump_json_run
["config-dump"]="config_dump_json" # implemented in cli_config_dump_json_pre_run and cli_config_dump_json_run
["config-dump-json"]="config_dump_json" # implemented in cli_config_dump_json_pre_run and cli_config_dump_json_run
["config-dump-no-json"]="config_dump_json" # implemented in cli_config_dump_json_pre_run and cli_config_dump_json_run
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
pipeline: inventory all board vars; add `not-eos-with-video`; introduce `TARGETS_FILTER_INCLUDE` > How to use: > > `./compile.sh inventory` - does just the board inventory; look for output in `output/info` > > `./compile.sh targets-dashboard` - does inventory, targets compositing, and images info; look for output in `output/info`, read the instructions output by the command if you want to load the OpenSearch dashboards. > > `./compile.sh targets` - does the full targets compositing and artifacts, look for output in `output/info` > > If you don't have a `userpatches/targets.yaml`, _one will be provided for you_ defaulting to Jammy minimal CLI > and Jammy xfce desktop, for all boards in all branches. You can pass filters via `TARGETS_FILTER_INCLUDE=...` to narrow. > - board JSON inventory: - more generic regex parsing of variables from board files: - all top-level (non-indented) variables are parsed and included in the JSON board inventory - this allows us to add new variables to the board files without having to update the parser - variables can be bare, `export` or `declare -g`, but **_must_ be quoted** (single or double) and UPPER_CASE - some special treatment for certain variables: - `KERNEL_TARGET` is parsed as a _comma-separated_ list of valid BRANCH'es - `BOARD_MAINTAINER` is parsed as _space-separated_ list of valid maintainer GH usernames as `BOARD_MAINTAINERS: [...]` in the JSON - script complains if `BOARD_MAINTAINER` is not set in core boards. Empty is still allowed. - `HAS_VIDEO_OUTPUT="no"` causes `BOARD_HAS_VIDEO: false` in the JSON (for desktop-only inventorying, see below) - introduce `not-eos-with-video` in `items-from-inventory` at the targets compositor - the same as `not-eos`, but with added `BOARD_HAS_VIDEO: true` filter, see above - introduce `TARGETS_FILTER_INCLUDE` for targets compositor - this filters the targets _after_ compositing (but before getting image info), based on the board inventory data - it's a comma-separated list of `key:value` pairs, which are OR-ed together - new virtual info `BOARD_SLASH_BRANCH` post-compositing inventory for filtering of a specific BOARD/BRANCH combo (e.g. `odroidhc4/edge`) - some interesting possible filters: - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4"`: _only_ build a single board, all branches. JIRA [AR-1806] - `TARGETS_FILTER_INCLUDE="BOARD_SLASH_BRANCH:odroidhc4/current"`: _only_ build a single board/branch combo - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4,BOARD:odroidn2"`: _only_ build _two_ boards, all branches. - `TARGETS_FILTER_INCLUDE="BOARD_MAINTAINERS:rpardini"`: build all boards and branches where rpardini is a maintainer - `TARGETS_FILTER_INCLUDE="BOARDFAMILY:rockchip64"`: build all boards and branches in the rockchip64 family - image-info-only variables like `LINUXFAMILY` is **not** available for filtering at this stage - rename `config/templates` `targets-all-cli.yaml` to `targets-default.yaml` - this is used when no `userpatches/targets.yaml` is found - new default includes all boards vs branches for non-EOS boards - also desktop for all boards that _don't_ have `HAS_VIDEO_OUTPUT='no`` - introduce simplified `targets-dashboard` CLI: - does only inventory, compositing, and image info, but not artifact reducing, etc. - ignore desktop builds in the OpenSearch indexer - update the OpenSearch Dashboards, including new information now available - invert the logic used for `CLEAN_INFO` and `CLEAN_MATRIX` - defaults to `yes` now, so new users/CI don't get hit by stale caches by default - repo pipeline CLI stuff is usually run on saved/restored artifacts for `output/info`, so don't clean by default via the CLI
2023-07-25 14:50:22 +00:00
["inventory"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
["targets"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
["targets-dashboard"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
cli: introduce `inventory-boards` cli command, for hopefully painless & useful one-board-per-line CSV output - `inventory-boards-csv.py`: moar Python atrocities done to data; idea here is to output one line per board, with <<varies>> for fields that, well, vary (mainly across BRANCH'es) - `inventory-boards` sets `TARGETS_FILE` to something that doesn't exist, so the `default-targets.yaml` is used (so same list for everyone, save for userpatched-boards)
2023-10-04 11:43:15 +00:00
["inventory-boards"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
pipeline: add userspace inventory capabilities - digs into config/distributions and config/desktops for info - this produces `output/info/all_userspace_inventory.json` - this is now passed down to the `targets-compositor` in `cli-jsoninfo` - `targets-compositor` now accepts `userspace:` as `items-from-inventory` - extra: add `targets-composed` CLI command, to stop after targets-compositor
2023-08-23 16:26:42 +00:00
["targets-composed"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
pipeline: inventory all board vars; add `not-eos-with-video`; introduce `TARGETS_FILTER_INCLUDE` > How to use: > > `./compile.sh inventory` - does just the board inventory; look for output in `output/info` > > `./compile.sh targets-dashboard` - does inventory, targets compositing, and images info; look for output in `output/info`, read the instructions output by the command if you want to load the OpenSearch dashboards. > > `./compile.sh targets` - does the full targets compositing and artifacts, look for output in `output/info` > > If you don't have a `userpatches/targets.yaml`, _one will be provided for you_ defaulting to Jammy minimal CLI > and Jammy xfce desktop, for all boards in all branches. You can pass filters via `TARGETS_FILTER_INCLUDE=...` to narrow. > - board JSON inventory: - more generic regex parsing of variables from board files: - all top-level (non-indented) variables are parsed and included in the JSON board inventory - this allows us to add new variables to the board files without having to update the parser - variables can be bare, `export` or `declare -g`, but **_must_ be quoted** (single or double) and UPPER_CASE - some special treatment for certain variables: - `KERNEL_TARGET` is parsed as a _comma-separated_ list of valid BRANCH'es - `BOARD_MAINTAINER` is parsed as _space-separated_ list of valid maintainer GH usernames as `BOARD_MAINTAINERS: [...]` in the JSON - script complains if `BOARD_MAINTAINER` is not set in core boards. Empty is still allowed. - `HAS_VIDEO_OUTPUT="no"` causes `BOARD_HAS_VIDEO: false` in the JSON (for desktop-only inventorying, see below) - introduce `not-eos-with-video` in `items-from-inventory` at the targets compositor - the same as `not-eos`, but with added `BOARD_HAS_VIDEO: true` filter, see above - introduce `TARGETS_FILTER_INCLUDE` for targets compositor - this filters the targets _after_ compositing (but before getting image info), based on the board inventory data - it's a comma-separated list of `key:value` pairs, which are OR-ed together - new virtual info `BOARD_SLASH_BRANCH` post-compositing inventory for filtering of a specific BOARD/BRANCH combo (e.g. `odroidhc4/edge`) - some interesting possible filters: - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4"`: _only_ build a single board, all branches. JIRA [AR-1806] - `TARGETS_FILTER_INCLUDE="BOARD_SLASH_BRANCH:odroidhc4/current"`: _only_ build a single board/branch combo - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4,BOARD:odroidn2"`: _only_ build _two_ boards, all branches. - `TARGETS_FILTER_INCLUDE="BOARD_MAINTAINERS:rpardini"`: build all boards and branches where rpardini is a maintainer - `TARGETS_FILTER_INCLUDE="BOARDFAMILY:rockchip64"`: build all boards and branches in the rockchip64 family - image-info-only variables like `LINUXFAMILY` is **not** available for filtering at this stage - rename `config/templates` `targets-all-cli.yaml` to `targets-default.yaml` - this is used when no `userpatches/targets.yaml` is found - new default includes all boards vs branches for non-EOS boards - also desktop for all boards that _don't_ have `HAS_VIDEO_OUTPUT='no`` - introduce simplified `targets-dashboard` CLI: - does only inventory, compositing, and image info, but not artifact reducing, etc. - ignore desktop builds in the OpenSearch indexer - update the OpenSearch Dashboards, including new information now available - invert the logic used for `CLEAN_INFO` and `CLEAN_MATRIX` - defaults to `yes` now, so new users/CI don't get hit by stale caches by default - repo pipeline CLI stuff is usually run on saved/restored artifacts for `output/info`, so don't clean by default via the CLI
2023-07-25 14:50:22 +00:00
["debs-to-repo-json"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
["gha-matrix"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
["gha-workflow"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
["gha-template"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
pipeline: `debs-to-repo` (v7, working with reprepro) - pipeline: artifacts: use better description for artifacts (artifact_name+artifact_version instead of artifact_final_file_basename)
2023-05-05 12:03:18 +00:00
# These probably should be in their own separate CLI commands file, but for now they're together in jsoninfo.
["debs-to-repo-download"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
["debs-to-repo-reprepro"]="json_info" # implemented in cli_json_info_pre_run and cli_json_info_run
armbian-next: cli: half-assed, legacy based, `kernel` and `u-boot` cli commands - @TODO: a placeholder for now, needs refactoring after patching-hash/versioned-kernel work
2023-01-13 03:03:51 +00:00
cli: introduce `rewrite-uboot-patches` cli command, works similar to the kernel one - it prepares git, applies patches to git, and re-writes them back from git - same as kernel, it does git archeology for mbox-less patches, etc - `uboot-patches-to-git` alias is also added, but my guess is rewrite is more useful - refactor a common config function for both kernel and uboot - example invocation: `./compile.sh BOARD=xxxx BRANCH=edge rewrite-uboot-patches`
2024-03-06 23:49:16 +00:00
# Patch to git & patch rewrite, for kernel
cli: introduce `rewrite-kernel-patches`; fix `kernel-patches-to-git`; refactor `obtain_kernel_git_info_and_makefile()` - `kernel-patches-to-git` wasn't providing the needed `KERNEL_GIT_SHA1` for kernel drivers - refactor `obtain_kernel_git_info_and_makefile()` out of `artifact_kernel_prepare_version()` so we can reuse - introduce `rewrite-kernel-patches`, which is just an alias to `kernel-patches-to-git` with `REWRITE_PATCHES=yes`
2023-09-28 11:15:28 +00:00
["kernel-patches-to-git"]="patch_kernel" # implemented in cli_patch_kernel_pre_run and cli_patch_kernel_run
["rewrite-kernel-patches"]="patch_kernel" # implemented in cli_patch_kernel_pre_run and cli_patch_kernel_run
armbian-next: `json-info` command, wrapping up the Python tooling; running in Docker; fix config dump
2022-11-03 18:24:00 +00:00
cli: introduce `rewrite-uboot-patches` cli command, works similar to the kernel one - it prepares git, applies patches to git, and re-writes them back from git - same as kernel, it does git archeology for mbox-less patches, etc - `uboot-patches-to-git` alias is also added, but my guess is rewrite is more useful - refactor a common config function for both kernel and uboot - example invocation: `./compile.sh BOARD=xxxx BRANCH=edge rewrite-uboot-patches`
2024-03-06 23:49:16 +00:00
# Patch to git & patch rewrite, for u-boot
["uboot-patches-to-git"]="patch_uboot" # implemented in cli_patch_uboot_pre_run and cli_patch_uboot_run
["rewrite-uboot-patches"]="patch_uboot" # implemented in cli_patch_uboot_pre_run and cli_patch_uboot_run
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
["build"]="standard_build" # implemented in cli_standard_build_pre_run and cli_standard_build_run
armbian-next: `cli`: `distccd` launcher for build farm - relaunched under docker - possibly much nicer if zeroconf enabled? - this thing is ignoring signals. can't ctrl-c out of it. why? @TODO - pt 1, subpt 3: distccd - distcc-pump - avahi - pt 1, subpt 3: distccd + distcc-pump - pt 1, subpt 2: distccd + avahi - pt 1, subpt 1: distccd
2022-10-27 22:13:12 +00:00
["distccd"]="distccd" # implemented in cli_distccd_pre_run and cli_distccd_run
lib: shellfmt & fix gen-library to match
2023-04-16 14:26:18 +00:00
["flash"]="flash" # implemented in cli_flash_pre_run and cli_flash_run
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
armbian-next: GHA & rootfs adventures, pt 1 - rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version) - add "oras-upload" CLI command (dumb, no retries, single target/single source) - docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker - introduce `github-actions.sh::github_actions_add_output()` - during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-01-24 02:24:38 +00:00
# external tooling, made easy.
["oras-upload"]="oras" # implemented in cli_oras_pre_run and cli_oras_run; up/down/info are the same, see vars below
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
armbian-next: artifacts: introduce kernel/u-boot artifacts; git ref2info; "memoizer"; some hashing - artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()` - artifacts: u-boot - pt6: add artifact for u-boot - use artifact version / reason in actual u-boot .deb if present - artifacts: kernel - pt5: tune kernel version, refactor - artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build - artifacts: kernel - pt3: drivers+patches+.config hashing - split file hashing function from drivers-harness; fix it so filenames are relative and sorted; sort from ${SRC}, always - aplit prepare_kernel_config_core_or_userpatches() from `kernel_config_initialize()` - artifacts: kernel - pt2: memoizing git ref2info - artifacts: kernel - pt1: versioning
2023-01-29 11:47:40 +00:00
# all-around artifact wrapper
artifacts/cli: introduce CLI command `artifact-config-dump-json` (alias to `artifact`, but sets `CONFIG_DEFS_ONLY=yes`) - if CONFIG_DEFS_ONLY during `obtain_complete_artifact()`, just dump JSON to stdout after calculating version/coordinates - don't prepare ORAS tooling if CONFIG_DEFS_ONLY - don't run in standard build (and thus don't prepare_host()) if under CONFIG_DEFS_ONLY - if CONFIG_DEFS_ONLY, don't run `check_basic_host()`
2023-04-09 10:44:59 +00:00
["artifact"]="artifact" # implemented in cli_artifact_pre_run and cli_artifact_run
["artifact-config-dump-json"]="artifact" # implemented in cli_artifact_pre_run and cli_artifact_run
artifacts: `download-artifact` CLI. makes sure to only used local .deb, or download from OCI, never build
2023-05-09 19:00:14 +00:00
["download-artifact"]="artifact" # implemented in cli_artifact_pre_run and cli_artifact_run
armbian-next: cli: add CLI alias command `rootfs-cache` for creating rootfs cache
2023-01-21 09:24:23 +00:00
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
# shortcuts, see vars set below. the use legacy single build, and try to control it via variables
["rootfs"]="artifact"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["kernel"]="artifact"
cli: introduce `kernel-dtb` cli command, to build only DTB, and output full preprocessed dts source - example: `./compile.sh BOARD=xxxxx BRANCH=edge kernel-dtb` - outputs preprocessed DTS source for board in question to `output/` - also outputs same preprocessed DTS source, ran through `dtc` with input and output DTS formats for "normalized" comparisions
2024-03-03 14:37:07 +00:00
["kernel-dtb"]="artifact"
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
["kernel-patch"]="artifact"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["kernel-config"]="artifact"
cli: add command to rewrite/refresh kernel configs
2023-10-05 19:05:48 +00:00
["rewrite-kernel-config"]="artifact"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["uboot"]="artifact"
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
["uboot-patch"]="artifact"
["atf-patch"]="artifact"
allwinner: Enable crust compilation
2023-06-14 11:54:07 +00:00
["crust-patch"]="artifact"
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
["uboot-config"]="artifact"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
["firmware"]="artifact"
["firmware-full"]="artifact"
[AR-1620 + AR-1639 squashed]:: @igorpec's armbian-config/zsh/bsp/desktop/ubuntu-advantage artifacts
2023-03-19 12:44:09 +00:00
["armbian-config"]="artifact"
["armbian-zsh"]="artifact"
["armbian-plymouth-theme"]="artifact"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["fake-ubuntu-advantage-tools"]="artifact"
introduce `armbian-base-files` artifact, which downloads & repacks `base-files` from upstream distro - bsp-cli: now depends on `base-files (>= ${REVISION})`, this way upgrading the bsp-cli causes our base-files to be installed - bsp-cli no longer does gymnastics with /etc/os-release et al, all done in armbian-base-files now - general/apt-utils.sh: introduce `apt_find_upstream_package_version_and_download_url()` - base-files: add release to version, in order to comply with repo restrictions (valid repos can't have two different debs with same name and version, md5 must match)
2023-05-12 16:11:51 +00:00
["armbian-base-files"]="artifact"
[AR-1620 + AR-1639 squashed]:: @igorpec's armbian-config/zsh/bsp/desktop/ubuntu-advantage artifacts
2023-03-19 12:44:09 +00:00
["armbian-bsp-cli"]="artifact"
["armbian-bsp-desktop"]="artifact"
["armbian-desktop"]="artifact"
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
["undecided"]="undecided" # implemented in cli_undecided_pre_run and cli_undecided_run - relaunches either build or docker
)
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
# common for all CLI-based artifact shortcuts
declare common_cli_artifact_vars=""
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
# common for interactive artifact shortcuts (configure, patch, etc)
declare common_cli_artifact_interactive_vars="ARTIFACT_WILL_NOT_BUILD='yes' ARTIFACT_BUILD_INTERACTIVE='yes' ARTIFACT_IGNORE_CACHE='yes'"
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
# Vars to be set for each command. Optional.
declare -g -A ARMBIAN_COMMANDS_TO_VARS_DICT=(
armbian-next: docker: implement `docker-purge` and `docker-shell` cli commands; deprecated mountpoints
2022-11-30 10:16:17 +00:00
["docker-purge"]="DOCKER_SUBCMD='purge'"
["dockerpurge"]="DOCKER_SUBCMD='purge'"
["docker-shell"]="DOCKER_SUBCMD='shell'"
["dockershell"]="DOCKER_SUBCMD='shell'"
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
["generate-dockerfile"]="DOCKERFILE_GENERATE_ONLY='yes'"
artifacts/cli: introduce CLI command `artifact-config-dump-json` (alias to `artifact`, but sets `CONFIG_DEFS_ONLY=yes`) - if CONFIG_DEFS_ONLY during `obtain_complete_artifact()`, just dump JSON to stdout after calculating version/coordinates - don't prepare ORAS tooling if CONFIG_DEFS_ONLY - don't run in standard build (and thus don't prepare_host()) if under CONFIG_DEFS_ONLY - if CONFIG_DEFS_ONLY, don't run `check_basic_host()`
2023-04-09 10:44:59 +00:00
["artifact-config-dump-json"]='CONFIG_DEFS_ONLY="yes"'
pipeline: inventory all board vars; add `not-eos-with-video`; introduce `TARGETS_FILTER_INCLUDE` > How to use: > > `./compile.sh inventory` - does just the board inventory; look for output in `output/info` > > `./compile.sh targets-dashboard` - does inventory, targets compositing, and images info; look for output in `output/info`, read the instructions output by the command if you want to load the OpenSearch dashboards. > > `./compile.sh targets` - does the full targets compositing and artifacts, look for output in `output/info` > > If you don't have a `userpatches/targets.yaml`, _one will be provided for you_ defaulting to Jammy minimal CLI > and Jammy xfce desktop, for all boards in all branches. You can pass filters via `TARGETS_FILTER_INCLUDE=...` to narrow. > - board JSON inventory: - more generic regex parsing of variables from board files: - all top-level (non-indented) variables are parsed and included in the JSON board inventory - this allows us to add new variables to the board files without having to update the parser - variables can be bare, `export` or `declare -g`, but **_must_ be quoted** (single or double) and UPPER_CASE - some special treatment for certain variables: - `KERNEL_TARGET` is parsed as a _comma-separated_ list of valid BRANCH'es - `BOARD_MAINTAINER` is parsed as _space-separated_ list of valid maintainer GH usernames as `BOARD_MAINTAINERS: [...]` in the JSON - script complains if `BOARD_MAINTAINER` is not set in core boards. Empty is still allowed. - `HAS_VIDEO_OUTPUT="no"` causes `BOARD_HAS_VIDEO: false` in the JSON (for desktop-only inventorying, see below) - introduce `not-eos-with-video` in `items-from-inventory` at the targets compositor - the same as `not-eos`, but with added `BOARD_HAS_VIDEO: true` filter, see above - introduce `TARGETS_FILTER_INCLUDE` for targets compositor - this filters the targets _after_ compositing (but before getting image info), based on the board inventory data - it's a comma-separated list of `key:value` pairs, which are OR-ed together - new virtual info `BOARD_SLASH_BRANCH` post-compositing inventory for filtering of a specific BOARD/BRANCH combo (e.g. `odroidhc4/edge`) - some interesting possible filters: - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4"`: _only_ build a single board, all branches. JIRA [AR-1806] - `TARGETS_FILTER_INCLUDE="BOARD_SLASH_BRANCH:odroidhc4/current"`: _only_ build a single board/branch combo - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4,BOARD:odroidn2"`: _only_ build _two_ boards, all branches. - `TARGETS_FILTER_INCLUDE="BOARD_MAINTAINERS:rpardini"`: build all boards and branches where rpardini is a maintainer - `TARGETS_FILTER_INCLUDE="BOARDFAMILY:rockchip64"`: build all boards and branches in the rockchip64 family - image-info-only variables like `LINUXFAMILY` is **not** available for filtering at this stage - rename `config/templates` `targets-all-cli.yaml` to `targets-default.yaml` - this is used when no `userpatches/targets.yaml` is found - new default includes all boards vs branches for non-EOS boards - also desktop for all boards that _don't_ have `HAS_VIDEO_OUTPUT='no`` - introduce simplified `targets-dashboard` CLI: - does only inventory, compositing, and image info, but not artifact reducing, etc. - ignore desktop builds in the OpenSearch indexer - update the OpenSearch Dashboards, including new information now available - invert the logic used for `CLEAN_INFO` and `CLEAN_MATRIX` - defaults to `yes` now, so new users/CI don't get hit by stale caches by default - repo pipeline CLI stuff is usually run on saved/restored artifacts for `output/info`, so don't clean by default via the CLI
2023-07-25 14:50:22 +00:00
# repo pipeline stuff is usually run on saved/restored artifacts for output/info, so don't clean them by default
["debs-to-repo-download"]="CLEAN_MATRIX='no' CLEAN_INFO='no'"
["debs-to-repo-reprepro"]="CLEAN_MATRIX='no' CLEAN_INFO='no'"
cli: introduce `inventory-boards` cli command, for hopefully painless & useful one-board-per-line CSV output - `inventory-boards-csv.py`: moar Python atrocities done to data; idea here is to output one line per board, with <<varies>> for fields that, well, vary (mainly across BRANCH'es) - `inventory-boards` sets `TARGETS_FILE` to something that doesn't exist, so the `default-targets.yaml` is used (so same list for everyone, save for userpatched-boards)
2023-10-04 11:43:15 +00:00
# inventory
["inventory-boards"]="TARGETS_FILE='something_that_does_not_exist_so_defaults_are_used'"
cli: introduce `rewrite-kernel-patches`; fix `kernel-patches-to-git`; refactor `obtain_kernel_git_info_and_makefile()` - `kernel-patches-to-git` wasn't providing the needed `KERNEL_GIT_SHA1` for kernel drivers - refactor `obtain_kernel_git_info_and_makefile()` out of `artifact_kernel_prepare_version()` so we can reuse - introduce `rewrite-kernel-patches`, which is just an alias to `kernel-patches-to-git` with `REWRITE_PATCHES=yes`
2023-09-28 11:15:28 +00:00
# patching
["rewrite-kernel-patches"]="REWRITE_PATCHES=yes" # rewrite the patches after round-tripping to git: "rebase patches"
cli: introduce `rewrite-uboot-patches` cli command, works similar to the kernel one - it prepares git, applies patches to git, and re-writes them back from git - same as kernel, it does git archeology for mbox-less patches, etc - `uboot-patches-to-git` alias is also added, but my guess is rewrite is more useful - refactor a common config function for both kernel and uboot - example invocation: `./compile.sh BOARD=xxxx BRANCH=edge rewrite-uboot-patches`
2024-03-06 23:49:16 +00:00
["rewrite-uboot-patches"]="REWRITE_PATCHES=yes" # rewrite the patches after round-tripping to git: "rebase patches"
cli: introduce `rewrite-kernel-patches`; fix `kernel-patches-to-git`; refactor `obtain_kernel_git_info_and_makefile()` - `kernel-patches-to-git` wasn't providing the needed `KERNEL_GIT_SHA1` for kernel drivers - refactor `obtain_kernel_git_info_and_makefile()` out of `artifact_kernel_prepare_version()` so we can reuse - introduce `rewrite-kernel-patches`, which is just an alias to `kernel-patches-to-git` with `REWRITE_PATCHES=yes`
2023-09-28 11:15:28 +00:00
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
# artifact shortcuts
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["rootfs"]="WHAT='rootfs' ${common_cli_artifact_vars}"
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
["kernel"]="WHAT='kernel' ${common_cli_artifact_vars}"
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
["kernel-config"]="WHAT='kernel' KERNEL_CONFIGURE='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
cli: add command to rewrite/refresh kernel configs
2023-10-05 19:05:48 +00:00
["rewrite-kernel-config"]="WHAT='kernel' KERNEL_CONFIGURE='yes' ARTIFACT_WILL_NOT_BUILD='yes' ARTIFACT_IGNORE_CACHE='yes' ${common_cli_artifact_vars}"
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
["kernel-patch"]="WHAT='kernel' CREATE_PATCHES='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
cli: introduce `kernel-dtb` cli command, to build only DTB, and output full preprocessed dts source - example: `./compile.sh BOARD=xxxxx BRANCH=edge kernel-dtb` - outputs preprocessed DTS source for board in question to `output/` - also outputs same preprocessed DTS source, ran through `dtc` with input and output DTS formats for "normalized" comparisions
2024-03-03 14:37:07 +00:00
["kernel-dtb"]="WHAT='kernel' KERNEL_DTB_ONLY='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
["uboot"]="WHAT='uboot' ${common_cli_artifact_vars}"
`kernel`/`uboot`/`atf`: introduce `kernel-patch`, `uboot-patch`, `atf-patch`, `uboot-config`, `kernel-config` CLI commands; enhanced manual patching; block deprecated ways - all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends - user is required to put the produced patches in the right place and build again, for full consistency - split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot - non-interactive artifact builds are now 100% sans-stdin - introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch - reworked `userpatch_create()` to be (hopefully) more useful: - detects a previous patch and offers to apply it before continuing - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers) - don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 13:15:42 +00:00
["uboot-config"]="WHAT='uboot' UBOOT_CONFIGURE='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
["uboot-patch"]="WHAT='uboot' CREATE_PATCHES='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
["atf-patch"]="WHAT='uboot' CREATE_PATCHES_ATF='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
allwinner: Enable crust compilation
2023-06-14 11:54:07 +00:00
["crust-patch"]="WHAT='uboot' CREATE_PATCHES_CRUST='yes' ${common_cli_artifact_interactive_vars} ${common_cli_artifact_vars}"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
["firmware"]="WHAT='firmware' ${common_cli_artifact_vars}"
["firmware-full"]="WHAT='full_firmware' ${common_cli_artifact_vars}"
[AR-1620 + AR-1639 squashed]:: @igorpec's armbian-config/zsh/bsp/desktop/ubuntu-advantage artifacts
2023-03-19 12:44:09 +00:00
["armbian-config"]="WHAT='armbian-config' ${common_cli_artifact_vars}"
["armbian-zsh"]="WHAT='armbian-zsh' ${common_cli_artifact_vars}"
["armbian-plymouth-theme"]="WHAT='armbian-plymouth-theme' ${common_cli_artifact_vars}"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["fake-ubuntu-advantage-tools"]="WHAT='fake_ubuntu_advantage_tools' ${common_cli_artifact_vars}"
introduce `armbian-base-files` artifact, which downloads & repacks `base-files` from upstream distro - bsp-cli: now depends on `base-files (>= ${REVISION})`, this way upgrading the bsp-cli causes our base-files to be installed - bsp-cli no longer does gymnastics with /etc/os-release et al, all done in armbian-base-files now - general/apt-utils.sh: introduce `apt_find_upstream_package_version_and_download_url()` - base-files: add release to version, in order to comply with repo restrictions (valid repos can't have two different debs with same name and version, md5 must match)
2023-05-12 16:11:51 +00:00
["armbian-base-files"]="WHAT='armbian-base-files' ${common_cli_artifact_vars}"
[AR-1620 + AR-1639 squashed]:: @igorpec's armbian-config/zsh/bsp/desktop/ubuntu-advantage artifacts
2023-03-19 12:44:09 +00:00
["armbian-bsp-cli"]="WHAT='armbian-bsp-cli' ${common_cli_artifact_vars}"
artifacts: bsp-cli, bsp-desktop, desktop: rework, with @igorpec - bsp-cli: does not require aggregation - bsp-desktop / desktop: requires aggregation - 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp - make consistent naming/folders - extract .sh generation workflow into 'declare -f' magic functions - `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen - artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD - artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore - cli: commands: fix artifact aliases, reorder - desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name - artifact bsps: require aggregation to build, not to list what needs to be built - distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`) - armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check - distro-specific: fix installation of `fake-ubuntu-advantage-tools`
2023-03-28 12:36:11 +00:00
["armbian-bsp-desktop"]="WHAT='armbian-bsp-desktop' BUILD_DESKTOP='yes' ${common_cli_artifact_vars}"
["armbian-desktop"]="WHAT='armbian-desktop' BUILD_DESKTOP='yes' ${common_cli_artifact_vars}"
armbian-next: make sure temporary `kernel` and `u-boot` CLI commands always build a kernel/u-boot even if cached - really, it's time to split off those into actual, proper CLIs - remove a forgotten `KDEB_CHANGELOG_DIST` left there
2023-01-15 21:32:18 +00:00
armbian-next: GHA & rootfs adventures, pt 1 - rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version) - add "oras-upload" CLI command (dumb, no retries, single target/single source) - docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker - introduce `github-actions.sh::github_actions_add_output()` - during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-01-24 02:24:38 +00:00
["oras-upload"]="ORAS_OPERATION='upload'"
armbian-next: make sure temporary `kernel` and `u-boot` CLI commands always build a kernel/u-boot even if cached - really, it's time to split off those into actual, proper CLIs - remove a forgotten `KDEB_CHANGELOG_DIST` left there
2023-01-15 21:32:18 +00:00
["undecided"]="UNDECIDED='yes'"
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
)
# Override the LOG_CLI_ID to change the log file name.
# Will be set to ARMBIAN_COMMAND if not set after all pre-runs done.
declare -g ARMBIAN_LOG_CLI_ID
# Keep a running dict of params/variables. Can't repeat stuff here. Dict.
declare -g -A ARMBIAN_CLI_RELAUNCH_PARAMS=(["ARMBIAN_RELAUNCHED"]="yes")
docker/sudo: better control/info over CLI relaunch args; introduce CLI relaunch envs - allow adding re-launch environments by populating `ARMBIAN_CLI_RELAUNCH_PARAMS` dict - similar to `ARMBIAN_CLI_RELAUNCH_PARAMS` - rename/consolidate `()`'s output globals:   - `ARMBIAN_CLI_FINAL_RELAUNCH_ARGS` (rename)   - `ARMBIAN_CLI_FINAL_RELAUNCH_ENVS` (new)     - includes `ARMBIAN_HIDE_REPEAT_PARAMS` automatically - `sudo`: include envs when relaunching, use bash explicitly - `docker`: better logging - `docker`: include envs when relaunching via `--env` args
2023-04-20 16:36:00 +00:00
declare -g -A ARMBIAN_CLI_RELAUNCH_ENVS=(["ARMBIAN_RELAUNCHED"]="yes")
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` - armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker) - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages - debootstrap, by passing it `--cache-dir` - utility function to help understand what is happening to cache during usage - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_ - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot - clean up `install_deb_chroot()` a little, find an unrelated bug there - WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG` - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile - it's `drastic red` on non-buildx dockers - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging - WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x - source configs in a logging section. - Docker: silent, fast retries to make sure `docker system df` works - shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID` - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow - use green apple for Mac logging, instead of red apple which might imply error... - WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root. - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID` - use it for userpatches files created, logs, and output files, including images and debs. - @TODOs ref. `$SUDO_USER` which I think the old version of this? - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command. - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS` - WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3 - WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented - WiP: the great cli entrypoint (+docker) rewrite, Phase 1 - armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs. - using image in my private repo. - this has significant speedup to "start building time" on the 1st run - move some Linux specific stuff to its own if - add comments and todo - armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences - armbian-next: WiP: Docker, configure `BUILDKIT_COLORS` - armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers - armbian-next: `logging`: add whale indicator if build running under Docker - armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only - armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy - armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots - WiP: try with debian:bullseye -- can't detect docker at all - armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working - gen .dockerignore together with Dockerfile - split in funcs - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes - mac still works, Linux stuff would break it but I if'fed - armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing - this fails for the right reasons now, causing retries, which are then retried and work ;-) - this is related to building under Docker on Linux, using docker.io package (not docker-ce) - armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated - armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal - armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore - armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY - uses REQUIREMENTS_DEFS_ONLY - works on Docker Desktop on Mac; - linux TBA - armbian-next: don't error out if `.git` not present; other small fixes - armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance" - can't assume things, for example: - that `sudo` will be available; it might not, and might be already root, no reason to fail - that `/etc/timezone` will exist - that `systemd-detect-virt` will be available - that `git` will be available - that `locale-gen` will be available
2022-10-09 15:58:23 +00:00
# Keep a running array of config files needed for relaunch.
declare -g -a ARMBIAN_CLI_RELAUNCH_CONFIGS=()
}