armbian-build/lib/functions/cli/cli-artifact.sh

75 lines
2.5 KiB
Bash
Raw Permalink Normal View History

#!/usr/bin/env bash
#
# SPDX-License-Identifier: GPL-2.0
#
# Copyright (c) 2013-2023 Igor Pecovnik, igor@armbian.com
#
# This file is a part of the Armbian Build Framework
# https://github.com/armbian/build/
function cli_artifact_pre_run() {
case "${ARMBIAN_COMMAND}" in
download-artifact)
display_alert "download-only mode:" "won't build '${WHAT}'" "info"
adapt `obtain_complete_artifact()` and friends to new reversioned scheme; introduce `UPLOAD_TO_OCI_ONLY=yes` for deploying to remote cache > tl-dr: only deploys to remote OCI if `UPLOAD_TO_OCI_ONLY=yes`; stop leaving junk behind in local cache in many situations - simplify CLI artifact building parameters and behaviour - `ARTIFACT_USE_CACHE` is now deprecated, and its behaviour is the default - for _any_ uploading to OCI to occur, `UPLOAD_TO_OCI_ONLY=yes` **must** be present; in this case, reversioning is not done - `FORCE_ARTIFACTS_DOWNLOAD` is completely removed (use `download-artifact` instead) - `cli_obtain_complete_artifact()`'s and `build_artifact_for_image()`'s reversioning is now moved to common `obtain_complete_artifact()` - `standard_artifact_reversion_for_deployment()`: - check for hashed deb existence only if reversioned does not exist - touch the reversioned file if it already exists; helps to clean up junk later - delete hashed version after reversioning, so we don't leave trash behind - unless in `download-artifact` mode, which `touch`es the hashed version - we can later delete old files from packages-hashed to keep junk under control - refactor `obtain_complete_artifact()` - extract function `artifact_dump_json_info()` since obtain is large enough already - when deploying to remote, always ignore all local and remote caches - introduce `artifact_is_available_in_revisioned_local_cache()` - if not deploying to remote, and revisioned cache exists, use it directly - if deploying to remote, reversioned is not checked and not created - if deploying to remote, force `DEB_COMPRESS=xz` - if deploying to remote, completely remove the local cache base dir after upload is done (no more junk leftover)
2023-08-10 14:17:40 +00:00
declare -g DONT_BUILD_ARTIFACTS="${WHAT}"
declare -g KEEP_HASHED_DEB_ARTIFACTS="yes"
;;
esac
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
initialize_artifact "${WHAT}"
# Run the pre run adapter
artifact_cli_adapter_pre_run
}
function cli_artifact_run() {
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
: "${chosen_artifact:?chosen_artifact is not set}"
: "${chosen_artifact_impl:?chosen_artifact_impl is not set}"
if [[ "${CONFIG_DEFS_ONLY}" != "yes" ]]; then
# Make sure ORAS tooling is installed before starting.
run_tool_oras
fi
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
display_alert "artifact" "${chosen_artifact}" "debug"
display_alert "artifact" "${chosen_artifact} :: ${chosen_artifact_impl}()" "debug"
track_general_config_variables "in cli_artifact_run before artifact_cli_adapter_config_prep"
declare -g artifact_version_requires_aggregation="no" # marker
artifact_cli_adapter_config_prep # only if in cli.
track_general_config_variables "in cli_artifact_run after artifact_cli_adapter_config_prep"
# if asked by _config_prep to aggregate, and HOSTRELEASE is not set, obtain it.
if [[ "${artifact_version_requires_aggregation}" == "yes" ]] && [[ -z "${HOSTRELEASE}" ]]; then
obtain_hostrelease_only # Sets HOSTRELEASE
fi
adapt `obtain_complete_artifact()` and friends to new reversioned scheme; introduce `UPLOAD_TO_OCI_ONLY=yes` for deploying to remote cache > tl-dr: only deploys to remote OCI if `UPLOAD_TO_OCI_ONLY=yes`; stop leaving junk behind in local cache in many situations - simplify CLI artifact building parameters and behaviour - `ARTIFACT_USE_CACHE` is now deprecated, and its behaviour is the default - for _any_ uploading to OCI to occur, `UPLOAD_TO_OCI_ONLY=yes` **must** be present; in this case, reversioning is not done - `FORCE_ARTIFACTS_DOWNLOAD` is completely removed (use `download-artifact` instead) - `cli_obtain_complete_artifact()`'s and `build_artifact_for_image()`'s reversioning is now moved to common `obtain_complete_artifact()` - `standard_artifact_reversion_for_deployment()`: - check for hashed deb existence only if reversioned does not exist - touch the reversioned file if it already exists; helps to clean up junk later - delete hashed version after reversioning, so we don't leave trash behind - unless in `download-artifact` mode, which `touch`es the hashed version - we can later delete old files from packages-hashed to keep junk under control - refactor `obtain_complete_artifact()` - extract function `artifact_dump_json_info()` since obtain is large enough already - when deploying to remote, always ignore all local and remote caches - introduce `artifact_is_available_in_revisioned_local_cache()` - if not deploying to remote, and revisioned cache exists, use it directly - if deploying to remote, reversioned is not checked and not created - if deploying to remote, force `DEB_COMPRESS=xz` - if deploying to remote, completely remove the local cache base dir after upload is done (no more junk leftover)
2023-08-10 14:17:40 +00:00
declare deploy_to_remote="no"
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build - artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes` - artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build - kernel_package_source() is no more - a long dissertation about kernels, families, and the universe - artifacts: actually use rootfs artifact for image build - artifacts: detangle via artifact_base_dir - artifacts: rootfs: use folders in artifact_name; include cache_type - artifacts: some cleanups / validations - rootfs artifact; drop old rootfs cli - artifacts: new CLI shortcuts; remove old firmware CLI - artifacts: full firmware & usage - use firmware artifacts in image build and install its debs - kernel artifact sans legacy; use tmpdir for .deb target for all packages - legacy artifact versions is no more; pack/unpack now in common obtain; - artifacts: uboot: cleanup legacy renaming, use artifact version directly - artifacts: add firmware (small) artifact - deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning - fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()` - don't use realpath for artifact_file_relative - curb some warnings - fix: only install headers & dtbs if such exist - kernel .config hook modification hash versioning - OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB - artifact revolver & installing into image - add artifact_map_packages and artifact_map_debs dicts - revolver accumulates all info - REPOSITORY_INSTALL is no more (for uboot/kernel, later others) - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-03 14:36:28 +00:00
case "${ARMBIAN_COMMAND}" in
download-artifact)
display_alert "Running in download-artifact mode" "download-artifact" "ext"
;;
*)
adapt `obtain_complete_artifact()` and friends to new reversioned scheme; introduce `UPLOAD_TO_OCI_ONLY=yes` for deploying to remote cache > tl-dr: only deploys to remote OCI if `UPLOAD_TO_OCI_ONLY=yes`; stop leaving junk behind in local cache in many situations - simplify CLI artifact building parameters and behaviour - `ARTIFACT_USE_CACHE` is now deprecated, and its behaviour is the default - for _any_ uploading to OCI to occur, `UPLOAD_TO_OCI_ONLY=yes` **must** be present; in this case, reversioning is not done - `FORCE_ARTIFACTS_DOWNLOAD` is completely removed (use `download-artifact` instead) - `cli_obtain_complete_artifact()`'s and `build_artifact_for_image()`'s reversioning is now moved to common `obtain_complete_artifact()` - `standard_artifact_reversion_for_deployment()`: - check for hashed deb existence only if reversioned does not exist - touch the reversioned file if it already exists; helps to clean up junk later - delete hashed version after reversioning, so we don't leave trash behind - unless in `download-artifact` mode, which `touch`es the hashed version - we can later delete old files from packages-hashed to keep junk under control - refactor `obtain_complete_artifact()` - extract function `artifact_dump_json_info()` since obtain is large enough already - when deploying to remote, always ignore all local and remote caches - introduce `artifact_is_available_in_revisioned_local_cache()` - if not deploying to remote, and revisioned cache exists, use it directly - if deploying to remote, reversioned is not checked and not created - if deploying to remote, force `DEB_COMPRESS=xz` - if deploying to remote, completely remove the local cache base dir after upload is done (no more junk leftover)
2023-08-10 14:17:40 +00:00
# Warn of deprecation...
if [[ "${ARTIFACT_USE_CACHE}" == "yes" ]]; then
display_alert "deprecated!" "ARTIFACT_USE_CACHE=yes is deprecated, its behaviour is now the default." "warn"
fi
adapt `obtain_complete_artifact()` and friends to new reversioned scheme; introduce `UPLOAD_TO_OCI_ONLY=yes` for deploying to remote cache > tl-dr: only deploys to remote OCI if `UPLOAD_TO_OCI_ONLY=yes`; stop leaving junk behind in local cache in many situations - simplify CLI artifact building parameters and behaviour - `ARTIFACT_USE_CACHE` is now deprecated, and its behaviour is the default - for _any_ uploading to OCI to occur, `UPLOAD_TO_OCI_ONLY=yes` **must** be present; in this case, reversioning is not done - `FORCE_ARTIFACTS_DOWNLOAD` is completely removed (use `download-artifact` instead) - `cli_obtain_complete_artifact()`'s and `build_artifact_for_image()`'s reversioning is now moved to common `obtain_complete_artifact()` - `standard_artifact_reversion_for_deployment()`: - check for hashed deb existence only if reversioned does not exist - touch the reversioned file if it already exists; helps to clean up junk later - delete hashed version after reversioning, so we don't leave trash behind - unless in `download-artifact` mode, which `touch`es the hashed version - we can later delete old files from packages-hashed to keep junk under control - refactor `obtain_complete_artifact()` - extract function `artifact_dump_json_info()` since obtain is large enough already - when deploying to remote, always ignore all local and remote caches - introduce `artifact_is_available_in_revisioned_local_cache()` - if not deploying to remote, and revisioned cache exists, use it directly - if deploying to remote, reversioned is not checked and not created - if deploying to remote, force `DEB_COMPRESS=xz` - if deploying to remote, completely remove the local cache base dir after upload is done (no more junk leftover)
2023-08-10 14:17:40 +00:00
# If UPLOAD_TO_OCI_ONLY=yes is explicitly set; deploy to remote.
if [[ "${UPLOAD_TO_OCI_ONLY}" == "yes" ]]; then
display_alert "UPLOAD_TO_OCI_ONLY=yes is set" "UPLOAD_TO_OCI_ONLY=yes; ignoring local cache and deploying to remote" "info"
deploy_to_remote="yes"
fi
;;
esac
if [[ "${ARTIFACT_BUILD_INTERACTIVE}" == "yes" ]]; then # Set by `kernel-config`, `kernel-patch`, `uboot-config`, `uboot-patch`, etc.
display_alert "Running artifact build in interactive mode" "log file will be incomplete" "info"
adapt `obtain_complete_artifact()` and friends to new reversioned scheme; introduce `UPLOAD_TO_OCI_ONLY=yes` for deploying to remote cache > tl-dr: only deploys to remote OCI if `UPLOAD_TO_OCI_ONLY=yes`; stop leaving junk behind in local cache in many situations - simplify CLI artifact building parameters and behaviour - `ARTIFACT_USE_CACHE` is now deprecated, and its behaviour is the default - for _any_ uploading to OCI to occur, `UPLOAD_TO_OCI_ONLY=yes` **must** be present; in this case, reversioning is not done - `FORCE_ARTIFACTS_DOWNLOAD` is completely removed (use `download-artifact` instead) - `cli_obtain_complete_artifact()`'s and `build_artifact_for_image()`'s reversioning is now moved to common `obtain_complete_artifact()` - `standard_artifact_reversion_for_deployment()`: - check for hashed deb existence only if reversioned does not exist - touch the reversioned file if it already exists; helps to clean up junk later - delete hashed version after reversioning, so we don't leave trash behind - unless in `download-artifact` mode, which `touch`es the hashed version - we can later delete old files from packages-hashed to keep junk under control - refactor `obtain_complete_artifact()` - extract function `artifact_dump_json_info()` since obtain is large enough already - when deploying to remote, always ignore all local and remote caches - introduce `artifact_is_available_in_revisioned_local_cache()` - if not deploying to remote, and revisioned cache exists, use it directly - if deploying to remote, reversioned is not checked and not created - if deploying to remote, force `DEB_COMPRESS=xz` - if deploying to remote, completely remove the local cache base dir after upload is done (no more junk leftover)
2023-08-10 14:17:40 +00:00
do_with_default_build obtain_complete_artifact
else
adapt `obtain_complete_artifact()` and friends to new reversioned scheme; introduce `UPLOAD_TO_OCI_ONLY=yes` for deploying to remote cache > tl-dr: only deploys to remote OCI if `UPLOAD_TO_OCI_ONLY=yes`; stop leaving junk behind in local cache in many situations - simplify CLI artifact building parameters and behaviour - `ARTIFACT_USE_CACHE` is now deprecated, and its behaviour is the default - for _any_ uploading to OCI to occur, `UPLOAD_TO_OCI_ONLY=yes` **must** be present; in this case, reversioning is not done - `FORCE_ARTIFACTS_DOWNLOAD` is completely removed (use `download-artifact` instead) - `cli_obtain_complete_artifact()`'s and `build_artifact_for_image()`'s reversioning is now moved to common `obtain_complete_artifact()` - `standard_artifact_reversion_for_deployment()`: - check for hashed deb existence only if reversioned does not exist - touch the reversioned file if it already exists; helps to clean up junk later - delete hashed version after reversioning, so we don't leave trash behind - unless in `download-artifact` mode, which `touch`es the hashed version - we can later delete old files from packages-hashed to keep junk under control - refactor `obtain_complete_artifact()` - extract function `artifact_dump_json_info()` since obtain is large enough already - when deploying to remote, always ignore all local and remote caches - introduce `artifact_is_available_in_revisioned_local_cache()` - if not deploying to remote, and revisioned cache exists, use it directly - if deploying to remote, reversioned is not checked and not created - if deploying to remote, force `DEB_COMPRESS=xz` - if deploying to remote, completely remove the local cache base dir after upload is done (no more junk leftover)
2023-08-10 14:17:40 +00:00
do_with_default_build obtain_complete_artifact < /dev/null
fi
}