armbian-build/extensions/detect-unused-extensions.sh

51 lines
2.7 KiB
Bash
Raw Normal View History

extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
## Configuration
export LOG_ALL_HOOK_TRACES=no # Should we log all hook function traces to stdout? (no, or level: wrn info)
## Hooks
# A honeypot wishful hooking. To make sure the whole thing works.
# This is exactly the kind of hooking this extension is meant to detect.
# This will never run, and should be detected below, but is handled specially and ignored.
# Note: this will never run, since we (hopefully) don't have a hook_point called 'wishful_hooking_example'.
function wishful_hooking_example__this_will_never_run() {
echo "WISHFUL HOOKING -- this will never run. I promise."
}
# Run super late, hopefully at the last possible moment.
function extension_metadata_ready__999_detect_wishful_hooking() {
display_alert "Checking extensions and hooks for uncalled hook points"
declare -i found_honeypot_function=0
# Loop over the defined functions' keys. Find the info about the call. If not found, warn the user.
# shellcheck disable=SC2154 # hook-exported variable
for one_defined_function in ${!defined_hook_point_functions[*]}; do
local source_info defined_info line_info
defined_info="${defined_hook_point_functions["${one_defined_function}"]}"
source_info="${hook_point_function_trace_sources["${one_defined_function}"]}"
line_info="${hook_point_function_trace_lines["${one_defined_function}"]}"
stack="$(get_extension_hook_stracktrace "${source_info}" "${line_info}")"
if [[ "$source_info" != "" ]]; then
# log to debug log. it's reassuring.
2022-10-08 11:17:42 +00:00
echo "\$\$\$ Hook function stacktrace for '${one_defined_function}': '${stack}' (${defined_info})" >> "${EXTENSION_MANAGER_LOG_FILE}"
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
if [[ "${LOG_ALL_HOOK_TRACES}" != "no" ]]; then
display_alert "Hook function stacktrace for '${one_defined_function}'" "${stack}" "${LOG_ALL_HOOK_TRACES}"
fi
continue # found a caller, move on.
fi
# special handling for the honeypot function. it is supposed to be always detected as uncalled.
if [[ "${one_defined_function}" == "wishful_hooking_example__this_will_never_run" ]]; then
# we expect this wishful hooking, it is done on purpose below, to make sure this code works.
found_honeypot_function=1
else
# unexpected wishful hooking. Log and wrn the user.
2022-10-08 11:17:42 +00:00
echo "\$\$\$ Wishful hooking detected" "Function '${one_defined_function}' is defined (${defined_info}) but never called by the build." >> "${EXTENSION_MANAGER_LOG_FILE}"
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "Wishful hooking detected" "Function '${one_defined_function}' is defined (${defined_info}) but never called by the build." "wrn"
fi
done
if [[ $found_honeypot_function -lt 1 ]]; then
display_alert "Wishful hook DETECTION FAILED" "detect-wishful-hooking is not working. Good chance the environment vars are corrupted. Avoid child shells. Sorry." "wrn" | tee -a "${EXTENSION_MANAGER_LOG_FILE}"
fi
}