mirror of https://github.com/armbian/build.git
29 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Ricardo Pardini
|
bef7f8fee4
|
armbian-next: artifacts: introduce kernel/u-boot artifacts; git ref2info; "memoizer"; some hashing
- artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()`
- artifacts: u-boot - pt6: add artifact for u-boot
- use artifact version / reason in actual u-boot .deb if present
- artifacts: kernel - pt5: tune kernel version, refactor
- artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build
- artifacts: kernel - pt3: drivers+patches+.config hashing
- split file hashing function from drivers-harness; fix it so filenames are relative and sorted; sort from ${SRC}, always
- aplit prepare_kernel_config_core_or_userpatches() from `kernel_config_initialize()`
- artifacts: kernel - pt2: memoizing git ref2info
- artifacts: kernel - pt1: versioning
|
|
|
Ricardo Pardini
|
9dc24840c3
|
armbian-next: firmware-deb: reworked: Provides/Conflicts instead of Replaces; `cp` -> `git archive`; new firmware CLI; FULL version .deb out of tmpfs | |
|
Ricardo Pardini
|
817deb0c53
|
armbian-next: drop buildpkg code | |
|
Ricardo Pardini
|
1cec42392e
|
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version) - add "oras-upload" CLI command (dumb, no retries, single target/single source) - docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker - introduce `github-actions.sh::github_actions_add_output()` - during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available |
|
|
Ricardo Pardini
|
28d89e2c91
|
armbian-next: `rootfs`: new CLI command for rootfs cache building; further refactorings
- _told ya it would all make sense eventually..._ - introduce new, real, non-alias `rootfs` command in `cli-rootfs` -- this _only_ builds rootfs, nothing else - no more `main_default_build_single()`; instead `full_build_packages_rootfs_and_image()` - introduce `do_with_default_build()` wrapper, for things that need host prepared + aggregation to run - CLI-specific parts of `main_default_build_single()` moved to cli-build - put prepare/cleanup pair of `prepare_rootfs_build_params_and_trap()` and `trap_handler_cleanup_rootfs_and_image()` into new `trap-rootfs.sh` (this needs further splitting for the image part) - refactor `get_or_create_rootfs_cache_chroot_sdcard()` into `build_rootfs_only()`, `calculate_rootfs_cache_id()` - update library with new files |
|
|
Ricardo Pardini
|
eb30aac310
|
armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function
- wraps globals in `extension_manager_declare_globals()` -- and make them actually global - whoever wrote this original code is insane, add copyright |
|
|
Ricardo Pardini
|
88ff66ae11
|
armbian-next: adaptive shallow/full Kernel git bare git (3gb -> 300mb download)
- allows using pre-shallow-ed bare trees, specific to the KERNEL_MAJOR_MINOR - uses shallow by default if storage is "slow" (mmc), or free disk space < 32g (@TODO tweak this) - if full tree already there keeps using it - allow forcing decision with `KERNEL_GIT=full` or `KERNEL_GIT=shallow` - countdown to confirm if decision based on storage type/free space and user is interactive - this has some of the worst copy ever written - @TODO refactor the device type / free space code, reuse across codebase (3-4 other spots detect free space) - @TODO GHA self-hosted (full) vs hosted (shallow) runners decision - sprinkle with `wait_for_disk_sync()` around clean/download/extract - sprinkle git with debugs, and an info between fetch and checkout (otherwise it seems "fetch" takes a long time) |
|
|
Ricardo Pardini
|
8af14d4f8c
|
armbian-next: `countdown`: split off file for countdown functions (countdown-to-abort and countdown-to-continue)
- introduce `countdown.sh`, move `traps.sh::exit_if_countdown_not_aborted()` there, introduce `exit_if_countdown_not_aborted()` |
|
|
Ricardo Pardini
|
0ea0fe3528
|
armbian-next: `mktemp` and `tmpfs` related utility functions with automatic cleanup handlers | |
|
Ricardo Pardini
|
92d902c883
|
armbian-next: drop dead/duplicated code for bootsplash and mark kernel-drivers.sh (which includes bootsplash) as dead code | |
|
Ricardo Pardini
|
2455c55dac
|
armbian-next: firmware: don't build `-full` firmware if not on CI/noninteractive and board's not going to use it
- move `fakeroot_dpkg_deb_build()` from runners.sh to new utils-dpkgdeb.sh - and make it debug show the size of the source - clean off `.git` directory earlier for full build - @TODO: get rid of the "hardlinks" funky biz in there and not-copy `.git` to begin with |
|
Tony
|
f549ddb273
|
armbian-next: officially support WSL2; pester user for UTF-8 terminal
- new `wsl2.sh` file with WSL2-related functions; checking/validating/pestering user; allow WSL2 (not WSL1) Co-authored-by: Ricardo Pardini <ricardo@pardini.net> |
|
|
Ricardo Pardini
|
2d9f9216eb
|
armbian-next: introduce `tmpfs-utils.sh`; put `LOGDIR` and `WORKDIR` under tmpfs; set `CCACHE_TEMPDIR` under `WORKDIR`
- introduce generic `prepare_tmpfs_for()`, which manages it's own cleanup/dir removal - use it for `WORKDIR` (which is `TMPDIR`) and `LOGDIR` - adapt previous cleanup handlers for those, so they delete their contents but not the dir itself (which might be mounted) - also: make `ARMBIAN_LOG_CLI_ID` readonly together with other superglobals - set `XDG_RUNTIME_DIR` & `XDG_RUNTIME_DIR` together with `TMPDIR` - kernel-make.sh: pass `CCACHE_TEMPDIR` down to Kernel make (thanks @the-Going) |
|
|
Ricardo Pardini
|
4d17aa2a98
|
armbian-next: `rootfs`: bunch'o'fixes, introduce `disable_systemd_service_sdcard()` to stop repeating incantantion
- fix a bunch of quoting issues - use runners for chroot stuff - don't error out of failed disable - makes building `sid` again possible - add a bunch of TODO's |
|
|
Ricardo Pardini
|
53266feff8
|
armbian-next: kernel: deterministic `.config` mtime handling; kernel build logging sections reorg
- .config is now compared with previous one byte-wise, diff shown, and mtime preserved directly via `cp -p` - this does away with the previous mtime-based BS I had written during the fasthash era (and thus remove file-mtime.sh, regen lib) - split some functions, to better control the interactive piece in the _middle_ of kernel config process - no more `kernel_config_maybe_interactive()`, now `kernel_config()` that controls its own logging sections - extract `kernel_determine_toolchain()` out of config into make - introduce internal hook `armbian_kernel_config()` that runs before custom version (for .config defaults, coming soon) |
|
|
Ricardo Pardini
|
d979c73c1d
|
armbian-next: split `compilation/debs.sh`; `compile_xilinx_bootgen()` moved to family | |
|
Ricardo Pardini
|
6cb12a3397
|
armbian-next: [focal-host] try to support `focal`-like host OS's again, via `python3.9` (and drop `buster` completely)
- completely remove support for building under `buster` -- that's way too old, sorry. - de-hardcode `python3` invocations, instead use `python3_binary_path` set by `prepare_python3_binary_for_python_tools()` - juggle `$HOSTRELEASE`: read from actual host, or determined from Docker image name (during Dockerfile build) - TL;DR: include and use `python3.9` for focal-like host OS's |
|
|
Ricardo Pardini
|
f167864f2c
|
armbian-next: split of `config-prepare` into multiple functions, allowing for logging of all non-interactive sections
- introduce `do_with_conditional_logging()` which only starts logging sections if `do_logging=no` - with this we should get complete logs (ofc except for the interactive sections) |
|
|
Ricardo Pardini
|
8399c5d72b
|
armbian-next: **regen library** after some 13 squashes (and repo.sh removal) | |
|
Ricardo Pardini
|
074857ede3
|
armbian-next: u-boot: use `git worktree` for u-boot; full mainline clone for bare
- share a single `.git` for all u-boots - all fetches are done against it, maximizing cache hit ratio drastically - also reduces the size of each working copy by more than 70% - split uboot's git stuff into uboot-git.sh - split `uboot_prepare_git()` from `compile_uboot()` |
|
|
Ricardo Pardini
|
84c0bf0f7d
|
armbian-next: kernel: use `git worktree` & kernel.org clone bundles for kernel working copies
- now we've a single `.git` for all kernels with all branches; a "master" cache - back to full stable git bundle usage from kernel.org - bye bye, "gitballs" - split kernel-related git stuff into `kernel-git.sh` - introduce `ARMBIAN_MOUNTPOINTS_DEPRECATED` @TODO actually use for cleaning non-Docker - add `axel` hostdep for multi-conn download of bundle, which comes from a CDN. - deprecate old mountpoints for gitballs and linux-kernel - move cleaning of old sources/kernel to mountpoints code |
|
|
Ricardo Pardini
|
adce39d3b7
|
armbian-next: `aggregation.py` rewrite; introduce `package-lists.sh`; usage in core lib | |
|
Ricardo Pardini
|
b2d02071bd
|
armbian-next: better `ccache` logs/stats/Docker-support/use in u-boot/kernel
- introduce `SHOW_CCACHE=yes` for detailed ccache statistics and logging
- fix: kernel build is done under "env -i", so pass CCACHE_DIR down to Make if CCACHE_DIR is set...
- split from kernel.sh, show stats also for u-boot targets; show compile time
- add volume definition (under `${SRC}/cache/ccache`); auto-use that in Docker builds via `CCACHE_DIR`
- better logging for `do_with_ccache_statistics()`
- there's some CCACHE_DIR code from before; unify @TODO
|
|
|
Ricardo Pardini
|
093336e914
|
armbian-next: `cli`: `distccd` launcher for build farm
- relaunched under docker - possibly much nicer if zeroconf enabled? - this thing is ignoring signals. can't ctrl-c out of it. why? @TODO - pt 1, subpt 3: distccd - distcc-pump - avahi - pt 1, subpt 3: distccd + distcc-pump - pt 1, subpt 2: distccd + avahi - pt 1, subpt 1: distccd |
|
|
Ricardo Pardini
|
da0b3080a0
|
armbian-next: `distcc`: optionally using `distcc` for kernel build | |
|
Ricardo Pardini
|
a9e88f58e5
|
armbian-next: `json-info` command, wrapping up the Python tooling; running in Docker; fix config dump | |
|
Ricardo Pardini
|
aaa59261bb
|
armbian-next: `mountpoints.sh` with all bind/volume directories definitions and looping func; use it to determine best bind/volume combination for each host OS in Docker | |
|
Ricardo Pardini
|
d24d3327a8
|
armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng`
- armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker)
- this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for
- apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages
- debootstrap, by passing it `--cache-dir`
- utility function to help understand what is happening to cache during usage
- apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_
- introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s
- don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot
- clean up `install_deb_chroot()` a little, find an unrelated bug there
- WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile
- it's `drastic red` on non-buildx dockers
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x
- source configs in a logging section.
- Docker: silent, fast retries to make sure `docker system df` works
- shut-up `chown` (no `-v`) output related to `SET_OWNER_TO_UID`
- ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow
- use green apple for Mac logging, instead of red apple which might imply error...
- WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker
- introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol
- introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root.
- introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID`
- use it for userpatches files created, logs, and output files, including images and debs.
- @TODOs ref. `$SUDO_USER` which I think the old version of this?
- add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command.
- initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3
- WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented
- WiP: the great cli entrypoint (+docker) rewrite, Phase 1
- armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs.
- using image in my private repo.
- this has significant speedup to "start building time" on the 1st run
- move some Linux specific stuff to its own if
- add comments and todo
- armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences
- armbian-next: WiP: Docker, configure `BUILDKIT_COLORS`
- armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers
- armbian-next: `logging`: add whale indicator if build running under Docker
- armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only
- armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy
- armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots
- WiP: try with debian:bullseye -- can't detect docker at all
- armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working
- gen .dockerignore together with Dockerfile
- split in funcs
- hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes
- mac still works, Linux stuff would break it but I if'fed
- armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing
- this fails for the right reasons now, causing retries, which are then retried and work ;-)
- this is related to building under Docker on Linux, using docker.io package (not docker-ce)
- armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated
- armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build
- IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal
- armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore
- armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY
- uses REQUIREMENTS_DEFS_ONLY
- works on Docker Desktop on Mac;
- linux TBA
- armbian-next: don't error out if `.git` not present; other small fixes
- armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance"
- can't assume things, for example:
- that `sudo` will be available; it might not, and might be already root, no reason to fail
- that `/etc/timezone` will exist
- that `systemd-detect-virt` will be available
- that `git` will be available
- that `locale-gen` will be available
|
|
|
Ricardo Pardini
|
b25b3cf499
|
armbian-next: `lib` changes - MEGASQUASH - squashed changes from c9cf3fc241cfb4c872f4aef7bbc41d5854db7ea3 to 6809de3d6063cb041205a8318e19da6a4dee68c9 ref extensions_08_10_2022_pre_v30
- also compile.sh - shellfmt lib - split off shell and python tools under lib - revert removal of stuff a-n no longer uses (ref. compilation): general packaging, mkdeb etc - editoconfig split off - extension changes split off - sources and sources/families split off - some undue stuff removed or split armbian-next: manual merge (30) of lib changes between |