armbian-build/lib/tools/common/armbian_utils.py

70 lines
1.8 KiB
Python
Raw Normal View History

armbian-next: Python patching delusion, pt1 & pt2 & pt3 - WiP: Python patching delusion, pt 1: finding & parsing patches; apply & git commit with pygit2; Markdown summaries (also for aggregation); git-to-patches tool - Python: Markdown aggregation and patching summaries; collapsible; SummarizedMarkdownWriter - Python: Markdown aggregation and patching summaries - Python: reorg a bit into common/armbian_utils; define the `ASSET_LOG_BASE` in preparation for Markdown delusion - Python patching: initial apply patches & initial commit patches to git (using pygit2) - Python patching: add basic `series.conf` support - Python patching: force use of utf-8; better error handling; use realpath of dirs - Python patching: `git-to-patches` initial hack. not proud. half-reused some of the patches-to-git - Python patching: "tag" the git commits with info for extracting later; introduce REWRITE_PATCHES/rewrite_patches_in_place - Python patching: commented-out, recover-bad-patches hacks - Python patching: shorten the signature - Python patching: allow BASE_GIT_TAG as well as BASE_GIT_REVISION - Python patching: git-archeology for patches missing descriptions; avoid UTF-8 in header/desc (not diff) - Python patching: use modern-er email.utils.parsedate_to_datetime to parse commit date - Python patching: unify PatchInPatchFile; better git-commiting; re-exporting patches from Git (directly) - Python patching: switch to GitPython - GitPython is like 100x slower than pygit2, but actually allows for date & committer - also allows to remove untracked files before starting - Python aggregation: fix missing `AGGREGATED_APT_SOURCES_DICT` - Python patching: add `unidecode` dependency to pip3 install - Python patching: don't try archeology if SRC is not a Git Repo (eg, in Docker) - Python patching: don't try archeology if not applying patches to git - WiP: Python patching delusion, pt2: actually use for u-boot & kernel patching - Python patching: much better problem handling/logging; lenient with recreations (kernel) - Python patching: don't force SHOW_LOG for u-boot patching - Python patching: don't bomb for no reason when there are no patches to apply - Python patching: fully (?) switch kernel patching to Python - Python patching: more logging fixups - Python patching: capture `kernel_git_revision` from `fetch_from_repo()`'s `checked_out_revision` - Python patching: fully switch u-boot patching to Python - Python aggregation/patching: colored logging; patching: always reset to git revision - Python aggregation/patching: better logging; introduce u-boot Python patching - Python patching pt3: recovers and better Markdown - Python patching: detect, and rescue, `wrong_strip_level` problem; don't try to export patches that didn't apply, bitch instead - Python patching: Markdown patching summary table, complete with emoji - Python patching: include the problem breakdown in Markdown summary - Python patching: sanity check against half-bare, half-mbox patches - Python patching: try to recover from 1) bad utf-8 encoded patches; 2) bad unidiff patches; add a few sanity checks - Python patching: try, and fail, to apply badly utf-8 encoded patches directly as bytes [reverted] - Python patching: try to recover from patch *parse* failures; show summary; better logging - set `GIT_ARCHEOLOGY=yes` to do archeology, default not - armbian-next: Python `pip` dependencies handling, similar to `hostdeps` - same scheme for Dockerfile caching - @TODO: still using global/shared environment; should move to a dir under `cache` or some kinda venv - WiP: add `python3-pip` to hostdeps; remove `python-setuptools` - remove `python-setuptools` (Python2, no longer exists in Sid) from hostdeps - add `python3-pip` to hostdeps; part of virtualenv saga - WiP: split `kernel.sh` a bit, into `kernel-patching.sh`, `kernel-config.sh` and `kernel-make.sh` - `advanced_patch()`: rename vars for clarity; no real changes - Python patching: introduce FAST_ARCHEOLOGY; still trying for Markdown links
2022-12-01 15:09:26 +00:00
import logging
import os
import sys
log: logging.Logger = logging.getLogger("armbian_utils")
def parse_env_for_tokens(env_name):
result = []
# Read the environment; if None, return an empty list.
val = os.environ.get(env_name, None)
if val is None:
return result
# tokenize val; split by whitespace, line breaks, commas, and semicolons.
# trim whitespace from tokens.
return [token for token in [token.strip() for token in (val.split())] if token != ""]
def get_from_env(env_name):
value = os.environ.get(env_name, None)
if value is not None:
value = value.strip()
return value
def get_from_env_or_bomb(env_name):
value = get_from_env(env_name)
if value is None:
raise Exception(f"{env_name} environment var not set")
if value == "":
raise Exception(f"{env_name} environment var is empty")
return value
def yes_or_no_or_bomb(value):
if value == "yes":
return True
if value == "no":
return False
raise Exception(f"Expected yes or no, got {value}")
def show_incoming_environment():
log.debug("--ENV-- Environment:")
for key in os.environ:
log.debug(f"--ENV-- {key}={os.environ[key]}")
def setup_logging():
try:
import coloredlogs
level = "INFO"
if get_from_env("LOG_DEBUG") == "yes":
level = "DEBUG"
format = "%(message)s"
styles = {
'trace': {'color': 'white', },
'debug': {'color': 'white'},
'info': {'color': 'white', 'bold': True},
'warning': {'color': 'yellow', 'bold': True},
'error': {'color': 'red'},
'critical': {'bold': True, 'color': 'red'}
}
coloredlogs.install(level=level, stream=sys.stderr, isatty=True, fmt=format, level_styles=styles)
except ImportError:
level = logging.INFO
if get_from_env("LOG_DEBUG") == "yes":
level = logging.DEBUG
logging.basicConfig(level=level, stream=sys.stderr)