Commit bb03b896 (remove linear scan) moved cross-CPU scheduler
wakeups into a per-CPU ipi_context_wakeup_list. Its x86 wakeup handler
drains that list before completing the IPI, but the AArch64 handler
only completed the SGI.
As a result, a context assigned to another AArch64 CPU could remain
blocked indefinitely. Which process and CPU encountered the missing
wakeup varied between boots, producing stalls at unrelated points
during userspace startup.
Mirror the x86 contract by creating a CleanLockToken and draining
the current CPU wakeup queue before ending the SGI.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
irq_trigger() notifies event queues when an interrupt arrives, but an
interrupt can occur after a driver opens the IRQ handle and before it
subscribes the descriptor to an event queue. In that window kreadoff()
reports data while fevent() reports no readiness, so the driver can
sleep with an unacknowledged interrupt.
Make fevent() use the same per-handle acknowledged count and global
interrupt count as kreadoff(). Report EVENT_READ when the requested
handle has a pending count, closing the subscription race without
changing the existing acknowledgement protocol.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
IRQ handles currently reconstruct their reservation when they are
closed. That assumes every handle stores an x86 IRQ number and applies
irq_to_vector(), but devicetree phandle handles already store a
virtual IRQ. Closing such a handle therefore releases the wrong
descriptor. The devicetree implementation also enables the interrupt
in both branches of set_reserved(), so a release cannot mask it.
Store the exact CPU and index when a reservation is acquired and use
that information when the handle is closed. Handles opened only for
stat or through unreserved legacy paths carry no reservation. Also
reject phandle IRQ mappings that cannot be represented by the scheme
counter table, and disable devicetree IRQs when their reservation is
released.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
Select the ACPI timer GSIV that matches the timer being programmed,
derive the GICv2 CPU target mask instead of assuming interface zero,
and route SPIs exclusively to the BSP when that mapping is available.
Preserve firmware routing and disable SMP safely when no valid target
exists, remove blocking diagnostics from interrupt handlers, skip
unnecessary boot-resource and TCR validation on uniprocessor builds,
remove unused CPUECTLR helpers, and document the current GICv2-only
secondary boot limitation.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
Treat a zero GICD_ITARGETSR target mask as valid during BSP
initialization because GICv2 target registers may be RAZ/WI on
uniprocessor implementations, while retaining fail-closed SMP
behavior by requiring a valid target mask before starting secondary
CPUs and after each AP initializes its local GIC interface.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
Initialize the PSCI conduit from the FADT ARM boot flags, discover
enabled CPUs and their MPIDRs from MADT GICC entries, and feed the
resulting topology into the existing physical trampoline boot path,
enabling SMP on UEFI/ACPI platforms while preserving single-CPU
operation.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
Use the StallWatch helper added earlier to report a spin loop that has
run far longer than any legitimate SMP handshake should take -- the
TLB shootdown wait and the context-switch-lock acquisition -- with one
diagnostic naming which CPU and, for the shootdown case, how many
acknowledgements are outstanding. Purely diagnostic: the loop keeps
spinning exactly as before, this only makes a stuck wait visible
instead of looking like a silent hang.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
Prepare, during BSP initialization, everything a secondary CPU needs:
a 16-page kernel stack, a PercpuBlock, and an args page for every
PSCI-enabled secondary in the discovered topology. Also prepare a
physical trampoline page, copied from a small hand-written assembly
routine, cleaned to PoC, instruction-cache synchronized, and identity
mapped through a dedicated TTBR0.
Each BootArgs contains the CPU identity, physical and virtual stack
tops, the virtual per-CPU address, the trampoline identity-map TTBR0,
the kernel TTBR1, the BSP's TCR/MAIR/SCTLR/VBAR/CPACR values, the
virtual Rust entry point, and an atomic state word.
Fail closed and leave the board running on one CPU if resources cannot
be reserved, if any secondary does not use PSCI, or if TCR_EL1 does
not make both TTBR0 and TTBR1 walks Inner Shareable. PTE.SH alone
cannot make page-table entries coherent between walkers, and this boot
path inherits TCR from the bootloader instead of reprogramming it
while the kernel is live.
start_secondaries() calls PSCI_CPU_ON for each slot with the physical
trampoline and BootArgs addresses. It observes the shared state word
advance through STARTING -> LOCAL_READY -> READY -> ONLINE and sends a
directed SGI after LOCAL_READY to exercise actual IPI delivery before
publishing the CPU. CPU startup and local interrupt self-tests are
bounded by two-second timeouts; failures stop further CPU activation
instead of exposing a partially initialized SMP topology.
The trampoline accepts PSCI entry at EL1 or EL2, normalizing EL2 to
EL1h when necessary. PSCI supplies the context address in x0 and
enters with stage-1 translation and caches disabled. The trampoline
loads the prepared translation registers, invalidates the local TLB,
enables the kernel translation regime, installs the stack, per-CPU
pointer, exception vector and FP configuration, and branches to the
virtual Rust entry point.
Cache initialization and coherency-domain management are deliberately
left to the PSCI implementation, as required by the PSCI contract.
The trampoline therefore does not probe CPUECTLR_EL1.SMPEN, inspect
cache geometry, or invalidate private cache levels.
secondary_start() validates the magic, MPIDR, logical ID, state and
per-CPU pointer before continuing. It installs the per-CPU block,
initializes the local GIC interface and timer, removes the temporary
identity map, and requires both a directed SGI and a local timer
interrupt before advancing from LOCAL_READY to READY.
CPU_COUNT is advanced only by publish_cpu(), whose compare_exchange
requires the published logical ID to equal the current CPU count.
Because secondaries are started in order, the visible CPU set remains
a contiguous prefix of logical IDs.
shootdown_tlb_ipi() skips the requesting CPU in its broadcast path.
Callers invalidate their local TLB directly, so sending an IPI back to
the requester would be redundant.
Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
4lDO2
Ibuki Omatsu
Valentin Lorentz
Wildan M
MJ Pooladkhay
R Aadarsh
Akshit Gaur
aarch
Luiz Fernando Becher de Araujo