AArch64: Drain scheduler wakeups in wakeup IPIs

Commit bb03b896 (remove linear scan) moved cross-CPU scheduler
wakeups into a per-CPU ipi_context_wakeup_list. Its x86 wakeup handler
drains that list before completing the IPI, but the AArch64 handler
only completed the SGI.

As a result, a context assigned to another AArch64 CPU could remain
blocked indefinitely. Which process and CPU encountered the missing
wakeup varied between boots, producing stalls at unrelated points
during userspace startup.

Mirror the x86 contract by creating a CleanLockToken and draining
the current CPU wakeup queue before ending the SGI.

Signed-off-by: Luiz Fernando Becher de Araujo <luiz.becher.araujo@gmail.com>
This commit is contained in:
Luiz Fernando Becher de Araujo
2026-08-23 19:37:08 +00:00
parent f2b6b6b803
commit 0787855dce
+11 -2
View File
@@ -1,6 +1,8 @@
use core::sync::atomic::Ordering;
use crate::percpu::PercpuBlock;
use crate::{
context::switch::drain_ipi_context_wakeups, percpu::PercpuBlock, sync::CleanLockToken,
};
#[derive(Clone, Copy, Debug)]
#[repr(u8)]
@@ -50,7 +52,14 @@ pub fn ipi_single(kind: IpiKind, target: &PercpuBlock) {
pub(crate) fn handle(hwirq: u32, raw_iar: u32) -> bool {
let handled = match hwirq {
x if x == IpiKind::Wakeup as u32 => true,
x if x == IpiKind::Wakeup as u32 => {
// Cross-CPU scheduler wakeups are queued per CPU. Drain this CPU's
// queue before completing the SGI, otherwise contexts assigned to
// this CPU can remain blocked indefinitely.
let mut token = unsafe { CleanLockToken::new() };
drain_ipi_context_wakeups(&mut token);
true
}
x if x == IpiKind::Tlb as u32 => {
unsafe { core::arch::asm!("dmb ish", options(nostack, preserves_flags)) };
PercpuBlock::current().maybe_handle_tlb_shootdown();