qiurui
/
Centos-kernel-stream-9
mirror of https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

rcu-tasks: Stop RCU Tasks Trace from scanning full tasks list 

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2169516

commit 1a4a8153e0df1705397aa3ff561dd8fdc2e6fc23
Author: Paul E. McKenney <paulmck@kernel.org>
Date:   Fri, 20 May 2022 10:21:00 -0700

    rcu-tasks: Stop RCU Tasks Trace from scanning full tasks list

    This commit takes off the training wheels and relies only on scanning
    currently running tasks and tasks that have blocked or been preempted
    within their current RCU Tasks Trace read-side critical section.

    Before this commit, the time complexity of an RCU Tasks Trace grace
    period is O(T), where T is the number of tasks.  After this commit,
    this time complexity is O(C+B), where C is the number of CPUs and B
    is the number of tasks that have blocked (or been preempted) at least
    once during their current RCU Tasks Trace read-side critical sections.
    Of course, if all tasks have blocked (or been preempted) at least once
    during their current RCU Tasks Trace read-side critical sections, this is
    still O(T), but current expectations are that RCU Tasks Trace read-side
    critical section will be short and that there will normally not be large
    numbers of tasks blocked within such a critical section.

    Dave Marchevsky kindly measured the effects of this commit on the RCU
    Tasks Trace grace-period latency and the rcu_tasks_trace_kthread task's
    CPU consumption per RCU Tasks Trace grace period over the course of a
    fixed test, all in milliseconds:

                    Before                  After

    GP latency      22.3 ms stddev > 0.1    17.0 ms stddev < 0.1

    GP CPU           2.3 ms stddev 0.3       1.1 ms stddev 0.2

    This was on a system with 15,000 tasks, so it is reasonable to expect
    much larger savings on the systems on which this issue was first noted,
    given that they sport well in excess of 100,000 tasks.  CPU consumption
    was measured using profiling techniques.

    Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
    Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
    Cc: Eric Dumazet <edumazet@google.com>
    Cc: Alexei Starovoitov <ast@kernel.org>
    Cc: Andrii Nakryiko <andrii@kernel.org>
    Cc: Martin KaFai Lau <kafai@fb.com>
    Cc: KP Singh <kpsingh@kernel.org>
    Tested-by: Dave Marchevsky <davemarchevsky@fb.com>

Signed-off-by: Waiman Long <longman@redhat.com>
This commit is contained in:
Waiman Long 2023-02-26 22:21:30 -05:00
parent d7888512e9
commit 335b311cba
1 changed files with 6 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
kernel/rcu/tasks.h
View File

@ -671,10 +671,12 @@ static void rcu_tasks_wait_gp(struct rcu_tasks *rtp)
* and make a list of them in holdouts. * and make a list of them in holdouts.
*/ */
set_tasks_gp_state(rtp, RTGS_SCAN_TASKLIST); set_tasks_gp_state(rtp, RTGS_SCAN_TASKLIST);
rcu_read_lock(); if (rtp->pertask_func) {
for_each_process_thread(g, t) rcu_read_lock();
rtp->pertask_func(t, &holdouts); for_each_process_thread(g, t)
rcu_read_unlock(); rtp->pertask_func(t, &holdouts);
rcu_read_unlock();
}
set_tasks_gp_state(rtp, RTGS_POST_SCAN_TASKLIST); set_tasks_gp_state(rtp, RTGS_POST_SCAN_TASKLIST);
rtp->postscan_func(&holdouts); rtp->postscan_func(&holdouts);
@ -1747,7 +1749,6 @@ static int __init rcu_spawn_tasks_trace_kthread(void)
rcu_tasks_trace.init_fract = 1; rcu_tasks_trace.init_fract = 1;
} }
rcu_tasks_trace.pregp_func = rcu_tasks_trace_pregp_step; rcu_tasks_trace.pregp_func = rcu_tasks_trace_pregp_step;
rcu_tasks_trace.pertask_func = rcu_tasks_trace_pertask;
rcu_tasks_trace.postscan_func = rcu_tasks_trace_postscan; rcu_tasks_trace.postscan_func = rcu_tasks_trace_postscan;
rcu_tasks_trace.holdouts_func = check_all_holdout_tasks_trace; rcu_tasks_trace.holdouts_func = check_all_holdout_tasks_trace;
rcu_tasks_trace.postgp_func = rcu_tasks_trace_postgp; rcu_tasks_trace.postgp_func = rcu_tasks_trace_postgp;