qiurui
/
Centos-kernel-stream-9
mirror of https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

rcu: Throttle callback invocation based on number of ready callbacks 

JIRA: https://issues.redhat.com/browse/RHEL-5228

commit 253cbbff621407a6265ce7a6a03c3766f8846f02
Author: Paul E. McKenney <paulmck@kernel.org>
Date:   Mon, 14 Nov 2022 09:40:19 -0800

    rcu: Throttle callback invocation based on number of ready callbacks

    Currently, rcu_do_batch() sizes its batches based on the total number
    of callbacks in the callback list.  This can result in some strange
    choices, for example, if there was 12,800 callbacks in the list, but
    only 200 were ready to invoke, RCU would invoke 100 at a time (12,800
    shifted down by seven bits).

    A more measured approach would use the number that were actually ready
    to invoke, an approach that has become feasible only recently given the
    per-segment ->seglen counts in ->cblist.

    This commit therefore bases the batch limit on the number of callbacks
    ready to invoke instead of on the total number of callbacks.

    Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

Signed-off-by: Waiman Long <longman@redhat.com>
This commit is contained in:
Waiman Long 2023-09-22 13:21:21 -04:00
parent b2f1e8fb35
commit 268ded33a8
3 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
kernel/rcu/rcu_segcblist.c
View File

@ -89,7 +89,7 @@ static void rcu_segcblist_set_len(struct rcu_segcblist *rsclp, long v)
}
/* Get the length of a segment of the rcu_segcblist structure. */
static long rcu_segcblist_get_seglen(struct rcu_segcblist *rsclp, int seg)
long rcu_segcblist_get_seglen(struct rcu_segcblist *rsclp, int seg)
{
return READ_ONCE(rsclp->seglen[seg]);
}

2
kernel/rcu/rcu_segcblist.h
View File

@ -15,6 +15,8 @@ static inline long rcu_cblist_n_cbs(struct rcu_cblist *rclp)
return READ_ONCE(rclp->len);
}
long rcu_segcblist_get_seglen(struct rcu_segcblist *rsclp, int seg);
/* Return number of callbacks in segmented callback list by summing seglen. */
long rcu_segcblist_n_segment_cbs(struct rcu_segcblist *rsclp);

2
kernel/rcu/tree.c
View File

@ -2063,7 +2063,7 @@ static void rcu_do_batch(struct rcu_data *rdp)
*/
rcu_nocb_lock_irqsave(rdp, flags);
WARN_ON_ONCE(cpu_is_offline(smp_processor_id()));
pending = rcu_segcblist_n_cbs(&rdp->cblist);
pending = rcu_segcblist_get_seglen(&rdp->cblist, RCU_DONE_TAIL);
div = READ_ONCE(rcu_divisor);
div = div < 0 ? 7 : div > sizeof(long) * 8 - 2 ? sizeof(long) * 8 - 2 : div;
bl = max(rdp->blimit, pending >> div);