diff options
author | Eric Dumazet <dada1@cosmosbay.com> | 2006-12-07 04:38:44 (GMT) |
---|---|---|
committer | Linus Torvalds <torvalds@woody.osdl.org> | 2006-12-07 16:39:40 (GMT) |
commit | 1c69d921ed9cc6593ad4f60c0f9951cb0d62b0b4 (patch) | |
tree | 669149c112fd917d68a301b2bcceae681af17a1c | |
parent | 9a0efbb8795e8c5acb279d984b53d6af6ecdd00f (diff) | |
download | linux-1c69d921ed9cc6593ad4f60c0f9951cb0d62b0b4.tar.xz |
[PATCH] rcu: add a prefetch() in rcu_do_batch()
On some workloads, (for example when lot of close() syscalls are done), RCU
qlen can be quite large, and RCU heads are no longer in cpu cache when
rcu_do_batch() is called.
This patch adds a prefetch() in rcu_do_batch() to give CPU a hint to bring
back cache lines containing 'struct rcu_head's.
Most list manipulations macros include prefetch(), but not open coded ones
(at least with current C compilers :) )
I got a nice speedup on a trivial benchmark (3.48 us per iteration instead
of 3.95 us on a 1.6 GHz Pentium-M)
while (1) { pipe(p); close(fd[0]); close(fd[1]);}
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
-rw-r--r-- | kernel/rcupdate.c | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/kernel/rcupdate.c b/kernel/rcupdate.c index 26bb5ff..3554b76 100644 --- a/kernel/rcupdate.c +++ b/kernel/rcupdate.c @@ -235,12 +235,14 @@ static void rcu_do_batch(struct rcu_data *rdp) list = rdp->donelist; while (list) { - next = rdp->donelist = list->next; + next = list->next; + prefetch(next); list->func(list); list = next; if (++count >= rdp->blimit) break; } + rdp->donelist = list; local_irq_disable(); rdp->qlen -= count; |