summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorEric Dumazet <dada1@cosmosbay.com>2008-11-14 08:53:54 (GMT)
committerDavid S. Miller <davem@davemloft.net>2008-11-14 08:53:54 (GMT)
commitef711cf1d156428d4c2911b8c86c6ce90519dc45 (patch)
tree3c5f5f34d70585d7bd2455f288fe7efe4b272710
parentf30ab418a1d3c5a8b83493e7d70d6876a74aa0ce (diff)
downloadlinux-ef711cf1d156428d4c2911b8c86c6ce90519dc45.tar.xz
net: speedup dst_release()
During tbench/oprofile sessions, I found that dst_release() was in third position. CPU: Core 2, speed 2999.68 MHz (estimated) Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 100000 samples % symbol name 483726 9.0185 __copy_user_zeroing_intel 191466 3.5697 __copy_user_intel 185475 3.4580 dst_release 175114 3.2648 ip_queue_xmit 153447 2.8608 tcp_sendmsg 108775 2.0280 tcp_recvmsg 102659 1.9140 sysenter_past_esp 101450 1.8914 tcp_current_mss 95067 1.7724 __copy_from_user_ll 86531 1.6133 tcp_transmit_skb Of course, all CPUS fight on the dst_entry associated with 127.0.0.1 Instead of first checking the refcount value, then decrement it, we use atomic_dec_return() to help CPU to make the right memory transaction (ie getting the cache line in exclusive mode) dst_release() is now at the fifth position, and tbench a litle bit faster ;) CPU: Core 2, speed 3000.1 MHz (estimated) Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 100000 samples % symbol name 647107 8.8072 __copy_user_zeroing_intel 258840 3.5229 ip_queue_xmit 258302 3.5155 __copy_user_intel 209629 2.8531 tcp_sendmsg 165632 2.2543 dst_release 149232 2.0311 tcp_current_mss 147821 2.0119 tcp_recvmsg 137893 1.8767 sysenter_past_esp 127473 1.7349 __copy_from_user_ll 121308 1.6510 ip_finish_output 118510 1.6129 tcp_transmit_skb 109295 1.4875 tcp_v4_rcv Signed-off-by: Eric Dumazet <dada1@cosmosbay.com> Signed-off-by: David S. Miller <davem@davemloft.net>
-rw-r--r--net/core/dst.c6
1 files changed, 4 insertions, 2 deletions
diff --git a/net/core/dst.c b/net/core/dst.c
index 09c1530..57bc4d5 100644
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -263,9 +263,11 @@ again:
void dst_release(struct dst_entry *dst)
{
if (dst) {
- WARN_ON(atomic_read(&dst->__refcnt) < 1);
+ int newrefcnt;
+
smp_mb__before_atomic_dec();
- atomic_dec(&dst->__refcnt);
+ newrefcnt = atomic_dec_return(&dst->__refcnt);
+ WARN_ON(newrefcnt < 0);
}
}
EXPORT_SYMBOL(dst_release);