From: Or Kehati Date: Wed, 29 Oct 2014 14:32:04 +0000 (+0200) Subject: IB/addr: Improve address resolution callback scheduling X-Git-Url: https://git.karo-electronics.de/?a=commitdiff_plain;h=346f98b41b76281bd0b748fb86bc1953c9fd9fe2;p=linux-beck.git IB/addr: Improve address resolution callback scheduling Address resolution always does a context switch to a work-queue to deliver the address resolution event. When the IP address is already cached in the system ARP table, we're going through the following: chain: rdma_resolve_ip --> addr_resolve (cache hit) --> which ends up with: queue_req --> set_timeout (now) --> mod_delayed_work(,, delay=1) We actually do realize that the timeout should be zero, but the code forces it to a minimum of one jiffie. Using one jiffie as the minimum delay value results in sub-optimal scheduling of executing this work item by the workqueue, which on the below testbed costs about 3-4ms out of 12ms total time. To fix that, we let the minimum delay to be zero. Note that the connect step times change too, as there are address resolution calls from that flow. The results were taken from running both client and server on the same node, over mlx4 RoCE port. before --> step total ms max ms min us us / conn create id : 0.01 0.01 6.00 6.00 resolve addr : 4.02 4.01 4013.00 4016.00 resolve route: 0.18 0.18 182.00 183.00 create qp : 1.15 1.15 1150.00 1150.00 connect : 6.73 6.73 6730.00 6731.00 disconnect : 0.55 0.55 549.00 550.00 destroy : 0.01 0.01 9.00 9.00 after --> step total ms max ms min us us / conn create id : 0.01 0.01 6.00 6.00 resolve addr : 0.05 0.05 49.00 52.00 resolve route: 0.21 0.21 207.00 208.00 create qp : 1.10 1.10 1104.00 1104.00 connect : 1.22 1.22 1220.00 1221.00 disconnect : 0.71 0.71 713.00 713.00 destroy : 0.01 0.01 9.00 9.00 Signed-off-by: Or Kehati Signed-off-by: Or Gerlitz Acked-by: Sean Hefty Signed-off-by: Roland Dreier --- diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c index 8172d37f9add..f80da50d84a5 100644 --- a/drivers/infiniband/core/addr.c +++ b/drivers/infiniband/core/addr.c @@ -176,8 +176,8 @@ static void set_timeout(unsigned long time) unsigned long delay; delay = time - jiffies; - if ((long)delay <= 0) - delay = 1; + if ((long)delay < 0) + delay = 0; mod_delayed_work(addr_wq, &work, delay); }