From: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Date: Wed, 10 Jun 2015 20:34:41 +0000 (-0700)
Subject: cpu: Wait for RCU grace periods concurrently
X-Git-Url: https://git.karo-electronics.de/?a=commitdiff_plain;h=779de6ce54f627f955d4a3d0c5b3dcfaab74fea8;p=linux-beck.git

cpu: Wait for RCU grace periods concurrently

In kernels built with CONFIG_PREEMPT, _cpu_down() waits for RCU and
RCU-sched grace periods back-to-back, incurring quite a bit more latency
than required.  This commit therefore uses the new synchronize_rcu_mult()
to allow waiting for both grace periods concurrently.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 9c9c9fab16cc..d63b062b6267 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -380,14 +380,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 	 * will observe it.
 	 *
 	 * For CONFIG_PREEMPT we have preemptible RCU and its sync_rcu() might
-	 * not imply sync_sched(), so explicitly call both.
+	 * not imply sync_sched(), so wait for both.
 	 *
 	 * Do sync before park smpboot threads to take care the rcu boost case.
 	 */
-#ifdef CONFIG_PREEMPT
-	synchronize_sched();
-#endif
-	synchronize_rcu();
+	if (IS_ENABLED(CONFIG_PREEMPT))
+		synchronize_rcu_mult(call_rcu, call_rcu_sched);
+	else
+		synchronize_rcu();
 
 	smpboot_park_threads(cpu);