cpu-hotplug: convert cpu_hotplug_disabled to a counter

As a prerequisite to exporting cpu_hotplug_enable/cpu_hotplug_disable
functions to modules we need to convert cpu_hotplug_disabled to a counter
to properly support disable -> disable -> enable call sequences. E.g.
after Hyper-V vmbus module (which is supposed to be the first user of
exported cpu_hotplug_enable/cpu_hotplug_disable) did cpu_hotplug_disable()
hibernate path calls disable_nonboot_cpus() and if we hit an error in
_cpu_down() enable_nonboot_cpus() will be called on the failure path (thus
making cpu_hotplug_disabled = 0 and leaving cpu hotplug in 'enabled'
state). Same problem is possible if more than 1 module use
cpu_hotplug_disable/cpu_hotplug_enable on their load/unload paths. When
one of these modules is been unloaded it is logical to leave cpu hotplug
in 'disabled' state.

To support the change we need to increse cpu_hotplug_disabled counter
in disable_nonboot_cpus() unconditionally as all users of
disable_nonboot_cpus() are supposed to do enable_nonboot_cpus() in case
an error was returned.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/Documentation/power/suspend-and-cpuhotplug.txt b/Documentation/power/suspend-and-cpuhotplug.txt
index 2850df3..2fc9095 100644 (file)
--- a/Documentation/power/suspend-and-cpuhotplug.txt
+++ b/Documentation/power/suspend-and-cpuhotplug.txt
@@ -72,7 +72,7 @@ More details follow:
                                         |
                                         v
                            Disable regular cpu hotplug
-                        by setting cpu_hotplug_disabled=1
+                        by increasing cpu_hotplug_disabled
                                         |
                                         v
                             Release cpu_add_remove_lock
@@ -89,7 +89,7 @@ Resuming back is likewise, with the counterparts being (in the order of
 execution during resume):
 * enable_nonboot_cpus() which involves:
    |  Acquire cpu_add_remove_lock
-   |  Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug
+   |  Decrease cpu_hotplug_disabled, thereby enabling regular cpu hotplug
    |  Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop]
    |  Release cpu_add_remove_lock
    v
@@ -120,7 +120,7 @@ after the entire cycle is complete (i.e., suspend + resume).
                            Acquire cpu_add_remove_lock
                                         |
                                         v
-                          If cpu_hotplug_disabled is 1
+                          If cpu_hotplug_disabled > 0
                                 return gracefully
                                         |
                                         |

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 5644ec5..0fca2ba 100644 (file)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -191,14 +191,14 @@ void cpu_hotplug_done(void)
 void cpu_hotplug_disable(void)
 {
        cpu_maps_update_begin();
-       cpu_hotplug_disabled = 1;
+       cpu_hotplug_disabled++;
        cpu_maps_update_done();
 }
 
 void cpu_hotplug_enable(void)
 {
        cpu_maps_update_begin();
-       cpu_hotplug_disabled = 0;
+       WARN_ON(--cpu_hotplug_disabled < 0);
        cpu_maps_update_done();
 }
 
@@ -608,13 +608,18 @@ int disable_nonboot_cpus(void)
                }
        }
 
-       if (!error) {
+       if (!error)
                BUG_ON(num_online_cpus() > 1);
-               /* Make sure the CPUs won't be enabled by someone else */
-               cpu_hotplug_disabled = 1;
-       } else {
+       else
                pr_err("Non-boot CPUs are not disabled\n");
-       }
+
+       /*
+        * Make sure the CPUs won't be enabled by someone else. We need to do
+        * this even in case of failure as all disable_nonboot_cpus() users are
+        * supposed to do enable_nonboot_cpus() on the failure path.
+        */
+       cpu_hotplug_disabled++;
+
        cpu_maps_update_done();
        return error;
 }
@@ -633,7 +638,7 @@ void __ref enable_nonboot_cpus(void)
 
        /* Allow everyone to use the CPU hotplug again */
        cpu_maps_update_begin();
-       cpu_hotplug_disabled = 0;
+       WARN_ON(--cpu_hotplug_disabled < 0);
        if (cpumask_empty(frozen_cpus))
                goto out;