habanalabs: handle race in driver fini
authorDani Liberman <dliberman@habana.ai>
Wed, 27 Apr 2022 11:14:17 +0000 (14:14 +0300)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Sun, 22 May 2022 19:01:20 +0000 (21:01 +0200)
Scenario:

1. During hard reset, driver executes device_kill_open_processes.
2. Drivers file descriptor is not closed yet (user process is alive),
   hence we are starting loop on all open file descriptors.
3. Just before getting task struct of user process, according to
   pid, SIGKILL is sent to the user process, hence get_pid_task
   fails, driver prints a warning and device_kill_open_processes
   returns an error.
4. Returned error causing driver fini do disable the device object
   of the process which causes a kernel crash.

The fix is to handle this case not as an error and continue fini flow
as normal, since the killed process (by the SIGKILL) will release its
resources just like it will do when the driver sends him the sigkill.

Signed-off-by: Dani Liberman <dliberman@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
drivers/misc/habanalabs/common/device.c

index dbec987..15df89b 100644 (file)
@@ -1024,10 +1024,13 @@ static int device_kill_open_processes(struct hl_device *hdev, u32 timeout, bool
 
                        put_task_struct(task);
                } else {
-                       dev_warn(hdev->dev,
-                               "Can't get task struct for PID so giving up on killing process\n");
-                       mutex_unlock(fd_lock);
-                       return -ETIME;
+                       /*
+                        * If we got here, it means that process was killed from outside the driver
+                        * right after it started looping on fd_list and before get_pid_task, thus
+                        * we don't need to kill it.
+                        */
+                       dev_dbg(hdev->dev,
+                               "Can't get task struct for user process, assuming process was killed from outside the driver\n");
                }
        }