From: Tian Lan Date: Mon, 22 May 2023 21:05:55 +0000 (-0400) Subject: blk-mq: fix race condition in active queue accounting X-Git-Tag: v6.1.37~748 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=6793a3c6326eb3769a99e52908b22df8786776f9;p=platform%2Fkernel%2Flinux-starfive.git blk-mq: fix race condition in active queue accounting [ Upstream commit 3e94d54e83cafd2b562bb6d15bb2f72d76200fb5 ] If multiple CPUs are sharing the same hardware queue, it can cause leak in the active queue counter tracking when __blk_mq_tag_busy() is executed simultaneously. Fixes: ee78ec1077d3 ("blk-mq: blk_mq_tag_busy is no need to return a value") Signed-off-by: Tian Lan Reviewed-by: Ming Lei Reviewed-by: Damien Le Moal Reviewed-by: John Garry Link: https://lore.kernel.org/r/20230522210555.794134-1-tilan7663@gmail.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin --- diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 9eb968e..a80d7c6 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -41,16 +41,20 @@ void __blk_mq_tag_busy(struct blk_mq_hw_ctx *hctx) { unsigned int users; + /* + * calling test_bit() prior to test_and_set_bit() is intentional, + * it avoids dirtying the cacheline if the queue is already active. + */ if (blk_mq_is_shared_tags(hctx->flags)) { struct request_queue *q = hctx->queue; - if (test_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags)) + if (test_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags) || + test_and_set_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags)) return; - set_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags); } else { - if (test_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state)) + if (test_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state) || + test_and_set_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state)) return; - set_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state); } users = atomic_inc_return(&hctx->tags->active_queues);