[Service] Fix memory leak with worker isolation