compositor/blend: fix blending of subsampled components
authorMathieu Duponchelle <mathieu@centricular.com>
Tue, 29 Dec 2020 19:24:51 +0000 (20:24 +0100)
committerMathieu Duponchelle <mduponchelle1@gmail.com>
Tue, 29 Dec 2020 22:48:08 +0000 (22:48 +0000)
commit8ff5079e5eef37b9bd5b212350f0cefbd9546b1b
tree53e541cb0db2a5d9e8c8fc765932421e39635471
parent15fb391401f300b6a63e117aaaa24a308148b754
compositor/blend: fix blending of subsampled components

The correct way to determine the byte offset at a certain yoffset
in a subsampled component is to shift the yoffset by the component's
hsub

This fixes out-of-bounds memory accesses and visible artefacts,
example pipeline with the samples from #802:

gst-launch-1.0 compositor name=vmixer sink_1::xpos=1910 sink_1::ypos=1080 ! \
    videoconvert ! videorate ! xvimagesink \
  filesrc location=VID_20200723_203606.mp4 ! decodebin name=demux1 ! \
    queue ! videoflip method=vertical-flip ! vmixer. \
  filesrc location=bridgeoverstubbledwater.mp4 ! decodebin name=demux2 ! \
    queue ! vmixer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/988>
gst/compositor/blend.c