[WebRTC] Trigger HTML media element resize when surface layer changes

When media stream changes resolution, it needs to notify
HTML media element so it can adjust bounds according to
set object fit property. Together with rendering process
it involves multiple threads and it's important to keep
state consistent to prevent video from displaying with
wrong geometry.

Issue fixed in this CL was about bounds update potentially
few frames after surface id was updated, resulting in
wrong geometry calculated in Viz. It was due to the
asynchronous nature of processing and subtle bug with it.

To fix this issue, bounds recalculation should be done
together with changing surface id for HTML media element,
in single task in message loop. This way they should always
be consistent when sending as compositor frame.

Bug: https://jira-eu.sec.samsung.net/browse/VDGAME-698
Change-Id: I8f45cdfbe0558a5510364b2c09aed1ba07732edd
Signed-off-by: Jakub Gajownik <j.gajownik2@samsung.com>

diff --git a/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms.cc b/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms.cc
index 8e459d5a74a28a04a2eb09fa36c19336daab5fc9..1d370aefc0ae401998f7065e062fde535432f7b8 100644 (file)
--- a/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms.cc
+++ b/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms.cc
@@ -638,7 +638,15 @@ WebMediaPlayer::LoadTiming WebMediaPlayerMS::Load(
   return WebMediaPlayer::LoadTiming::kImmediate;
 }
 
-void WebMediaPlayerMS::OnWebLayerUpdated() {}
+void WebMediaPlayerMS::OnWebLayerUpdated() {
+#if defined(TIZEN_TV_UPSTREAM_MULTIMEDIA)
+  // As surface layer was updated (e.g video frame resolution has changed),
+  // we need to update it together with setting new surface id. It's important,
+  // as they are strictly connected, so sending old bounds with new surface id
+  // results in wrong geometry of video.
+  TriggerResize();
+#endif  // defined(TIZEN_TV_UPSTREAM_MULTIMEDIA)
+}
 
 void WebMediaPlayerMS::RegisterContentsLayer(cc::Layer* layer) {
   DCHECK_CALLED_ON_VALID_THREAD(thread_checker_);

diff --git a/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms_compositor.cc b/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms_compositor.cc
index d261cb855a522d6f57ec48c4c0c2b603ff3227cd..7b60abfede9ae78de64b7af1f4b8b2e6751c88a7 100644 (file)
--- a/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms_compositor.cc
+++ b/third_party/blink/renderer/modules/mediastream/webmediaplayer_ms_compositor.cc
@@ -858,9 +858,20 @@ void WebMediaPlayerMSCompositor::CheckForFrameChanges(
                                             player_, *new_frame_opacity));
   }
   if (has_frame_size_changed) {
+#if defined(TIZEN_TV_UPSTREAM_MULTIMEDIA)
+    // Resize should be done together with surface layer id update to ensure
+    // video natural size matches with the one in compositor frame generated
+    // from |VideoFrameSubmitter|.
+    if (!submitter_) {
+      PostCrossThreadTask(
+          *main_task_runner_, FROM_HERE,
+          CrossThreadBindOnce(&WebMediaPlayerMS::TriggerResize, player_));
+    }
+#else
     PostCrossThreadTask(
         *main_task_runner_, FROM_HERE,
         CrossThreadBindOnce(&WebMediaPlayerMS::TriggerResize, player_));
+#endif  // defined(TIZEN_TV_UPSTREAM_MULTIMEDIA)
     PostCrossThreadTask(
         *main_task_runner_, FROM_HERE,
         CrossThreadBindOnce(&WebMediaPlayerMS::ResetCanvasCache, player_));