libs: h265enc: Set max_transform_hierarchy_depth_{inter, intra} to 2

Intel HW has limitation on max_transform_hierarchy_depth_inter and
max_transform_hierarchy_depth_intra (see [1]). We can provide a quirk for
other HWs if other HWs may support other values

[1] https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-kbl-vol10-hevc.pdf

diff --git a/gst-libs/gst/vaapi/gstvaapiencoder_h265.c b/gst-libs/gst/vaapi/gstvaapiencoder_h265.c
index 2c9b79a..86ab5f8 100644 (file)
--- a/gst-libs/gst/vaapi/gstvaapiencoder_h265.c
+++ b/gst-libs/gst/vaapi/gstvaapiencoder_h265.c
@@ -1640,8 +1640,14 @@ fill_sequence (GstVaapiEncoderH265 * encoder, GstVaapiEncSequence * sequence)
     seq_param->log2_diff_max_min_luma_coding_block_size = 2;
   seq_param->log2_min_transform_block_size_minus2 = 0;
   seq_param->log2_diff_max_min_transform_block_size = 3;
-  seq_param->max_transform_hierarchy_depth_inter = 3;
-  seq_param->max_transform_hierarchy_depth_intra = 3;
+  /*
+   * Intel HW supports up to 2, we can provide a quirk for other HWs in future
+   * if other HW may support other values
+   *
+   * Refer to https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-kbl-vol10-hevc.pdf
+   */
+  seq_param->max_transform_hierarchy_depth_inter = 2;
+  seq_param->max_transform_hierarchy_depth_intra = 2;
 
   seq_param->pcm_sample_bit_depth_luma_minus1 = 0;
   seq_param->pcm_sample_bit_depth_chroma_minus1 = 0;