Javan Whistling Duck release.
Change-Id: If44c9ca16a8188b68759325fbacc771365cb4af8
Adrian Grange <agrange@google.com>
-Alex Converse <aconverse@google.com> <alex.converse@gmail.com>
+Adrian Grange <agrange@google.com> <agrange@agrange-macbookpro.roam.corp.google.com>
+Aℓex Converse <aconverse@google.com>
+Aℓex Converse <aconverse@google.com> <alex.converse@gmail.com>
Alexis Ballier <aballier@gentoo.org> <alexis.ballier@gmail.com>
Alpha Lam <hclam@google.com> <hclam@chromium.org>
Deb Mukherjee <debargha@google.com>
Erik Niemeyer <erik.a.niemeyer@intel.com> <erik.a.niemeyer@gmail.com>
Guillaume Martres <gmartres@google.com> <smarter3@gmail.com>
Hangyu Kuang <hkuang@google.com>
+Hangyu Kuang <hkuang@google.com> <hkuang@hkuang-macbookpro.roam.corp.google.com>
+Hui Su <huisu@google.com>
+Jacky Chen <jackychen@google.com>
Jim Bankoski <jimbankoski@google.com>
Johann Koenig <johannkoenig@google.com>
Johann Koenig <johannkoenig@google.com> <johann.koenig@duck.com>
+Johann Koenig <johannkoenig@google.com> <johannkoenig@dhcp-172-19-7-52.mtv.corp.google.com>
+Johann Koenig <johannkoenig@google.com> <johann.koenig@gmail.com>
John Koleszar <jkoleszar@google.com>
Joshua Litt <joshualitt@google.com> <joshualitt@chromium.org>
Marco Paniconi <marpan@google.com>
Paul Wilkins <paulwilkins@google.com>
Ralph Giles <giles@xiph.org> <giles@entropywave.com>
Ralph Giles <giles@xiph.org> <giles@mozilla.com>
+Ronald S. Bultje <rsbultje@gmail.com> <rbultje@google.com>
Sami Pietilä <samipietila@google.com>
Tamar Levy <tamar.levy@intel.com>
Tamar Levy <tamar.levy@intel.com> <levytamar82@gmail.com>
Tero Rintaluoma <teror@google.com> <tero.rintaluoma@on2.com>
Timothy B. Terriberry <tterribe@xiph.org> Tim Terriberry <tterriberry@mozilla.com>
Tom Finegan <tomfinegan@google.com>
+Tom Finegan <tomfinegan@google.com> <tomfinegan@chromium.org>
Yaowu Xu <yaowu@google.com> <yaowu@xuyaowu.com>
+Yaowu Xu <yaowu@google.com> <yaowu@YAOWU2-W.ad.corp.google.com>
Abo Talib Mahfoodh <ab.mahfoodh@gmail.com>
Adam Xu <adam@xuyaowu.com>
Adrian Grange <agrange@google.com>
+Aℓex Converse <aconverse@google.com>
Ahmad Sharif <asharif@google.com>
Alexander Voronov <avoronov@graphics.cs.msu.ru>
-Alex Converse <aconverse@google.com>
Alexis Ballier <aballier@gentoo.org>
Alok Ahuja <waveletcoeff@gmail.com>
Alpha Lam <hclam@google.com>
Andoni Morales Alastruey <ylatuya@gmail.com>
Andres Mejia <mcitadel@gmail.com>
Andrew Russell <anrussell@google.com>
+Angie Chiang <angiebird@google.com>
Aron Rosenberg <arosenberg@logitech.com>
Attila Nagy <attilanagy@google.com>
+Brion Vibber <bvibber@wikimedia.org>
changjun.yang <changjun.yang@intel.com>
Charles 'Buck' Krasic <ckrasic@google.com>
chm <chm@rock-chips.com>
Dim Temp <dimtemp0@gmail.com>
Dmitry Kovalev <dkovalev@google.com>
Dragan Mrdjan <dmrdjan@mips.com>
+Ed Baker <edward.baker@intel.com>
Ehsan Akhgari <ehsan.akhgari@gmail.com>
Erik Niemeyer <erik.a.niemeyer@intel.com>
Fabio Pedretti <fabio.ped@libero.it>
Fredrik Söderquist <fs@opera.com>
Fritz Koenig <frkoenig@google.com>
Gaute Strokkenes <gaute.strokkenes@broadcom.com>
+Geza Lore <gezalore@gmail.com>
+Ghislain MARY <ghislainmary2@gmail.com>
Giuseppe Scrivano <gscrivano@gnu.org>
Gordana Cmiljanovic <gordana.cmiljanovic@imgtec.com>
Guillaume Martres <gmartres@google.com>
Hui Su <huisu@google.com>
Ivan Maltz <ivanmaltz@google.com>
Jacek Caban <cjacek@gmail.com>
-JackyChen <jackychen@google.com>
+Jacky Chen <jackychen@google.com>
James Berry <jamesberry@google.com>
James Yu <james.yu@linaro.org>
James Zern <jzern@google.com>
Joey Parrish <joeyparrish@google.com>
Johann Koenig <johannkoenig@google.com>
John Koleszar <jkoleszar@google.com>
+Johnny Klonaris <google@jawknee.com>
John Stark <jhnstrk@gmail.com>
Joshua Bleecher Snyder <josh@treelinelabs.com>
Joshua Litt <joshualitt@google.com>
+Julia Robson <juliamrobson@gmail.com>
Justin Clift <justin@salasaga.org>
Justin Lebar <justin.lebar@gmail.com>
KO Myung-Hun <komh@chollian.net>
Mikhal Shemer <mikhal@google.com>
Minghai Shang <minghai@google.com>
Morton Jonuschat <yabawock@gmail.com>
+Nico Weber <thakis@chromium.org>
Parag Salasakar <img.mips1@gmail.com>
Pascal Massimino <pascal.massimino@gmail.com>
Patrik Westin <patrik.westin@gmail.com>
Rafaël Carré <funman@videolan.org>
Ralph Giles <giles@xiph.org>
Rob Bradford <rob@linux.intel.com>
-Ronald S. Bultje <rbultje@google.com>
+Ronald S. Bultje <rsbultje@gmail.com>
Rui Ueyama <ruiu@google.com>
Sami Pietilä <samipietila@google.com>
Scott Graham <scottmg@chromium.org>
Sean McGovern <gseanmcg@gmail.com>
Sergey Ulanov <sergeyu@chromium.org>
Shimon Doodkin <helpmepro1@gmail.com>
+Shunyao Li <shunyaoli@google.com>
Stefan Holmer <holmer@google.com>
Suman Sunkara <sunkaras@google.com>
Taekhyun Kim <takim@nvidia.com>
-xxxx-yy-zz v1.4.0 "Changes for next release"
- vpxenc is changed to use VP9 by default.
- Encoder controls added for 1 pass SVC.
- Decoder control to toggle on/off loopfilter.
+2015-11-09 v1.5.0 "Javan Whistling Duck"
+ This release improves upon the VP9 encoder and speeds up the encoding and
+ decoding processes.
+
+ - Upgrading:
+ This release is ABI incompatible with 1.4.0. It drops deprecated VP8
+ controls and adds a variety of VP9 controls for testing.
+
+ The vpxenc utility now prefers VP9 by default.
+
+ - Enhancements:
+ Faster VP9 encoding and decoding
+ Smaller library size by combining functions used by VP8 and VP9
+
+ - Bug Fixes:
+ A variety of fuzzing issues
2015-04-03 v1.4.0 "Indian Runner Duck"
This release includes significant improvements to the VP9 codec.
LIBS-$(if yes,$(CONFIG_STATIC)) += $(BUILD_PFX)libvpx.a $(BUILD_PFX)libvpx_g.a
$(BUILD_PFX)libvpx_g.a: $(LIBVPX_OBJS)
-SO_VERSION_MAJOR := 2
+SO_VERSION_MAJOR := 3
SO_VERSION_MINOR := 0
SO_VERSION_PATCH := 0
ifeq ($(filter darwin%,$(TGT_OS)),$(TGT_OS))
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_config.h"
-#include "vpx_mem/vpx_mem.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/entropymv.h"
-#include "vp10/common/onyxc_int.h"
-
-void vp10_set_mb_mi(VP10_COMMON *cm, int width, int height) {
- const int aligned_width = ALIGN_POWER_OF_TWO(width, MI_SIZE_LOG2);
- const int aligned_height = ALIGN_POWER_OF_TWO(height, MI_SIZE_LOG2);
-
- cm->mi_cols = aligned_width >> MI_SIZE_LOG2;
- cm->mi_rows = aligned_height >> MI_SIZE_LOG2;
- cm->mi_stride = calc_mi_size(cm->mi_cols);
-
- cm->mb_cols = (cm->mi_cols + 1) >> 1;
- cm->mb_rows = (cm->mi_rows + 1) >> 1;
- cm->MBs = cm->mb_rows * cm->mb_cols;
-}
-
-static int alloc_seg_map(VP10_COMMON *cm, int seg_map_size) {
- int i;
-
- for (i = 0; i < NUM_PING_PONG_BUFFERS; ++i) {
- cm->seg_map_array[i] = (uint8_t *)vpx_calloc(seg_map_size, 1);
- if (cm->seg_map_array[i] == NULL)
- return 1;
- }
- cm->seg_map_alloc_size = seg_map_size;
-
- // Init the index.
- cm->seg_map_idx = 0;
- cm->prev_seg_map_idx = 1;
-
- cm->current_frame_seg_map = cm->seg_map_array[cm->seg_map_idx];
- if (!cm->frame_parallel_decode)
- cm->last_frame_seg_map = cm->seg_map_array[cm->prev_seg_map_idx];
-
- return 0;
-}
-
-static void free_seg_map(VP10_COMMON *cm) {
- int i;
-
- for (i = 0; i < NUM_PING_PONG_BUFFERS; ++i) {
- vpx_free(cm->seg_map_array[i]);
- cm->seg_map_array[i] = NULL;
- }
-
- cm->current_frame_seg_map = NULL;
-
- if (!cm->frame_parallel_decode) {
- cm->last_frame_seg_map = NULL;
- }
-}
-
-void vp10_free_ref_frame_buffers(BufferPool *pool) {
- int i;
-
- for (i = 0; i < FRAME_BUFFERS; ++i) {
- if (pool->frame_bufs[i].ref_count > 0 &&
- pool->frame_bufs[i].raw_frame_buffer.data != NULL) {
- pool->release_fb_cb(pool->cb_priv, &pool->frame_bufs[i].raw_frame_buffer);
- pool->frame_bufs[i].ref_count = 0;
- }
- vpx_free(pool->frame_bufs[i].mvs);
- pool->frame_bufs[i].mvs = NULL;
- vpx_free_frame_buffer(&pool->frame_bufs[i].buf);
- }
-}
-
-void vp10_free_postproc_buffers(VP10_COMMON *cm) {
-#if CONFIG_VP9_POSTPROC
- vpx_free_frame_buffer(&cm->post_proc_buffer);
- vpx_free_frame_buffer(&cm->post_proc_buffer_int);
-#else
- (void)cm;
-#endif
-}
-
-void vp10_free_context_buffers(VP10_COMMON *cm) {
- cm->free_mi(cm);
- free_seg_map(cm);
- vpx_free(cm->above_context);
- cm->above_context = NULL;
- vpx_free(cm->above_seg_context);
- cm->above_seg_context = NULL;
-}
-
-int vp10_alloc_context_buffers(VP10_COMMON *cm, int width, int height) {
- int new_mi_size;
-
- vp10_set_mb_mi(cm, width, height);
- new_mi_size = cm->mi_stride * calc_mi_size(cm->mi_rows);
- if (cm->mi_alloc_size < new_mi_size) {
- cm->free_mi(cm);
- if (cm->alloc_mi(cm, new_mi_size))
- goto fail;
- }
-
- if (cm->seg_map_alloc_size < cm->mi_rows * cm->mi_cols) {
- // Create the segmentation map structure and set to 0.
- free_seg_map(cm);
- if (alloc_seg_map(cm, cm->mi_rows * cm->mi_cols))
- goto fail;
- }
-
- if (cm->above_context_alloc_cols < cm->mi_cols) {
- vpx_free(cm->above_context);
- cm->above_context = (ENTROPY_CONTEXT *)vpx_calloc(
- 2 * mi_cols_aligned_to_sb(cm->mi_cols) * MAX_MB_PLANE,
- sizeof(*cm->above_context));
- if (!cm->above_context) goto fail;
-
- vpx_free(cm->above_seg_context);
- cm->above_seg_context = (PARTITION_CONTEXT *)vpx_calloc(
- mi_cols_aligned_to_sb(cm->mi_cols), sizeof(*cm->above_seg_context));
- if (!cm->above_seg_context) goto fail;
- cm->above_context_alloc_cols = cm->mi_cols;
- }
-
- return 0;
-
- fail:
- vp10_free_context_buffers(cm);
- return 1;
-}
-
-void vp10_remove_common(VP10_COMMON *cm) {
- vp10_free_context_buffers(cm);
-
- vpx_free(cm->fc);
- cm->fc = NULL;
- vpx_free(cm->frame_contexts);
- cm->frame_contexts = NULL;
-}
-
-void vp10_init_context_buffers(VP10_COMMON *cm) {
- cm->setup_mi(cm);
- if (cm->last_frame_seg_map && !cm->frame_parallel_decode)
- memset(cm->last_frame_seg_map, 0, cm->mi_rows * cm->mi_cols);
-}
-
-void vp10_swap_current_and_last_seg_map(VP10_COMMON *cm) {
- // Swap indices.
- const int tmp = cm->seg_map_idx;
- cm->seg_map_idx = cm->prev_seg_map_idx;
- cm->prev_seg_map_idx = tmp;
-
- cm->current_frame_seg_map = cm->seg_map_array[cm->seg_map_idx];
- cm->last_frame_seg_map = cm->seg_map_array[cm->prev_seg_map_idx];
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_COMMON_ALLOCCOMMON_H_
-#define VP10_COMMON_ALLOCCOMMON_H_
-
-#define INVALID_IDX -1 // Invalid buffer index.
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10Common;
-struct BufferPool;
-
-void vp10_remove_common(struct VP10Common *cm);
-
-int vp10_alloc_context_buffers(struct VP10Common *cm, int width, int height);
-void vp10_init_context_buffers(struct VP10Common *cm);
-void vp10_free_context_buffers(struct VP10Common *cm);
-
-void vp10_free_ref_frame_buffers(struct BufferPool *pool);
-void vp10_free_postproc_buffers(struct VP10Common *cm);
-
-int vp10_alloc_state_buffers(struct VP10Common *cm, int width, int height);
-void vp10_free_state_buffers(struct VP10Common *cm);
-
-void vp10_set_mb_mi(struct VP10Common *cm, int width, int height);
-
-void vp10_swap_current_and_last_seg_map(struct VP10Common *cm);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_ALLOCCOMMON_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <arm_neon.h>
-#include <assert.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-#include "vp10/common/common.h"
-
-static int16_t sinpi_1_9 = 0x14a3;
-static int16_t sinpi_2_9 = 0x26c9;
-static int16_t sinpi_3_9 = 0x3441;
-static int16_t sinpi_4_9 = 0x3b6c;
-static int16_t cospi_8_64 = 0x3b21;
-static int16_t cospi_16_64 = 0x2d41;
-static int16_t cospi_24_64 = 0x187e;
-
-static INLINE void TRANSPOSE4X4(
- int16x8_t *q8s16,
- int16x8_t *q9s16) {
- int32x4_t q8s32, q9s32;
- int16x4x2_t d0x2s16, d1x2s16;
- int32x4x2_t q0x2s32;
-
- d0x2s16 = vtrn_s16(vget_low_s16(*q8s16), vget_high_s16(*q8s16));
- d1x2s16 = vtrn_s16(vget_low_s16(*q9s16), vget_high_s16(*q9s16));
-
- q8s32 = vreinterpretq_s32_s16(vcombine_s16(d0x2s16.val[0], d0x2s16.val[1]));
- q9s32 = vreinterpretq_s32_s16(vcombine_s16(d1x2s16.val[0], d1x2s16.val[1]));
- q0x2s32 = vtrnq_s32(q8s32, q9s32);
-
- *q8s16 = vreinterpretq_s16_s32(q0x2s32.val[0]);
- *q9s16 = vreinterpretq_s16_s32(q0x2s32.val[1]);
- return;
-}
-
-static INLINE void GENERATE_COSINE_CONSTANTS(
- int16x4_t *d0s16,
- int16x4_t *d1s16,
- int16x4_t *d2s16) {
- *d0s16 = vdup_n_s16(cospi_8_64);
- *d1s16 = vdup_n_s16(cospi_16_64);
- *d2s16 = vdup_n_s16(cospi_24_64);
- return;
-}
-
-static INLINE void GENERATE_SINE_CONSTANTS(
- int16x4_t *d3s16,
- int16x4_t *d4s16,
- int16x4_t *d5s16,
- int16x8_t *q3s16) {
- *d3s16 = vdup_n_s16(sinpi_1_9);
- *d4s16 = vdup_n_s16(sinpi_2_9);
- *q3s16 = vdupq_n_s16(sinpi_3_9);
- *d5s16 = vdup_n_s16(sinpi_4_9);
- return;
-}
-
-static INLINE void IDCT4x4_1D(
- int16x4_t *d0s16,
- int16x4_t *d1s16,
- int16x4_t *d2s16,
- int16x8_t *q8s16,
- int16x8_t *q9s16) {
- int16x4_t d16s16, d17s16, d18s16, d19s16, d23s16, d24s16;
- int16x4_t d26s16, d27s16, d28s16, d29s16;
- int32x4_t q10s32, q13s32, q14s32, q15s32;
- int16x8_t q13s16, q14s16;
-
- d16s16 = vget_low_s16(*q8s16);
- d17s16 = vget_high_s16(*q8s16);
- d18s16 = vget_low_s16(*q9s16);
- d19s16 = vget_high_s16(*q9s16);
-
- d23s16 = vadd_s16(d16s16, d18s16);
- d24s16 = vsub_s16(d16s16, d18s16);
-
- q15s32 = vmull_s16(d17s16, *d2s16);
- q10s32 = vmull_s16(d17s16, *d0s16);
- q13s32 = vmull_s16(d23s16, *d1s16);
- q14s32 = vmull_s16(d24s16, *d1s16);
- q15s32 = vmlsl_s16(q15s32, d19s16, *d0s16);
- q10s32 = vmlal_s16(q10s32, d19s16, *d2s16);
-
- d26s16 = vqrshrn_n_s32(q13s32, 14);
- d27s16 = vqrshrn_n_s32(q14s32, 14);
- d29s16 = vqrshrn_n_s32(q15s32, 14);
- d28s16 = vqrshrn_n_s32(q10s32, 14);
-
- q13s16 = vcombine_s16(d26s16, d27s16);
- q14s16 = vcombine_s16(d28s16, d29s16);
- *q8s16 = vaddq_s16(q13s16, q14s16);
- *q9s16 = vsubq_s16(q13s16, q14s16);
- *q9s16 = vcombine_s16(vget_high_s16(*q9s16),
- vget_low_s16(*q9s16)); // vswp
- return;
-}
-
-static INLINE void IADST4x4_1D(
- int16x4_t *d3s16,
- int16x4_t *d4s16,
- int16x4_t *d5s16,
- int16x8_t *q3s16,
- int16x8_t *q8s16,
- int16x8_t *q9s16) {
- int16x4_t d6s16, d16s16, d17s16, d18s16, d19s16;
- int32x4_t q8s32, q9s32, q10s32, q11s32, q12s32, q13s32, q14s32, q15s32;
-
- d6s16 = vget_low_s16(*q3s16);
-
- d16s16 = vget_low_s16(*q8s16);
- d17s16 = vget_high_s16(*q8s16);
- d18s16 = vget_low_s16(*q9s16);
- d19s16 = vget_high_s16(*q9s16);
-
- q10s32 = vmull_s16(*d3s16, d16s16);
- q11s32 = vmull_s16(*d4s16, d16s16);
- q12s32 = vmull_s16(d6s16, d17s16);
- q13s32 = vmull_s16(*d5s16, d18s16);
- q14s32 = vmull_s16(*d3s16, d18s16);
- q15s32 = vmovl_s16(d16s16);
- q15s32 = vaddw_s16(q15s32, d19s16);
- q8s32 = vmull_s16(*d4s16, d19s16);
- q15s32 = vsubw_s16(q15s32, d18s16);
- q9s32 = vmull_s16(*d5s16, d19s16);
-
- q10s32 = vaddq_s32(q10s32, q13s32);
- q10s32 = vaddq_s32(q10s32, q8s32);
- q11s32 = vsubq_s32(q11s32, q14s32);
- q8s32 = vdupq_n_s32(sinpi_3_9);
- q11s32 = vsubq_s32(q11s32, q9s32);
- q15s32 = vmulq_s32(q15s32, q8s32);
-
- q13s32 = vaddq_s32(q10s32, q12s32);
- q10s32 = vaddq_s32(q10s32, q11s32);
- q14s32 = vaddq_s32(q11s32, q12s32);
- q10s32 = vsubq_s32(q10s32, q12s32);
-
- d16s16 = vqrshrn_n_s32(q13s32, 14);
- d17s16 = vqrshrn_n_s32(q14s32, 14);
- d18s16 = vqrshrn_n_s32(q15s32, 14);
- d19s16 = vqrshrn_n_s32(q10s32, 14);
-
- *q8s16 = vcombine_s16(d16s16, d17s16);
- *q9s16 = vcombine_s16(d18s16, d19s16);
- return;
-}
-
-void vp10_iht4x4_16_add_neon(const tran_low_t *input, uint8_t *dest,
- int dest_stride, int tx_type) {
- uint8x8_t d26u8, d27u8;
- int16x4_t d0s16, d1s16, d2s16, d3s16, d4s16, d5s16;
- uint32x2_t d26u32, d27u32;
- int16x8_t q3s16, q8s16, q9s16;
- uint16x8_t q8u16, q9u16;
-
- d26u32 = d27u32 = vdup_n_u32(0);
-
- q8s16 = vld1q_s16(input);
- q9s16 = vld1q_s16(input + 8);
-
- TRANSPOSE4X4(&q8s16, &q9s16);
-
- switch (tx_type) {
- case 0: // idct_idct is not supported. Fall back to C
- vp10_iht4x4_16_add_c(input, dest, dest_stride, tx_type);
- return;
- break;
- case 1: // iadst_idct
- // generate constants
- GENERATE_COSINE_CONSTANTS(&d0s16, &d1s16, &d2s16);
- GENERATE_SINE_CONSTANTS(&d3s16, &d4s16, &d5s16, &q3s16);
-
- // first transform rows
- IDCT4x4_1D(&d0s16, &d1s16, &d2s16, &q8s16, &q9s16);
-
- // transpose the matrix
- TRANSPOSE4X4(&q8s16, &q9s16);
-
- // then transform columns
- IADST4x4_1D(&d3s16, &d4s16, &d5s16, &q3s16, &q8s16, &q9s16);
- break;
- case 2: // idct_iadst
- // generate constantsyy
- GENERATE_COSINE_CONSTANTS(&d0s16, &d1s16, &d2s16);
- GENERATE_SINE_CONSTANTS(&d3s16, &d4s16, &d5s16, &q3s16);
-
- // first transform rows
- IADST4x4_1D(&d3s16, &d4s16, &d5s16, &q3s16, &q8s16, &q9s16);
-
- // transpose the matrix
- TRANSPOSE4X4(&q8s16, &q9s16);
-
- // then transform columns
- IDCT4x4_1D(&d0s16, &d1s16, &d2s16, &q8s16, &q9s16);
- break;
- case 3: // iadst_iadst
- // generate constants
- GENERATE_SINE_CONSTANTS(&d3s16, &d4s16, &d5s16, &q3s16);
-
- // first transform rows
- IADST4x4_1D(&d3s16, &d4s16, &d5s16, &q3s16, &q8s16, &q9s16);
-
- // transpose the matrix
- TRANSPOSE4X4(&q8s16, &q9s16);
-
- // then transform columns
- IADST4x4_1D(&d3s16, &d4s16, &d5s16, &q3s16, &q8s16, &q9s16);
- break;
- default: // iadst_idct
- assert(0);
- break;
- }
-
- q8s16 = vrshrq_n_s16(q8s16, 4);
- q9s16 = vrshrq_n_s16(q9s16, 4);
-
- d26u32 = vld1_lane_u32((const uint32_t *)dest, d26u32, 0);
- dest += dest_stride;
- d26u32 = vld1_lane_u32((const uint32_t *)dest, d26u32, 1);
- dest += dest_stride;
- d27u32 = vld1_lane_u32((const uint32_t *)dest, d27u32, 0);
- dest += dest_stride;
- d27u32 = vld1_lane_u32((const uint32_t *)dest, d27u32, 1);
-
- q8u16 = vaddw_u8(vreinterpretq_u16_s16(q8s16), vreinterpret_u8_u32(d26u32));
- q9u16 = vaddw_u8(vreinterpretq_u16_s16(q9s16), vreinterpret_u8_u32(d27u32));
-
- d26u8 = vqmovun_s16(vreinterpretq_s16_u16(q8u16));
- d27u8 = vqmovun_s16(vreinterpretq_s16_u16(q9u16));
-
- vst1_lane_u32((uint32_t *)dest, vreinterpret_u32_u8(d27u8), 1);
- dest -= dest_stride;
- vst1_lane_u32((uint32_t *)dest, vreinterpret_u32_u8(d27u8), 0);
- dest -= dest_stride;
- vst1_lane_u32((uint32_t *)dest, vreinterpret_u32_u8(d26u8), 1);
- dest -= dest_stride;
- vst1_lane_u32((uint32_t *)dest, vreinterpret_u32_u8(d26u8), 0);
- return;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <arm_neon.h>
-#include <assert.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-#include "vp10/common/common.h"
-
-static int16_t cospi_2_64 = 16305;
-static int16_t cospi_4_64 = 16069;
-static int16_t cospi_6_64 = 15679;
-static int16_t cospi_8_64 = 15137;
-static int16_t cospi_10_64 = 14449;
-static int16_t cospi_12_64 = 13623;
-static int16_t cospi_14_64 = 12665;
-static int16_t cospi_16_64 = 11585;
-static int16_t cospi_18_64 = 10394;
-static int16_t cospi_20_64 = 9102;
-static int16_t cospi_22_64 = 7723;
-static int16_t cospi_24_64 = 6270;
-static int16_t cospi_26_64 = 4756;
-static int16_t cospi_28_64 = 3196;
-static int16_t cospi_30_64 = 1606;
-
-static INLINE void TRANSPOSE8X8(
- int16x8_t *q8s16,
- int16x8_t *q9s16,
- int16x8_t *q10s16,
- int16x8_t *q11s16,
- int16x8_t *q12s16,
- int16x8_t *q13s16,
- int16x8_t *q14s16,
- int16x8_t *q15s16) {
- int16x4_t d16s16, d17s16, d18s16, d19s16, d20s16, d21s16, d22s16, d23s16;
- int16x4_t d24s16, d25s16, d26s16, d27s16, d28s16, d29s16, d30s16, d31s16;
- int32x4x2_t q0x2s32, q1x2s32, q2x2s32, q3x2s32;
- int16x8x2_t q0x2s16, q1x2s16, q2x2s16, q3x2s16;
-
- d16s16 = vget_low_s16(*q8s16);
- d17s16 = vget_high_s16(*q8s16);
- d18s16 = vget_low_s16(*q9s16);
- d19s16 = vget_high_s16(*q9s16);
- d20s16 = vget_low_s16(*q10s16);
- d21s16 = vget_high_s16(*q10s16);
- d22s16 = vget_low_s16(*q11s16);
- d23s16 = vget_high_s16(*q11s16);
- d24s16 = vget_low_s16(*q12s16);
- d25s16 = vget_high_s16(*q12s16);
- d26s16 = vget_low_s16(*q13s16);
- d27s16 = vget_high_s16(*q13s16);
- d28s16 = vget_low_s16(*q14s16);
- d29s16 = vget_high_s16(*q14s16);
- d30s16 = vget_low_s16(*q15s16);
- d31s16 = vget_high_s16(*q15s16);
-
- *q8s16 = vcombine_s16(d16s16, d24s16); // vswp d17, d24
- *q9s16 = vcombine_s16(d18s16, d26s16); // vswp d19, d26
- *q10s16 = vcombine_s16(d20s16, d28s16); // vswp d21, d28
- *q11s16 = vcombine_s16(d22s16, d30s16); // vswp d23, d30
- *q12s16 = vcombine_s16(d17s16, d25s16);
- *q13s16 = vcombine_s16(d19s16, d27s16);
- *q14s16 = vcombine_s16(d21s16, d29s16);
- *q15s16 = vcombine_s16(d23s16, d31s16);
-
- q0x2s32 = vtrnq_s32(vreinterpretq_s32_s16(*q8s16),
- vreinterpretq_s32_s16(*q10s16));
- q1x2s32 = vtrnq_s32(vreinterpretq_s32_s16(*q9s16),
- vreinterpretq_s32_s16(*q11s16));
- q2x2s32 = vtrnq_s32(vreinterpretq_s32_s16(*q12s16),
- vreinterpretq_s32_s16(*q14s16));
- q3x2s32 = vtrnq_s32(vreinterpretq_s32_s16(*q13s16),
- vreinterpretq_s32_s16(*q15s16));
-
- q0x2s16 = vtrnq_s16(vreinterpretq_s16_s32(q0x2s32.val[0]), // q8
- vreinterpretq_s16_s32(q1x2s32.val[0])); // q9
- q1x2s16 = vtrnq_s16(vreinterpretq_s16_s32(q0x2s32.val[1]), // q10
- vreinterpretq_s16_s32(q1x2s32.val[1])); // q11
- q2x2s16 = vtrnq_s16(vreinterpretq_s16_s32(q2x2s32.val[0]), // q12
- vreinterpretq_s16_s32(q3x2s32.val[0])); // q13
- q3x2s16 = vtrnq_s16(vreinterpretq_s16_s32(q2x2s32.val[1]), // q14
- vreinterpretq_s16_s32(q3x2s32.val[1])); // q15
-
- *q8s16 = q0x2s16.val[0];
- *q9s16 = q0x2s16.val[1];
- *q10s16 = q1x2s16.val[0];
- *q11s16 = q1x2s16.val[1];
- *q12s16 = q2x2s16.val[0];
- *q13s16 = q2x2s16.val[1];
- *q14s16 = q3x2s16.val[0];
- *q15s16 = q3x2s16.val[1];
- return;
-}
-
-static INLINE void IDCT8x8_1D(
- int16x8_t *q8s16,
- int16x8_t *q9s16,
- int16x8_t *q10s16,
- int16x8_t *q11s16,
- int16x8_t *q12s16,
- int16x8_t *q13s16,
- int16x8_t *q14s16,
- int16x8_t *q15s16) {
- int16x4_t d0s16, d1s16, d2s16, d3s16;
- int16x4_t d8s16, d9s16, d10s16, d11s16, d12s16, d13s16, d14s16, d15s16;
- int16x4_t d16s16, d17s16, d18s16, d19s16, d20s16, d21s16, d22s16, d23s16;
- int16x4_t d24s16, d25s16, d26s16, d27s16, d28s16, d29s16, d30s16, d31s16;
- int16x8_t q0s16, q1s16, q2s16, q3s16, q4s16, q5s16, q6s16, q7s16;
- int32x4_t q2s32, q3s32, q5s32, q6s32, q8s32, q9s32;
- int32x4_t q10s32, q11s32, q12s32, q13s32, q15s32;
-
- d0s16 = vdup_n_s16(cospi_28_64);
- d1s16 = vdup_n_s16(cospi_4_64);
- d2s16 = vdup_n_s16(cospi_12_64);
- d3s16 = vdup_n_s16(cospi_20_64);
-
- d16s16 = vget_low_s16(*q8s16);
- d17s16 = vget_high_s16(*q8s16);
- d18s16 = vget_low_s16(*q9s16);
- d19s16 = vget_high_s16(*q9s16);
- d20s16 = vget_low_s16(*q10s16);
- d21s16 = vget_high_s16(*q10s16);
- d22s16 = vget_low_s16(*q11s16);
- d23s16 = vget_high_s16(*q11s16);
- d24s16 = vget_low_s16(*q12s16);
- d25s16 = vget_high_s16(*q12s16);
- d26s16 = vget_low_s16(*q13s16);
- d27s16 = vget_high_s16(*q13s16);
- d28s16 = vget_low_s16(*q14s16);
- d29s16 = vget_high_s16(*q14s16);
- d30s16 = vget_low_s16(*q15s16);
- d31s16 = vget_high_s16(*q15s16);
-
- q2s32 = vmull_s16(d18s16, d0s16);
- q3s32 = vmull_s16(d19s16, d0s16);
- q5s32 = vmull_s16(d26s16, d2s16);
- q6s32 = vmull_s16(d27s16, d2s16);
-
- q2s32 = vmlsl_s16(q2s32, d30s16, d1s16);
- q3s32 = vmlsl_s16(q3s32, d31s16, d1s16);
- q5s32 = vmlsl_s16(q5s32, d22s16, d3s16);
- q6s32 = vmlsl_s16(q6s32, d23s16, d3s16);
-
- d8s16 = vqrshrn_n_s32(q2s32, 14);
- d9s16 = vqrshrn_n_s32(q3s32, 14);
- d10s16 = vqrshrn_n_s32(q5s32, 14);
- d11s16 = vqrshrn_n_s32(q6s32, 14);
- q4s16 = vcombine_s16(d8s16, d9s16);
- q5s16 = vcombine_s16(d10s16, d11s16);
-
- q2s32 = vmull_s16(d18s16, d1s16);
- q3s32 = vmull_s16(d19s16, d1s16);
- q9s32 = vmull_s16(d26s16, d3s16);
- q13s32 = vmull_s16(d27s16, d3s16);
-
- q2s32 = vmlal_s16(q2s32, d30s16, d0s16);
- q3s32 = vmlal_s16(q3s32, d31s16, d0s16);
- q9s32 = vmlal_s16(q9s32, d22s16, d2s16);
- q13s32 = vmlal_s16(q13s32, d23s16, d2s16);
-
- d14s16 = vqrshrn_n_s32(q2s32, 14);
- d15s16 = vqrshrn_n_s32(q3s32, 14);
- d12s16 = vqrshrn_n_s32(q9s32, 14);
- d13s16 = vqrshrn_n_s32(q13s32, 14);
- q6s16 = vcombine_s16(d12s16, d13s16);
- q7s16 = vcombine_s16(d14s16, d15s16);
-
- d0s16 = vdup_n_s16(cospi_16_64);
-
- q2s32 = vmull_s16(d16s16, d0s16);
- q3s32 = vmull_s16(d17s16, d0s16);
- q13s32 = vmull_s16(d16s16, d0s16);
- q15s32 = vmull_s16(d17s16, d0s16);
-
- q2s32 = vmlal_s16(q2s32, d24s16, d0s16);
- q3s32 = vmlal_s16(q3s32, d25s16, d0s16);
- q13s32 = vmlsl_s16(q13s32, d24s16, d0s16);
- q15s32 = vmlsl_s16(q15s32, d25s16, d0s16);
-
- d0s16 = vdup_n_s16(cospi_24_64);
- d1s16 = vdup_n_s16(cospi_8_64);
-
- d18s16 = vqrshrn_n_s32(q2s32, 14);
- d19s16 = vqrshrn_n_s32(q3s32, 14);
- d22s16 = vqrshrn_n_s32(q13s32, 14);
- d23s16 = vqrshrn_n_s32(q15s32, 14);
- *q9s16 = vcombine_s16(d18s16, d19s16);
- *q11s16 = vcombine_s16(d22s16, d23s16);
-
- q2s32 = vmull_s16(d20s16, d0s16);
- q3s32 = vmull_s16(d21s16, d0s16);
- q8s32 = vmull_s16(d20s16, d1s16);
- q12s32 = vmull_s16(d21s16, d1s16);
-
- q2s32 = vmlsl_s16(q2s32, d28s16, d1s16);
- q3s32 = vmlsl_s16(q3s32, d29s16, d1s16);
- q8s32 = vmlal_s16(q8s32, d28s16, d0s16);
- q12s32 = vmlal_s16(q12s32, d29s16, d0s16);
-
- d26s16 = vqrshrn_n_s32(q2s32, 14);
- d27s16 = vqrshrn_n_s32(q3s32, 14);
- d30s16 = vqrshrn_n_s32(q8s32, 14);
- d31s16 = vqrshrn_n_s32(q12s32, 14);
- *q13s16 = vcombine_s16(d26s16, d27s16);
- *q15s16 = vcombine_s16(d30s16, d31s16);
-
- q0s16 = vaddq_s16(*q9s16, *q15s16);
- q1s16 = vaddq_s16(*q11s16, *q13s16);
- q2s16 = vsubq_s16(*q11s16, *q13s16);
- q3s16 = vsubq_s16(*q9s16, *q15s16);
-
- *q13s16 = vsubq_s16(q4s16, q5s16);
- q4s16 = vaddq_s16(q4s16, q5s16);
- *q14s16 = vsubq_s16(q7s16, q6s16);
- q7s16 = vaddq_s16(q7s16, q6s16);
- d26s16 = vget_low_s16(*q13s16);
- d27s16 = vget_high_s16(*q13s16);
- d28s16 = vget_low_s16(*q14s16);
- d29s16 = vget_high_s16(*q14s16);
-
- d16s16 = vdup_n_s16(cospi_16_64);
-
- q9s32 = vmull_s16(d28s16, d16s16);
- q10s32 = vmull_s16(d29s16, d16s16);
- q11s32 = vmull_s16(d28s16, d16s16);
- q12s32 = vmull_s16(d29s16, d16s16);
-
- q9s32 = vmlsl_s16(q9s32, d26s16, d16s16);
- q10s32 = vmlsl_s16(q10s32, d27s16, d16s16);
- q11s32 = vmlal_s16(q11s32, d26s16, d16s16);
- q12s32 = vmlal_s16(q12s32, d27s16, d16s16);
-
- d10s16 = vqrshrn_n_s32(q9s32, 14);
- d11s16 = vqrshrn_n_s32(q10s32, 14);
- d12s16 = vqrshrn_n_s32(q11s32, 14);
- d13s16 = vqrshrn_n_s32(q12s32, 14);
- q5s16 = vcombine_s16(d10s16, d11s16);
- q6s16 = vcombine_s16(d12s16, d13s16);
-
- *q8s16 = vaddq_s16(q0s16, q7s16);
- *q9s16 = vaddq_s16(q1s16, q6s16);
- *q10s16 = vaddq_s16(q2s16, q5s16);
- *q11s16 = vaddq_s16(q3s16, q4s16);
- *q12s16 = vsubq_s16(q3s16, q4s16);
- *q13s16 = vsubq_s16(q2s16, q5s16);
- *q14s16 = vsubq_s16(q1s16, q6s16);
- *q15s16 = vsubq_s16(q0s16, q7s16);
- return;
-}
-
-static INLINE void IADST8X8_1D(
- int16x8_t *q8s16,
- int16x8_t *q9s16,
- int16x8_t *q10s16,
- int16x8_t *q11s16,
- int16x8_t *q12s16,
- int16x8_t *q13s16,
- int16x8_t *q14s16,
- int16x8_t *q15s16) {
- int16x4_t d0s16, d1s16, d2s16, d3s16, d4s16, d5s16, d6s16, d7s16;
- int16x4_t d8s16, d9s16, d10s16, d11s16, d12s16, d13s16, d14s16, d15s16;
- int16x4_t d16s16, d17s16, d18s16, d19s16, d20s16, d21s16, d22s16, d23s16;
- int16x4_t d24s16, d25s16, d26s16, d27s16, d28s16, d29s16, d30s16, d31s16;
- int16x8_t q2s16, q4s16, q5s16, q6s16;
- int32x4_t q0s32, q1s32, q2s32, q3s32, q4s32, q5s32, q6s32, q7s32, q8s32;
- int32x4_t q9s32, q10s32, q11s32, q12s32, q13s32, q14s32, q15s32;
-
- d16s16 = vget_low_s16(*q8s16);
- d17s16 = vget_high_s16(*q8s16);
- d18s16 = vget_low_s16(*q9s16);
- d19s16 = vget_high_s16(*q9s16);
- d20s16 = vget_low_s16(*q10s16);
- d21s16 = vget_high_s16(*q10s16);
- d22s16 = vget_low_s16(*q11s16);
- d23s16 = vget_high_s16(*q11s16);
- d24s16 = vget_low_s16(*q12s16);
- d25s16 = vget_high_s16(*q12s16);
- d26s16 = vget_low_s16(*q13s16);
- d27s16 = vget_high_s16(*q13s16);
- d28s16 = vget_low_s16(*q14s16);
- d29s16 = vget_high_s16(*q14s16);
- d30s16 = vget_low_s16(*q15s16);
- d31s16 = vget_high_s16(*q15s16);
-
- d14s16 = vdup_n_s16(cospi_2_64);
- d15s16 = vdup_n_s16(cospi_30_64);
-
- q1s32 = vmull_s16(d30s16, d14s16);
- q2s32 = vmull_s16(d31s16, d14s16);
- q3s32 = vmull_s16(d30s16, d15s16);
- q4s32 = vmull_s16(d31s16, d15s16);
-
- d30s16 = vdup_n_s16(cospi_18_64);
- d31s16 = vdup_n_s16(cospi_14_64);
-
- q1s32 = vmlal_s16(q1s32, d16s16, d15s16);
- q2s32 = vmlal_s16(q2s32, d17s16, d15s16);
- q3s32 = vmlsl_s16(q3s32, d16s16, d14s16);
- q4s32 = vmlsl_s16(q4s32, d17s16, d14s16);
-
- q5s32 = vmull_s16(d22s16, d30s16);
- q6s32 = vmull_s16(d23s16, d30s16);
- q7s32 = vmull_s16(d22s16, d31s16);
- q8s32 = vmull_s16(d23s16, d31s16);
-
- q5s32 = vmlal_s16(q5s32, d24s16, d31s16);
- q6s32 = vmlal_s16(q6s32, d25s16, d31s16);
- q7s32 = vmlsl_s16(q7s32, d24s16, d30s16);
- q8s32 = vmlsl_s16(q8s32, d25s16, d30s16);
-
- q11s32 = vaddq_s32(q1s32, q5s32);
- q12s32 = vaddq_s32(q2s32, q6s32);
- q1s32 = vsubq_s32(q1s32, q5s32);
- q2s32 = vsubq_s32(q2s32, q6s32);
-
- d22s16 = vqrshrn_n_s32(q11s32, 14);
- d23s16 = vqrshrn_n_s32(q12s32, 14);
- *q11s16 = vcombine_s16(d22s16, d23s16);
-
- q12s32 = vaddq_s32(q3s32, q7s32);
- q15s32 = vaddq_s32(q4s32, q8s32);
- q3s32 = vsubq_s32(q3s32, q7s32);
- q4s32 = vsubq_s32(q4s32, q8s32);
-
- d2s16 = vqrshrn_n_s32(q1s32, 14);
- d3s16 = vqrshrn_n_s32(q2s32, 14);
- d24s16 = vqrshrn_n_s32(q12s32, 14);
- d25s16 = vqrshrn_n_s32(q15s32, 14);
- d6s16 = vqrshrn_n_s32(q3s32, 14);
- d7s16 = vqrshrn_n_s32(q4s32, 14);
- *q12s16 = vcombine_s16(d24s16, d25s16);
-
- d0s16 = vdup_n_s16(cospi_10_64);
- d1s16 = vdup_n_s16(cospi_22_64);
- q4s32 = vmull_s16(d26s16, d0s16);
- q5s32 = vmull_s16(d27s16, d0s16);
- q2s32 = vmull_s16(d26s16, d1s16);
- q6s32 = vmull_s16(d27s16, d1s16);
-
- d30s16 = vdup_n_s16(cospi_26_64);
- d31s16 = vdup_n_s16(cospi_6_64);
-
- q4s32 = vmlal_s16(q4s32, d20s16, d1s16);
- q5s32 = vmlal_s16(q5s32, d21s16, d1s16);
- q2s32 = vmlsl_s16(q2s32, d20s16, d0s16);
- q6s32 = vmlsl_s16(q6s32, d21s16, d0s16);
-
- q0s32 = vmull_s16(d18s16, d30s16);
- q13s32 = vmull_s16(d19s16, d30s16);
-
- q0s32 = vmlal_s16(q0s32, d28s16, d31s16);
- q13s32 = vmlal_s16(q13s32, d29s16, d31s16);
-
- q10s32 = vmull_s16(d18s16, d31s16);
- q9s32 = vmull_s16(d19s16, d31s16);
-
- q10s32 = vmlsl_s16(q10s32, d28s16, d30s16);
- q9s32 = vmlsl_s16(q9s32, d29s16, d30s16);
-
- q14s32 = vaddq_s32(q2s32, q10s32);
- q15s32 = vaddq_s32(q6s32, q9s32);
- q2s32 = vsubq_s32(q2s32, q10s32);
- q6s32 = vsubq_s32(q6s32, q9s32);
-
- d28s16 = vqrshrn_n_s32(q14s32, 14);
- d29s16 = vqrshrn_n_s32(q15s32, 14);
- d4s16 = vqrshrn_n_s32(q2s32, 14);
- d5s16 = vqrshrn_n_s32(q6s32, 14);
- *q14s16 = vcombine_s16(d28s16, d29s16);
-
- q9s32 = vaddq_s32(q4s32, q0s32);
- q10s32 = vaddq_s32(q5s32, q13s32);
- q4s32 = vsubq_s32(q4s32, q0s32);
- q5s32 = vsubq_s32(q5s32, q13s32);
-
- d30s16 = vdup_n_s16(cospi_8_64);
- d31s16 = vdup_n_s16(cospi_24_64);
-
- d18s16 = vqrshrn_n_s32(q9s32, 14);
- d19s16 = vqrshrn_n_s32(q10s32, 14);
- d8s16 = vqrshrn_n_s32(q4s32, 14);
- d9s16 = vqrshrn_n_s32(q5s32, 14);
- *q9s16 = vcombine_s16(d18s16, d19s16);
-
- q5s32 = vmull_s16(d2s16, d30s16);
- q6s32 = vmull_s16(d3s16, d30s16);
- q7s32 = vmull_s16(d2s16, d31s16);
- q0s32 = vmull_s16(d3s16, d31s16);
-
- q5s32 = vmlal_s16(q5s32, d6s16, d31s16);
- q6s32 = vmlal_s16(q6s32, d7s16, d31s16);
- q7s32 = vmlsl_s16(q7s32, d6s16, d30s16);
- q0s32 = vmlsl_s16(q0s32, d7s16, d30s16);
-
- q1s32 = vmull_s16(d4s16, d30s16);
- q3s32 = vmull_s16(d5s16, d30s16);
- q10s32 = vmull_s16(d4s16, d31s16);
- q2s32 = vmull_s16(d5s16, d31s16);
-
- q1s32 = vmlsl_s16(q1s32, d8s16, d31s16);
- q3s32 = vmlsl_s16(q3s32, d9s16, d31s16);
- q10s32 = vmlal_s16(q10s32, d8s16, d30s16);
- q2s32 = vmlal_s16(q2s32, d9s16, d30s16);
-
- *q8s16 = vaddq_s16(*q11s16, *q9s16);
- *q11s16 = vsubq_s16(*q11s16, *q9s16);
- q4s16 = vaddq_s16(*q12s16, *q14s16);
- *q12s16 = vsubq_s16(*q12s16, *q14s16);
-
- q14s32 = vaddq_s32(q5s32, q1s32);
- q15s32 = vaddq_s32(q6s32, q3s32);
- q5s32 = vsubq_s32(q5s32, q1s32);
- q6s32 = vsubq_s32(q6s32, q3s32);
-
- d18s16 = vqrshrn_n_s32(q14s32, 14);
- d19s16 = vqrshrn_n_s32(q15s32, 14);
- d10s16 = vqrshrn_n_s32(q5s32, 14);
- d11s16 = vqrshrn_n_s32(q6s32, 14);
- *q9s16 = vcombine_s16(d18s16, d19s16);
-
- q1s32 = vaddq_s32(q7s32, q10s32);
- q3s32 = vaddq_s32(q0s32, q2s32);
- q7s32 = vsubq_s32(q7s32, q10s32);
- q0s32 = vsubq_s32(q0s32, q2s32);
-
- d28s16 = vqrshrn_n_s32(q1s32, 14);
- d29s16 = vqrshrn_n_s32(q3s32, 14);
- d14s16 = vqrshrn_n_s32(q7s32, 14);
- d15s16 = vqrshrn_n_s32(q0s32, 14);
- *q14s16 = vcombine_s16(d28s16, d29s16);
-
- d30s16 = vdup_n_s16(cospi_16_64);
-
- d22s16 = vget_low_s16(*q11s16);
- d23s16 = vget_high_s16(*q11s16);
- q2s32 = vmull_s16(d22s16, d30s16);
- q3s32 = vmull_s16(d23s16, d30s16);
- q13s32 = vmull_s16(d22s16, d30s16);
- q1s32 = vmull_s16(d23s16, d30s16);
-
- d24s16 = vget_low_s16(*q12s16);
- d25s16 = vget_high_s16(*q12s16);
- q2s32 = vmlal_s16(q2s32, d24s16, d30s16);
- q3s32 = vmlal_s16(q3s32, d25s16, d30s16);
- q13s32 = vmlsl_s16(q13s32, d24s16, d30s16);
- q1s32 = vmlsl_s16(q1s32, d25s16, d30s16);
-
- d4s16 = vqrshrn_n_s32(q2s32, 14);
- d5s16 = vqrshrn_n_s32(q3s32, 14);
- d24s16 = vqrshrn_n_s32(q13s32, 14);
- d25s16 = vqrshrn_n_s32(q1s32, 14);
- q2s16 = vcombine_s16(d4s16, d5s16);
- *q12s16 = vcombine_s16(d24s16, d25s16);
-
- q13s32 = vmull_s16(d10s16, d30s16);
- q1s32 = vmull_s16(d11s16, d30s16);
- q11s32 = vmull_s16(d10s16, d30s16);
- q0s32 = vmull_s16(d11s16, d30s16);
-
- q13s32 = vmlal_s16(q13s32, d14s16, d30s16);
- q1s32 = vmlal_s16(q1s32, d15s16, d30s16);
- q11s32 = vmlsl_s16(q11s32, d14s16, d30s16);
- q0s32 = vmlsl_s16(q0s32, d15s16, d30s16);
-
- d20s16 = vqrshrn_n_s32(q13s32, 14);
- d21s16 = vqrshrn_n_s32(q1s32, 14);
- d12s16 = vqrshrn_n_s32(q11s32, 14);
- d13s16 = vqrshrn_n_s32(q0s32, 14);
- *q10s16 = vcombine_s16(d20s16, d21s16);
- q6s16 = vcombine_s16(d12s16, d13s16);
-
- q5s16 = vdupq_n_s16(0);
-
- *q9s16 = vsubq_s16(q5s16, *q9s16);
- *q11s16 = vsubq_s16(q5s16, q2s16);
- *q13s16 = vsubq_s16(q5s16, q6s16);
- *q15s16 = vsubq_s16(q5s16, q4s16);
- return;
-}
-
-void vp10_iht8x8_64_add_neon(const tran_low_t *input, uint8_t *dest,
- int dest_stride, int tx_type) {
- int i;
- uint8_t *d1, *d2;
- uint8x8_t d0u8, d1u8, d2u8, d3u8;
- uint64x1_t d0u64, d1u64, d2u64, d3u64;
- int16x8_t q8s16, q9s16, q10s16, q11s16, q12s16, q13s16, q14s16, q15s16;
- uint16x8_t q8u16, q9u16, q10u16, q11u16;
-
- q8s16 = vld1q_s16(input);
- q9s16 = vld1q_s16(input + 8);
- q10s16 = vld1q_s16(input + 8 * 2);
- q11s16 = vld1q_s16(input + 8 * 3);
- q12s16 = vld1q_s16(input + 8 * 4);
- q13s16 = vld1q_s16(input + 8 * 5);
- q14s16 = vld1q_s16(input + 8 * 6);
- q15s16 = vld1q_s16(input + 8 * 7);
-
- TRANSPOSE8X8(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- switch (tx_type) {
- case 0: // idct_idct is not supported. Fall back to C
- vp10_iht8x8_64_add_c(input, dest, dest_stride, tx_type);
- return;
- break;
- case 1: // iadst_idct
- // generate IDCT constants
- // GENERATE_IDCT_CONSTANTS
-
- // first transform rows
- IDCT8x8_1D(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- // transpose the matrix
- TRANSPOSE8X8(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- // generate IADST constants
- // GENERATE_IADST_CONSTANTS
-
- // then transform columns
- IADST8X8_1D(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
- break;
- case 2: // idct_iadst
- // generate IADST constants
- // GENERATE_IADST_CONSTANTS
-
- // first transform rows
- IADST8X8_1D(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- // transpose the matrix
- TRANSPOSE8X8(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- // generate IDCT constants
- // GENERATE_IDCT_CONSTANTS
-
- // then transform columns
- IDCT8x8_1D(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
- break;
- case 3: // iadst_iadst
- // generate IADST constants
- // GENERATE_IADST_CONSTANTS
-
- // first transform rows
- IADST8X8_1D(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- // transpose the matrix
- TRANSPOSE8X8(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
-
- // then transform columns
- IADST8X8_1D(&q8s16, &q9s16, &q10s16, &q11s16,
- &q12s16, &q13s16, &q14s16, &q15s16);
- break;
- default: // iadst_idct
- assert(0);
- break;
- }
-
- q8s16 = vrshrq_n_s16(q8s16, 5);
- q9s16 = vrshrq_n_s16(q9s16, 5);
- q10s16 = vrshrq_n_s16(q10s16, 5);
- q11s16 = vrshrq_n_s16(q11s16, 5);
- q12s16 = vrshrq_n_s16(q12s16, 5);
- q13s16 = vrshrq_n_s16(q13s16, 5);
- q14s16 = vrshrq_n_s16(q14s16, 5);
- q15s16 = vrshrq_n_s16(q15s16, 5);
-
- for (d1 = d2 = dest, i = 0; i < 2; i++) {
- if (i != 0) {
- q8s16 = q12s16;
- q9s16 = q13s16;
- q10s16 = q14s16;
- q11s16 = q15s16;
- }
-
- d0u64 = vld1_u64((uint64_t *)d1);
- d1 += dest_stride;
- d1u64 = vld1_u64((uint64_t *)d1);
- d1 += dest_stride;
- d2u64 = vld1_u64((uint64_t *)d1);
- d1 += dest_stride;
- d3u64 = vld1_u64((uint64_t *)d1);
- d1 += dest_stride;
-
- q8u16 = vaddw_u8(vreinterpretq_u16_s16(q8s16),
- vreinterpret_u8_u64(d0u64));
- q9u16 = vaddw_u8(vreinterpretq_u16_s16(q9s16),
- vreinterpret_u8_u64(d1u64));
- q10u16 = vaddw_u8(vreinterpretq_u16_s16(q10s16),
- vreinterpret_u8_u64(d2u64));
- q11u16 = vaddw_u8(vreinterpretq_u16_s16(q11s16),
- vreinterpret_u8_u64(d3u64));
-
- d0u8 = vqmovun_s16(vreinterpretq_s16_u16(q8u16));
- d1u8 = vqmovun_s16(vreinterpretq_s16_u16(q9u16));
- d2u8 = vqmovun_s16(vreinterpretq_s16_u16(q10u16));
- d3u8 = vqmovun_s16(vreinterpretq_s16_u16(q11u16));
-
- vst1_u64((uint64_t *)d2, vreinterpret_u64_u8(d0u8));
- d2 += dest_stride;
- vst1_u64((uint64_t *)d2, vreinterpret_u64_u8(d1u8));
- d2 += dest_stride;
- vst1_u64((uint64_t *)d2, vreinterpret_u64_u8(d2u8));
- d2 += dest_stride;
- vst1_u64((uint64_t *)d2, vreinterpret_u64_u8(d3u8));
- d2 += dest_stride;
- }
- return;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/blockd.h"
-
-PREDICTION_MODE vp10_left_block_mode(const MODE_INFO *cur_mi,
- const MODE_INFO *left_mi, int b) {
- if (b == 0 || b == 2) {
- if (!left_mi || is_inter_block(&left_mi->mbmi))
- return DC_PRED;
-
- return get_y_mode(left_mi, b + 1);
- } else {
- assert(b == 1 || b == 3);
- return cur_mi->bmi[b - 1].as_mode;
- }
-}
-
-PREDICTION_MODE vp10_above_block_mode(const MODE_INFO *cur_mi,
- const MODE_INFO *above_mi, int b) {
- if (b == 0 || b == 1) {
- if (!above_mi || is_inter_block(&above_mi->mbmi))
- return DC_PRED;
-
- return get_y_mode(above_mi, b + 2);
- } else {
- assert(b == 2 || b == 3);
- return cur_mi->bmi[b - 2].as_mode;
- }
-}
-
-void vp10_foreach_transformed_block_in_plane(
- const MACROBLOCKD *const xd, BLOCK_SIZE bsize, int plane,
- foreach_transformed_block_visitor visit, void *arg) {
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- const MB_MODE_INFO* mbmi = &xd->mi[0]->mbmi;
- // block and transform sizes, in number of 4x4 blocks log 2 ("*_b")
- // 4x4=0, 8x8=2, 16x16=4, 32x32=6, 64x64=8
- // transform size varies per plane, look it up in a common way.
- const TX_SIZE tx_size = plane ? get_uv_tx_size(mbmi, pd)
- : mbmi->tx_size;
- const BLOCK_SIZE plane_bsize = get_plane_block_size(bsize, pd);
- const int num_4x4_w = num_4x4_blocks_wide_lookup[plane_bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[plane_bsize];
- const int step = 1 << (tx_size << 1);
- int i = 0, r, c;
-
- // If mb_to_right_edge is < 0 we are in a situation in which
- // the current block size extends into the UMV and we won't
- // visit the sub blocks that are wholly within the UMV.
- const int max_blocks_wide = num_4x4_w + (xd->mb_to_right_edge >= 0 ? 0 :
- xd->mb_to_right_edge >> (5 + pd->subsampling_x));
- const int max_blocks_high = num_4x4_h + (xd->mb_to_bottom_edge >= 0 ? 0 :
- xd->mb_to_bottom_edge >> (5 + pd->subsampling_y));
- const int extra_step = ((num_4x4_w - max_blocks_wide) >> tx_size) * step;
-
- // Keep track of the row and column of the blocks we use so that we know
- // if we are in the unrestricted motion border.
- for (r = 0; r < max_blocks_high; r += (1 << tx_size)) {
- // Skip visiting the sub blocks that are wholly within the UMV.
- for (c = 0; c < max_blocks_wide; c += (1 << tx_size)) {
- visit(plane, i, r, c, plane_bsize, tx_size, arg);
- i += step;
- }
- i += extra_step;
- }
-}
-
-void vp10_foreach_transformed_block(const MACROBLOCKD* const xd,
- BLOCK_SIZE bsize,
- foreach_transformed_block_visitor visit,
- void *arg) {
- int plane;
-
- for (plane = 0; plane < MAX_MB_PLANE; ++plane)
- vp10_foreach_transformed_block_in_plane(xd, bsize, plane, visit, arg);
-}
-
-void vp10_set_contexts(const MACROBLOCKD *xd, struct macroblockd_plane *pd,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size, int has_eob,
- int aoff, int loff) {
- ENTROPY_CONTEXT *const a = pd->above_context + aoff;
- ENTROPY_CONTEXT *const l = pd->left_context + loff;
- const int tx_size_in_blocks = 1 << tx_size;
-
- // above
- if (has_eob && xd->mb_to_right_edge < 0) {
- int i;
- const int blocks_wide = num_4x4_blocks_wide_lookup[plane_bsize] +
- (xd->mb_to_right_edge >> (5 + pd->subsampling_x));
- int above_contexts = tx_size_in_blocks;
- if (above_contexts + aoff > blocks_wide)
- above_contexts = blocks_wide - aoff;
-
- for (i = 0; i < above_contexts; ++i)
- a[i] = has_eob;
- for (i = above_contexts; i < tx_size_in_blocks; ++i)
- a[i] = 0;
- } else {
- memset(a, has_eob, sizeof(ENTROPY_CONTEXT) * tx_size_in_blocks);
- }
-
- // left
- if (has_eob && xd->mb_to_bottom_edge < 0) {
- int i;
- const int blocks_high = num_4x4_blocks_high_lookup[plane_bsize] +
- (xd->mb_to_bottom_edge >> (5 + pd->subsampling_y));
- int left_contexts = tx_size_in_blocks;
- if (left_contexts + loff > blocks_high)
- left_contexts = blocks_high - loff;
-
- for (i = 0; i < left_contexts; ++i)
- l[i] = has_eob;
- for (i = left_contexts; i < tx_size_in_blocks; ++i)
- l[i] = 0;
- } else {
- memset(l, has_eob, sizeof(ENTROPY_CONTEXT) * tx_size_in_blocks);
- }
-}
-
-void vp10_setup_block_planes(MACROBLOCKD *xd, int ss_x, int ss_y) {
- int i;
-
- for (i = 0; i < MAX_MB_PLANE; i++) {
- xd->plane[i].plane_type = i ? PLANE_TYPE_UV : PLANE_TYPE_Y;
- xd->plane[i].subsampling_x = i ? ss_x : 0;
- xd->plane[i].subsampling_y = i ? ss_y : 0;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_COMMON_BLOCKD_H_
-#define VP10_COMMON_BLOCKD_H_
-
-#include "./vpx_config.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_ports/mem.h"
-#include "vpx_scale/yv12config.h"
-
-#include "vp10/common/common_data.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/mv.h"
-#include "vp10/common/scale.h"
-#include "vp10/common/seg_common.h"
-#include "vp10/common/tile_common.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MAX_MB_PLANE 3
-
-typedef enum {
- KEY_FRAME = 0,
- INTER_FRAME = 1,
- FRAME_TYPES,
-} FRAME_TYPE;
-
-static INLINE int is_inter_mode(PREDICTION_MODE mode) {
- return mode >= NEARESTMV && mode <= NEWMV;
-}
-
-/* For keyframes, intra block modes are predicted by the (already decoded)
- modes for the Y blocks to the left and above us; for interframes, there
- is a single probability table. */
-
-typedef struct {
- PREDICTION_MODE as_mode;
- int_mv as_mv[2]; // first, second inter predictor motion vectors
-} b_mode_info;
-
-// Note that the rate-distortion optimization loop, bit-stream writer, and
-// decoder implementation modules critically rely on the defined entry values
-// specified herein. They should be refactored concurrently.
-
-#define NONE -1
-#define INTRA_FRAME 0
-#define LAST_FRAME 1
-#define GOLDEN_FRAME 2
-#define ALTREF_FRAME 3
-#define MAX_REF_FRAMES 4
-typedef int8_t MV_REFERENCE_FRAME;
-
-typedef struct {
- // Number of base colors for Y (0) and UV (1)
- uint8_t palette_size[2];
- // Value of base colors for Y, U, and V
-#if CONFIG_VP9_HIGHBITDEPTH
- uint16_t palette_colors[3 * PALETTE_MAX_SIZE];
-#else
- uint8_t palette_colors[3 * PALETTE_MAX_SIZE];
-#endif // CONFIG_VP9_HIGHBITDEPTH
- // Only used by encoder to store the color index of the top left pixel.
- // TODO(huisu): move this to encoder
- uint8_t palette_first_color_idx[2];
-} PALETTE_MODE_INFO;
-
-// This structure now relates to 8x8 block regions.
-typedef struct {
- // Common for both INTER and INTRA blocks
- BLOCK_SIZE sb_type;
- PREDICTION_MODE mode;
- TX_SIZE tx_size;
- int8_t skip;
-#if CONFIG_MISC_FIXES
- int8_t has_no_coeffs;
-#endif
- int8_t segment_id;
- int8_t seg_id_predicted; // valid only when temporal_update is enabled
-
- // Only for INTRA blocks
- PREDICTION_MODE uv_mode;
- PALETTE_MODE_INFO palette_mode_info;
-
- // Only for INTER blocks
- INTERP_FILTER interp_filter;
- MV_REFERENCE_FRAME ref_frame[2];
-
- // TODO(slavarnway): Delete and use bmi[3].as_mv[] instead.
- int_mv mv[2];
-} MB_MODE_INFO;
-
-typedef struct MODE_INFO {
- MB_MODE_INFO mbmi;
- b_mode_info bmi[4];
-} MODE_INFO;
-
-static INLINE PREDICTION_MODE get_y_mode(const MODE_INFO *mi, int block) {
- return mi->mbmi.sb_type < BLOCK_8X8 ? mi->bmi[block].as_mode
- : mi->mbmi.mode;
-}
-
-static INLINE int is_inter_block(const MB_MODE_INFO *mbmi) {
- return mbmi->ref_frame[0] > INTRA_FRAME;
-}
-
-static INLINE int has_second_ref(const MB_MODE_INFO *mbmi) {
- return mbmi->ref_frame[1] > INTRA_FRAME;
-}
-
-PREDICTION_MODE vp10_left_block_mode(const MODE_INFO *cur_mi,
- const MODE_INFO *left_mi, int b);
-
-PREDICTION_MODE vp10_above_block_mode(const MODE_INFO *cur_mi,
- const MODE_INFO *above_mi, int b);
-
-enum mv_precision {
- MV_PRECISION_Q3,
- MV_PRECISION_Q4
-};
-
-struct buf_2d {
- uint8_t *buf;
- int stride;
-};
-
-struct macroblockd_plane {
- tran_low_t *dqcoeff;
- PLANE_TYPE plane_type;
- int subsampling_x;
- int subsampling_y;
- struct buf_2d dst;
- struct buf_2d pre[2];
- ENTROPY_CONTEXT *above_context;
- ENTROPY_CONTEXT *left_context;
- int16_t seg_dequant[MAX_SEGMENTS][2];
- uint8_t *color_index_map;
-
- // number of 4x4s in current block
- uint16_t n4_w, n4_h;
- // log2 of n4_w, n4_h
- uint8_t n4_wl, n4_hl;
-
- // encoder
- const int16_t *dequant;
-};
-
-#define BLOCK_OFFSET(x, i) ((x) + (i) * 16)
-
-typedef struct RefBuffer {
- // TODO(dkovalev): idx is not really required and should be removed, now it
- // is used in vp10_onyxd_if.c
- int idx;
- YV12_BUFFER_CONFIG *buf;
- struct scale_factors sf;
-} RefBuffer;
-
-typedef struct macroblockd {
- struct macroblockd_plane plane[MAX_MB_PLANE];
- uint8_t bmode_blocks_wl;
- uint8_t bmode_blocks_hl;
-
- FRAME_COUNTS *counts;
- TileInfo tile;
-
- int mi_stride;
-
- MODE_INFO **mi;
- MODE_INFO *left_mi;
- MODE_INFO *above_mi;
- MB_MODE_INFO *left_mbmi;
- MB_MODE_INFO *above_mbmi;
-
- int up_available;
- int left_available;
-
- /* Distance of MB away from frame edges */
- int mb_to_left_edge;
- int mb_to_right_edge;
- int mb_to_top_edge;
- int mb_to_bottom_edge;
-
- FRAME_CONTEXT *fc;
-
- /* pointers to reference frames */
- RefBuffer *block_refs[2];
-
- /* pointer to current frame */
- const YV12_BUFFER_CONFIG *cur_buf;
-
- ENTROPY_CONTEXT *above_context[MAX_MB_PLANE];
- ENTROPY_CONTEXT left_context[MAX_MB_PLANE][16];
-
- PARTITION_CONTEXT *above_seg_context;
- PARTITION_CONTEXT left_seg_context[8];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- /* Bit depth: 8, 10, 12 */
- int bd;
-#endif
-
- int lossless[MAX_SEGMENTS];
- int corrupted;
-
- struct vpx_internal_error_info *error_info;
-} MACROBLOCKD;
-
-static INLINE BLOCK_SIZE get_subsize(BLOCK_SIZE bsize,
- PARTITION_TYPE partition) {
- return subsize_lookup[partition][bsize];
-}
-
-static const TX_TYPE intra_mode_to_tx_type_lookup[INTRA_MODES] = {
- DCT_DCT, // DC
- ADST_DCT, // V
- DCT_ADST, // H
- DCT_DCT, // D45
- ADST_ADST, // D135
- ADST_DCT, // D117
- DCT_ADST, // D153
- DCT_ADST, // D207
- ADST_DCT, // D63
- ADST_ADST, // TM
-};
-
-static INLINE TX_TYPE get_tx_type(PLANE_TYPE plane_type, const MACROBLOCKD *xd,
- int block_idx) {
- const MODE_INFO *const mi = xd->mi[0];
- const MB_MODE_INFO *const mbmi = &mi->mbmi;
-
- if (plane_type != PLANE_TYPE_Y || xd->lossless[mbmi->segment_id] ||
- is_inter_block(mbmi) || mbmi->tx_size >= TX_32X32)
- return DCT_DCT;
-
- return intra_mode_to_tx_type_lookup[get_y_mode(mi, block_idx)];
-}
-
-void vp10_setup_block_planes(MACROBLOCKD *xd, int ss_x, int ss_y);
-
-static INLINE TX_SIZE get_uv_tx_size_impl(TX_SIZE y_tx_size, BLOCK_SIZE bsize,
- int xss, int yss) {
- if (bsize < BLOCK_8X8) {
- return TX_4X4;
- } else {
- const BLOCK_SIZE plane_bsize = ss_size_lookup[bsize][xss][yss];
- return VPXMIN(y_tx_size, max_txsize_lookup[plane_bsize]);
- }
-}
-
-static INLINE TX_SIZE get_uv_tx_size(const MB_MODE_INFO *mbmi,
- const struct macroblockd_plane *pd) {
- return get_uv_tx_size_impl(mbmi->tx_size, mbmi->sb_type, pd->subsampling_x,
- pd->subsampling_y);
-}
-
-static INLINE BLOCK_SIZE get_plane_block_size(BLOCK_SIZE bsize,
- const struct macroblockd_plane *pd) {
- return ss_size_lookup[bsize][pd->subsampling_x][pd->subsampling_y];
-}
-
-static INLINE void reset_skip_context(MACROBLOCKD *xd, BLOCK_SIZE bsize) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++) {
- struct macroblockd_plane *const pd = &xd->plane[i];
- const BLOCK_SIZE plane_bsize = get_plane_block_size(bsize, pd);
- memset(pd->above_context, 0,
- sizeof(ENTROPY_CONTEXT) * num_4x4_blocks_wide_lookup[plane_bsize]);
- memset(pd->left_context, 0,
- sizeof(ENTROPY_CONTEXT) * num_4x4_blocks_high_lookup[plane_bsize]);
- }
-}
-
-typedef void (*foreach_transformed_block_visitor)(int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size,
- void *arg);
-
-void vp10_foreach_transformed_block_in_plane(
- const MACROBLOCKD *const xd, BLOCK_SIZE bsize, int plane,
- foreach_transformed_block_visitor visit, void *arg);
-
-
-void vp10_foreach_transformed_block(
- const MACROBLOCKD* const xd, BLOCK_SIZE bsize,
- foreach_transformed_block_visitor visit, void *arg);
-
-void vp10_set_contexts(const MACROBLOCKD *xd, struct macroblockd_plane *pd,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size, int has_eob,
- int aoff, int loff);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_BLOCKD_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_COMMON_H_
-#define VP10_COMMON_COMMON_H_
-
-/* Interface header for common constant data structures and lookup tables */
-
-#include <assert.h>
-
-#include "./vpx_config.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx/vpx_integer.h"
-#include "vpx_ports/bitops.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// Only need this for fixed-size arrays, for structs just assign.
-#define vp10_copy(dest, src) { \
- assert(sizeof(dest) == sizeof(src)); \
- memcpy(dest, src, sizeof(src)); \
- }
-
-// Use this for variably-sized arrays.
-#define vp10_copy_array(dest, src, n) { \
- assert(sizeof(*dest) == sizeof(*src)); \
- memcpy(dest, src, n * sizeof(*src)); \
- }
-
-#define vp10_zero(dest) memset(&(dest), 0, sizeof(dest))
-#define vp10_zero_array(dest, n) memset(dest, 0, n * sizeof(*dest))
-
-static INLINE int get_unsigned_bits(unsigned int num_values) {
- return num_values > 0 ? get_msb(num_values) + 1 : 0;
-}
-
-#if CONFIG_DEBUG
-#define CHECK_MEM_ERROR(cm, lval, expr) do { \
- lval = (expr); \
- if (!lval) \
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR, \
- "Failed to allocate "#lval" at %s:%d", \
- __FILE__, __LINE__); \
- } while (0)
-#else
-#define CHECK_MEM_ERROR(cm, lval, expr) do { \
- lval = (expr); \
- if (!lval) \
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR, \
- "Failed to allocate "#lval); \
- } while (0)
-#endif
-// TODO(yaowu: validate the usage of these codes or develop new ones.)
-#define VP10_SYNC_CODE_0 0x49
-#define VP10_SYNC_CODE_1 0x83
-#define VP10_SYNC_CODE_2 0x43
-
-#define VP9_FRAME_MARKER 0x2
-
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_COMMON_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_COMMON_DATA_H_
-#define VP10_COMMON_COMMON_DATA_H_
-
-#include "vp10/common/enums.h"
-#include "vpx/vpx_integer.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// Log 2 conversion lookup tables for block width and height
-static const uint8_t b_width_log2_lookup[BLOCK_SIZES] =
- {0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4};
-static const uint8_t b_height_log2_lookup[BLOCK_SIZES] =
- {0, 1, 0, 1, 2, 1, 2, 3, 2, 3, 4, 3, 4};
-static const uint8_t num_4x4_blocks_wide_lookup[BLOCK_SIZES] =
- {1, 1, 2, 2, 2, 4, 4, 4, 8, 8, 8, 16, 16};
-static const uint8_t num_4x4_blocks_high_lookup[BLOCK_SIZES] =
- {1, 2, 1, 2, 4, 2, 4, 8, 4, 8, 16, 8, 16};
-// Log 2 conversion lookup tables for modeinfo width and height
-static const uint8_t mi_width_log2_lookup[BLOCK_SIZES] =
- {0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3};
-static const uint8_t mi_height_log2_lookup[BLOCK_SIZES] =
- {0, 0, 0, 0, 1, 0, 1, 2, 1, 2, 3, 2, 3};
-static const uint8_t num_8x8_blocks_wide_lookup[BLOCK_SIZES] =
- {1, 1, 1, 1, 1, 2, 2, 2, 4, 4, 4, 8, 8};
-static const uint8_t num_8x8_blocks_high_lookup[BLOCK_SIZES] =
- {1, 1, 1, 1, 2, 1, 2, 4, 2, 4, 8, 4, 8};
-
-// VPXMIN(3, VPXMIN(b_width_log2(bsize), b_height_log2(bsize)))
-static const uint8_t size_group_lookup[BLOCK_SIZES] =
- {0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3};
-
-static const uint8_t num_pels_log2_lookup[BLOCK_SIZES] =
- {4, 5, 5, 6, 7, 7, 8, 9, 9, 10, 11, 11, 12};
-
-static const PARTITION_TYPE partition_lookup[][BLOCK_SIZES] = {
- { // 4X4
- // 4X4, 4X8,8X4,8X8,8X16,16X8,16X16,16X32,32X16,32X32,32X64,64X32,64X64
- PARTITION_NONE, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID
- }, { // 8X8
- // 4X4, 4X8,8X4,8X8,8X16,16X8,16X16,16X32,32X16,32X32,32X64,64X32,64X64
- PARTITION_SPLIT, PARTITION_VERT, PARTITION_HORZ, PARTITION_NONE,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID
- }, { // 16X16
- // 4X4, 4X8,8X4,8X8,8X16,16X8,16X16,16X32,32X16,32X32,32X64,64X32,64X64
- PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT,
- PARTITION_VERT, PARTITION_HORZ, PARTITION_NONE, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID
- }, { // 32X32
- // 4X4, 4X8,8X4,8X8,8X16,16X8,16X16,16X32,32X16,32X32,32X64,64X32,64X64
- PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT,
- PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_VERT,
- PARTITION_HORZ, PARTITION_NONE, PARTITION_INVALID,
- PARTITION_INVALID, PARTITION_INVALID
- }, { // 64X64
- // 4X4, 4X8,8X4,8X8,8X16,16X8,16X16,16X32,32X16,32X32,32X64,64X32,64X64
- PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT,
- PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_SPLIT,
- PARTITION_SPLIT, PARTITION_SPLIT, PARTITION_VERT, PARTITION_HORZ,
- PARTITION_NONE
- }
-};
-
-static const BLOCK_SIZE subsize_lookup[PARTITION_TYPES][BLOCK_SIZES] = {
- { // PARTITION_NONE
- BLOCK_4X4, BLOCK_4X8, BLOCK_8X4,
- BLOCK_8X8, BLOCK_8X16, BLOCK_16X8,
- BLOCK_16X16, BLOCK_16X32, BLOCK_32X16,
- BLOCK_32X32, BLOCK_32X64, BLOCK_64X32,
- BLOCK_64X64,
- }, { // PARTITION_HORZ
- BLOCK_INVALID, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_8X4, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_16X8, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_32X16, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_64X32,
- }, { // PARTITION_VERT
- BLOCK_INVALID, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_4X8, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_8X16, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_16X32, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_32X64,
- }, { // PARTITION_SPLIT
- BLOCK_INVALID, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_4X4, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_8X8, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_16X16, BLOCK_INVALID, BLOCK_INVALID,
- BLOCK_32X32,
- }
-};
-
-static const TX_SIZE max_txsize_lookup[BLOCK_SIZES] = {
- TX_4X4, TX_4X4, TX_4X4,
- TX_8X8, TX_8X8, TX_8X8,
- TX_16X16, TX_16X16, TX_16X16,
- TX_32X32, TX_32X32, TX_32X32, TX_32X32
-};
-
-static const BLOCK_SIZE txsize_to_bsize[TX_SIZES] = {
- BLOCK_4X4, // TX_4X4
- BLOCK_8X8, // TX_8X8
- BLOCK_16X16, // TX_16X16
- BLOCK_32X32, // TX_32X32
-};
-
-static const TX_SIZE tx_mode_to_biggest_tx_size[TX_MODES] = {
- TX_4X4, // ONLY_4X4
- TX_8X8, // ALLOW_8X8
- TX_16X16, // ALLOW_16X16
- TX_32X32, // ALLOW_32X32
- TX_32X32, // TX_MODE_SELECT
-};
-
-static const BLOCK_SIZE ss_size_lookup[BLOCK_SIZES][2][2] = {
-// ss_x == 0 ss_x == 0 ss_x == 1 ss_x == 1
-// ss_y == 0 ss_y == 1 ss_y == 0 ss_y == 1
- {{BLOCK_4X4, BLOCK_INVALID}, {BLOCK_INVALID, BLOCK_INVALID}},
- {{BLOCK_4X8, BLOCK_4X4}, {BLOCK_INVALID, BLOCK_INVALID}},
- {{BLOCK_8X4, BLOCK_INVALID}, {BLOCK_4X4, BLOCK_INVALID}},
- {{BLOCK_8X8, BLOCK_8X4}, {BLOCK_4X8, BLOCK_4X4}},
- {{BLOCK_8X16, BLOCK_8X8}, {BLOCK_INVALID, BLOCK_4X8}},
- {{BLOCK_16X8, BLOCK_INVALID}, {BLOCK_8X8, BLOCK_8X4}},
- {{BLOCK_16X16, BLOCK_16X8}, {BLOCK_8X16, BLOCK_8X8}},
- {{BLOCK_16X32, BLOCK_16X16}, {BLOCK_INVALID, BLOCK_8X16}},
- {{BLOCK_32X16, BLOCK_INVALID}, {BLOCK_16X16, BLOCK_16X8}},
- {{BLOCK_32X32, BLOCK_32X16}, {BLOCK_16X32, BLOCK_16X16}},
- {{BLOCK_32X64, BLOCK_32X32}, {BLOCK_INVALID, BLOCK_16X32}},
- {{BLOCK_64X32, BLOCK_INVALID}, {BLOCK_32X32, BLOCK_32X16}},
- {{BLOCK_64X64, BLOCK_64X32}, {BLOCK_32X64, BLOCK_32X32}},
-};
-
-// Generates 4 bit field in which each bit set to 1 represents
-// a blocksize partition 1111 means we split 64x64, 32x32, 16x16
-// and 8x8. 1000 means we just split the 64x64 to 32x32
-static const struct {
- PARTITION_CONTEXT above;
- PARTITION_CONTEXT left;
-} partition_context_lookup[BLOCK_SIZES]= {
- {15, 15}, // 4X4 - {0b1111, 0b1111}
- {15, 14}, // 4X8 - {0b1111, 0b1110}
- {14, 15}, // 8X4 - {0b1110, 0b1111}
- {14, 14}, // 8X8 - {0b1110, 0b1110}
- {14, 12}, // 8X16 - {0b1110, 0b1100}
- {12, 14}, // 16X8 - {0b1100, 0b1110}
- {12, 12}, // 16X16 - {0b1100, 0b1100}
- {12, 8 }, // 16X32 - {0b1100, 0b1000}
- {8, 12}, // 32X16 - {0b1000, 0b1100}
- {8, 8 }, // 32X32 - {0b1000, 0b1000}
- {8, 0 }, // 32X64 - {0b1000, 0b0000}
- {0, 8 }, // 64X32 - {0b0000, 0b1000}
- {0, 0 }, // 64X64 - {0b0000, 0b0000}
-};
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_COMMON_DATA_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <stdio.h>
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/onyxc_int.h"
-
-static void log_frame_info(VP10_COMMON *cm, const char *str, FILE *f) {
- fprintf(f, "%s", str);
- fprintf(f, "(Frame %d, Show:%d, Q:%d): \n", cm->current_video_frame,
- cm->show_frame, cm->base_qindex);
-}
-/* This function dereferences a pointer to the mbmi structure
- * and uses the passed in member offset to print out the value of an integer
- * for each mbmi member value in the mi structure.
- */
-static void print_mi_data(VP10_COMMON *cm, FILE *file, const char *descriptor,
- size_t member_offset) {
- int mi_row, mi_col;
- MODE_INFO **mi = cm->mi_grid_visible;
- int rows = cm->mi_rows;
- int cols = cm->mi_cols;
- char prefix = descriptor[0];
-
- log_frame_info(cm, descriptor, file);
- for (mi_row = 0; mi_row < rows; mi_row++) {
- fprintf(file, "%c ", prefix);
- for (mi_col = 0; mi_col < cols; mi_col++) {
- fprintf(file, "%2d ",
- *((int*) ((char *) (&mi[0]->mbmi) +
- member_offset)));
- mi++;
- }
- fprintf(file, "\n");
- mi += 8;
- }
- fprintf(file, "\n");
-}
-
-void vp10_print_modes_and_motion_vectors(VP10_COMMON *cm, const char *file) {
- int mi_row;
- int mi_col;
- FILE *mvs = fopen(file, "a");
- MODE_INFO **mi = cm->mi_grid_visible;
- int rows = cm->mi_rows;
- int cols = cm->mi_cols;
-
- print_mi_data(cm, mvs, "Partitions:", offsetof(MB_MODE_INFO, sb_type));
- print_mi_data(cm, mvs, "Modes:", offsetof(MB_MODE_INFO, mode));
- print_mi_data(cm, mvs, "Ref frame:", offsetof(MB_MODE_INFO, ref_frame[0]));
- print_mi_data(cm, mvs, "Transform:", offsetof(MB_MODE_INFO, tx_size));
- print_mi_data(cm, mvs, "UV Modes:", offsetof(MB_MODE_INFO, uv_mode));
-
- // output skip infomation.
- log_frame_info(cm, "Skips:", mvs);
- for (mi_row = 0; mi_row < rows; mi_row++) {
- fprintf(mvs, "S ");
- for (mi_col = 0; mi_col < cols; mi_col++) {
- fprintf(mvs, "%2d ", mi[0]->mbmi.skip);
- mi++;
- }
- fprintf(mvs, "\n");
- mi += 8;
- }
- fprintf(mvs, "\n");
-
- // output motion vectors.
- log_frame_info(cm, "Vectors ", mvs);
- mi = cm->mi_grid_visible;
- for (mi_row = 0; mi_row < rows; mi_row++) {
- fprintf(mvs, "V ");
- for (mi_col = 0; mi_col < cols; mi_col++) {
- fprintf(mvs, "%4d:%4d ", mi[0]->mbmi.mv[0].as_mv.row,
- mi[0]->mbmi.mv[0].as_mv.col);
- mi++;
- }
- fprintf(mvs, "\n");
- mi += 8;
- }
- fprintf(mvs, "\n");
-
- fclose(mvs);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/entropy.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/entropymode.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx/vpx_integer.h"
-
-// Unconstrained Node Tree
-const vpx_tree_index vp10_coef_con_tree[TREE_SIZE(ENTROPY_TOKENS)] = {
- 2, 6, // 0 = LOW_VAL
- -TWO_TOKEN, 4, // 1 = TWO
- -THREE_TOKEN, -FOUR_TOKEN, // 2 = THREE
- 8, 10, // 3 = HIGH_LOW
- -CATEGORY1_TOKEN, -CATEGORY2_TOKEN, // 4 = CAT_ONE
- 12, 14, // 5 = CAT_THREEFOUR
- -CATEGORY3_TOKEN, -CATEGORY4_TOKEN, // 6 = CAT_THREE
- -CATEGORY5_TOKEN, -CATEGORY6_TOKEN // 7 = CAT_FIVE
-};
-
-const vpx_prob vp10_cat1_prob[] = { 159 };
-const vpx_prob vp10_cat2_prob[] = { 165, 145 };
-const vpx_prob vp10_cat3_prob[] = { 173, 148, 140 };
-const vpx_prob vp10_cat4_prob[] = { 176, 155, 140, 135 };
-const vpx_prob vp10_cat5_prob[] = { 180, 157, 141, 134, 130 };
-const vpx_prob vp10_cat6_prob[] = {
- 254, 254, 254, 252, 249, 243, 230, 196, 177, 153, 140, 133, 130, 129
-};
-#if CONFIG_VP9_HIGHBITDEPTH
-const vpx_prob vp10_cat1_prob_high10[] = { 159 };
-const vpx_prob vp10_cat2_prob_high10[] = { 165, 145 };
-const vpx_prob vp10_cat3_prob_high10[] = { 173, 148, 140 };
-const vpx_prob vp10_cat4_prob_high10[] = { 176, 155, 140, 135 };
-const vpx_prob vp10_cat5_prob_high10[] = { 180, 157, 141, 134, 130 };
-const vpx_prob vp10_cat6_prob_high10[] = {
- 255, 255, 254, 254, 254, 252, 249, 243,
- 230, 196, 177, 153, 140, 133, 130, 129
-};
-const vpx_prob vp10_cat1_prob_high12[] = { 159 };
-const vpx_prob vp10_cat2_prob_high12[] = { 165, 145 };
-const vpx_prob vp10_cat3_prob_high12[] = { 173, 148, 140 };
-const vpx_prob vp10_cat4_prob_high12[] = { 176, 155, 140, 135 };
-const vpx_prob vp10_cat5_prob_high12[] = { 180, 157, 141, 134, 130 };
-const vpx_prob vp10_cat6_prob_high12[] = {
- 255, 255, 255, 255, 254, 254, 254, 252, 249,
- 243, 230, 196, 177, 153, 140, 133, 130, 129
-};
-#endif
-
-const uint8_t vp10_coefband_trans_8x8plus[1024] = {
- 0, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4,
- 4, 4, 4, 4, 4, 5,
- // beyond MAXBAND_INDEX+1 all values are filled as 5
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
-};
-
-const uint8_t vp10_coefband_trans_4x4[16] = {
- 0, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 5, 5, 5,
-};
-
-const uint8_t vp10_pt_energy_class[ENTROPY_TOKENS] = {
- 0, 1, 2, 3, 3, 4, 4, 5, 5, 5, 5, 5
-};
-
-// Model obtained from a 2-sided zero-centerd distribuition derived
-// from a Pareto distribution. The cdf of the distribution is:
-// cdf(x) = 0.5 + 0.5 * sgn(x) * [1 - {alpha/(alpha + |x|)} ^ beta]
-//
-// For a given beta and a given probablity of the 1-node, the alpha
-// is first solved, and then the {alpha, beta} pair is used to generate
-// the probabilities for the rest of the nodes.
-
-// beta = 8
-
-// Every odd line in this table can be generated from the even lines
-// by averaging :
-// vp10_pareto8_full[l][node] = (vp10_pareto8_full[l-1][node] +
-// vp10_pareto8_full[l+1][node] ) >> 1;
-const vpx_prob vp10_pareto8_full[COEFF_PROB_MODELS][MODEL_NODES] = {
- { 3, 86, 128, 6, 86, 23, 88, 29},
- { 6, 86, 128, 11, 87, 42, 91, 52},
- { 9, 86, 129, 17, 88, 61, 94, 76},
- { 12, 86, 129, 22, 88, 77, 97, 93},
- { 15, 87, 129, 28, 89, 93, 100, 110},
- { 17, 87, 129, 33, 90, 105, 103, 123},
- { 20, 88, 130, 38, 91, 118, 106, 136},
- { 23, 88, 130, 43, 91, 128, 108, 146},
- { 26, 89, 131, 48, 92, 139, 111, 156},
- { 28, 89, 131, 53, 93, 147, 114, 163},
- { 31, 90, 131, 58, 94, 156, 117, 171},
- { 34, 90, 131, 62, 94, 163, 119, 177},
- { 37, 90, 132, 66, 95, 171, 122, 184},
- { 39, 90, 132, 70, 96, 177, 124, 189},
- { 42, 91, 132, 75, 97, 183, 127, 194},
- { 44, 91, 132, 79, 97, 188, 129, 198},
- { 47, 92, 133, 83, 98, 193, 132, 202},
- { 49, 92, 133, 86, 99, 197, 134, 205},
- { 52, 93, 133, 90, 100, 201, 137, 208},
- { 54, 93, 133, 94, 100, 204, 139, 211},
- { 57, 94, 134, 98, 101, 208, 142, 214},
- { 59, 94, 134, 101, 102, 211, 144, 216},
- { 62, 94, 135, 105, 103, 214, 146, 218},
- { 64, 94, 135, 108, 103, 216, 148, 220},
- { 66, 95, 135, 111, 104, 219, 151, 222},
- { 68, 95, 135, 114, 105, 221, 153, 223},
- { 71, 96, 136, 117, 106, 224, 155, 225},
- { 73, 96, 136, 120, 106, 225, 157, 226},
- { 76, 97, 136, 123, 107, 227, 159, 228},
- { 78, 97, 136, 126, 108, 229, 160, 229},
- { 80, 98, 137, 129, 109, 231, 162, 231},
- { 82, 98, 137, 131, 109, 232, 164, 232},
- { 84, 98, 138, 134, 110, 234, 166, 233},
- { 86, 98, 138, 137, 111, 235, 168, 234},
- { 89, 99, 138, 140, 112, 236, 170, 235},
- { 91, 99, 138, 142, 112, 237, 171, 235},
- { 93, 100, 139, 145, 113, 238, 173, 236},
- { 95, 100, 139, 147, 114, 239, 174, 237},
- { 97, 101, 140, 149, 115, 240, 176, 238},
- { 99, 101, 140, 151, 115, 241, 177, 238},
- {101, 102, 140, 154, 116, 242, 179, 239},
- {103, 102, 140, 156, 117, 242, 180, 239},
- {105, 103, 141, 158, 118, 243, 182, 240},
- {107, 103, 141, 160, 118, 243, 183, 240},
- {109, 104, 141, 162, 119, 244, 185, 241},
- {111, 104, 141, 164, 119, 244, 186, 241},
- {113, 104, 142, 166, 120, 245, 187, 242},
- {114, 104, 142, 168, 121, 245, 188, 242},
- {116, 105, 143, 170, 122, 246, 190, 243},
- {118, 105, 143, 171, 122, 246, 191, 243},
- {120, 106, 143, 173, 123, 247, 192, 244},
- {121, 106, 143, 175, 124, 247, 193, 244},
- {123, 107, 144, 177, 125, 248, 195, 244},
- {125, 107, 144, 178, 125, 248, 196, 244},
- {127, 108, 145, 180, 126, 249, 197, 245},
- {128, 108, 145, 181, 127, 249, 198, 245},
- {130, 109, 145, 183, 128, 249, 199, 245},
- {132, 109, 145, 184, 128, 249, 200, 245},
- {134, 110, 146, 186, 129, 250, 201, 246},
- {135, 110, 146, 187, 130, 250, 202, 246},
- {137, 111, 147, 189, 131, 251, 203, 246},
- {138, 111, 147, 190, 131, 251, 204, 246},
- {140, 112, 147, 192, 132, 251, 205, 247},
- {141, 112, 147, 193, 132, 251, 206, 247},
- {143, 113, 148, 194, 133, 251, 207, 247},
- {144, 113, 148, 195, 134, 251, 207, 247},
- {146, 114, 149, 197, 135, 252, 208, 248},
- {147, 114, 149, 198, 135, 252, 209, 248},
- {149, 115, 149, 199, 136, 252, 210, 248},
- {150, 115, 149, 200, 137, 252, 210, 248},
- {152, 115, 150, 201, 138, 252, 211, 248},
- {153, 115, 150, 202, 138, 252, 212, 248},
- {155, 116, 151, 204, 139, 253, 213, 249},
- {156, 116, 151, 205, 139, 253, 213, 249},
- {158, 117, 151, 206, 140, 253, 214, 249},
- {159, 117, 151, 207, 141, 253, 215, 249},
- {161, 118, 152, 208, 142, 253, 216, 249},
- {162, 118, 152, 209, 142, 253, 216, 249},
- {163, 119, 153, 210, 143, 253, 217, 249},
- {164, 119, 153, 211, 143, 253, 217, 249},
- {166, 120, 153, 212, 144, 254, 218, 250},
- {167, 120, 153, 212, 145, 254, 219, 250},
- {168, 121, 154, 213, 146, 254, 220, 250},
- {169, 121, 154, 214, 146, 254, 220, 250},
- {171, 122, 155, 215, 147, 254, 221, 250},
- {172, 122, 155, 216, 147, 254, 221, 250},
- {173, 123, 155, 217, 148, 254, 222, 250},
- {174, 123, 155, 217, 149, 254, 222, 250},
- {176, 124, 156, 218, 150, 254, 223, 250},
- {177, 124, 156, 219, 150, 254, 223, 250},
- {178, 125, 157, 220, 151, 254, 224, 251},
- {179, 125, 157, 220, 151, 254, 224, 251},
- {180, 126, 157, 221, 152, 254, 225, 251},
- {181, 126, 157, 221, 152, 254, 225, 251},
- {183, 127, 158, 222, 153, 254, 226, 251},
- {184, 127, 158, 223, 154, 254, 226, 251},
- {185, 128, 159, 224, 155, 255, 227, 251},
- {186, 128, 159, 224, 155, 255, 227, 251},
- {187, 129, 160, 225, 156, 255, 228, 251},
- {188, 130, 160, 225, 156, 255, 228, 251},
- {189, 131, 160, 226, 157, 255, 228, 251},
- {190, 131, 160, 226, 158, 255, 228, 251},
- {191, 132, 161, 227, 159, 255, 229, 251},
- {192, 132, 161, 227, 159, 255, 229, 251},
- {193, 133, 162, 228, 160, 255, 230, 252},
- {194, 133, 162, 229, 160, 255, 230, 252},
- {195, 134, 163, 230, 161, 255, 231, 252},
- {196, 134, 163, 230, 161, 255, 231, 252},
- {197, 135, 163, 231, 162, 255, 231, 252},
- {198, 135, 163, 231, 162, 255, 231, 252},
- {199, 136, 164, 232, 163, 255, 232, 252},
- {200, 136, 164, 232, 164, 255, 232, 252},
- {201, 137, 165, 233, 165, 255, 233, 252},
- {201, 137, 165, 233, 165, 255, 233, 252},
- {202, 138, 166, 233, 166, 255, 233, 252},
- {203, 138, 166, 233, 166, 255, 233, 252},
- {204, 139, 166, 234, 167, 255, 234, 252},
- {205, 139, 166, 234, 167, 255, 234, 252},
- {206, 140, 167, 235, 168, 255, 235, 252},
- {206, 140, 167, 235, 168, 255, 235, 252},
- {207, 141, 168, 236, 169, 255, 235, 252},
- {208, 141, 168, 236, 170, 255, 235, 252},
- {209, 142, 169, 237, 171, 255, 236, 252},
- {209, 143, 169, 237, 171, 255, 236, 252},
- {210, 144, 169, 237, 172, 255, 236, 252},
- {211, 144, 169, 237, 172, 255, 236, 252},
- {212, 145, 170, 238, 173, 255, 237, 252},
- {213, 145, 170, 238, 173, 255, 237, 252},
- {214, 146, 171, 239, 174, 255, 237, 253},
- {214, 146, 171, 239, 174, 255, 237, 253},
- {215, 147, 172, 240, 175, 255, 238, 253},
- {215, 147, 172, 240, 175, 255, 238, 253},
- {216, 148, 173, 240, 176, 255, 238, 253},
- {217, 148, 173, 240, 176, 255, 238, 253},
- {218, 149, 173, 241, 177, 255, 239, 253},
- {218, 149, 173, 241, 178, 255, 239, 253},
- {219, 150, 174, 241, 179, 255, 239, 253},
- {219, 151, 174, 241, 179, 255, 239, 253},
- {220, 152, 175, 242, 180, 255, 240, 253},
- {221, 152, 175, 242, 180, 255, 240, 253},
- {222, 153, 176, 242, 181, 255, 240, 253},
- {222, 153, 176, 242, 181, 255, 240, 253},
- {223, 154, 177, 243, 182, 255, 240, 253},
- {223, 154, 177, 243, 182, 255, 240, 253},
- {224, 155, 178, 244, 183, 255, 241, 253},
- {224, 155, 178, 244, 183, 255, 241, 253},
- {225, 156, 178, 244, 184, 255, 241, 253},
- {225, 157, 178, 244, 184, 255, 241, 253},
- {226, 158, 179, 244, 185, 255, 242, 253},
- {227, 158, 179, 244, 185, 255, 242, 253},
- {228, 159, 180, 245, 186, 255, 242, 253},
- {228, 159, 180, 245, 186, 255, 242, 253},
- {229, 160, 181, 245, 187, 255, 242, 253},
- {229, 160, 181, 245, 187, 255, 242, 253},
- {230, 161, 182, 246, 188, 255, 243, 253},
- {230, 162, 182, 246, 188, 255, 243, 253},
- {231, 163, 183, 246, 189, 255, 243, 253},
- {231, 163, 183, 246, 189, 255, 243, 253},
- {232, 164, 184, 247, 190, 255, 243, 253},
- {232, 164, 184, 247, 190, 255, 243, 253},
- {233, 165, 185, 247, 191, 255, 244, 253},
- {233, 165, 185, 247, 191, 255, 244, 253},
- {234, 166, 185, 247, 192, 255, 244, 253},
- {234, 167, 185, 247, 192, 255, 244, 253},
- {235, 168, 186, 248, 193, 255, 244, 253},
- {235, 168, 186, 248, 193, 255, 244, 253},
- {236, 169, 187, 248, 194, 255, 244, 253},
- {236, 169, 187, 248, 194, 255, 244, 253},
- {236, 170, 188, 248, 195, 255, 245, 253},
- {236, 170, 188, 248, 195, 255, 245, 253},
- {237, 171, 189, 249, 196, 255, 245, 254},
- {237, 172, 189, 249, 196, 255, 245, 254},
- {238, 173, 190, 249, 197, 255, 245, 254},
- {238, 173, 190, 249, 197, 255, 245, 254},
- {239, 174, 191, 249, 198, 255, 245, 254},
- {239, 174, 191, 249, 198, 255, 245, 254},
- {240, 175, 192, 249, 199, 255, 246, 254},
- {240, 176, 192, 249, 199, 255, 246, 254},
- {240, 177, 193, 250, 200, 255, 246, 254},
- {240, 177, 193, 250, 200, 255, 246, 254},
- {241, 178, 194, 250, 201, 255, 246, 254},
- {241, 178, 194, 250, 201, 255, 246, 254},
- {242, 179, 195, 250, 202, 255, 246, 254},
- {242, 180, 195, 250, 202, 255, 246, 254},
- {242, 181, 196, 250, 203, 255, 247, 254},
- {242, 181, 196, 250, 203, 255, 247, 254},
- {243, 182, 197, 251, 204, 255, 247, 254},
- {243, 183, 197, 251, 204, 255, 247, 254},
- {244, 184, 198, 251, 205, 255, 247, 254},
- {244, 184, 198, 251, 205, 255, 247, 254},
- {244, 185, 199, 251, 206, 255, 247, 254},
- {244, 185, 199, 251, 206, 255, 247, 254},
- {245, 186, 200, 251, 207, 255, 247, 254},
- {245, 187, 200, 251, 207, 255, 247, 254},
- {246, 188, 201, 252, 207, 255, 248, 254},
- {246, 188, 201, 252, 207, 255, 248, 254},
- {246, 189, 202, 252, 208, 255, 248, 254},
- {246, 190, 202, 252, 208, 255, 248, 254},
- {247, 191, 203, 252, 209, 255, 248, 254},
- {247, 191, 203, 252, 209, 255, 248, 254},
- {247, 192, 204, 252, 210, 255, 248, 254},
- {247, 193, 204, 252, 210, 255, 248, 254},
- {248, 194, 205, 252, 211, 255, 248, 254},
- {248, 194, 205, 252, 211, 255, 248, 254},
- {248, 195, 206, 252, 212, 255, 249, 254},
- {248, 196, 206, 252, 212, 255, 249, 254},
- {249, 197, 207, 253, 213, 255, 249, 254},
- {249, 197, 207, 253, 213, 255, 249, 254},
- {249, 198, 208, 253, 214, 255, 249, 254},
- {249, 199, 209, 253, 214, 255, 249, 254},
- {250, 200, 210, 253, 215, 255, 249, 254},
- {250, 200, 210, 253, 215, 255, 249, 254},
- {250, 201, 211, 253, 215, 255, 249, 254},
- {250, 202, 211, 253, 215, 255, 249, 254},
- {250, 203, 212, 253, 216, 255, 249, 254},
- {250, 203, 212, 253, 216, 255, 249, 254},
- {251, 204, 213, 253, 217, 255, 250, 254},
- {251, 205, 213, 253, 217, 255, 250, 254},
- {251, 206, 214, 254, 218, 255, 250, 254},
- {251, 206, 215, 254, 218, 255, 250, 254},
- {252, 207, 216, 254, 219, 255, 250, 254},
- {252, 208, 216, 254, 219, 255, 250, 254},
- {252, 209, 217, 254, 220, 255, 250, 254},
- {252, 210, 217, 254, 220, 255, 250, 254},
- {252, 211, 218, 254, 221, 255, 250, 254},
- {252, 212, 218, 254, 221, 255, 250, 254},
- {253, 213, 219, 254, 222, 255, 250, 254},
- {253, 213, 220, 254, 222, 255, 250, 254},
- {253, 214, 221, 254, 223, 255, 250, 254},
- {253, 215, 221, 254, 223, 255, 250, 254},
- {253, 216, 222, 254, 224, 255, 251, 254},
- {253, 217, 223, 254, 224, 255, 251, 254},
- {253, 218, 224, 254, 225, 255, 251, 254},
- {253, 219, 224, 254, 225, 255, 251, 254},
- {254, 220, 225, 254, 225, 255, 251, 254},
- {254, 221, 226, 254, 225, 255, 251, 254},
- {254, 222, 227, 255, 226, 255, 251, 254},
- {254, 223, 227, 255, 226, 255, 251, 254},
- {254, 224, 228, 255, 227, 255, 251, 254},
- {254, 225, 229, 255, 227, 255, 251, 254},
- {254, 226, 230, 255, 228, 255, 251, 254},
- {254, 227, 230, 255, 229, 255, 251, 254},
- {255, 228, 231, 255, 230, 255, 251, 254},
- {255, 229, 232, 255, 230, 255, 251, 254},
- {255, 230, 233, 255, 231, 255, 252, 254},
- {255, 231, 234, 255, 231, 255, 252, 254},
- {255, 232, 235, 255, 232, 255, 252, 254},
- {255, 233, 236, 255, 232, 255, 252, 254},
- {255, 235, 237, 255, 233, 255, 252, 254},
- {255, 236, 238, 255, 234, 255, 252, 254},
- {255, 238, 240, 255, 235, 255, 252, 255},
- {255, 239, 241, 255, 235, 255, 252, 254},
- {255, 241, 243, 255, 236, 255, 252, 254},
- {255, 243, 245, 255, 237, 255, 252, 254},
- {255, 246, 247, 255, 239, 255, 253, 255},
- {255, 246, 247, 255, 239, 255, 253, 255},
-};
-
-static const vp10_coeff_probs_model default_coef_probs_4x4[PLANE_TYPES] = {
- { // Y plane
- { // Intra
- { // Band 0
- { 195, 29, 183 }, { 84, 49, 136 }, { 8, 42, 71 }
- }, { // Band 1
- { 31, 107, 169 }, { 35, 99, 159 }, { 17, 82, 140 },
- { 8, 66, 114 }, { 2, 44, 76 }, { 1, 19, 32 }
- }, { // Band 2
- { 40, 132, 201 }, { 29, 114, 187 }, { 13, 91, 157 },
- { 7, 75, 127 }, { 3, 58, 95 }, { 1, 28, 47 }
- }, { // Band 3
- { 69, 142, 221 }, { 42, 122, 201 }, { 15, 91, 159 },
- { 6, 67, 121 }, { 1, 42, 77 }, { 1, 17, 31 }
- }, { // Band 4
- { 102, 148, 228 }, { 67, 117, 204 }, { 17, 82, 154 },
- { 6, 59, 114 }, { 2, 39, 75 }, { 1, 15, 29 }
- }, { // Band 5
- { 156, 57, 233 }, { 119, 57, 212 }, { 58, 48, 163 },
- { 29, 40, 124 }, { 12, 30, 81 }, { 3, 12, 31 }
- }
- }, { // Inter
- { // Band 0
- { 191, 107, 226 }, { 124, 117, 204 }, { 25, 99, 155 }
- }, { // Band 1
- { 29, 148, 210 }, { 37, 126, 194 }, { 8, 93, 157 },
- { 2, 68, 118 }, { 1, 39, 69 }, { 1, 17, 33 }
- }, { // Band 2
- { 41, 151, 213 }, { 27, 123, 193 }, { 3, 82, 144 },
- { 1, 58, 105 }, { 1, 32, 60 }, { 1, 13, 26 }
- }, { // Band 3
- { 59, 159, 220 }, { 23, 126, 198 }, { 4, 88, 151 },
- { 1, 66, 114 }, { 1, 38, 71 }, { 1, 18, 34 }
- }, { // Band 4
- { 114, 136, 232 }, { 51, 114, 207 }, { 11, 83, 155 },
- { 3, 56, 105 }, { 1, 33, 65 }, { 1, 17, 34 }
- }, { // Band 5
- { 149, 65, 234 }, { 121, 57, 215 }, { 61, 49, 166 },
- { 28, 36, 114 }, { 12, 25, 76 }, { 3, 16, 42 }
- }
- }
- }, { // UV plane
- { // Intra
- { // Band 0
- { 214, 49, 220 }, { 132, 63, 188 }, { 42, 65, 137 }
- }, { // Band 1
- { 85, 137, 221 }, { 104, 131, 216 }, { 49, 111, 192 },
- { 21, 87, 155 }, { 2, 49, 87 }, { 1, 16, 28 }
- }, { // Band 2
- { 89, 163, 230 }, { 90, 137, 220 }, { 29, 100, 183 },
- { 10, 70, 135 }, { 2, 42, 81 }, { 1, 17, 33 }
- }, { // Band 3
- { 108, 167, 237 }, { 55, 133, 222 }, { 15, 97, 179 },
- { 4, 72, 135 }, { 1, 45, 85 }, { 1, 19, 38 }
- }, { // Band 4
- { 124, 146, 240 }, { 66, 124, 224 }, { 17, 88, 175 },
- { 4, 58, 122 }, { 1, 36, 75 }, { 1, 18, 37 }
- }, { // Band 5
- { 141, 79, 241 }, { 126, 70, 227 }, { 66, 58, 182 },
- { 30, 44, 136 }, { 12, 34, 96 }, { 2, 20, 47 }
- }
- }, { // Inter
- { // Band 0
- { 229, 99, 249 }, { 143, 111, 235 }, { 46, 109, 192 }
- }, { // Band 1
- { 82, 158, 236 }, { 94, 146, 224 }, { 25, 117, 191 },
- { 9, 87, 149 }, { 3, 56, 99 }, { 1, 33, 57 }
- }, { // Band 2
- { 83, 167, 237 }, { 68, 145, 222 }, { 10, 103, 177 },
- { 2, 72, 131 }, { 1, 41, 79 }, { 1, 20, 39 }
- }, { // Band 3
- { 99, 167, 239 }, { 47, 141, 224 }, { 10, 104, 178 },
- { 2, 73, 133 }, { 1, 44, 85 }, { 1, 22, 47 }
- }, { // Band 4
- { 127, 145, 243 }, { 71, 129, 228 }, { 17, 93, 177 },
- { 3, 61, 124 }, { 1, 41, 84 }, { 1, 21, 52 }
- }, { // Band 5
- { 157, 78, 244 }, { 140, 72, 231 }, { 69, 58, 184 },
- { 31, 44, 137 }, { 14, 38, 105 }, { 8, 23, 61 }
- }
- }
- }
-};
-
-static const vp10_coeff_probs_model default_coef_probs_8x8[PLANE_TYPES] = {
- { // Y plane
- { // Intra
- { // Band 0
- { 125, 34, 187 }, { 52, 41, 133 }, { 6, 31, 56 }
- }, { // Band 1
- { 37, 109, 153 }, { 51, 102, 147 }, { 23, 87, 128 },
- { 8, 67, 101 }, { 1, 41, 63 }, { 1, 19, 29 }
- }, { // Band 2
- { 31, 154, 185 }, { 17, 127, 175 }, { 6, 96, 145 },
- { 2, 73, 114 }, { 1, 51, 82 }, { 1, 28, 45 }
- }, { // Band 3
- { 23, 163, 200 }, { 10, 131, 185 }, { 2, 93, 148 },
- { 1, 67, 111 }, { 1, 41, 69 }, { 1, 14, 24 }
- }, { // Band 4
- { 29, 176, 217 }, { 12, 145, 201 }, { 3, 101, 156 },
- { 1, 69, 111 }, { 1, 39, 63 }, { 1, 14, 23 }
- }, { // Band 5
- { 57, 192, 233 }, { 25, 154, 215 }, { 6, 109, 167 },
- { 3, 78, 118 }, { 1, 48, 69 }, { 1, 21, 29 }
- }
- }, { // Inter
- { // Band 0
- { 202, 105, 245 }, { 108, 106, 216 }, { 18, 90, 144 }
- }, { // Band 1
- { 33, 172, 219 }, { 64, 149, 206 }, { 14, 117, 177 },
- { 5, 90, 141 }, { 2, 61, 95 }, { 1, 37, 57 }
- }, { // Band 2
- { 33, 179, 220 }, { 11, 140, 198 }, { 1, 89, 148 },
- { 1, 60, 104 }, { 1, 33, 57 }, { 1, 12, 21 }
- }, { // Band 3
- { 30, 181, 221 }, { 8, 141, 198 }, { 1, 87, 145 },
- { 1, 58, 100 }, { 1, 31, 55 }, { 1, 12, 20 }
- }, { // Band 4
- { 32, 186, 224 }, { 7, 142, 198 }, { 1, 86, 143 },
- { 1, 58, 100 }, { 1, 31, 55 }, { 1, 12, 22 }
- }, { // Band 5
- { 57, 192, 227 }, { 20, 143, 204 }, { 3, 96, 154 },
- { 1, 68, 112 }, { 1, 42, 69 }, { 1, 19, 32 }
- }
- }
- }, { // UV plane
- { // Intra
- { // Band 0
- { 212, 35, 215 }, { 113, 47, 169 }, { 29, 48, 105 }
- }, { // Band 1
- { 74, 129, 203 }, { 106, 120, 203 }, { 49, 107, 178 },
- { 19, 84, 144 }, { 4, 50, 84 }, { 1, 15, 25 }
- }, { // Band 2
- { 71, 172, 217 }, { 44, 141, 209 }, { 15, 102, 173 },
- { 6, 76, 133 }, { 2, 51, 89 }, { 1, 24, 42 }
- }, { // Band 3
- { 64, 185, 231 }, { 31, 148, 216 }, { 8, 103, 175 },
- { 3, 74, 131 }, { 1, 46, 81 }, { 1, 18, 30 }
- }, { // Band 4
- { 65, 196, 235 }, { 25, 157, 221 }, { 5, 105, 174 },
- { 1, 67, 120 }, { 1, 38, 69 }, { 1, 15, 30 }
- }, { // Band 5
- { 65, 204, 238 }, { 30, 156, 224 }, { 7, 107, 177 },
- { 2, 70, 124 }, { 1, 42, 73 }, { 1, 18, 34 }
- }
- }, { // Inter
- { // Band 0
- { 225, 86, 251 }, { 144, 104, 235 }, { 42, 99, 181 }
- }, { // Band 1
- { 85, 175, 239 }, { 112, 165, 229 }, { 29, 136, 200 },
- { 12, 103, 162 }, { 6, 77, 123 }, { 2, 53, 84 }
- }, { // Band 2
- { 75, 183, 239 }, { 30, 155, 221 }, { 3, 106, 171 },
- { 1, 74, 128 }, { 1, 44, 76 }, { 1, 17, 28 }
- }, { // Band 3
- { 73, 185, 240 }, { 27, 159, 222 }, { 2, 107, 172 },
- { 1, 75, 127 }, { 1, 42, 73 }, { 1, 17, 29 }
- }, { // Band 4
- { 62, 190, 238 }, { 21, 159, 222 }, { 2, 107, 172 },
- { 1, 72, 122 }, { 1, 40, 71 }, { 1, 18, 32 }
- }, { // Band 5
- { 61, 199, 240 }, { 27, 161, 226 }, { 4, 113, 180 },
- { 1, 76, 129 }, { 1, 46, 80 }, { 1, 23, 41 }
- }
- }
- }
-};
-
-static const vp10_coeff_probs_model default_coef_probs_16x16[PLANE_TYPES] = {
- { // Y plane
- { // Intra
- { // Band 0
- { 7, 27, 153 }, { 5, 30, 95 }, { 1, 16, 30 }
- }, { // Band 1
- { 50, 75, 127 }, { 57, 75, 124 }, { 27, 67, 108 },
- { 10, 54, 86 }, { 1, 33, 52 }, { 1, 12, 18 }
- }, { // Band 2
- { 43, 125, 151 }, { 26, 108, 148 }, { 7, 83, 122 },
- { 2, 59, 89 }, { 1, 38, 60 }, { 1, 17, 27 }
- }, { // Band 3
- { 23, 144, 163 }, { 13, 112, 154 }, { 2, 75, 117 },
- { 1, 50, 81 }, { 1, 31, 51 }, { 1, 14, 23 }
- }, { // Band 4
- { 18, 162, 185 }, { 6, 123, 171 }, { 1, 78, 125 },
- { 1, 51, 86 }, { 1, 31, 54 }, { 1, 14, 23 }
- }, { // Band 5
- { 15, 199, 227 }, { 3, 150, 204 }, { 1, 91, 146 },
- { 1, 55, 95 }, { 1, 30, 53 }, { 1, 11, 20 }
- }
- }, { // Inter
- { // Band 0
- { 19, 55, 240 }, { 19, 59, 196 }, { 3, 52, 105 }
- }, { // Band 1
- { 41, 166, 207 }, { 104, 153, 199 }, { 31, 123, 181 },
- { 14, 101, 152 }, { 5, 72, 106 }, { 1, 36, 52 }
- }, { // Band 2
- { 35, 176, 211 }, { 12, 131, 190 }, { 2, 88, 144 },
- { 1, 60, 101 }, { 1, 36, 60 }, { 1, 16, 28 }
- }, { // Band 3
- { 28, 183, 213 }, { 8, 134, 191 }, { 1, 86, 142 },
- { 1, 56, 96 }, { 1, 30, 53 }, { 1, 12, 20 }
- }, { // Band 4
- { 20, 190, 215 }, { 4, 135, 192 }, { 1, 84, 139 },
- { 1, 53, 91 }, { 1, 28, 49 }, { 1, 11, 20 }
- }, { // Band 5
- { 13, 196, 216 }, { 2, 137, 192 }, { 1, 86, 143 },
- { 1, 57, 99 }, { 1, 32, 56 }, { 1, 13, 24 }
- }
- }
- }, { // UV plane
- { // Intra
- { // Band 0
- { 211, 29, 217 }, { 96, 47, 156 }, { 22, 43, 87 }
- }, { // Band 1
- { 78, 120, 193 }, { 111, 116, 186 }, { 46, 102, 164 },
- { 15, 80, 128 }, { 2, 49, 76 }, { 1, 18, 28 }
- }, { // Band 2
- { 71, 161, 203 }, { 42, 132, 192 }, { 10, 98, 150 },
- { 3, 69, 109 }, { 1, 44, 70 }, { 1, 18, 29 }
- }, { // Band 3
- { 57, 186, 211 }, { 30, 140, 196 }, { 4, 93, 146 },
- { 1, 62, 102 }, { 1, 38, 65 }, { 1, 16, 27 }
- }, { // Band 4
- { 47, 199, 217 }, { 14, 145, 196 }, { 1, 88, 142 },
- { 1, 57, 98 }, { 1, 36, 62 }, { 1, 15, 26 }
- }, { // Band 5
- { 26, 219, 229 }, { 5, 155, 207 }, { 1, 94, 151 },
- { 1, 60, 104 }, { 1, 36, 62 }, { 1, 16, 28 }
- }
- }, { // Inter
- { // Band 0
- { 233, 29, 248 }, { 146, 47, 220 }, { 43, 52, 140 }
- }, { // Band 1
- { 100, 163, 232 }, { 179, 161, 222 }, { 63, 142, 204 },
- { 37, 113, 174 }, { 26, 89, 137 }, { 18, 68, 97 }
- }, { // Band 2
- { 85, 181, 230 }, { 32, 146, 209 }, { 7, 100, 164 },
- { 3, 71, 121 }, { 1, 45, 77 }, { 1, 18, 30 }
- }, { // Band 3
- { 65, 187, 230 }, { 20, 148, 207 }, { 2, 97, 159 },
- { 1, 68, 116 }, { 1, 40, 70 }, { 1, 14, 29 }
- }, { // Band 4
- { 40, 194, 227 }, { 8, 147, 204 }, { 1, 94, 155 },
- { 1, 65, 112 }, { 1, 39, 66 }, { 1, 14, 26 }
- }, { // Band 5
- { 16, 208, 228 }, { 3, 151, 207 }, { 1, 98, 160 },
- { 1, 67, 117 }, { 1, 41, 74 }, { 1, 17, 31 }
- }
- }
- }
-};
-
-static const vp10_coeff_probs_model default_coef_probs_32x32[PLANE_TYPES] = {
- { // Y plane
- { // Intra
- { // Band 0
- { 17, 38, 140 }, { 7, 34, 80 }, { 1, 17, 29 }
- }, { // Band 1
- { 37, 75, 128 }, { 41, 76, 128 }, { 26, 66, 116 },
- { 12, 52, 94 }, { 2, 32, 55 }, { 1, 10, 16 }
- }, { // Band 2
- { 50, 127, 154 }, { 37, 109, 152 }, { 16, 82, 121 },
- { 5, 59, 85 }, { 1, 35, 54 }, { 1, 13, 20 }
- }, { // Band 3
- { 40, 142, 167 }, { 17, 110, 157 }, { 2, 71, 112 },
- { 1, 44, 72 }, { 1, 27, 45 }, { 1, 11, 17 }
- }, { // Band 4
- { 30, 175, 188 }, { 9, 124, 169 }, { 1, 74, 116 },
- { 1, 48, 78 }, { 1, 30, 49 }, { 1, 11, 18 }
- }, { // Band 5
- { 10, 222, 223 }, { 2, 150, 194 }, { 1, 83, 128 },
- { 1, 48, 79 }, { 1, 27, 45 }, { 1, 11, 17 }
- }
- }, { // Inter
- { // Band 0
- { 36, 41, 235 }, { 29, 36, 193 }, { 10, 27, 111 }
- }, { // Band 1
- { 85, 165, 222 }, { 177, 162, 215 }, { 110, 135, 195 },
- { 57, 113, 168 }, { 23, 83, 120 }, { 10, 49, 61 }
- }, { // Band 2
- { 85, 190, 223 }, { 36, 139, 200 }, { 5, 90, 146 },
- { 1, 60, 103 }, { 1, 38, 65 }, { 1, 18, 30 }
- }, { // Band 3
- { 72, 202, 223 }, { 23, 141, 199 }, { 2, 86, 140 },
- { 1, 56, 97 }, { 1, 36, 61 }, { 1, 16, 27 }
- }, { // Band 4
- { 55, 218, 225 }, { 13, 145, 200 }, { 1, 86, 141 },
- { 1, 57, 99 }, { 1, 35, 61 }, { 1, 13, 22 }
- }, { // Band 5
- { 15, 235, 212 }, { 1, 132, 184 }, { 1, 84, 139 },
- { 1, 57, 97 }, { 1, 34, 56 }, { 1, 14, 23 }
- }
- }
- }, { // UV plane
- { // Intra
- { // Band 0
- { 181, 21, 201 }, { 61, 37, 123 }, { 10, 38, 71 }
- }, { // Band 1
- { 47, 106, 172 }, { 95, 104, 173 }, { 42, 93, 159 },
- { 18, 77, 131 }, { 4, 50, 81 }, { 1, 17, 23 }
- }, { // Band 2
- { 62, 147, 199 }, { 44, 130, 189 }, { 28, 102, 154 },
- { 18, 75, 115 }, { 2, 44, 65 }, { 1, 12, 19 }
- }, { // Band 3
- { 55, 153, 210 }, { 24, 130, 194 }, { 3, 93, 146 },
- { 1, 61, 97 }, { 1, 31, 50 }, { 1, 10, 16 }
- }, { // Band 4
- { 49, 186, 223 }, { 17, 148, 204 }, { 1, 96, 142 },
- { 1, 53, 83 }, { 1, 26, 44 }, { 1, 11, 17 }
- }, { // Band 5
- { 13, 217, 212 }, { 2, 136, 180 }, { 1, 78, 124 },
- { 1, 50, 83 }, { 1, 29, 49 }, { 1, 14, 23 }
- }
- }, { // Inter
- { // Band 0
- { 197, 13, 247 }, { 82, 17, 222 }, { 25, 17, 162 }
- }, { // Band 1
- { 126, 186, 247 }, { 234, 191, 243 }, { 176, 177, 234 },
- { 104, 158, 220 }, { 66, 128, 186 }, { 55, 90, 137 }
- }, { // Band 2
- { 111, 197, 242 }, { 46, 158, 219 }, { 9, 104, 171 },
- { 2, 65, 125 }, { 1, 44, 80 }, { 1, 17, 91 }
- }, { // Band 3
- { 104, 208, 245 }, { 39, 168, 224 }, { 3, 109, 162 },
- { 1, 79, 124 }, { 1, 50, 102 }, { 1, 43, 102 }
- }, { // Band 4
- { 84, 220, 246 }, { 31, 177, 231 }, { 2, 115, 180 },
- { 1, 79, 134 }, { 1, 55, 77 }, { 1, 60, 79 }
- }, { // Band 5
- { 43, 243, 240 }, { 8, 180, 217 }, { 1, 115, 166 },
- { 1, 84, 121 }, { 1, 51, 67 }, { 1, 16, 6 }
- }
- }
- }
-};
-
-static void extend_to_full_distribution(vpx_prob *probs, vpx_prob p) {
- memcpy(probs, vp10_pareto8_full[p = 0 ? 0 : p - 1],
- MODEL_NODES * sizeof(vpx_prob));
-}
-
-void vp10_model_to_full_probs(const vpx_prob *model, vpx_prob *full) {
- if (full != model)
- memcpy(full, model, sizeof(vpx_prob) * UNCONSTRAINED_NODES);
- extend_to_full_distribution(&full[UNCONSTRAINED_NODES], model[PIVOT_NODE]);
-}
-
-void vp10_default_coef_probs(VP10_COMMON *cm) {
- vp10_copy(cm->fc->coef_probs[TX_4X4], default_coef_probs_4x4);
- vp10_copy(cm->fc->coef_probs[TX_8X8], default_coef_probs_8x8);
- vp10_copy(cm->fc->coef_probs[TX_16X16], default_coef_probs_16x16);
- vp10_copy(cm->fc->coef_probs[TX_32X32], default_coef_probs_32x32);
-}
-
-#define COEF_COUNT_SAT 24
-#define COEF_MAX_UPDATE_FACTOR 112
-#define COEF_COUNT_SAT_KEY 24
-#define COEF_MAX_UPDATE_FACTOR_KEY 112
-#define COEF_COUNT_SAT_AFTER_KEY 24
-#define COEF_MAX_UPDATE_FACTOR_AFTER_KEY 128
-
-static void adapt_coef_probs(VP10_COMMON *cm, TX_SIZE tx_size,
- unsigned int count_sat,
- unsigned int update_factor) {
- const FRAME_CONTEXT *pre_fc = &cm->frame_contexts[cm->frame_context_idx];
- vp10_coeff_probs_model *const probs = cm->fc->coef_probs[tx_size];
- const vp10_coeff_probs_model *const pre_probs = pre_fc->coef_probs[tx_size];
- vp10_coeff_count_model *counts = cm->counts.coef[tx_size];
- unsigned int (*eob_counts)[REF_TYPES][COEF_BANDS][COEFF_CONTEXTS] =
- cm->counts.eob_branch[tx_size];
- int i, j, k, l, m;
-
- for (i = 0; i < PLANE_TYPES; ++i)
- for (j = 0; j < REF_TYPES; ++j)
- for (k = 0; k < COEF_BANDS; ++k)
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l) {
- const int n0 = counts[i][j][k][l][ZERO_TOKEN];
- const int n1 = counts[i][j][k][l][ONE_TOKEN];
- const int n2 = counts[i][j][k][l][TWO_TOKEN];
- const int neob = counts[i][j][k][l][EOB_MODEL_TOKEN];
- const unsigned int branch_ct[UNCONSTRAINED_NODES][2] = {
- { neob, eob_counts[i][j][k][l] - neob },
- { n0, n1 + n2 },
- { n1, n2 }
- };
- for (m = 0; m < UNCONSTRAINED_NODES; ++m)
- probs[i][j][k][l][m] = merge_probs(pre_probs[i][j][k][l][m],
- branch_ct[m],
- count_sat, update_factor);
- }
-}
-
-void vp10_adapt_coef_probs(VP10_COMMON *cm) {
- TX_SIZE t;
- unsigned int count_sat, update_factor;
-
- if (frame_is_intra_only(cm)) {
- update_factor = COEF_MAX_UPDATE_FACTOR_KEY;
- count_sat = COEF_COUNT_SAT_KEY;
- } else if (cm->last_frame_type == KEY_FRAME) {
- update_factor = COEF_MAX_UPDATE_FACTOR_AFTER_KEY; /* adapt quickly */
- count_sat = COEF_COUNT_SAT_AFTER_KEY;
- } else {
- update_factor = COEF_MAX_UPDATE_FACTOR;
- count_sat = COEF_COUNT_SAT;
- }
- for (t = TX_4X4; t <= TX_32X32; t++)
- adapt_coef_probs(cm, t, count_sat, update_factor);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_ENTROPY_H_
-#define VP10_COMMON_ENTROPY_H_
-
-#include "vpx/vpx_integer.h"
-#include "vpx_dsp/prob.h"
-
-#include "vp10/common/common.h"
-#include "vp10/common/enums.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define DIFF_UPDATE_PROB 252
-
-// Coefficient token alphabet
-#define ZERO_TOKEN 0 // 0 Extra Bits 0+0
-#define ONE_TOKEN 1 // 1 Extra Bits 0+1
-#define TWO_TOKEN 2 // 2 Extra Bits 0+1
-#define THREE_TOKEN 3 // 3 Extra Bits 0+1
-#define FOUR_TOKEN 4 // 4 Extra Bits 0+1
-#define CATEGORY1_TOKEN 5 // 5-6 Extra Bits 1+1
-#define CATEGORY2_TOKEN 6 // 7-10 Extra Bits 2+1
-#define CATEGORY3_TOKEN 7 // 11-18 Extra Bits 3+1
-#define CATEGORY4_TOKEN 8 // 19-34 Extra Bits 4+1
-#define CATEGORY5_TOKEN 9 // 35-66 Extra Bits 5+1
-#define CATEGORY6_TOKEN 10 // 67+ Extra Bits 14+1
-#define EOB_TOKEN 11 // EOB Extra Bits 0+0
-
-#define ENTROPY_TOKENS 12
-
-#define ENTROPY_NODES 11
-
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_pt_energy_class[ENTROPY_TOKENS]);
-
-#define CAT1_MIN_VAL 5
-#define CAT2_MIN_VAL 7
-#define CAT3_MIN_VAL 11
-#define CAT4_MIN_VAL 19
-#define CAT5_MIN_VAL 35
-#define CAT6_MIN_VAL 67
-
-// Extra bit probabilities.
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat1_prob[1]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat2_prob[2]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat3_prob[3]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat4_prob[4]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat5_prob[5]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat6_prob[14]);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat1_prob_high10[1]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat2_prob_high10[2]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat3_prob_high10[3]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat4_prob_high10[4]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat5_prob_high10[5]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat6_prob_high10[16]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat1_prob_high12[1]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat2_prob_high12[2]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat3_prob_high12[3]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat4_prob_high12[4]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat5_prob_high12[5]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_cat6_prob_high12[18]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#define EOB_MODEL_TOKEN 3
-
-typedef struct {
- const vpx_tree_index *tree;
- const vpx_prob *prob;
- int len;
- int base_val;
- const int16_t *cost;
-} vp10_extra_bit;
-
-// indexed by token value
-extern const vp10_extra_bit vp10_extra_bits[ENTROPY_TOKENS];
-#if CONFIG_VP9_HIGHBITDEPTH
-extern const vp10_extra_bit vp10_extra_bits_high10[ENTROPY_TOKENS];
-extern const vp10_extra_bit vp10_extra_bits_high12[ENTROPY_TOKENS];
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#define DCT_MAX_VALUE 16384
-#if CONFIG_VP9_HIGHBITDEPTH
-#define DCT_MAX_VALUE_HIGH10 65536
-#define DCT_MAX_VALUE_HIGH12 262144
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-/* Coefficients are predicted via a 3-dimensional probability table. */
-
-#define REF_TYPES 2 // intra=0, inter=1
-
-/* Middle dimension reflects the coefficient position within the transform. */
-#define COEF_BANDS 6
-
-/* Inside dimension is measure of nearby complexity, that reflects the energy
- of nearby coefficients are nonzero. For the first coefficient (DC, unless
- block type is 0), we look at the (already encoded) blocks above and to the
- left of the current block. The context index is then the number (0,1,or 2)
- of these blocks having nonzero coefficients.
- After decoding a coefficient, the measure is determined by the size of the
- most recently decoded coefficient.
- Note that the intuitive meaning of this measure changes as coefficients
- are decoded, e.g., prior to the first token, a zero means that my neighbors
- are empty while, after the first token, because of the use of end-of-block,
- a zero means we just decoded a zero and hence guarantees that a non-zero
- coefficient will appear later in this block. However, this shift
- in meaning is perfectly OK because our context depends also on the
- coefficient band (and since zigzag positions 0, 1, and 2 are in
- distinct bands). */
-
-#define COEFF_CONTEXTS 6
-#define BAND_COEFF_CONTEXTS(band) ((band) == 0 ? 3 : COEFF_CONTEXTS)
-
-// #define ENTROPY_STATS
-
-typedef unsigned int vp10_coeff_count[REF_TYPES][COEF_BANDS][COEFF_CONTEXTS]
- [ENTROPY_TOKENS];
-typedef unsigned int vp10_coeff_stats[REF_TYPES][COEF_BANDS][COEFF_CONTEXTS]
- [ENTROPY_NODES][2];
-
-#define SUBEXP_PARAM 4 /* Subexponential code parameter */
-#define MODULUS_PARAM 13 /* Modulus parameter */
-
-struct VP10Common;
-void vp10_default_coef_probs(struct VP10Common *cm);
-void vp10_adapt_coef_probs(struct VP10Common *cm);
-
-// This is the index in the scan order beyond which all coefficients for
-// 8x8 transform and above are in the top band.
-// This macro is currently unused but may be used by certain implementations
-#define MAXBAND_INDEX 21
-
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_coefband_trans_8x8plus[1024]);
-DECLARE_ALIGNED(16, extern const uint8_t, vp10_coefband_trans_4x4[16]);
-
-static INLINE const uint8_t *get_band_translate(TX_SIZE tx_size) {
- return tx_size == TX_4X4 ? vp10_coefband_trans_4x4
- : vp10_coefband_trans_8x8plus;
-}
-
-// 128 lists of probabilities are stored for the following ONE node probs:
-// 1, 3, 5, 7, ..., 253, 255
-// In between probabilities are interpolated linearly
-
-#define COEFF_PROB_MODELS 256
-
-#define UNCONSTRAINED_NODES 3
-
-#define PIVOT_NODE 2 // which node is pivot
-
-#define MODEL_NODES (ENTROPY_NODES - UNCONSTRAINED_NODES)
-extern const vpx_tree_index vp10_coef_con_tree[TREE_SIZE(ENTROPY_TOKENS)];
-extern const vpx_prob vp10_pareto8_full[COEFF_PROB_MODELS][MODEL_NODES];
-
-typedef vpx_prob vp10_coeff_probs_model[REF_TYPES][COEF_BANDS]
- [COEFF_CONTEXTS][UNCONSTRAINED_NODES];
-
-typedef unsigned int vp10_coeff_count_model[REF_TYPES][COEF_BANDS]
- [COEFF_CONTEXTS]
- [UNCONSTRAINED_NODES + 1];
-
-void vp10_model_to_full_probs(const vpx_prob *model, vpx_prob *full);
-
-typedef char ENTROPY_CONTEXT;
-
-static INLINE int combine_entropy_contexts(ENTROPY_CONTEXT a,
- ENTROPY_CONTEXT b) {
- return (a != 0) + (b != 0);
-}
-
-static INLINE int get_entropy_context(TX_SIZE tx_size, const ENTROPY_CONTEXT *a,
- const ENTROPY_CONTEXT *l) {
- ENTROPY_CONTEXT above_ec = 0, left_ec = 0;
-
- switch (tx_size) {
- case TX_4X4:
- above_ec = a[0] != 0;
- left_ec = l[0] != 0;
- break;
- case TX_8X8:
- above_ec = !!*(const uint16_t *)a;
- left_ec = !!*(const uint16_t *)l;
- break;
- case TX_16X16:
- above_ec = !!*(const uint32_t *)a;
- left_ec = !!*(const uint32_t *)l;
- break;
- case TX_32X32:
- above_ec = !!*(const uint64_t *)a;
- left_ec = !!*(const uint64_t *)l;
- break;
- default:
- assert(0 && "Invalid transform size.");
- break;
- }
-
- return combine_entropy_contexts(above_ec, left_ec);
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_ENTROPY_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vpx_mem/vpx_mem.h"
-
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/seg_common.h"
-
-const vpx_prob vp10_kf_y_mode_prob[INTRA_MODES][INTRA_MODES][INTRA_MODES - 1] = {
- { // above = dc
- { 137, 30, 42, 148, 151, 207, 70, 52, 91 }, // left = dc
- { 92, 45, 102, 136, 116, 180, 74, 90, 100 }, // left = v
- { 73, 32, 19, 187, 222, 215, 46, 34, 100 }, // left = h
- { 91, 30, 32, 116, 121, 186, 93, 86, 94 }, // left = d45
- { 72, 35, 36, 149, 68, 206, 68, 63, 105 }, // left = d135
- { 73, 31, 28, 138, 57, 124, 55, 122, 151 }, // left = d117
- { 67, 23, 21, 140, 126, 197, 40, 37, 171 }, // left = d153
- { 86, 27, 28, 128, 154, 212, 45, 43, 53 }, // left = d207
- { 74, 32, 27, 107, 86, 160, 63, 134, 102 }, // left = d63
- { 59, 67, 44, 140, 161, 202, 78, 67, 119 } // left = tm
- }, { // above = v
- { 63, 36, 126, 146, 123, 158, 60, 90, 96 }, // left = dc
- { 43, 46, 168, 134, 107, 128, 69, 142, 92 }, // left = v
- { 44, 29, 68, 159, 201, 177, 50, 57, 77 }, // left = h
- { 58, 38, 76, 114, 97, 172, 78, 133, 92 }, // left = d45
- { 46, 41, 76, 140, 63, 184, 69, 112, 57 }, // left = d135
- { 38, 32, 85, 140, 46, 112, 54, 151, 133 }, // left = d117
- { 39, 27, 61, 131, 110, 175, 44, 75, 136 }, // left = d153
- { 52, 30, 74, 113, 130, 175, 51, 64, 58 }, // left = d207
- { 47, 35, 80, 100, 74, 143, 64, 163, 74 }, // left = d63
- { 36, 61, 116, 114, 128, 162, 80, 125, 82 } // left = tm
- }, { // above = h
- { 82, 26, 26, 171, 208, 204, 44, 32, 105 }, // left = dc
- { 55, 44, 68, 166, 179, 192, 57, 57, 108 }, // left = v
- { 42, 26, 11, 199, 241, 228, 23, 15, 85 }, // left = h
- { 68, 42, 19, 131, 160, 199, 55, 52, 83 }, // left = d45
- { 58, 50, 25, 139, 115, 232, 39, 52, 118 }, // left = d135
- { 50, 35, 33, 153, 104, 162, 64, 59, 131 }, // left = d117
- { 44, 24, 16, 150, 177, 202, 33, 19, 156 }, // left = d153
- { 55, 27, 12, 153, 203, 218, 26, 27, 49 }, // left = d207
- { 53, 49, 21, 110, 116, 168, 59, 80, 76 }, // left = d63
- { 38, 72, 19, 168, 203, 212, 50, 50, 107 } // left = tm
- }, { // above = d45
- { 103, 26, 36, 129, 132, 201, 83, 80, 93 }, // left = dc
- { 59, 38, 83, 112, 103, 162, 98, 136, 90 }, // left = v
- { 62, 30, 23, 158, 200, 207, 59, 57, 50 }, // left = h
- { 67, 30, 29, 84, 86, 191, 102, 91, 59 }, // left = d45
- { 60, 32, 33, 112, 71, 220, 64, 89, 104 }, // left = d135
- { 53, 26, 34, 130, 56, 149, 84, 120, 103 }, // left = d117
- { 53, 21, 23, 133, 109, 210, 56, 77, 172 }, // left = d153
- { 77, 19, 29, 112, 142, 228, 55, 66, 36 }, // left = d207
- { 61, 29, 29, 93, 97, 165, 83, 175, 162 }, // left = d63
- { 47, 47, 43, 114, 137, 181, 100, 99, 95 } // left = tm
- }, { // above = d135
- { 69, 23, 29, 128, 83, 199, 46, 44, 101 }, // left = dc
- { 53, 40, 55, 139, 69, 183, 61, 80, 110 }, // left = v
- { 40, 29, 19, 161, 180, 207, 43, 24, 91 }, // left = h
- { 60, 34, 19, 105, 61, 198, 53, 64, 89 }, // left = d45
- { 52, 31, 22, 158, 40, 209, 58, 62, 89 }, // left = d135
- { 44, 31, 29, 147, 46, 158, 56, 102, 198 }, // left = d117
- { 35, 19, 12, 135, 87, 209, 41, 45, 167 }, // left = d153
- { 55, 25, 21, 118, 95, 215, 38, 39, 66 }, // left = d207
- { 51, 38, 25, 113, 58, 164, 70, 93, 97 }, // left = d63
- { 47, 54, 34, 146, 108, 203, 72, 103, 151 } // left = tm
- }, { // above = d117
- { 64, 19, 37, 156, 66, 138, 49, 95, 133 }, // left = dc
- { 46, 27, 80, 150, 55, 124, 55, 121, 135 }, // left = v
- { 36, 23, 27, 165, 149, 166, 54, 64, 118 }, // left = h
- { 53, 21, 36, 131, 63, 163, 60, 109, 81 }, // left = d45
- { 40, 26, 35, 154, 40, 185, 51, 97, 123 }, // left = d135
- { 35, 19, 34, 179, 19, 97, 48, 129, 124 }, // left = d117
- { 36, 20, 26, 136, 62, 164, 33, 77, 154 }, // left = d153
- { 45, 18, 32, 130, 90, 157, 40, 79, 91 }, // left = d207
- { 45, 26, 28, 129, 45, 129, 49, 147, 123 }, // left = d63
- { 38, 44, 51, 136, 74, 162, 57, 97, 121 } // left = tm
- }, { // above = d153
- { 75, 17, 22, 136, 138, 185, 32, 34, 166 }, // left = dc
- { 56, 39, 58, 133, 117, 173, 48, 53, 187 }, // left = v
- { 35, 21, 12, 161, 212, 207, 20, 23, 145 }, // left = h
- { 56, 29, 19, 117, 109, 181, 55, 68, 112 }, // left = d45
- { 47, 29, 17, 153, 64, 220, 59, 51, 114 }, // left = d135
- { 46, 16, 24, 136, 76, 147, 41, 64, 172 }, // left = d117
- { 34, 17, 11, 108, 152, 187, 13, 15, 209 }, // left = d153
- { 51, 24, 14, 115, 133, 209, 32, 26, 104 }, // left = d207
- { 55, 30, 18, 122, 79, 179, 44, 88, 116 }, // left = d63
- { 37, 49, 25, 129, 168, 164, 41, 54, 148 } // left = tm
- }, { // above = d207
- { 82, 22, 32, 127, 143, 213, 39, 41, 70 }, // left = dc
- { 62, 44, 61, 123, 105, 189, 48, 57, 64 }, // left = v
- { 47, 25, 17, 175, 222, 220, 24, 30, 86 }, // left = h
- { 68, 36, 17, 106, 102, 206, 59, 74, 74 }, // left = d45
- { 57, 39, 23, 151, 68, 216, 55, 63, 58 }, // left = d135
- { 49, 30, 35, 141, 70, 168, 82, 40, 115 }, // left = d117
- { 51, 25, 15, 136, 129, 202, 38, 35, 139 }, // left = d153
- { 68, 26, 16, 111, 141, 215, 29, 28, 28 }, // left = d207
- { 59, 39, 19, 114, 75, 180, 77, 104, 42 }, // left = d63
- { 40, 61, 26, 126, 152, 206, 61, 59, 93 } // left = tm
- }, { // above = d63
- { 78, 23, 39, 111, 117, 170, 74, 124, 94 }, // left = dc
- { 48, 34, 86, 101, 92, 146, 78, 179, 134 }, // left = v
- { 47, 22, 24, 138, 187, 178, 68, 69, 59 }, // left = h
- { 56, 25, 33, 105, 112, 187, 95, 177, 129 }, // left = d45
- { 48, 31, 27, 114, 63, 183, 82, 116, 56 }, // left = d135
- { 43, 28, 37, 121, 63, 123, 61, 192, 169 }, // left = d117
- { 42, 17, 24, 109, 97, 177, 56, 76, 122 }, // left = d153
- { 58, 18, 28, 105, 139, 182, 70, 92, 63 }, // left = d207
- { 46, 23, 32, 74, 86, 150, 67, 183, 88 }, // left = d63
- { 36, 38, 48, 92, 122, 165, 88, 137, 91 } // left = tm
- }, { // above = tm
- { 65, 70, 60, 155, 159, 199, 61, 60, 81 }, // left = dc
- { 44, 78, 115, 132, 119, 173, 71, 112, 93 }, // left = v
- { 39, 38, 21, 184, 227, 206, 42, 32, 64 }, // left = h
- { 58, 47, 36, 124, 137, 193, 80, 82, 78 }, // left = d45
- { 49, 50, 35, 144, 95, 205, 63, 78, 59 }, // left = d135
- { 41, 53, 52, 148, 71, 142, 65, 128, 51 }, // left = d117
- { 40, 36, 28, 143, 143, 202, 40, 55, 137 }, // left = d153
- { 52, 34, 29, 129, 183, 227, 42, 35, 43 }, // left = d207
- { 42, 44, 44, 104, 105, 164, 64, 130, 80 }, // left = d63
- { 43, 81, 53, 140, 169, 204, 68, 84, 72 } // left = tm
- }
-};
-
-#if !CONFIG_MISC_FIXES
-const vpx_prob vp10_kf_uv_mode_prob[INTRA_MODES][INTRA_MODES - 1] = {
- { 144, 11, 54, 157, 195, 130, 46, 58, 108 }, // y = dc
- { 118, 15, 123, 148, 131, 101, 44, 93, 131 }, // y = v
- { 113, 12, 23, 188, 226, 142, 26, 32, 125 }, // y = h
- { 120, 11, 50, 123, 163, 135, 64, 77, 103 }, // y = d45
- { 113, 9, 36, 155, 111, 157, 32, 44, 161 }, // y = d135
- { 116, 9, 55, 176, 76, 96, 37, 61, 149 }, // y = d117
- { 115, 9, 28, 141, 161, 167, 21, 25, 193 }, // y = d153
- { 120, 12, 32, 145, 195, 142, 32, 38, 86 }, // y = d207
- { 116, 12, 64, 120, 140, 125, 49, 115, 121 }, // y = d63
- { 102, 19, 66, 162, 182, 122, 35, 59, 128 } // y = tm
-};
-#endif
-
-static const vpx_prob default_if_y_probs[BLOCK_SIZE_GROUPS][INTRA_MODES - 1] = {
- { 65, 32, 18, 144, 162, 194, 41, 51, 98 }, // block_size < 8x8
- { 132, 68, 18, 165, 217, 196, 45, 40, 78 }, // block_size < 16x16
- { 173, 80, 19, 176, 240, 193, 64, 35, 46 }, // block_size < 32x32
- { 221, 135, 38, 194, 248, 121, 96, 85, 29 } // block_size >= 32x32
-};
-
-static const vpx_prob default_uv_probs[INTRA_MODES][INTRA_MODES - 1] = {
- { 120, 7, 76, 176, 208, 126, 28, 54, 103 }, // y = dc
- { 48, 12, 154, 155, 139, 90, 34, 117, 119 }, // y = v
- { 67, 6, 25, 204, 243, 158, 13, 21, 96 }, // y = h
- { 97, 5, 44, 131, 176, 139, 48, 68, 97 }, // y = d45
- { 83, 5, 42, 156, 111, 152, 26, 49, 152 }, // y = d135
- { 80, 5, 58, 178, 74, 83, 33, 62, 145 }, // y = d117
- { 86, 5, 32, 154, 192, 168, 14, 22, 163 }, // y = d153
- { 85, 5, 32, 156, 216, 148, 19, 29, 73 }, // y = d207
- { 77, 7, 64, 116, 132, 122, 37, 126, 120 }, // y = d63
- { 101, 21, 107, 181, 192, 103, 19, 67, 125 } // y = tm
-};
-
-#if !CONFIG_MISC_FIXES
-const vpx_prob vp10_kf_partition_probs[PARTITION_CONTEXTS]
- [PARTITION_TYPES - 1] = {
- // 8x8 -> 4x4
- { 158, 97, 94 }, // a/l both not split
- { 93, 24, 99 }, // a split, l not split
- { 85, 119, 44 }, // l split, a not split
- { 62, 59, 67 }, // a/l both split
- // 16x16 -> 8x8
- { 149, 53, 53 }, // a/l both not split
- { 94, 20, 48 }, // a split, l not split
- { 83, 53, 24 }, // l split, a not split
- { 52, 18, 18 }, // a/l both split
- // 32x32 -> 16x16
- { 150, 40, 39 }, // a/l both not split
- { 78, 12, 26 }, // a split, l not split
- { 67, 33, 11 }, // l split, a not split
- { 24, 7, 5 }, // a/l both split
- // 64x64 -> 32x32
- { 174, 35, 49 }, // a/l both not split
- { 68, 11, 27 }, // a split, l not split
- { 57, 15, 9 }, // l split, a not split
- { 12, 3, 3 }, // a/l both split
-};
-#endif
-
-static const vpx_prob default_partition_probs[PARTITION_CONTEXTS]
- [PARTITION_TYPES - 1] = {
- // 8x8 -> 4x4
- { 199, 122, 141 }, // a/l both not split
- { 147, 63, 159 }, // a split, l not split
- { 148, 133, 118 }, // l split, a not split
- { 121, 104, 114 }, // a/l both split
- // 16x16 -> 8x8
- { 174, 73, 87 }, // a/l both not split
- { 92, 41, 83 }, // a split, l not split
- { 82, 99, 50 }, // l split, a not split
- { 53, 39, 39 }, // a/l both split
- // 32x32 -> 16x16
- { 177, 58, 59 }, // a/l both not split
- { 68, 26, 63 }, // a split, l not split
- { 52, 79, 25 }, // l split, a not split
- { 17, 14, 12 }, // a/l both split
- // 64x64 -> 32x32
- { 222, 34, 30 }, // a/l both not split
- { 72, 16, 44 }, // a split, l not split
- { 58, 32, 12 }, // l split, a not split
- { 10, 7, 6 }, // a/l both split
-};
-
-static const vpx_prob default_inter_mode_probs[INTER_MODE_CONTEXTS]
- [INTER_MODES - 1] = {
- {2, 173, 34}, // 0 = both zero mv
- {7, 145, 85}, // 1 = one zero mv + one a predicted mv
- {7, 166, 63}, // 2 = two predicted mvs
- {7, 94, 66}, // 3 = one predicted/zero and one new mv
- {8, 64, 46}, // 4 = two new mvs
- {17, 81, 31}, // 5 = one intra neighbour + x
- {25, 29, 30}, // 6 = two intra neighbours
-};
-
-/* Array indices are identical to previously-existing INTRAMODECONTEXTNODES. */
-const vpx_tree_index vp10_intra_mode_tree[TREE_SIZE(INTRA_MODES)] = {
- -DC_PRED, 2, /* 0 = DC_NODE */
- -TM_PRED, 4, /* 1 = TM_NODE */
- -V_PRED, 6, /* 2 = V_NODE */
- 8, 12, /* 3 = COM_NODE */
- -H_PRED, 10, /* 4 = H_NODE */
- -D135_PRED, -D117_PRED, /* 5 = D135_NODE */
- -D45_PRED, 14, /* 6 = D45_NODE */
- -D63_PRED, 16, /* 7 = D63_NODE */
- -D153_PRED, -D207_PRED /* 8 = D153_NODE */
-};
-
-const vpx_tree_index vp10_inter_mode_tree[TREE_SIZE(INTER_MODES)] = {
- -INTER_OFFSET(ZEROMV), 2,
- -INTER_OFFSET(NEARESTMV), 4,
- -INTER_OFFSET(NEARMV), -INTER_OFFSET(NEWMV)
-};
-
-const vpx_tree_index vp10_partition_tree[TREE_SIZE(PARTITION_TYPES)] = {
- -PARTITION_NONE, 2,
- -PARTITION_HORZ, 4,
- -PARTITION_VERT, -PARTITION_SPLIT
-};
-
-static const vpx_prob default_intra_inter_p[INTRA_INTER_CONTEXTS] = {
- 9, 102, 187, 225
-};
-
-static const vpx_prob default_comp_inter_p[COMP_INTER_CONTEXTS] = {
- 239, 183, 119, 96, 41
-};
-
-static const vpx_prob default_comp_ref_p[REF_CONTEXTS] = {
- 50, 126, 123, 221, 226
-};
-
-static const vpx_prob default_single_ref_p[REF_CONTEXTS][2] = {
- { 33, 16 },
- { 77, 74 },
- { 142, 142 },
- { 172, 170 },
- { 238, 247 }
-};
-
-static const struct tx_probs default_tx_probs = {
- { { 3, 136, 37 },
- { 5, 52, 13 } },
-
- { { 20, 152 },
- { 15, 101 } },
-
- { { 100 },
- { 66 } }
-};
-
-const vpx_tree_index vp10_palette_size_tree[TREE_SIZE(PALETTE_SIZES)] = {
- -TWO_COLORS, 2,
- -THREE_COLORS, 4,
- -FOUR_COLORS, 6,
- -FIVE_COLORS, 8,
- -SIX_COLORS, 10,
- -SEVEN_COLORS, -EIGHT_COLORS,
-};
-
-// TODO(huisu): tune these probs
-const vpx_prob
-vp10_default_palette_y_size_prob[PALETTE_BLOCK_SIZES][PALETTE_SIZES - 1] = {
- { 96, 89, 100, 64, 77, 130},
- { 22, 15, 44, 16, 34, 82},
- { 30, 19, 57, 18, 38, 86},
- { 94, 36, 104, 23, 43, 92},
- { 116, 76, 107, 46, 65, 105},
- { 112, 82, 94, 40, 70, 112},
- { 147, 124, 123, 58, 69, 103},
- { 180, 113, 136, 49, 45, 114},
- { 107, 70, 87, 49, 154, 156},
- { 98, 105, 142, 63, 64, 152},
-};
-
-const vpx_prob
-vp10_default_palette_uv_size_prob[PALETTE_BLOCK_SIZES][PALETTE_SIZES - 1] = {
- { 160, 196, 228, 213, 175, 230},
- { 87, 148, 208, 141, 166, 163},
- { 72, 151, 204, 139, 155, 161},
- { 78, 135, 171, 104, 120, 173},
- { 59, 92, 131, 78, 92, 142},
- { 75, 118, 149, 84, 90, 128},
- { 89, 87, 92, 66, 66, 128},
- { 67, 53, 54, 55, 66, 93},
- { 120, 130, 83, 171, 75, 214},
- { 72, 55, 66, 68, 79, 107},
-};
-
-const vpx_prob
-vp10_default_palette_y_mode_prob[PALETTE_BLOCK_SIZES][PALETTE_Y_MODE_CONTEXTS]
- = {
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
- { 240, 180, 100, },
-};
-
-
-const vpx_prob default_uv_palette_mode_prob[2] = {
- 253, 229
-};
-
-const vpx_tree_index
-vp10_palette_color_tree[PALETTE_MAX_SIZE - 1][TREE_SIZE(PALETTE_COLORS)] = {
- { // 2 colors
- -PALETTE_COLOR_ONE, -PALETTE_COLOR_TWO,
- },
- { // 3 colors
- -PALETTE_COLOR_ONE, 2,
- -PALETTE_COLOR_TWO, -PALETTE_COLOR_THREE,
- },
- { // 4 colors
- -PALETTE_COLOR_ONE, 2,
- -PALETTE_COLOR_TWO, 4,
- -PALETTE_COLOR_THREE, -PALETTE_COLOR_FOUR,
- },
- { // 5 colors
- -PALETTE_COLOR_ONE, 2,
- -PALETTE_COLOR_TWO, 4,
- -PALETTE_COLOR_THREE, 6,
- -PALETTE_COLOR_FOUR, -PALETTE_COLOR_FIVE,
- },
- { // 6 colors
- -PALETTE_COLOR_ONE, 2,
- -PALETTE_COLOR_TWO, 4,
- -PALETTE_COLOR_THREE, 6,
- -PALETTE_COLOR_FOUR, 8,
- -PALETTE_COLOR_FIVE, -PALETTE_COLOR_SIX,
- },
- { // 7 colors
- -PALETTE_COLOR_ONE, 2,
- -PALETTE_COLOR_TWO, 4,
- -PALETTE_COLOR_THREE, 6,
- -PALETTE_COLOR_FOUR, 8,
- -PALETTE_COLOR_FIVE, 10,
- -PALETTE_COLOR_SIX, -PALETTE_COLOR_SEVEN,
- },
- { // 8 colors
- -PALETTE_COLOR_ONE, 2,
- -PALETTE_COLOR_TWO, 4,
- -PALETTE_COLOR_THREE, 6,
- -PALETTE_COLOR_FOUR, 8,
- -PALETTE_COLOR_FIVE, 10,
- -PALETTE_COLOR_SIX, 12,
- -PALETTE_COLOR_SEVEN, -PALETTE_COLOR_EIGHT,
- },
-};
-
-const vpx_prob vp10_default_palette_y_color_prob
-[PALETTE_MAX_SIZE - 1][PALETTE_COLOR_CONTEXTS][PALETTE_COLORS - 1] = {
- { // 2 colors
- { 230, 255, 128, 128, 128, 128, 128 },
- { 214, 255, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 240, 255, 128, 128, 128, 128, 128 },
- { 73, 255, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 130, 255, 128, 128, 128, 128, 128 },
- { 227, 255, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 188, 255, 128, 128, 128, 128, 128 },
- { 75, 255, 128, 128, 128, 128, 128 },
- { 250, 255, 128, 128, 128, 128, 128 },
- { 223, 255, 128, 128, 128, 128, 128 },
- { 252, 255, 128, 128, 128, 128, 128 },
- }, { // 3 colors
- { 229, 137, 255, 128, 128, 128, 128 },
- { 197, 120, 255, 128, 128, 128, 128 },
- { 107, 195, 255, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 27, 151, 255, 128, 128, 128, 128 },
- { 230, 130, 255, 128, 128, 128, 128 },
- { 37, 230, 255, 128, 128, 128, 128 },
- { 67, 221, 255, 128, 128, 128, 128 },
- { 124, 230, 255, 128, 128, 128, 128 },
- { 195, 109, 255, 128, 128, 128, 128 },
- { 99, 122, 255, 128, 128, 128, 128 },
- { 205, 208, 255, 128, 128, 128, 128 },
- { 40, 235, 255, 128, 128, 128, 128 },
- { 251, 132, 255, 128, 128, 128, 128 },
- { 237, 186, 255, 128, 128, 128, 128 },
- { 253, 112, 255, 128, 128, 128, 128 },
- }, { // 4 colors
- { 195, 87, 128, 255, 128, 128, 128 },
- { 143, 100, 123, 255, 128, 128, 128 },
- { 94, 124, 119, 255, 128, 128, 128 },
- { 77, 91, 130, 255, 128, 128, 128 },
- { 39, 114, 178, 255, 128, 128, 128 },
- { 222, 94, 125, 255, 128, 128, 128 },
- { 44, 203, 132, 255, 128, 128, 128 },
- { 68, 175, 122, 255, 128, 128, 128 },
- { 110, 187, 124, 255, 128, 128, 128 },
- { 152, 91, 128, 255, 128, 128, 128 },
- { 70, 109, 181, 255, 128, 128, 128 },
- { 133, 113, 164, 255, 128, 128, 128 },
- { 47, 205, 133, 255, 128, 128, 128 },
- { 247, 94, 136, 255, 128, 128, 128 },
- { 205, 122, 146, 255, 128, 128, 128 },
- { 251, 100, 141, 255, 128, 128, 128 },
- }, { // 5 colors
- { 195, 65, 84, 125, 255, 128, 128 },
- { 150, 76, 84, 121, 255, 128, 128 },
- { 94, 110, 81, 117, 255, 128, 128 },
- { 79, 85, 91, 139, 255, 128, 128 },
- { 26, 102, 139, 127, 255, 128, 128 },
- { 220, 73, 91, 119, 255, 128, 128 },
- { 38, 203, 86, 127, 255, 128, 128 },
- { 61, 186, 72, 124, 255, 128, 128 },
- { 132, 199, 84, 128, 255, 128, 128 },
- { 172, 52, 62, 120, 255, 128, 128 },
- { 102, 89, 121, 122, 255, 128, 128 },
- { 182, 48, 69, 186, 255, 128, 128 },
- { 36, 206, 87, 126, 255, 128, 128 },
- { 249, 55, 67, 122, 255, 128, 128 },
- { 218, 88, 75, 122, 255, 128, 128 },
- { 253, 64, 80, 119, 255, 128, 128 },
- }, { // 6 colors
- { 182, 54, 64, 75, 118, 255, 128 },
- { 126, 67, 70, 76, 116, 255, 128 },
- { 79, 92, 67, 85, 120, 255, 128 },
- { 63, 61, 81, 118, 132, 255, 128 },
- { 21, 80, 105, 83, 119, 255, 128 },
- { 215, 72, 74, 74, 111, 255, 128 },
- { 50, 176, 63, 79, 120, 255, 128 },
- { 72, 148, 66, 77, 120, 255, 128 },
- { 105, 177, 57, 78, 130, 255, 128 },
- { 150, 66, 66, 80, 127, 255, 128 },
- { 81, 76, 109, 85, 116, 255, 128 },
- { 113, 81, 62, 96, 148, 255, 128 },
- { 54, 179, 69, 82, 121, 255, 128 },
- { 244, 47, 48, 67, 118, 255, 128 },
- { 198, 83, 53, 65, 121, 255, 128 },
- { 250, 42, 51, 69, 110, 255, 128 },
- }, { // 7 colors
- { 182, 45, 54, 62, 74, 113, 255 },
- { 124, 63, 57, 62, 77, 114, 255 },
- { 77, 80, 56, 66, 76, 117, 255 },
- { 63, 57, 69, 98, 85, 131, 255 },
- { 19, 81, 98, 63, 80, 116, 255 },
- { 215, 56, 60, 63, 68, 105, 255 },
- { 50, 174, 50, 60, 79, 118, 255 },
- { 68, 151, 50, 58, 73, 117, 255 },
- { 104, 182, 53, 57, 79, 127, 255 },
- { 156, 50, 51, 63, 77, 111, 255 },
- { 88, 67, 97, 59, 82, 120, 255 },
- { 114, 81, 46, 65, 103, 132, 255 },
- { 55, 166, 57, 66, 82, 120, 255 },
- { 245, 34, 38, 43, 63, 114, 255 },
- { 203, 68, 45, 47, 60, 118, 255 },
- { 250, 35, 37, 47, 66, 110, 255 },
- }, { // 8 colors
- { 180, 43, 46, 50, 56, 69, 109 },
- { 116, 53, 51, 49, 57, 73, 115 },
- { 79, 70, 49, 50, 59, 74, 117 },
- { 60, 54, 57, 70, 62, 83, 129 },
- { 20, 73, 85, 52, 66, 81, 119 },
- { 213, 56, 52, 49, 53, 62, 104 },
- { 48, 161, 41, 45, 56, 77, 116 },
- { 68, 139, 40, 47, 54, 71, 116 },
- { 123, 166, 42, 43, 52, 76, 130 },
- { 153, 44, 44, 47, 54, 79, 129 },
- { 87, 64, 83, 49, 60, 75, 127 },
- { 131, 68, 43, 48, 73, 96, 130 },
- { 55, 152, 45, 51, 64, 77, 113 },
- { 243, 30, 28, 33, 41, 65, 114 },
- { 202, 56, 35, 36, 42, 63, 123 },
- { 249, 31, 29, 32, 45, 68, 111 },
- }
-};
-
-const vpx_prob vp10_default_palette_uv_color_prob
-[PALETTE_MAX_SIZE - 1][PALETTE_COLOR_CONTEXTS][PALETTE_COLORS - 1] = {
- { // 2 colors
- { 228, 255, 128, 128, 128, 128, 128 },
- { 195, 255, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 228, 255, 128, 128, 128, 128, 128 },
- { 71, 255, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 129, 255, 128, 128, 128, 128, 128 },
- { 206, 255, 128, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 136, 255, 128, 128, 128, 128, 128 },
- { 98, 255, 128, 128, 128, 128, 128 },
- { 236, 255, 128, 128, 128, 128, 128 },
- { 222, 255, 128, 128, 128, 128, 128 },
- { 249, 255, 128, 128, 128, 128, 128 },
- }, { // 3 colors
- { 198, 136, 255, 128, 128, 128, 128 },
- { 178, 105, 255, 128, 128, 128, 128 },
- { 100, 206, 255, 128, 128, 128, 128 },
- { 128, 128, 128, 128, 128, 128, 128 },
- { 12, 136, 255, 128, 128, 128, 128 },
- { 219, 134, 255, 128, 128, 128, 128 },
- { 50, 198, 255, 128, 128, 128, 128 },
- { 61, 231, 255, 128, 128, 128, 128 },
- { 110, 209, 255, 128, 128, 128, 128 },
- { 173, 106, 255, 128, 128, 128, 128 },
- { 145, 166, 255, 128, 128, 128, 128 },
- { 156, 175, 255, 128, 128, 128, 128 },
- { 69, 183, 255, 128, 128, 128, 128 },
- { 241, 163, 255, 128, 128, 128, 128 },
- { 224, 160, 255, 128, 128, 128, 128 },
- { 246, 154, 255, 128, 128, 128, 128 },
- }, { // 4 colors
- { 173, 88, 143, 255, 128, 128, 128 },
- { 146, 81, 127, 255, 128, 128, 128 },
- { 84, 134, 102, 255, 128, 128, 128 },
- { 69, 138, 140, 255, 128, 128, 128 },
- { 31, 103, 200, 255, 128, 128, 128 },
- { 217, 101, 139, 255, 128, 128, 128 },
- { 51, 174, 121, 255, 128, 128, 128 },
- { 64, 177, 109, 255, 128, 128, 128 },
- { 96, 179, 145, 255, 128, 128, 128 },
- { 164, 77, 114, 255, 128, 128, 128 },
- { 87, 94, 156, 255, 128, 128, 128 },
- { 105, 57, 173, 255, 128, 128, 128 },
- { 63, 158, 137, 255, 128, 128, 128 },
- { 236, 102, 156, 255, 128, 128, 128 },
- { 197, 115, 153, 255, 128, 128, 128 },
- { 245, 106, 154, 255, 128, 128, 128 },
- }, { // 5 colors
- { 179, 64, 97, 129, 255, 128, 128 },
- { 137, 56, 88, 125, 255, 128, 128 },
- { 82, 107, 61, 118, 255, 128, 128 },
- { 59, 113, 86, 115, 255, 128, 128 },
- { 23, 88, 118, 130, 255, 128, 128 },
- { 213, 66, 90, 125, 255, 128, 128 },
- { 37, 181, 103, 121, 255, 128, 128 },
- { 47, 188, 61, 131, 255, 128, 128 },
- { 104, 185, 103, 144, 255, 128, 128 },
- { 163, 39, 76, 112, 255, 128, 128 },
- { 94, 74, 131, 126, 255, 128, 128 },
- { 142, 42, 103, 163, 255, 128, 128 },
- { 53, 162, 99, 149, 255, 128, 128 },
- { 239, 54, 84, 108, 255, 128, 128 },
- { 203, 84, 110, 147, 255, 128, 128 },
- { 248, 70, 105, 151, 255, 128, 128 },
- }, { // 6 colors
- { 189, 50, 67, 90, 130, 255, 128 },
- { 114, 50, 55, 90, 123, 255, 128 },
- { 66, 76, 54, 82, 128, 255, 128 },
- { 43, 69, 69, 80, 129, 255, 128 },
- { 22, 59, 87, 88, 141, 255, 128 },
- { 203, 49, 68, 87, 122, 255, 128 },
- { 43, 157, 74, 104, 146, 255, 128 },
- { 54, 138, 51, 95, 138, 255, 128 },
- { 82, 171, 58, 102, 146, 255, 128 },
- { 129, 38, 59, 64, 168, 255, 128 },
- { 56, 67, 119, 92, 112, 255, 128 },
- { 96, 62, 53, 132, 82, 255, 128 },
- { 60, 147, 77, 108, 145, 255, 128 },
- { 238, 76, 73, 93, 148, 255, 128 },
- { 189, 86, 73, 103, 157, 255, 128 },
- { 246, 62, 75, 83, 167, 255, 128 },
- }, { // 7 colors
- { 179, 42, 51, 73, 99, 134, 255 },
- { 119, 52, 52, 61, 64, 114, 255 },
- { 53, 77, 35, 65, 71, 131, 255 },
- { 38, 70, 51, 68, 89, 144, 255 },
- { 23, 65, 128, 73, 97, 131, 255 },
- { 210, 47, 52, 63, 81, 143, 255 },
- { 42, 159, 57, 68, 98, 143, 255 },
- { 49, 153, 45, 82, 93, 143, 255 },
- { 81, 169, 52, 72, 113, 151, 255 },
- { 136, 46, 35, 56, 75, 96, 255 },
- { 57, 84, 109, 47, 107, 131, 255 },
- { 128, 78, 57, 36, 128, 85, 255 },
- { 54, 149, 68, 77, 94, 153, 255 },
- { 243, 58, 50, 71, 81, 167, 255 },
- { 189, 92, 64, 70, 121, 173, 255 },
- { 248, 35, 38, 51, 82, 201, 255 },
- }, { // 8 colors
- { 201, 40, 36, 42, 64, 92, 123 },
- { 116, 43, 33, 43, 73, 102, 128 },
- { 46, 77, 37, 69, 62, 78, 150 },
- { 40, 65, 52, 50, 76, 89, 133 },
- { 28, 48, 91, 17, 64, 77, 133 },
- { 218, 43, 43, 37, 56, 72, 163 },
- { 41, 155, 44, 83, 82, 129, 180 },
- { 44, 141, 29, 55, 64, 89, 147 },
- { 92, 166, 48, 45, 59, 126, 179 },
- { 169, 35, 49, 41, 36, 99, 139 },
- { 55, 77, 77, 56, 60, 75, 156 },
- { 155, 81, 51, 64, 57, 182, 255 },
- { 60, 134, 49, 49, 93, 128, 174 },
- { 244, 98, 51, 46, 22, 73, 238 },
- { 189, 70, 40, 87, 93, 79, 201 },
- { 248, 54, 49, 40, 29, 42, 227 },
- }
-};
-
-static const int palette_color_context_lookup[PALETTE_COLOR_CONTEXTS] = {
- // (3, 0, 0, 0), (3, 2, 0, 0), (3, 3, 2, 0), (3, 3, 2, 2),
- 3993, 4235, 4378, 4380,
- // (4, 3, 3, 0), (5, 0, 0, 0), (5, 3, 0, 0), (5, 3, 2, 0),
- 5720, 6655, 7018, 7040,
- // (5, 5, 0, 0), (6, 2, 0, 0), (6, 2, 2, 0), (6, 4, 0, 0),
- 7260, 8228, 8250, 8470,
- // (7, 3, 0, 0), (8, 0, 0, 0), (8, 2, 0, 0), (10, 0, 0, 0)
- 9680, 10648, 10890, 13310
-};
-
-int vp10_get_palette_color_context(const uint8_t *color_map, int cols,
- int r, int c, int n, int *color_order) {
- int i, j, max, max_idx, temp;
- int scores[PALETTE_MAX_SIZE + 10];
- int weights[4] = {3, 2, 3, 2};
- int color_ctx = 0;
- int color_neighbors[4];
-
- assert(n <= PALETTE_MAX_SIZE);
-
- if (c - 1 >= 0)
- color_neighbors[0] = color_map[r * cols + c - 1];
- else
- color_neighbors[0] = -1;
- if (c - 1 >= 0 && r - 1 >= 0)
- color_neighbors[1] = color_map[(r - 1) * cols + c - 1];
- else
- color_neighbors[1] = -1;
- if (r - 1 >= 0)
- color_neighbors[2] = color_map[(r - 1) * cols + c];
- else
- color_neighbors[2] = -1;
- if (r - 1 >= 0 && c + 1 <= cols - 1)
- color_neighbors[3] = color_map[(r - 1) * cols + c + 1];
- else
- color_neighbors[3] = -1;
-
- for (i = 0; i < PALETTE_MAX_SIZE; ++i)
- color_order[i] = i;
- memset(scores, 0, PALETTE_MAX_SIZE * sizeof(scores[0]));
- for (i = 0; i < 4; ++i) {
- if (color_neighbors[i] >= 0)
- scores[color_neighbors[i]] += weights[i];
- }
-
- for (i = 0; i < 4; ++i) {
- max = scores[i];
- max_idx = i;
- j = i + 1;
- while (j < n) {
- if (scores[j] > max) {
- max = scores[j];
- max_idx = j;
- }
- ++j;
- }
-
- if (max_idx != i) {
- temp = scores[i];
- scores[i] = scores[max_idx];
- scores[max_idx] = temp;
-
- temp = color_order[i];
- color_order[i] = color_order[max_idx];
- color_order[max_idx] = temp;
- }
- }
-
- for (i = 0; i < 4; ++i)
- color_ctx = color_ctx * 11 + scores[i];
-
- for (i = 0; i < PALETTE_COLOR_CONTEXTS; ++i)
- if (color_ctx == palette_color_context_lookup[i]) {
- color_ctx = i;
- break;
- }
-
- if (color_ctx >= PALETTE_COLOR_CONTEXTS)
- color_ctx = 0;
-
- return color_ctx;
-}
-
-void vp10_tx_counts_to_branch_counts_32x32(const unsigned int *tx_count_32x32p,
- unsigned int (*ct_32x32p)[2]) {
- ct_32x32p[0][0] = tx_count_32x32p[TX_4X4];
- ct_32x32p[0][1] = tx_count_32x32p[TX_8X8] +
- tx_count_32x32p[TX_16X16] +
- tx_count_32x32p[TX_32X32];
- ct_32x32p[1][0] = tx_count_32x32p[TX_8X8];
- ct_32x32p[1][1] = tx_count_32x32p[TX_16X16] +
- tx_count_32x32p[TX_32X32];
- ct_32x32p[2][0] = tx_count_32x32p[TX_16X16];
- ct_32x32p[2][1] = tx_count_32x32p[TX_32X32];
-}
-
-void vp10_tx_counts_to_branch_counts_16x16(const unsigned int *tx_count_16x16p,
- unsigned int (*ct_16x16p)[2]) {
- ct_16x16p[0][0] = tx_count_16x16p[TX_4X4];
- ct_16x16p[0][1] = tx_count_16x16p[TX_8X8] + tx_count_16x16p[TX_16X16];
- ct_16x16p[1][0] = tx_count_16x16p[TX_8X8];
- ct_16x16p[1][1] = tx_count_16x16p[TX_16X16];
-}
-
-void vp10_tx_counts_to_branch_counts_8x8(const unsigned int *tx_count_8x8p,
- unsigned int (*ct_8x8p)[2]) {
- ct_8x8p[0][0] = tx_count_8x8p[TX_4X4];
- ct_8x8p[0][1] = tx_count_8x8p[TX_8X8];
-}
-
-static const vpx_prob default_skip_probs[SKIP_CONTEXTS] = {
- 192, 128, 64
-};
-
-static const vpx_prob default_switchable_interp_prob[SWITCHABLE_FILTER_CONTEXTS]
- [SWITCHABLE_FILTERS - 1] = {
- { 235, 162, },
- { 36, 255, },
- { 34, 3, },
- { 149, 144, },
-};
-
-#if CONFIG_MISC_FIXES
-// FIXME(someone) need real defaults here
-static const struct segmentation_probs default_seg_probs = {
- { 128, 128, 128, 128, 128, 128, 128 },
- { 128, 128, 128 },
-};
-#endif
-
-static void init_mode_probs(FRAME_CONTEXT *fc) {
- vp10_copy(fc->uv_mode_prob, default_uv_probs);
- vp10_copy(fc->y_mode_prob, default_if_y_probs);
- vp10_copy(fc->switchable_interp_prob, default_switchable_interp_prob);
- vp10_copy(fc->partition_prob, default_partition_probs);
- vp10_copy(fc->intra_inter_prob, default_intra_inter_p);
- vp10_copy(fc->comp_inter_prob, default_comp_inter_p);
- vp10_copy(fc->comp_ref_prob, default_comp_ref_p);
- vp10_copy(fc->single_ref_prob, default_single_ref_p);
- fc->tx_probs = default_tx_probs;
- vp10_copy(fc->skip_probs, default_skip_probs);
- vp10_copy(fc->inter_mode_probs, default_inter_mode_probs);
-#if CONFIG_MISC_FIXES
- vp10_copy(fc->seg.tree_probs, default_seg_probs.tree_probs);
- vp10_copy(fc->seg.pred_probs, default_seg_probs.pred_probs);
-#endif
-}
-
-const vpx_tree_index vp10_switchable_interp_tree
- [TREE_SIZE(SWITCHABLE_FILTERS)] = {
- -EIGHTTAP, 2,
- -EIGHTTAP_SMOOTH, -EIGHTTAP_SHARP
-};
-
-void vp10_adapt_inter_frame_probs(VP10_COMMON *cm) {
- int i, j;
- FRAME_CONTEXT *fc = cm->fc;
- const FRAME_CONTEXT *pre_fc = &cm->frame_contexts[cm->frame_context_idx];
- const FRAME_COUNTS *counts = &cm->counts;
-
- for (i = 0; i < INTRA_INTER_CONTEXTS; i++)
- fc->intra_inter_prob[i] = mode_mv_merge_probs(pre_fc->intra_inter_prob[i],
- counts->intra_inter[i]);
- for (i = 0; i < COMP_INTER_CONTEXTS; i++)
- fc->comp_inter_prob[i] = mode_mv_merge_probs(pre_fc->comp_inter_prob[i],
- counts->comp_inter[i]);
- for (i = 0; i < REF_CONTEXTS; i++)
- fc->comp_ref_prob[i] = mode_mv_merge_probs(pre_fc->comp_ref_prob[i],
- counts->comp_ref[i]);
- for (i = 0; i < REF_CONTEXTS; i++)
- for (j = 0; j < 2; j++)
- fc->single_ref_prob[i][j] = mode_mv_merge_probs(
- pre_fc->single_ref_prob[i][j], counts->single_ref[i][j]);
-
- for (i = 0; i < INTER_MODE_CONTEXTS; i++)
- vpx_tree_merge_probs(vp10_inter_mode_tree, pre_fc->inter_mode_probs[i],
- counts->inter_mode[i], fc->inter_mode_probs[i]);
-
- for (i = 0; i < BLOCK_SIZE_GROUPS; i++)
- vpx_tree_merge_probs(vp10_intra_mode_tree, pre_fc->y_mode_prob[i],
- counts->y_mode[i], fc->y_mode_prob[i]);
-
-#if !CONFIG_MISC_FIXES
- for (i = 0; i < INTRA_MODES; ++i)
- vpx_tree_merge_probs(vp10_intra_mode_tree, pre_fc->uv_mode_prob[i],
- counts->uv_mode[i], fc->uv_mode_prob[i]);
-
- for (i = 0; i < PARTITION_CONTEXTS; i++)
- vpx_tree_merge_probs(vp10_partition_tree, pre_fc->partition_prob[i],
- counts->partition[i], fc->partition_prob[i]);
-#endif
-
- if (cm->interp_filter == SWITCHABLE) {
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- vpx_tree_merge_probs(vp10_switchable_interp_tree,
- pre_fc->switchable_interp_prob[i],
- counts->switchable_interp[i],
- fc->switchable_interp_prob[i]);
- }
-}
-
-void vp10_adapt_intra_frame_probs(VP10_COMMON *cm) {
- int i;
- FRAME_CONTEXT *fc = cm->fc;
- const FRAME_CONTEXT *pre_fc = &cm->frame_contexts[cm->frame_context_idx];
- const FRAME_COUNTS *counts = &cm->counts;
-
- if (cm->tx_mode == TX_MODE_SELECT) {
- int j;
- unsigned int branch_ct_8x8p[TX_SIZES - 3][2];
- unsigned int branch_ct_16x16p[TX_SIZES - 2][2];
- unsigned int branch_ct_32x32p[TX_SIZES - 1][2];
-
- for (i = 0; i < TX_SIZE_CONTEXTS; ++i) {
- vp10_tx_counts_to_branch_counts_8x8(counts->tx.p8x8[i], branch_ct_8x8p);
- for (j = 0; j < TX_SIZES - 3; ++j)
- fc->tx_probs.p8x8[i][j] = mode_mv_merge_probs(
- pre_fc->tx_probs.p8x8[i][j], branch_ct_8x8p[j]);
-
- vp10_tx_counts_to_branch_counts_16x16(counts->tx.p16x16[i], branch_ct_16x16p);
- for (j = 0; j < TX_SIZES - 2; ++j)
- fc->tx_probs.p16x16[i][j] = mode_mv_merge_probs(
- pre_fc->tx_probs.p16x16[i][j], branch_ct_16x16p[j]);
-
- vp10_tx_counts_to_branch_counts_32x32(counts->tx.p32x32[i], branch_ct_32x32p);
- for (j = 0; j < TX_SIZES - 1; ++j)
- fc->tx_probs.p32x32[i][j] = mode_mv_merge_probs(
- pre_fc->tx_probs.p32x32[i][j], branch_ct_32x32p[j]);
- }
- }
-
- for (i = 0; i < SKIP_CONTEXTS; ++i)
- fc->skip_probs[i] = mode_mv_merge_probs(
- pre_fc->skip_probs[i], counts->skip[i]);
-
-#if CONFIG_MISC_FIXES
- if (cm->seg.temporal_update) {
- for (i = 0; i < PREDICTION_PROBS; i++)
- fc->seg.pred_probs[i] = mode_mv_merge_probs(pre_fc->seg.pred_probs[i],
- counts->seg.pred[i]);
-
- vpx_tree_merge_probs(vp10_segment_tree, pre_fc->seg.tree_probs,
- counts->seg.tree_mispred, fc->seg.tree_probs);
- } else {
- vpx_tree_merge_probs(vp10_segment_tree, pre_fc->seg.tree_probs,
- counts->seg.tree_total, fc->seg.tree_probs);
- }
-
- for (i = 0; i < INTRA_MODES; ++i)
- vpx_tree_merge_probs(vp10_intra_mode_tree, pre_fc->uv_mode_prob[i],
- counts->uv_mode[i], fc->uv_mode_prob[i]);
-
- for (i = 0; i < PARTITION_CONTEXTS; i++)
- vpx_tree_merge_probs(vp10_partition_tree, pre_fc->partition_prob[i],
- counts->partition[i], fc->partition_prob[i]);
-#endif
-}
-
-static void set_default_lf_deltas(struct loopfilter *lf) {
- lf->mode_ref_delta_enabled = 1;
- lf->mode_ref_delta_update = 1;
-
- lf->ref_deltas[INTRA_FRAME] = 1;
- lf->ref_deltas[LAST_FRAME] = 0;
- lf->ref_deltas[GOLDEN_FRAME] = -1;
- lf->ref_deltas[ALTREF_FRAME] = -1;
-
- lf->mode_deltas[0] = 0;
- lf->mode_deltas[1] = 0;
-}
-
-void vp10_setup_past_independence(VP10_COMMON *cm) {
- // Reset the segment feature data to the default stats:
- // Features disabled, 0, with delta coding (Default state).
- struct loopfilter *const lf = &cm->lf;
-
- int i;
- vp10_clearall_segfeatures(&cm->seg);
- cm->seg.abs_delta = SEGMENT_DELTADATA;
-
- if (cm->last_frame_seg_map && !cm->frame_parallel_decode)
- memset(cm->last_frame_seg_map, 0, (cm->mi_rows * cm->mi_cols));
-
- if (cm->current_frame_seg_map)
- memset(cm->current_frame_seg_map, 0, (cm->mi_rows * cm->mi_cols));
-
- // Reset the mode ref deltas for loop filter
- vp10_zero(lf->last_ref_deltas);
- vp10_zero(lf->last_mode_deltas);
- set_default_lf_deltas(lf);
-
- // To force update of the sharpness
- lf->last_sharpness_level = -1;
-
- vp10_default_coef_probs(cm);
- init_mode_probs(cm->fc);
- vp10_init_mv_probs(cm);
- cm->fc->initialized = 1;
-
- if (cm->frame_type == KEY_FRAME || cm->error_resilient_mode ||
- cm->reset_frame_context == RESET_FRAME_CONTEXT_ALL) {
- // Reset all frame contexts.
- for (i = 0; i < FRAME_CONTEXTS; ++i)
- cm->frame_contexts[i] = *cm->fc;
- } else if (cm->reset_frame_context == RESET_FRAME_CONTEXT_CURRENT) {
- // Reset only the frame context specified in the frame header.
- cm->frame_contexts[cm->frame_context_idx] = *cm->fc;
- }
-
- // prev_mip will only be allocated in encoder.
- if (frame_is_intra_only(cm) && cm->prev_mip && !cm->frame_parallel_decode)
- memset(cm->prev_mip, 0,
- cm->mi_stride * (cm->mi_rows + 1) * sizeof(*cm->prev_mip));
-
- cm->frame_context_idx = 0;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_ENTROPYMODE_H_
-#define VP10_COMMON_ENTROPYMODE_H_
-
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymv.h"
-#include "vp10/common/filter.h"
-#include "vp10/common/seg_common.h"
-#include "vpx_dsp/vpx_filter.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define BLOCK_SIZE_GROUPS 4
-
-#define TX_SIZE_CONTEXTS 2
-
-#define INTER_OFFSET(mode) ((mode) - NEARESTMV)
-
-#define PALETTE_COLOR_CONTEXTS 16
-#define PALETTE_MAX_SIZE 8
-#define PALETTE_BLOCK_SIZES (BLOCK_64X64 - BLOCK_8X8 + 1)
-#define PALETTE_Y_MODE_CONTEXTS 3
-
-
-struct VP10Common;
-
-struct tx_probs {
- vpx_prob p32x32[TX_SIZE_CONTEXTS][TX_SIZES - 1];
- vpx_prob p16x16[TX_SIZE_CONTEXTS][TX_SIZES - 2];
- vpx_prob p8x8[TX_SIZE_CONTEXTS][TX_SIZES - 3];
-};
-
-struct tx_counts {
- unsigned int p32x32[TX_SIZE_CONTEXTS][TX_SIZES];
- unsigned int p16x16[TX_SIZE_CONTEXTS][TX_SIZES - 1];
- unsigned int p8x8[TX_SIZE_CONTEXTS][TX_SIZES - 2];
- unsigned int tx_totals[TX_SIZES];
-};
-
-struct seg_counts {
- unsigned int tree_total[MAX_SEGMENTS];
- unsigned int tree_mispred[MAX_SEGMENTS];
- unsigned int pred[PREDICTION_PROBS][2];
-};
-
-typedef struct frame_contexts {
- vpx_prob y_mode_prob[BLOCK_SIZE_GROUPS][INTRA_MODES - 1];
- vpx_prob uv_mode_prob[INTRA_MODES][INTRA_MODES - 1];
- vpx_prob partition_prob[PARTITION_CONTEXTS][PARTITION_TYPES - 1];
- vp10_coeff_probs_model coef_probs[TX_SIZES][PLANE_TYPES];
- vpx_prob switchable_interp_prob[SWITCHABLE_FILTER_CONTEXTS]
- [SWITCHABLE_FILTERS - 1];
- vpx_prob inter_mode_probs[INTER_MODE_CONTEXTS][INTER_MODES - 1];
- vpx_prob intra_inter_prob[INTRA_INTER_CONTEXTS];
- vpx_prob comp_inter_prob[COMP_INTER_CONTEXTS];
- vpx_prob single_ref_prob[REF_CONTEXTS][2];
- vpx_prob comp_ref_prob[REF_CONTEXTS];
- struct tx_probs tx_probs;
- vpx_prob skip_probs[SKIP_CONTEXTS];
- nmv_context nmvc;
-#if CONFIG_MISC_FIXES
- struct segmentation_probs seg;
-#endif
- int initialized;
-} FRAME_CONTEXT;
-
-typedef struct FRAME_COUNTS {
- unsigned int kf_y_mode[INTRA_MODES][INTRA_MODES][INTRA_MODES];
- unsigned int y_mode[BLOCK_SIZE_GROUPS][INTRA_MODES];
- unsigned int uv_mode[INTRA_MODES][INTRA_MODES];
- unsigned int partition[PARTITION_CONTEXTS][PARTITION_TYPES];
- vp10_coeff_count_model coef[TX_SIZES][PLANE_TYPES];
- unsigned int eob_branch[TX_SIZES][PLANE_TYPES][REF_TYPES]
- [COEF_BANDS][COEFF_CONTEXTS];
- unsigned int switchable_interp[SWITCHABLE_FILTER_CONTEXTS]
- [SWITCHABLE_FILTERS];
- unsigned int inter_mode[INTER_MODE_CONTEXTS][INTER_MODES];
- unsigned int intra_inter[INTRA_INTER_CONTEXTS][2];
- unsigned int comp_inter[COMP_INTER_CONTEXTS][2];
- unsigned int single_ref[REF_CONTEXTS][2][2];
- unsigned int comp_ref[REF_CONTEXTS][2];
- struct tx_counts tx;
- unsigned int skip[SKIP_CONTEXTS][2];
- nmv_context_counts mv;
-#if CONFIG_MISC_FIXES
- struct seg_counts seg;
-#endif
-} FRAME_COUNTS;
-
-extern const vpx_prob vp10_kf_y_mode_prob[INTRA_MODES][INTRA_MODES]
- [INTRA_MODES - 1];
-#if !CONFIG_MISC_FIXES
-extern const vpx_prob vp10_kf_uv_mode_prob[INTRA_MODES][INTRA_MODES - 1];
-extern const vpx_prob vp10_kf_partition_probs[PARTITION_CONTEXTS]
- [PARTITION_TYPES - 1];
-#endif
-extern const vpx_prob
-vp10_default_palette_y_mode_prob[PALETTE_BLOCK_SIZES][PALETTE_Y_MODE_CONTEXTS];
-extern const vpx_prob
-vp10_default_palette_y_size_prob[PALETTE_BLOCK_SIZES][PALETTE_SIZES - 1];
-extern const vpx_prob
-vp10_default_palette_uv_size_prob[PALETTE_BLOCK_SIZES][PALETTE_SIZES - 1];
-extern const vpx_prob vp10_default_palette_y_color_prob
-[PALETTE_MAX_SIZE - 1][PALETTE_COLOR_CONTEXTS][PALETTE_COLORS - 1];
-extern const vpx_prob vp10_default_palette_uv_color_prob
-[PALETTE_MAX_SIZE - 1][PALETTE_COLOR_CONTEXTS][PALETTE_COLORS - 1];
-
-extern const vpx_tree_index vp10_intra_mode_tree[TREE_SIZE(INTRA_MODES)];
-extern const vpx_tree_index vp10_inter_mode_tree[TREE_SIZE(INTER_MODES)];
-extern const vpx_tree_index vp10_partition_tree[TREE_SIZE(PARTITION_TYPES)];
-extern const vpx_tree_index vp10_switchable_interp_tree
- [TREE_SIZE(SWITCHABLE_FILTERS)];
-extern const vpx_tree_index vp10_palette_size_tree[TREE_SIZE(PALETTE_SIZES)];
-extern const vpx_tree_index
-vp10_palette_color_tree[PALETTE_MAX_SIZE - 1][TREE_SIZE(PALETTE_COLORS)];
-
-
-void vp10_setup_past_independence(struct VP10Common *cm);
-
-void vp10_adapt_intra_frame_probs(struct VP10Common *cm);
-void vp10_adapt_inter_frame_probs(struct VP10Common *cm);
-
-void vp10_tx_counts_to_branch_counts_32x32(const unsigned int *tx_count_32x32p,
- unsigned int (*ct_32x32p)[2]);
-void vp10_tx_counts_to_branch_counts_16x16(const unsigned int *tx_count_16x16p,
- unsigned int (*ct_16x16p)[2]);
-void vp10_tx_counts_to_branch_counts_8x8(const unsigned int *tx_count_8x8p,
- unsigned int (*ct_8x8p)[2]);
-
-static INLINE int vp10_ceil_log2(int n) {
- int i = 1, p = 2;
- while (p < n) {
- i++;
- p = p << 1;
- }
- return i;
-}
-
-int vp10_get_palette_color_context(const uint8_t *color_map, int cols,
- int r, int c, int n, int *color_order);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_ENTROPYMODE_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/entropymv.h"
-
-// Integer pel reference mv threshold for use of high-precision 1/8 mv
-#define COMPANDED_MVREF_THRESH 8
-
-const vpx_tree_index vp10_mv_joint_tree[TREE_SIZE(MV_JOINTS)] = {
- -MV_JOINT_ZERO, 2,
- -MV_JOINT_HNZVZ, 4,
- -MV_JOINT_HZVNZ, -MV_JOINT_HNZVNZ
-};
-
-const vpx_tree_index vp10_mv_class_tree[TREE_SIZE(MV_CLASSES)] = {
- -MV_CLASS_0, 2,
- -MV_CLASS_1, 4,
- 6, 8,
- -MV_CLASS_2, -MV_CLASS_3,
- 10, 12,
- -MV_CLASS_4, -MV_CLASS_5,
- -MV_CLASS_6, 14,
- 16, 18,
- -MV_CLASS_7, -MV_CLASS_8,
- -MV_CLASS_9, -MV_CLASS_10,
-};
-
-const vpx_tree_index vp10_mv_class0_tree[TREE_SIZE(CLASS0_SIZE)] = {
- -0, -1,
-};
-
-const vpx_tree_index vp10_mv_fp_tree[TREE_SIZE(MV_FP_SIZE)] = {
- -0, 2,
- -1, 4,
- -2, -3
-};
-
-static const nmv_context default_nmv_context = {
- {32, 64, 96},
- {
- { // Vertical component
- 128, // sign
- {224, 144, 192, 168, 192, 176, 192, 198, 198, 245}, // class
- {216}, // class0
- {136, 140, 148, 160, 176, 192, 224, 234, 234, 240}, // bits
- {{128, 128, 64}, {96, 112, 64}}, // class0_fp
- {64, 96, 64}, // fp
- 160, // class0_hp bit
- 128, // hp
- },
- { // Horizontal component
- 128, // sign
- {216, 128, 176, 160, 176, 176, 192, 198, 198, 208}, // class
- {208}, // class0
- {136, 140, 148, 160, 176, 192, 224, 234, 234, 240}, // bits
- {{128, 128, 64}, {96, 112, 64}}, // class0_fp
- {64, 96, 64}, // fp
- 160, // class0_hp bit
- 128, // hp
- }
- },
-};
-
-static const uint8_t log_in_base_2[] = {
- 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
- 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6,
- 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
- 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
- 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
- 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
- 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
- 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
- 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
- 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
- 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10
-};
-
-static INLINE int mv_class_base(MV_CLASS_TYPE c) {
- return c ? CLASS0_SIZE << (c + 2) : 0;
-}
-
-MV_CLASS_TYPE vp10_get_mv_class(int z, int *offset) {
- const MV_CLASS_TYPE c = (z >= CLASS0_SIZE * 4096) ?
- MV_CLASS_10 : (MV_CLASS_TYPE)log_in_base_2[z >> 3];
- if (offset)
- *offset = z - mv_class_base(c);
- return c;
-}
-
-int vp10_use_mv_hp(const MV *ref) {
-#if CONFIG_MISC_FIXES
- (void) ref;
- return 1;
-#else
- return (abs(ref->row) >> 3) < COMPANDED_MVREF_THRESH &&
- (abs(ref->col) >> 3) < COMPANDED_MVREF_THRESH;
-#endif
-}
-
-static void inc_mv_component(int v, nmv_component_counts *comp_counts,
- int incr, int usehp) {
- int s, z, c, o, d, e, f;
- assert(v != 0); /* should not be zero */
- s = v < 0;
- comp_counts->sign[s] += incr;
- z = (s ? -v : v) - 1; /* magnitude - 1 */
-
- c = vp10_get_mv_class(z, &o);
- comp_counts->classes[c] += incr;
-
- d = (o >> 3); /* int mv data */
- f = (o >> 1) & 3; /* fractional pel mv data */
- e = (o & 1); /* high precision mv data */
-
- if (c == MV_CLASS_0) {
- comp_counts->class0[d] += incr;
- comp_counts->class0_fp[d][f] += incr;
- comp_counts->class0_hp[e] += usehp * incr;
- } else {
- int i;
- int b = c + CLASS0_BITS - 1; // number of bits
- for (i = 0; i < b; ++i)
- comp_counts->bits[i][((d >> i) & 1)] += incr;
- comp_counts->fp[f] += incr;
- comp_counts->hp[e] += usehp * incr;
- }
-}
-
-void vp10_inc_mv(const MV *mv, nmv_context_counts *counts, const int usehp) {
- if (counts != NULL) {
- const MV_JOINT_TYPE j = vp10_get_mv_joint(mv);
- ++counts->joints[j];
-
- if (mv_joint_vertical(j)) {
- inc_mv_component(mv->row, &counts->comps[0], 1,
- !CONFIG_MISC_FIXES || usehp);
- }
-
- if (mv_joint_horizontal(j)) {
- inc_mv_component(mv->col, &counts->comps[1], 1,
- !CONFIG_MISC_FIXES || usehp);
- }
- }
-}
-
-void vp10_adapt_mv_probs(VP10_COMMON *cm, int allow_hp) {
- int i, j;
-
- nmv_context *fc = &cm->fc->nmvc;
- const nmv_context *pre_fc = &cm->frame_contexts[cm->frame_context_idx].nmvc;
- const nmv_context_counts *counts = &cm->counts.mv;
-
- vpx_tree_merge_probs(vp10_mv_joint_tree, pre_fc->joints, counts->joints,
- fc->joints);
-
- for (i = 0; i < 2; ++i) {
- nmv_component *comp = &fc->comps[i];
- const nmv_component *pre_comp = &pre_fc->comps[i];
- const nmv_component_counts *c = &counts->comps[i];
-
- comp->sign = mode_mv_merge_probs(pre_comp->sign, c->sign);
- vpx_tree_merge_probs(vp10_mv_class_tree, pre_comp->classes, c->classes,
- comp->classes);
- vpx_tree_merge_probs(vp10_mv_class0_tree, pre_comp->class0, c->class0,
- comp->class0);
-
- for (j = 0; j < MV_OFFSET_BITS; ++j)
- comp->bits[j] = mode_mv_merge_probs(pre_comp->bits[j], c->bits[j]);
-
- for (j = 0; j < CLASS0_SIZE; ++j)
- vpx_tree_merge_probs(vp10_mv_fp_tree, pre_comp->class0_fp[j],
- c->class0_fp[j], comp->class0_fp[j]);
-
- vpx_tree_merge_probs(vp10_mv_fp_tree, pre_comp->fp, c->fp, comp->fp);
-
- if (allow_hp) {
- comp->class0_hp = mode_mv_merge_probs(pre_comp->class0_hp, c->class0_hp);
- comp->hp = mode_mv_merge_probs(pre_comp->hp, c->hp);
- }
- }
-}
-
-void vp10_init_mv_probs(VP10_COMMON *cm) {
- cm->fc->nmvc = default_nmv_context;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_COMMON_ENTROPYMV_H_
-#define VP10_COMMON_ENTROPYMV_H_
-
-#include "./vpx_config.h"
-
-#include "vpx_dsp/prob.h"
-
-#include "vp10/common/mv.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10Common;
-
-void vp10_init_mv_probs(struct VP10Common *cm);
-
-void vp10_adapt_mv_probs(struct VP10Common *cm, int usehp);
-int vp10_use_mv_hp(const MV *ref);
-
-#define MV_UPDATE_PROB 252
-
-/* Symbols for coding which components are zero jointly */
-#define MV_JOINTS 4
-typedef enum {
- MV_JOINT_ZERO = 0, /* Zero vector */
- MV_JOINT_HNZVZ = 1, /* Vert zero, hor nonzero */
- MV_JOINT_HZVNZ = 2, /* Hor zero, vert nonzero */
- MV_JOINT_HNZVNZ = 3, /* Both components nonzero */
-} MV_JOINT_TYPE;
-
-static INLINE int mv_joint_vertical(MV_JOINT_TYPE type) {
- return type == MV_JOINT_HZVNZ || type == MV_JOINT_HNZVNZ;
-}
-
-static INLINE int mv_joint_horizontal(MV_JOINT_TYPE type) {
- return type == MV_JOINT_HNZVZ || type == MV_JOINT_HNZVNZ;
-}
-
-/* Symbols for coding magnitude class of nonzero components */
-#define MV_CLASSES 11
-typedef enum {
- MV_CLASS_0 = 0, /* (0, 2] integer pel */
- MV_CLASS_1 = 1, /* (2, 4] integer pel */
- MV_CLASS_2 = 2, /* (4, 8] integer pel */
- MV_CLASS_3 = 3, /* (8, 16] integer pel */
- MV_CLASS_4 = 4, /* (16, 32] integer pel */
- MV_CLASS_5 = 5, /* (32, 64] integer pel */
- MV_CLASS_6 = 6, /* (64, 128] integer pel */
- MV_CLASS_7 = 7, /* (128, 256] integer pel */
- MV_CLASS_8 = 8, /* (256, 512] integer pel */
- MV_CLASS_9 = 9, /* (512, 1024] integer pel */
- MV_CLASS_10 = 10, /* (1024,2048] integer pel */
-} MV_CLASS_TYPE;
-
-#define CLASS0_BITS 1 /* bits at integer precision for class 0 */
-#define CLASS0_SIZE (1 << CLASS0_BITS)
-#define MV_OFFSET_BITS (MV_CLASSES + CLASS0_BITS - 2)
-#define MV_FP_SIZE 4
-
-#define MV_MAX_BITS (MV_CLASSES + CLASS0_BITS + 2)
-#define MV_MAX ((1 << MV_MAX_BITS) - 1)
-#define MV_VALS ((MV_MAX << 1) + 1)
-
-#define MV_IN_USE_BITS 14
-#define MV_UPP ((1 << MV_IN_USE_BITS) - 1)
-#define MV_LOW (-(1 << MV_IN_USE_BITS))
-
-extern const vpx_tree_index vp10_mv_joint_tree[];
-extern const vpx_tree_index vp10_mv_class_tree[];
-extern const vpx_tree_index vp10_mv_class0_tree[];
-extern const vpx_tree_index vp10_mv_fp_tree[];
-
-typedef struct {
- vpx_prob sign;
- vpx_prob classes[MV_CLASSES - 1];
- vpx_prob class0[CLASS0_SIZE - 1];
- vpx_prob bits[MV_OFFSET_BITS];
- vpx_prob class0_fp[CLASS0_SIZE][MV_FP_SIZE - 1];
- vpx_prob fp[MV_FP_SIZE - 1];
- vpx_prob class0_hp;
- vpx_prob hp;
-} nmv_component;
-
-typedef struct {
- vpx_prob joints[MV_JOINTS - 1];
- nmv_component comps[2];
-} nmv_context;
-
-static INLINE MV_JOINT_TYPE vp10_get_mv_joint(const MV *mv) {
- if (mv->row == 0) {
- return mv->col == 0 ? MV_JOINT_ZERO : MV_JOINT_HNZVZ;
- } else {
- return mv->col == 0 ? MV_JOINT_HZVNZ : MV_JOINT_HNZVNZ;
- }
-}
-
-MV_CLASS_TYPE vp10_get_mv_class(int z, int *offset);
-
-typedef struct {
- unsigned int sign[2];
- unsigned int classes[MV_CLASSES];
- unsigned int class0[CLASS0_SIZE];
- unsigned int bits[MV_OFFSET_BITS][2];
- unsigned int class0_fp[CLASS0_SIZE][MV_FP_SIZE];
- unsigned int fp[MV_FP_SIZE];
- unsigned int class0_hp[2];
- unsigned int hp[2];
-} nmv_component_counts;
-
-typedef struct {
- unsigned int joints[MV_JOINTS];
- nmv_component_counts comps[2];
-} nmv_context_counts;
-
-void vp10_inc_mv(const MV *mv, nmv_context_counts *mvctx, const int usehp);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_ENTROPYMV_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_ENUMS_H_
-#define VP10_COMMON_ENUMS_H_
-
-#include "./vpx_config.h"
-#include "vpx/vpx_integer.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MI_SIZE_LOG2 3
-#define MI_BLOCK_SIZE_LOG2 (6 - MI_SIZE_LOG2) // 64 = 2^6
-
-#define MI_SIZE (1 << MI_SIZE_LOG2) // pixels per mi-unit
-#define MI_BLOCK_SIZE (1 << MI_BLOCK_SIZE_LOG2) // mi-units per max block
-
-#define MI_MASK (MI_BLOCK_SIZE - 1)
-
-// Bitstream profiles indicated by 2-3 bits in the uncompressed header.
-// 00: Profile 0. 8-bit 4:2:0 only.
-// 10: Profile 1. 8-bit 4:4:4, 4:2:2, and 4:4:0.
-// 01: Profile 2. 10-bit and 12-bit color only, with 4:2:0 sampling.
-// 110: Profile 3. 10-bit and 12-bit color only, with 4:2:2/4:4:4/4:4:0
-// sampling.
-// 111: Undefined profile.
-typedef enum BITSTREAM_PROFILE {
- PROFILE_0,
- PROFILE_1,
- PROFILE_2,
- PROFILE_3,
- MAX_PROFILES
-} BITSTREAM_PROFILE;
-
-#define BLOCK_4X4 0
-#define BLOCK_4X8 1
-#define BLOCK_8X4 2
-#define BLOCK_8X8 3
-#define BLOCK_8X16 4
-#define BLOCK_16X8 5
-#define BLOCK_16X16 6
-#define BLOCK_16X32 7
-#define BLOCK_32X16 8
-#define BLOCK_32X32 9
-#define BLOCK_32X64 10
-#define BLOCK_64X32 11
-#define BLOCK_64X64 12
-#define BLOCK_SIZES 13
-#define BLOCK_INVALID BLOCK_SIZES
-typedef uint8_t BLOCK_SIZE;
-
-typedef enum PARTITION_TYPE {
- PARTITION_NONE,
- PARTITION_HORZ,
- PARTITION_VERT,
- PARTITION_SPLIT,
- PARTITION_TYPES,
- PARTITION_INVALID = PARTITION_TYPES
-} PARTITION_TYPE;
-
-typedef char PARTITION_CONTEXT;
-#define PARTITION_PLOFFSET 4 // number of probability models per block size
-#define PARTITION_CONTEXTS (4 * PARTITION_PLOFFSET)
-
-// block transform size
-typedef uint8_t TX_SIZE;
-#define TX_4X4 ((TX_SIZE)0) // 4x4 transform
-#define TX_8X8 ((TX_SIZE)1) // 8x8 transform
-#define TX_16X16 ((TX_SIZE)2) // 16x16 transform
-#define TX_32X32 ((TX_SIZE)3) // 32x32 transform
-#define TX_SIZES ((TX_SIZE)4)
-
-// frame transform mode
-typedef enum {
- ONLY_4X4 = 0, // only 4x4 transform used
- ALLOW_8X8 = 1, // allow block transform size up to 8x8
- ALLOW_16X16 = 2, // allow block transform size up to 16x16
- ALLOW_32X32 = 3, // allow block transform size up to 32x32
- TX_MODE_SELECT = 4, // transform specified for each block
- TX_MODES = 5,
-} TX_MODE;
-
-typedef enum {
- DCT_DCT = 0, // DCT in both horizontal and vertical
- ADST_DCT = 1, // ADST in vertical, DCT in horizontal
- DCT_ADST = 2, // DCT in vertical, ADST in horizontal
- ADST_ADST = 3, // ADST in both directions
- TX_TYPES = 4
-} TX_TYPE;
-
-typedef enum {
- VP9_LAST_FLAG = 1 << 0,
- VP9_GOLD_FLAG = 1 << 1,
- VP9_ALT_FLAG = 1 << 2,
-} VP9_REFFRAME;
-
-typedef enum {
- PLANE_TYPE_Y = 0,
- PLANE_TYPE_UV = 1,
- PLANE_TYPES
-} PLANE_TYPE;
-
-typedef enum {
- TWO_COLORS,
- THREE_COLORS,
- FOUR_COLORS,
- FIVE_COLORS,
- SIX_COLORS,
- SEVEN_COLORS,
- EIGHT_COLORS,
- PALETTE_SIZES
-} PALETTE_SIZE;
-
-typedef enum {
- PALETTE_COLOR_ONE,
- PALETTE_COLOR_TWO,
- PALETTE_COLOR_THREE,
- PALETTE_COLOR_FOUR,
- PALETTE_COLOR_FIVE,
- PALETTE_COLOR_SIX,
- PALETTE_COLOR_SEVEN,
- PALETTE_COLOR_EIGHT,
- PALETTE_COLORS
-} PALETTE_COLOR;
-
-#define DC_PRED 0 // Average of above and left pixels
-#define V_PRED 1 // Vertical
-#define H_PRED 2 // Horizontal
-#define D45_PRED 3 // Directional 45 deg = round(arctan(1/1) * 180/pi)
-#define D135_PRED 4 // Directional 135 deg = 180 - 45
-#define D117_PRED 5 // Directional 117 deg = 180 - 63
-#define D153_PRED 6 // Directional 153 deg = 180 - 27
-#define D207_PRED 7 // Directional 207 deg = 180 + 27
-#define D63_PRED 8 // Directional 63 deg = round(arctan(2/1) * 180/pi)
-#define TM_PRED 9 // True-motion
-#define NEARESTMV 10
-#define NEARMV 11
-#define ZEROMV 12
-#define NEWMV 13
-#define MB_MODE_COUNT 14
-typedef uint8_t PREDICTION_MODE;
-
-#define INTRA_MODES (TM_PRED + 1)
-
-#define INTER_MODES (1 + NEWMV - NEARESTMV)
-
-#define SKIP_CONTEXTS 3
-#define INTER_MODE_CONTEXTS 7
-
-/* Segment Feature Masks */
-#define MAX_MV_REF_CANDIDATES 2
-
-#define INTRA_INTER_CONTEXTS 4
-#define COMP_INTER_CONTEXTS 5
-#define REF_CONTEXTS 5
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_ENUMS_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/filter.h"
-
-DECLARE_ALIGNED(256, static const InterpKernel,
- bilinear_filters[SUBPEL_SHIFTS]) = {
- { 0, 0, 0, 128, 0, 0, 0, 0 },
- { 0, 0, 0, 120, 8, 0, 0, 0 },
- { 0, 0, 0, 112, 16, 0, 0, 0 },
- { 0, 0, 0, 104, 24, 0, 0, 0 },
- { 0, 0, 0, 96, 32, 0, 0, 0 },
- { 0, 0, 0, 88, 40, 0, 0, 0 },
- { 0, 0, 0, 80, 48, 0, 0, 0 },
- { 0, 0, 0, 72, 56, 0, 0, 0 },
- { 0, 0, 0, 64, 64, 0, 0, 0 },
- { 0, 0, 0, 56, 72, 0, 0, 0 },
- { 0, 0, 0, 48, 80, 0, 0, 0 },
- { 0, 0, 0, 40, 88, 0, 0, 0 },
- { 0, 0, 0, 32, 96, 0, 0, 0 },
- { 0, 0, 0, 24, 104, 0, 0, 0 },
- { 0, 0, 0, 16, 112, 0, 0, 0 },
- { 0, 0, 0, 8, 120, 0, 0, 0 }
-};
-
-// Lagrangian interpolation filter
-DECLARE_ALIGNED(256, static const InterpKernel,
- sub_pel_filters_8[SUBPEL_SHIFTS]) = {
- { 0, 0, 0, 128, 0, 0, 0, 0},
- { 0, 1, -5, 126, 8, -3, 1, 0},
- { -1, 3, -10, 122, 18, -6, 2, 0},
- { -1, 4, -13, 118, 27, -9, 3, -1},
- { -1, 4, -16, 112, 37, -11, 4, -1},
- { -1, 5, -18, 105, 48, -14, 4, -1},
- { -1, 5, -19, 97, 58, -16, 5, -1},
- { -1, 6, -19, 88, 68, -18, 5, -1},
- { -1, 6, -19, 78, 78, -19, 6, -1},
- { -1, 5, -18, 68, 88, -19, 6, -1},
- { -1, 5, -16, 58, 97, -19, 5, -1},
- { -1, 4, -14, 48, 105, -18, 5, -1},
- { -1, 4, -11, 37, 112, -16, 4, -1},
- { -1, 3, -9, 27, 118, -13, 4, -1},
- { 0, 2, -6, 18, 122, -10, 3, -1},
- { 0, 1, -3, 8, 126, -5, 1, 0}
-};
-
-// DCT based filter
-DECLARE_ALIGNED(256, static const InterpKernel,
- sub_pel_filters_8s[SUBPEL_SHIFTS]) = {
- {0, 0, 0, 128, 0, 0, 0, 0},
- {-1, 3, -7, 127, 8, -3, 1, 0},
- {-2, 5, -13, 125, 17, -6, 3, -1},
- {-3, 7, -17, 121, 27, -10, 5, -2},
- {-4, 9, -20, 115, 37, -13, 6, -2},
- {-4, 10, -23, 108, 48, -16, 8, -3},
- {-4, 10, -24, 100, 59, -19, 9, -3},
- {-4, 11, -24, 90, 70, -21, 10, -4},
- {-4, 11, -23, 80, 80, -23, 11, -4},
- {-4, 10, -21, 70, 90, -24, 11, -4},
- {-3, 9, -19, 59, 100, -24, 10, -4},
- {-3, 8, -16, 48, 108, -23, 10, -4},
- {-2, 6, -13, 37, 115, -20, 9, -4},
- {-2, 5, -10, 27, 121, -17, 7, -3},
- {-1, 3, -6, 17, 125, -13, 5, -2},
- {0, 1, -3, 8, 127, -7, 3, -1}
-};
-
-// freqmultiplier = 0.5
-DECLARE_ALIGNED(256, static const InterpKernel,
- sub_pel_filters_8lp[SUBPEL_SHIFTS]) = {
- { 0, 0, 0, 128, 0, 0, 0, 0},
- {-3, -1, 32, 64, 38, 1, -3, 0},
- {-2, -2, 29, 63, 41, 2, -3, 0},
- {-2, -2, 26, 63, 43, 4, -4, 0},
- {-2, -3, 24, 62, 46, 5, -4, 0},
- {-2, -3, 21, 60, 49, 7, -4, 0},
- {-1, -4, 18, 59, 51, 9, -4, 0},
- {-1, -4, 16, 57, 53, 12, -4, -1},
- {-1, -4, 14, 55, 55, 14, -4, -1},
- {-1, -4, 12, 53, 57, 16, -4, -1},
- { 0, -4, 9, 51, 59, 18, -4, -1},
- { 0, -4, 7, 49, 60, 21, -3, -2},
- { 0, -4, 5, 46, 62, 24, -3, -2},
- { 0, -4, 4, 43, 63, 26, -2, -2},
- { 0, -3, 2, 41, 63, 29, -2, -2},
- { 0, -3, 1, 38, 64, 32, -1, -3}
-};
-
-
-const InterpKernel *vp10_filter_kernels[4] = {
- sub_pel_filters_8,
- sub_pel_filters_8lp,
- sub_pel_filters_8s,
- bilinear_filters
-};
+++ /dev/null
-/*
- * Copyright (c) 2011 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_FILTER_H_
-#define VP10_COMMON_FILTER_H_
-
-#include "./vpx_config.h"
-#include "vpx/vpx_integer.h"
-#include "vpx_dsp/vpx_filter.h"
-#include "vpx_ports/mem.h"
-
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define EIGHTTAP 0
-#define EIGHTTAP_SMOOTH 1
-#define EIGHTTAP_SHARP 2
-#define SWITCHABLE_FILTERS 3 /* Number of switchable filters */
-#define BILINEAR 3
-// The codec can operate in four possible inter prediction filter mode:
-// 8-tap, 8-tap-smooth, 8-tap-sharp, and switching between the three.
-#define SWITCHABLE_FILTER_CONTEXTS (SWITCHABLE_FILTERS + 1)
-#define SWITCHABLE 4 /* should be the last one */
-
-typedef uint8_t INTERP_FILTER;
-
-extern const InterpKernel *vp10_filter_kernels[4];
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_FILTER_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/frame_buffers.h"
-#include "vpx_mem/vpx_mem.h"
-
-int vp10_alloc_internal_frame_buffers(InternalFrameBufferList *list) {
- assert(list != NULL);
- vp10_free_internal_frame_buffers(list);
-
- list->num_internal_frame_buffers =
- VP9_MAXIMUM_REF_BUFFERS + VPX_MAXIMUM_WORK_BUFFERS;
- list->int_fb =
- (InternalFrameBuffer *)vpx_calloc(list->num_internal_frame_buffers,
- sizeof(*list->int_fb));
- return (list->int_fb == NULL);
-}
-
-void vp10_free_internal_frame_buffers(InternalFrameBufferList *list) {
- int i;
-
- assert(list != NULL);
-
- for (i = 0; i < list->num_internal_frame_buffers; ++i) {
- vpx_free(list->int_fb[i].data);
- list->int_fb[i].data = NULL;
- }
- vpx_free(list->int_fb);
- list->int_fb = NULL;
-}
-
-int vp10_get_frame_buffer(void *cb_priv, size_t min_size,
- vpx_codec_frame_buffer_t *fb) {
- int i;
- InternalFrameBufferList *const int_fb_list =
- (InternalFrameBufferList *)cb_priv;
- if (int_fb_list == NULL)
- return -1;
-
- // Find a free frame buffer.
- for (i = 0; i < int_fb_list->num_internal_frame_buffers; ++i) {
- if (!int_fb_list->int_fb[i].in_use)
- break;
- }
-
- if (i == int_fb_list->num_internal_frame_buffers)
- return -1;
-
- if (int_fb_list->int_fb[i].size < min_size) {
- int_fb_list->int_fb[i].data =
- (uint8_t *)vpx_realloc(int_fb_list->int_fb[i].data, min_size);
- if (!int_fb_list->int_fb[i].data)
- return -1;
-
- // This memset is needed for fixing valgrind error from C loop filter
- // due to access uninitialized memory in frame border. It could be
- // removed if border is totally removed.
- memset(int_fb_list->int_fb[i].data, 0, min_size);
- int_fb_list->int_fb[i].size = min_size;
- }
-
- fb->data = int_fb_list->int_fb[i].data;
- fb->size = int_fb_list->int_fb[i].size;
- int_fb_list->int_fb[i].in_use = 1;
-
- // Set the frame buffer's private data to point at the internal frame buffer.
- fb->priv = &int_fb_list->int_fb[i];
- return 0;
-}
-
-int vp10_release_frame_buffer(void *cb_priv, vpx_codec_frame_buffer_t *fb) {
- InternalFrameBuffer *const int_fb = (InternalFrameBuffer *)fb->priv;
- (void)cb_priv;
- if (int_fb)
- int_fb->in_use = 0;
- return 0;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_FRAME_BUFFERS_H_
-#define VP10_COMMON_FRAME_BUFFERS_H_
-
-#include "vpx/vpx_frame_buffer.h"
-#include "vpx/vpx_integer.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct InternalFrameBuffer {
- uint8_t *data;
- size_t size;
- int in_use;
-} InternalFrameBuffer;
-
-typedef struct InternalFrameBufferList {
- int num_internal_frame_buffers;
- InternalFrameBuffer *int_fb;
-} InternalFrameBufferList;
-
-// Initializes |list|. Returns 0 on success.
-int vp10_alloc_internal_frame_buffers(InternalFrameBufferList *list);
-
-// Free any data allocated to the frame buffers.
-void vp10_free_internal_frame_buffers(InternalFrameBufferList *list);
-
-// Callback used by libvpx to request an external frame buffer. |cb_priv|
-// Callback private data, which points to an InternalFrameBufferList.
-// |min_size| is the minimum size in bytes needed to decode the next frame.
-// |fb| pointer to the frame buffer.
-int vp10_get_frame_buffer(void *cb_priv, size_t min_size,
- vpx_codec_frame_buffer_t *fb);
-
-// Callback used by libvpx when there are no references to the frame buffer.
-// |cb_priv| is not used. |fb| pointer to the frame buffer.
-int vp10_release_frame_buffer(void *cb_priv, vpx_codec_frame_buffer_t *fb);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_FRAME_BUFFERS_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/idct.h"
-#include "vpx_dsp/inv_txfm.h"
-#include "vpx_ports/mem.h"
-
-void vp10_iht4x4_16_add_c(const tran_low_t *input, uint8_t *dest, int stride,
- int tx_type) {
- const transform_2d IHT_4[] = {
- { idct4_c, idct4_c }, // DCT_DCT = 0
- { iadst4_c, idct4_c }, // ADST_DCT = 1
- { idct4_c, iadst4_c }, // DCT_ADST = 2
- { iadst4_c, iadst4_c } // ADST_ADST = 3
- };
-
- int i, j;
- tran_low_t out[4 * 4];
- tran_low_t *outptr = out;
- tran_low_t temp_in[4], temp_out[4];
-
- // inverse transform row vectors
- for (i = 0; i < 4; ++i) {
- IHT_4[tx_type].rows(input, outptr);
- input += 4;
- outptr += 4;
- }
-
- // inverse transform column vectors
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j * 4 + i];
- IHT_4[tx_type].cols(temp_in, temp_out);
- for (j = 0; j < 4; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 4));
- }
- }
-}
-
-static const transform_2d IHT_8[] = {
- { idct8_c, idct8_c }, // DCT_DCT = 0
- { iadst8_c, idct8_c }, // ADST_DCT = 1
- { idct8_c, iadst8_c }, // DCT_ADST = 2
- { iadst8_c, iadst8_c } // ADST_ADST = 3
-};
-
-void vp10_iht8x8_64_add_c(const tran_low_t *input, uint8_t *dest, int stride,
- int tx_type) {
- int i, j;
- tran_low_t out[8 * 8];
- tran_low_t *outptr = out;
- tran_low_t temp_in[8], temp_out[8];
- const transform_2d ht = IHT_8[tx_type];
-
- // inverse transform row vectors
- for (i = 0; i < 8; ++i) {
- ht.rows(input, outptr);
- input += 8;
- outptr += 8;
- }
-
- // inverse transform column vectors
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- ht.cols(temp_in, temp_out);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 5));
- }
- }
-}
-
-static const transform_2d IHT_16[] = {
- { idct16_c, idct16_c }, // DCT_DCT = 0
- { iadst16_c, idct16_c }, // ADST_DCT = 1
- { idct16_c, iadst16_c }, // DCT_ADST = 2
- { iadst16_c, iadst16_c } // ADST_ADST = 3
-};
-
-void vp10_iht16x16_256_add_c(const tran_low_t *input, uint8_t *dest, int stride,
- int tx_type) {
- int i, j;
- tran_low_t out[16 * 16];
- tran_low_t *outptr = out;
- tran_low_t temp_in[16], temp_out[16];
- const transform_2d ht = IHT_16[tx_type];
-
- // Rows
- for (i = 0; i < 16; ++i) {
- ht.rows(input, outptr);
- input += 16;
- outptr += 16;
- }
-
- // Columns
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- ht.cols(temp_in, temp_out);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 6));
- }
- }
-}
-
-// idct
-void vp10_idct4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob) {
- if (eob > 1)
- vpx_idct4x4_16_add(input, dest, stride);
- else
- vpx_idct4x4_1_add(input, dest, stride);
-}
-
-
-void vp10_iwht4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob) {
- if (eob > 1)
- vpx_iwht4x4_16_add(input, dest, stride);
- else
- vpx_iwht4x4_1_add(input, dest, stride);
-}
-
-void vp10_idct8x8_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob) {
- // If dc is 1, then input[0] is the reconstructed value, do not need
- // dequantization. Also, when dc is 1, dc is counted in eobs, namely eobs >=1.
-
- // The calculation can be simplified if there are not many non-zero dct
- // coefficients. Use eobs to decide what to do.
- // TODO(yunqingwang): "eobs = 1" case is also handled in vp10_short_idct8x8_c.
- // Combine that with code here.
- if (eob == 1)
- // DC only DCT coefficient
- vpx_idct8x8_1_add(input, dest, stride);
- else if (eob <= 12)
- vpx_idct8x8_12_add(input, dest, stride);
- else
- vpx_idct8x8_64_add(input, dest, stride);
-}
-
-void vp10_idct16x16_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob) {
- /* The calculation can be simplified if there are not many non-zero dct
- * coefficients. Use eobs to separate different cases. */
- if (eob == 1)
- /* DC only DCT coefficient. */
- vpx_idct16x16_1_add(input, dest, stride);
- else if (eob <= 10)
- vpx_idct16x16_10_add(input, dest, stride);
- else
- vpx_idct16x16_256_add(input, dest, stride);
-}
-
-void vp10_idct32x32_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob) {
- if (eob == 1)
- vpx_idct32x32_1_add(input, dest, stride);
- else if (eob <= 34)
- // non-zero coeff only in upper-left 8x8
- vpx_idct32x32_34_add(input, dest, stride);
- else
- vpx_idct32x32_1024_add(input, dest, stride);
-}
-
-void vp10_inv_txfm_add_4x4(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type, int lossless) {
- if (lossless) {
- assert(tx_type == DCT_DCT);
- vp10_iwht4x4_add(input, dest, stride, eob);
- } else {
- switch (tx_type) {
- case DCT_DCT:
- vp10_idct4x4_add(input, dest, stride, eob);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_iht4x4_16_add(input, dest, stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
- }
-}
-
-void vp10_inv_txfm_add_8x8(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vp10_idct8x8_add(input, dest, stride, eob);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_iht8x8_64_add(input, dest, stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_inv_txfm_add_16x16(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vp10_idct16x16_add(input, dest, stride, eob);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_iht16x16_256_add(input, dest, stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_inv_txfm_add_32x32(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vp10_idct32x32_add(input, dest, stride, eob);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- assert(0);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_iht4x4_16_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int tx_type, int bd) {
- const highbd_transform_2d IHT_4[] = {
- { vpx_highbd_idct4_c, vpx_highbd_idct4_c }, // DCT_DCT = 0
- { vpx_highbd_iadst4_c, vpx_highbd_idct4_c }, // ADST_DCT = 1
- { vpx_highbd_idct4_c, vpx_highbd_iadst4_c }, // DCT_ADST = 2
- { vpx_highbd_iadst4_c, vpx_highbd_iadst4_c } // ADST_ADST = 3
- };
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- int i, j;
- tran_low_t out[4 * 4];
- tran_low_t *outptr = out;
- tran_low_t temp_in[4], temp_out[4];
-
- // Inverse transform row vectors.
- for (i = 0; i < 4; ++i) {
- IHT_4[tx_type].rows(input, outptr, bd);
- input += 4;
- outptr += 4;
- }
-
- // Inverse transform column vectors.
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j * 4 + i];
- IHT_4[tx_type].cols(temp_in, temp_out, bd);
- for (j = 0; j < 4; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 4), bd);
- }
- }
-}
-
-static const highbd_transform_2d HIGH_IHT_8[] = {
- { vpx_highbd_idct8_c, vpx_highbd_idct8_c }, // DCT_DCT = 0
- { vpx_highbd_iadst8_c, vpx_highbd_idct8_c }, // ADST_DCT = 1
- { vpx_highbd_idct8_c, vpx_highbd_iadst8_c }, // DCT_ADST = 2
- { vpx_highbd_iadst8_c, vpx_highbd_iadst8_c } // ADST_ADST = 3
-};
-
-void vp10_highbd_iht8x8_64_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int tx_type, int bd) {
- int i, j;
- tran_low_t out[8 * 8];
- tran_low_t *outptr = out;
- tran_low_t temp_in[8], temp_out[8];
- const highbd_transform_2d ht = HIGH_IHT_8[tx_type];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // Inverse transform row vectors.
- for (i = 0; i < 8; ++i) {
- ht.rows(input, outptr, bd);
- input += 8;
- outptr += 8;
- }
-
- // Inverse transform column vectors.
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- ht.cols(temp_in, temp_out, bd);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 5), bd);
- }
- }
-}
-
-static const highbd_transform_2d HIGH_IHT_16[] = {
- { vpx_highbd_idct16_c, vpx_highbd_idct16_c }, // DCT_DCT = 0
- { vpx_highbd_iadst16_c, vpx_highbd_idct16_c }, // ADST_DCT = 1
- { vpx_highbd_idct16_c, vpx_highbd_iadst16_c }, // DCT_ADST = 2
- { vpx_highbd_iadst16_c, vpx_highbd_iadst16_c } // ADST_ADST = 3
-};
-
-void vp10_highbd_iht16x16_256_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int tx_type, int bd) {
- int i, j;
- tran_low_t out[16 * 16];
- tran_low_t *outptr = out;
- tran_low_t temp_in[16], temp_out[16];
- const highbd_transform_2d ht = HIGH_IHT_16[tx_type];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // Rows
- for (i = 0; i < 16; ++i) {
- ht.rows(input, outptr, bd);
- input += 16;
- outptr += 16;
- }
-
- // Columns
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- ht.cols(temp_in, temp_out, bd);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 6), bd);
- }
- }
-}
-
-// idct
-void vp10_highbd_idct4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob, int bd) {
- if (eob > 1)
- vpx_highbd_idct4x4_16_add(input, dest, stride, bd);
- else
- vpx_highbd_idct4x4_1_add(input, dest, stride, bd);
-}
-
-
-void vp10_highbd_iwht4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob, int bd) {
- if (eob > 1)
- vpx_highbd_iwht4x4_16_add(input, dest, stride, bd);
- else
- vpx_highbd_iwht4x4_1_add(input, dest, stride, bd);
-}
-
-void vp10_highbd_idct8x8_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob, int bd) {
- // If dc is 1, then input[0] is the reconstructed value, do not need
- // dequantization. Also, when dc is 1, dc is counted in eobs, namely eobs >=1.
-
- // The calculation can be simplified if there are not many non-zero dct
- // coefficients. Use eobs to decide what to do.
- // TODO(yunqingwang): "eobs = 1" case is also handled in vp10_short_idct8x8_c.
- // Combine that with code here.
- // DC only DCT coefficient
- if (eob == 1) {
- vpx_highbd_idct8x8_1_add(input, dest, stride, bd);
- } else if (eob <= 10) {
- vpx_highbd_idct8x8_10_add(input, dest, stride, bd);
- } else {
- vpx_highbd_idct8x8_64_add(input, dest, stride, bd);
- }
-}
-
-void vp10_highbd_idct16x16_add(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd) {
- // The calculation can be simplified if there are not many non-zero dct
- // coefficients. Use eobs to separate different cases.
- // DC only DCT coefficient.
- if (eob == 1) {
- vpx_highbd_idct16x16_1_add(input, dest, stride, bd);
- } else if (eob <= 10) {
- vpx_highbd_idct16x16_10_add(input, dest, stride, bd);
- } else {
- vpx_highbd_idct16x16_256_add(input, dest, stride, bd);
- }
-}
-
-void vp10_highbd_idct32x32_add(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd) {
- // Non-zero coeff only in upper-left 8x8
- if (eob == 1) {
- vpx_highbd_idct32x32_1_add(input, dest, stride, bd);
- } else if (eob <= 34) {
- vpx_highbd_idct32x32_34_add(input, dest, stride, bd);
- } else {
- vpx_highbd_idct32x32_1024_add(input, dest, stride, bd);
- }
-}
-
-void vp10_highbd_inv_txfm_add_4x4(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd, TX_TYPE tx_type,
- int lossless) {
- if (lossless) {
- assert(tx_type == DCT_DCT);
- vp10_highbd_iwht4x4_add(input, dest, stride, eob, bd);
- } else {
- switch (tx_type) {
- case DCT_DCT:
- vp10_highbd_idct4x4_add(input, dest, stride, eob, bd);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_highbd_iht4x4_16_add(input, dest, stride, tx_type, bd);
- break;
- default:
- assert(0);
- break;
- }
- }
-}
-
-void vp10_highbd_inv_txfm_add_8x8(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd,
- TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vp10_highbd_idct8x8_add(input, dest, stride, eob, bd);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_highbd_iht8x8_64_add(input, dest, stride, tx_type, bd);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_highbd_inv_txfm_add_16x16(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd,
- TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vp10_highbd_idct16x16_add(input, dest, stride, eob, bd);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_highbd_iht16x16_256_add(input, dest, stride, tx_type, bd);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_highbd_inv_txfm_add_32x32(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd,
- TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vp10_highbd_idct32x32_add(input, dest, stride, eob, bd);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- assert(0);
- break;
- default:
- assert(0);
- break;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_IDCT_H_
-#define VP10_COMMON_IDCT_H_
-
-#include <assert.h>
-
-#include "./vpx_config.h"
-#include "vp10/common/common.h"
-#include "vp10/common/enums.h"
-#include "vpx_dsp/inv_txfm.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_ports/mem.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef void (*transform_1d)(const tran_low_t*, tran_low_t*);
-
-typedef struct {
- transform_1d cols, rows; // vertical and horizontal
-} transform_2d;
-
-#if CONFIG_VP9_HIGHBITDEPTH
-typedef void (*highbd_transform_1d)(const tran_low_t*, tran_low_t*, int bd);
-
-typedef struct {
- highbd_transform_1d cols, rows; // vertical and horizontal
-} highbd_transform_2d;
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_iwht4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob);
-void vp10_idct4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob);
-
-void vp10_inv_txfm_add_4x4(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type, int lossless);
-void vp10_inv_txfm_add_8x8(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type);
-void vp10_inv_txfm_add_16x16(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type);
-void vp10_inv_txfm_add_32x32(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, TX_TYPE tx_type);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_iwht4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob, int bd);
-void vp10_highbd_idct4x4_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob, int bd);
-void vp10_highbd_idct8x8_add(const tran_low_t *input, uint8_t *dest, int stride,
- int eob, int bd);
-void vp10_highbd_idct16x16_add(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd);
-void vp10_highbd_idct32x32_add(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd);
-void vp10_highbd_inv_txfm_add_4x4(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd, TX_TYPE tx_type,
- int lossless);
-void vp10_highbd_inv_txfm_add_8x8(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd, TX_TYPE tx_type);
-void vp10_highbd_inv_txfm_add_16x16(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd,
- TX_TYPE tx_type);
-void vp10_highbd_inv_txfm_add_32x32(const tran_low_t *input, uint8_t *dest,
- int stride, int eob, int bd,
- TX_TYPE tx_type);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_IDCT_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-#include "vp10/common/loopfilter.h"
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/reconinter.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/seg_common.h"
-
-// 64 bit masks for left transform size. Each 1 represents a position where
-// we should apply a loop filter across the left border of an 8x8 block
-// boundary.
-//
-// In the case of TX_16X16-> ( in low order byte first we end up with
-// a mask that looks like this
-//
-// 10101010
-// 10101010
-// 10101010
-// 10101010
-// 10101010
-// 10101010
-// 10101010
-// 10101010
-//
-// A loopfilter should be applied to every other 8x8 horizontally.
-static const uint64_t left_64x64_txform_mask[TX_SIZES]= {
- 0xffffffffffffffffULL, // TX_4X4
- 0xffffffffffffffffULL, // TX_8x8
- 0x5555555555555555ULL, // TX_16x16
- 0x1111111111111111ULL, // TX_32x32
-};
-
-// 64 bit masks for above transform size. Each 1 represents a position where
-// we should apply a loop filter across the top border of an 8x8 block
-// boundary.
-//
-// In the case of TX_32x32 -> ( in low order byte first we end up with
-// a mask that looks like this
-//
-// 11111111
-// 00000000
-// 00000000
-// 00000000
-// 11111111
-// 00000000
-// 00000000
-// 00000000
-//
-// A loopfilter should be applied to every other 4 the row vertically.
-static const uint64_t above_64x64_txform_mask[TX_SIZES]= {
- 0xffffffffffffffffULL, // TX_4X4
- 0xffffffffffffffffULL, // TX_8x8
- 0x00ff00ff00ff00ffULL, // TX_16x16
- 0x000000ff000000ffULL, // TX_32x32
-};
-
-// 64 bit masks for prediction sizes (left). Each 1 represents a position
-// where left border of an 8x8 block. These are aligned to the right most
-// appropriate bit, and then shifted into place.
-//
-// In the case of TX_16x32 -> ( low order byte first ) we end up with
-// a mask that looks like this :
-//
-// 10000000
-// 10000000
-// 10000000
-// 10000000
-// 00000000
-// 00000000
-// 00000000
-// 00000000
-static const uint64_t left_prediction_mask[BLOCK_SIZES] = {
- 0x0000000000000001ULL, // BLOCK_4X4,
- 0x0000000000000001ULL, // BLOCK_4X8,
- 0x0000000000000001ULL, // BLOCK_8X4,
- 0x0000000000000001ULL, // BLOCK_8X8,
- 0x0000000000000101ULL, // BLOCK_8X16,
- 0x0000000000000001ULL, // BLOCK_16X8,
- 0x0000000000000101ULL, // BLOCK_16X16,
- 0x0000000001010101ULL, // BLOCK_16X32,
- 0x0000000000000101ULL, // BLOCK_32X16,
- 0x0000000001010101ULL, // BLOCK_32X32,
- 0x0101010101010101ULL, // BLOCK_32X64,
- 0x0000000001010101ULL, // BLOCK_64X32,
- 0x0101010101010101ULL, // BLOCK_64X64
-};
-
-// 64 bit mask to shift and set for each prediction size.
-static const uint64_t above_prediction_mask[BLOCK_SIZES] = {
- 0x0000000000000001ULL, // BLOCK_4X4
- 0x0000000000000001ULL, // BLOCK_4X8
- 0x0000000000000001ULL, // BLOCK_8X4
- 0x0000000000000001ULL, // BLOCK_8X8
- 0x0000000000000001ULL, // BLOCK_8X16,
- 0x0000000000000003ULL, // BLOCK_16X8
- 0x0000000000000003ULL, // BLOCK_16X16
- 0x0000000000000003ULL, // BLOCK_16X32,
- 0x000000000000000fULL, // BLOCK_32X16,
- 0x000000000000000fULL, // BLOCK_32X32,
- 0x000000000000000fULL, // BLOCK_32X64,
- 0x00000000000000ffULL, // BLOCK_64X32,
- 0x00000000000000ffULL, // BLOCK_64X64
-};
-// 64 bit mask to shift and set for each prediction size. A bit is set for
-// each 8x8 block that would be in the left most block of the given block
-// size in the 64x64 block.
-static const uint64_t size_mask[BLOCK_SIZES] = {
- 0x0000000000000001ULL, // BLOCK_4X4
- 0x0000000000000001ULL, // BLOCK_4X8
- 0x0000000000000001ULL, // BLOCK_8X4
- 0x0000000000000001ULL, // BLOCK_8X8
- 0x0000000000000101ULL, // BLOCK_8X16,
- 0x0000000000000003ULL, // BLOCK_16X8
- 0x0000000000000303ULL, // BLOCK_16X16
- 0x0000000003030303ULL, // BLOCK_16X32,
- 0x0000000000000f0fULL, // BLOCK_32X16,
- 0x000000000f0f0f0fULL, // BLOCK_32X32,
- 0x0f0f0f0f0f0f0f0fULL, // BLOCK_32X64,
- 0x00000000ffffffffULL, // BLOCK_64X32,
- 0xffffffffffffffffULL, // BLOCK_64X64
-};
-
-// These are used for masking the left and above borders.
-static const uint64_t left_border = 0x1111111111111111ULL;
-static const uint64_t above_border = 0x000000ff000000ffULL;
-
-// 16 bit masks for uv transform sizes.
-static const uint16_t left_64x64_txform_mask_uv[TX_SIZES]= {
- 0xffff, // TX_4X4
- 0xffff, // TX_8x8
- 0x5555, // TX_16x16
- 0x1111, // TX_32x32
-};
-
-static const uint16_t above_64x64_txform_mask_uv[TX_SIZES]= {
- 0xffff, // TX_4X4
- 0xffff, // TX_8x8
- 0x0f0f, // TX_16x16
- 0x000f, // TX_32x32
-};
-
-// 16 bit left mask to shift and set for each uv prediction size.
-static const uint16_t left_prediction_mask_uv[BLOCK_SIZES] = {
- 0x0001, // BLOCK_4X4,
- 0x0001, // BLOCK_4X8,
- 0x0001, // BLOCK_8X4,
- 0x0001, // BLOCK_8X8,
- 0x0001, // BLOCK_8X16,
- 0x0001, // BLOCK_16X8,
- 0x0001, // BLOCK_16X16,
- 0x0011, // BLOCK_16X32,
- 0x0001, // BLOCK_32X16,
- 0x0011, // BLOCK_32X32,
- 0x1111, // BLOCK_32X64
- 0x0011, // BLOCK_64X32,
- 0x1111, // BLOCK_64X64
-};
-// 16 bit above mask to shift and set for uv each prediction size.
-static const uint16_t above_prediction_mask_uv[BLOCK_SIZES] = {
- 0x0001, // BLOCK_4X4
- 0x0001, // BLOCK_4X8
- 0x0001, // BLOCK_8X4
- 0x0001, // BLOCK_8X8
- 0x0001, // BLOCK_8X16,
- 0x0001, // BLOCK_16X8
- 0x0001, // BLOCK_16X16
- 0x0001, // BLOCK_16X32,
- 0x0003, // BLOCK_32X16,
- 0x0003, // BLOCK_32X32,
- 0x0003, // BLOCK_32X64,
- 0x000f, // BLOCK_64X32,
- 0x000f, // BLOCK_64X64
-};
-
-// 64 bit mask to shift and set for each uv prediction size
-static const uint16_t size_mask_uv[BLOCK_SIZES] = {
- 0x0001, // BLOCK_4X4
- 0x0001, // BLOCK_4X8
- 0x0001, // BLOCK_8X4
- 0x0001, // BLOCK_8X8
- 0x0001, // BLOCK_8X16,
- 0x0001, // BLOCK_16X8
- 0x0001, // BLOCK_16X16
- 0x0011, // BLOCK_16X32,
- 0x0003, // BLOCK_32X16,
- 0x0033, // BLOCK_32X32,
- 0x3333, // BLOCK_32X64,
- 0x00ff, // BLOCK_64X32,
- 0xffff, // BLOCK_64X64
-};
-static const uint16_t left_border_uv = 0x1111;
-static const uint16_t above_border_uv = 0x000f;
-
-static const int mode_lf_lut[MB_MODE_COUNT] = {
- 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // INTRA_MODES
- 1, 1, 0, 1 // INTER_MODES (ZEROMV == 0)
-};
-
-static void update_sharpness(loop_filter_info_n *lfi, int sharpness_lvl) {
- int lvl;
-
- // For each possible value for the loop filter fill out limits
- for (lvl = 0; lvl <= MAX_LOOP_FILTER; lvl++) {
- // Set loop filter parameters that control sharpness.
- int block_inside_limit = lvl >> ((sharpness_lvl > 0) + (sharpness_lvl > 4));
-
- if (sharpness_lvl > 0) {
- if (block_inside_limit > (9 - sharpness_lvl))
- block_inside_limit = (9 - sharpness_lvl);
- }
-
- if (block_inside_limit < 1)
- block_inside_limit = 1;
-
- memset(lfi->lfthr[lvl].lim, block_inside_limit, SIMD_WIDTH);
- memset(lfi->lfthr[lvl].mblim, (2 * (lvl + 2) + block_inside_limit),
- SIMD_WIDTH);
- }
-}
-
-static uint8_t get_filter_level(const loop_filter_info_n *lfi_n,
- const MB_MODE_INFO *mbmi) {
- return lfi_n->lvl[mbmi->segment_id][mbmi->ref_frame[0]]
- [mode_lf_lut[mbmi->mode]];
-}
-
-void vp10_loop_filter_init(VP10_COMMON *cm) {
- loop_filter_info_n *lfi = &cm->lf_info;
- struct loopfilter *lf = &cm->lf;
- int lvl;
-
- // init limits for given sharpness
- update_sharpness(lfi, lf->sharpness_level);
- lf->last_sharpness_level = lf->sharpness_level;
-
- // init hev threshold const vectors
- for (lvl = 0; lvl <= MAX_LOOP_FILTER; lvl++)
- memset(lfi->lfthr[lvl].hev_thr, (lvl >> 4), SIMD_WIDTH);
-}
-
-void vp10_loop_filter_frame_init(VP10_COMMON *cm, int default_filt_lvl) {
- int seg_id;
- // n_shift is the multiplier for lf_deltas
- // the multiplier is 1 for when filter_lvl is between 0 and 31;
- // 2 when filter_lvl is between 32 and 63
- const int scale = 1 << (default_filt_lvl >> 5);
- loop_filter_info_n *const lfi = &cm->lf_info;
- struct loopfilter *const lf = &cm->lf;
- const struct segmentation *const seg = &cm->seg;
-
- // update limits if sharpness has changed
- if (lf->last_sharpness_level != lf->sharpness_level) {
- update_sharpness(lfi, lf->sharpness_level);
- lf->last_sharpness_level = lf->sharpness_level;
- }
-
- for (seg_id = 0; seg_id < MAX_SEGMENTS; seg_id++) {
- int lvl_seg = default_filt_lvl;
- if (segfeature_active(seg, seg_id, SEG_LVL_ALT_LF)) {
- const int data = get_segdata(seg, seg_id, SEG_LVL_ALT_LF);
- lvl_seg = clamp(seg->abs_delta == SEGMENT_ABSDATA ?
- data : default_filt_lvl + data,
- 0, MAX_LOOP_FILTER);
- }
-
- if (!lf->mode_ref_delta_enabled) {
- // we could get rid of this if we assume that deltas are set to
- // zero when not in use; encoder always uses deltas
- memset(lfi->lvl[seg_id], lvl_seg, sizeof(lfi->lvl[seg_id]));
- } else {
- int ref, mode;
- const int intra_lvl = lvl_seg + lf->ref_deltas[INTRA_FRAME] * scale;
- lfi->lvl[seg_id][INTRA_FRAME][0] = clamp(intra_lvl, 0, MAX_LOOP_FILTER);
-
- for (ref = LAST_FRAME; ref < MAX_REF_FRAMES; ++ref) {
- for (mode = 0; mode < MAX_MODE_LF_DELTAS; ++mode) {
- const int inter_lvl = lvl_seg + lf->ref_deltas[ref] * scale
- + lf->mode_deltas[mode] * scale;
- lfi->lvl[seg_id][ref][mode] = clamp(inter_lvl, 0, MAX_LOOP_FILTER);
- }
- }
- }
- }
-}
-
-static void filter_selectively_vert_row2(int subsampling_factor,
- uint8_t *s, int pitch,
- unsigned int mask_16x16_l,
- unsigned int mask_8x8_l,
- unsigned int mask_4x4_l,
- unsigned int mask_4x4_int_l,
- const loop_filter_info_n *lfi_n,
- const uint8_t *lfl) {
- const int mask_shift = subsampling_factor ? 4 : 8;
- const int mask_cutoff = subsampling_factor ? 0xf : 0xff;
- const int lfl_forward = subsampling_factor ? 4 : 8;
-
- unsigned int mask_16x16_0 = mask_16x16_l & mask_cutoff;
- unsigned int mask_8x8_0 = mask_8x8_l & mask_cutoff;
- unsigned int mask_4x4_0 = mask_4x4_l & mask_cutoff;
- unsigned int mask_4x4_int_0 = mask_4x4_int_l & mask_cutoff;
- unsigned int mask_16x16_1 = (mask_16x16_l >> mask_shift) & mask_cutoff;
- unsigned int mask_8x8_1 = (mask_8x8_l >> mask_shift) & mask_cutoff;
- unsigned int mask_4x4_1 = (mask_4x4_l >> mask_shift) & mask_cutoff;
- unsigned int mask_4x4_int_1 = (mask_4x4_int_l >> mask_shift) & mask_cutoff;
- unsigned int mask;
-
- for (mask = mask_16x16_0 | mask_8x8_0 | mask_4x4_0 | mask_4x4_int_0 |
- mask_16x16_1 | mask_8x8_1 | mask_4x4_1 | mask_4x4_int_1;
- mask; mask >>= 1) {
- const loop_filter_thresh *lfi0 = lfi_n->lfthr + *lfl;
- const loop_filter_thresh *lfi1 = lfi_n->lfthr + *(lfl + lfl_forward);
-
- // TODO(yunqingwang): count in loopfilter functions should be removed.
- if (mask & 1) {
- if ((mask_16x16_0 | mask_16x16_1) & 1) {
- if ((mask_16x16_0 & mask_16x16_1) & 1) {
- vpx_lpf_vertical_16_dual(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr);
- } else if (mask_16x16_0 & 1) {
- vpx_lpf_vertical_16(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr);
- } else {
- vpx_lpf_vertical_16(s + 8 *pitch, pitch, lfi1->mblim,
- lfi1->lim, lfi1->hev_thr);
- }
- }
-
- if ((mask_8x8_0 | mask_8x8_1) & 1) {
- if ((mask_8x8_0 & mask_8x8_1) & 1) {
- vpx_lpf_vertical_8_dual(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr);
- } else if (mask_8x8_0 & 1) {
- vpx_lpf_vertical_8(s, pitch, lfi0->mblim, lfi0->lim, lfi0->hev_thr,
- 1);
- } else {
- vpx_lpf_vertical_8(s + 8 * pitch, pitch, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr, 1);
- }
- }
-
- if ((mask_4x4_0 | mask_4x4_1) & 1) {
- if ((mask_4x4_0 & mask_4x4_1) & 1) {
- vpx_lpf_vertical_4_dual(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr);
- } else if (mask_4x4_0 & 1) {
- vpx_lpf_vertical_4(s, pitch, lfi0->mblim, lfi0->lim, lfi0->hev_thr,
- 1);
- } else {
- vpx_lpf_vertical_4(s + 8 * pitch, pitch, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr, 1);
- }
- }
-
- if ((mask_4x4_int_0 | mask_4x4_int_1) & 1) {
- if ((mask_4x4_int_0 & mask_4x4_int_1) & 1) {
- vpx_lpf_vertical_4_dual(s + 4, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr);
- } else if (mask_4x4_int_0 & 1) {
- vpx_lpf_vertical_4(s + 4, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, 1);
- } else {
- vpx_lpf_vertical_4(s + 8 * pitch + 4, pitch, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr, 1);
- }
- }
- }
-
- s += 8;
- lfl += 1;
- mask_16x16_0 >>= 1;
- mask_8x8_0 >>= 1;
- mask_4x4_0 >>= 1;
- mask_4x4_int_0 >>= 1;
- mask_16x16_1 >>= 1;
- mask_8x8_1 >>= 1;
- mask_4x4_1 >>= 1;
- mask_4x4_int_1 >>= 1;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void highbd_filter_selectively_vert_row2(int subsampling_factor,
- uint16_t *s, int pitch,
- unsigned int mask_16x16_l,
- unsigned int mask_8x8_l,
- unsigned int mask_4x4_l,
- unsigned int mask_4x4_int_l,
- const loop_filter_info_n *lfi_n,
- const uint8_t *lfl, int bd) {
- const int mask_shift = subsampling_factor ? 4 : 8;
- const int mask_cutoff = subsampling_factor ? 0xf : 0xff;
- const int lfl_forward = subsampling_factor ? 4 : 8;
-
- unsigned int mask_16x16_0 = mask_16x16_l & mask_cutoff;
- unsigned int mask_8x8_0 = mask_8x8_l & mask_cutoff;
- unsigned int mask_4x4_0 = mask_4x4_l & mask_cutoff;
- unsigned int mask_4x4_int_0 = mask_4x4_int_l & mask_cutoff;
- unsigned int mask_16x16_1 = (mask_16x16_l >> mask_shift) & mask_cutoff;
- unsigned int mask_8x8_1 = (mask_8x8_l >> mask_shift) & mask_cutoff;
- unsigned int mask_4x4_1 = (mask_4x4_l >> mask_shift) & mask_cutoff;
- unsigned int mask_4x4_int_1 = (mask_4x4_int_l >> mask_shift) & mask_cutoff;
- unsigned int mask;
-
- for (mask = mask_16x16_0 | mask_8x8_0 | mask_4x4_0 | mask_4x4_int_0 |
- mask_16x16_1 | mask_8x8_1 | mask_4x4_1 | mask_4x4_int_1;
- mask; mask >>= 1) {
- const loop_filter_thresh *lfi0 = lfi_n->lfthr + *lfl;
- const loop_filter_thresh *lfi1 = lfi_n->lfthr + *(lfl + lfl_forward);
-
- // TODO(yunqingwang): count in loopfilter functions should be removed.
- if (mask & 1) {
- if ((mask_16x16_0 | mask_16x16_1) & 1) {
- if ((mask_16x16_0 & mask_16x16_1) & 1) {
- vpx_highbd_lpf_vertical_16_dual(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, bd);
- } else if (mask_16x16_0 & 1) {
- vpx_highbd_lpf_vertical_16(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, bd);
- } else {
- vpx_highbd_lpf_vertical_16(s + 8 *pitch, pitch, lfi1->mblim,
- lfi1->lim, lfi1->hev_thr, bd);
- }
- }
-
- if ((mask_8x8_0 | mask_8x8_1) & 1) {
- if ((mask_8x8_0 & mask_8x8_1) & 1) {
- vpx_highbd_lpf_vertical_8_dual(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr, bd);
- } else if (mask_8x8_0 & 1) {
- vpx_highbd_lpf_vertical_8(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, 1, bd);
- } else {
- vpx_highbd_lpf_vertical_8(s + 8 * pitch, pitch, lfi1->mblim,
- lfi1->lim, lfi1->hev_thr, 1, bd);
- }
- }
-
- if ((mask_4x4_0 | mask_4x4_1) & 1) {
- if ((mask_4x4_0 & mask_4x4_1) & 1) {
- vpx_highbd_lpf_vertical_4_dual(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr, bd);
- } else if (mask_4x4_0 & 1) {
- vpx_highbd_lpf_vertical_4(s, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, 1, bd);
- } else {
- vpx_highbd_lpf_vertical_4(s + 8 * pitch, pitch, lfi1->mblim,
- lfi1->lim, lfi1->hev_thr, 1, bd);
- }
- }
-
- if ((mask_4x4_int_0 | mask_4x4_int_1) & 1) {
- if ((mask_4x4_int_0 & mask_4x4_int_1) & 1) {
- vpx_highbd_lpf_vertical_4_dual(s + 4, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, lfi1->mblim, lfi1->lim,
- lfi1->hev_thr, bd);
- } else if (mask_4x4_int_0 & 1) {
- vpx_highbd_lpf_vertical_4(s + 4, pitch, lfi0->mblim, lfi0->lim,
- lfi0->hev_thr, 1, bd);
- } else {
- vpx_highbd_lpf_vertical_4(s + 8 * pitch + 4, pitch, lfi1->mblim,
- lfi1->lim, lfi1->hev_thr, 1, bd);
- }
- }
- }
-
- s += 8;
- lfl += 1;
- mask_16x16_0 >>= 1;
- mask_8x8_0 >>= 1;
- mask_4x4_0 >>= 1;
- mask_4x4_int_0 >>= 1;
- mask_16x16_1 >>= 1;
- mask_8x8_1 >>= 1;
- mask_4x4_1 >>= 1;
- mask_4x4_int_1 >>= 1;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void filter_selectively_horiz(uint8_t *s, int pitch,
- unsigned int mask_16x16,
- unsigned int mask_8x8,
- unsigned int mask_4x4,
- unsigned int mask_4x4_int,
- const loop_filter_info_n *lfi_n,
- const uint8_t *lfl) {
- unsigned int mask;
- int count;
-
- for (mask = mask_16x16 | mask_8x8 | mask_4x4 | mask_4x4_int;
- mask; mask >>= count) {
- const loop_filter_thresh *lfi = lfi_n->lfthr + *lfl;
-
- count = 1;
- if (mask & 1) {
- if (mask_16x16 & 1) {
- if ((mask_16x16 & 3) == 3) {
- vpx_lpf_horizontal_16(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 2);
- count = 2;
- } else {
- vpx_lpf_horizontal_16(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1);
- }
- } else if (mask_8x8 & 1) {
- if ((mask_8x8 & 3) == 3) {
- // Next block's thresholds.
- const loop_filter_thresh *lfin = lfi_n->lfthr + *(lfl + 1);
-
- vpx_lpf_horizontal_8_dual(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, lfin->mblim, lfin->lim,
- lfin->hev_thr);
-
- if ((mask_4x4_int & 3) == 3) {
- vpx_lpf_horizontal_4_dual(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr, lfin->mblim,
- lfin->lim, lfin->hev_thr);
- } else {
- if (mask_4x4_int & 1)
- vpx_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1);
- else if (mask_4x4_int & 2)
- vpx_lpf_horizontal_4(s + 8 + 4 * pitch, pitch, lfin->mblim,
- lfin->lim, lfin->hev_thr, 1);
- }
- count = 2;
- } else {
- vpx_lpf_horizontal_8(s, pitch, lfi->mblim, lfi->lim, lfi->hev_thr, 1);
-
- if (mask_4x4_int & 1)
- vpx_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1);
- }
- } else if (mask_4x4 & 1) {
- if ((mask_4x4 & 3) == 3) {
- // Next block's thresholds.
- const loop_filter_thresh *lfin = lfi_n->lfthr + *(lfl + 1);
-
- vpx_lpf_horizontal_4_dual(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, lfin->mblim, lfin->lim,
- lfin->hev_thr);
- if ((mask_4x4_int & 3) == 3) {
- vpx_lpf_horizontal_4_dual(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr, lfin->mblim,
- lfin->lim, lfin->hev_thr);
- } else {
- if (mask_4x4_int & 1)
- vpx_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1);
- else if (mask_4x4_int & 2)
- vpx_lpf_horizontal_4(s + 8 + 4 * pitch, pitch, lfin->mblim,
- lfin->lim, lfin->hev_thr, 1);
- }
- count = 2;
- } else {
- vpx_lpf_horizontal_4(s, pitch, lfi->mblim, lfi->lim, lfi->hev_thr, 1);
-
- if (mask_4x4_int & 1)
- vpx_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1);
- }
- } else if (mask_4x4_int & 1) {
- vpx_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1);
- }
- }
- s += 8 * count;
- lfl += count;
- mask_16x16 >>= count;
- mask_8x8 >>= count;
- mask_4x4 >>= count;
- mask_4x4_int >>= count;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void highbd_filter_selectively_horiz(uint16_t *s, int pitch,
- unsigned int mask_16x16,
- unsigned int mask_8x8,
- unsigned int mask_4x4,
- unsigned int mask_4x4_int,
- const loop_filter_info_n *lfi_n,
- const uint8_t *lfl, int bd) {
- unsigned int mask;
- int count;
-
- for (mask = mask_16x16 | mask_8x8 | mask_4x4 | mask_4x4_int;
- mask; mask >>= count) {
- const loop_filter_thresh *lfi = lfi_n->lfthr + *lfl;
-
- count = 1;
- if (mask & 1) {
- if (mask_16x16 & 1) {
- if ((mask_16x16 & 3) == 3) {
- vpx_highbd_lpf_horizontal_16(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 2, bd);
- count = 2;
- } else {
- vpx_highbd_lpf_horizontal_16(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
- }
- } else if (mask_8x8 & 1) {
- if ((mask_8x8 & 3) == 3) {
- // Next block's thresholds.
- const loop_filter_thresh *lfin = lfi_n->lfthr + *(lfl + 1);
-
- vpx_highbd_lpf_horizontal_8_dual(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, lfin->mblim, lfin->lim,
- lfin->hev_thr, bd);
-
- if ((mask_4x4_int & 3) == 3) {
- vpx_highbd_lpf_horizontal_4_dual(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr,
- lfin->mblim, lfin->lim,
- lfin->hev_thr, bd);
- } else {
- if (mask_4x4_int & 1) {
- vpx_highbd_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr, 1, bd);
- } else if (mask_4x4_int & 2) {
- vpx_highbd_lpf_horizontal_4(s + 8 + 4 * pitch, pitch, lfin->mblim,
- lfin->lim, lfin->hev_thr, 1, bd);
- }
- }
- count = 2;
- } else {
- vpx_highbd_lpf_horizontal_8(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
-
- if (mask_4x4_int & 1) {
- vpx_highbd_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr, 1, bd);
- }
- }
- } else if (mask_4x4 & 1) {
- if ((mask_4x4 & 3) == 3) {
- // Next block's thresholds.
- const loop_filter_thresh *lfin = lfi_n->lfthr + *(lfl + 1);
-
- vpx_highbd_lpf_horizontal_4_dual(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, lfin->mblim, lfin->lim,
- lfin->hev_thr, bd);
- if ((mask_4x4_int & 3) == 3) {
- vpx_highbd_lpf_horizontal_4_dual(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr,
- lfin->mblim, lfin->lim,
- lfin->hev_thr, bd);
- } else {
- if (mask_4x4_int & 1) {
- vpx_highbd_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr, 1, bd);
- } else if (mask_4x4_int & 2) {
- vpx_highbd_lpf_horizontal_4(s + 8 + 4 * pitch, pitch, lfin->mblim,
- lfin->lim, lfin->hev_thr, 1, bd);
- }
- }
- count = 2;
- } else {
- vpx_highbd_lpf_horizontal_4(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
-
- if (mask_4x4_int & 1) {
- vpx_highbd_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim,
- lfi->lim, lfi->hev_thr, 1, bd);
- }
- }
- } else if (mask_4x4_int & 1) {
- vpx_highbd_lpf_horizontal_4(s + 4 * pitch, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
- }
- }
- s += 8 * count;
- lfl += count;
- mask_16x16 >>= count;
- mask_8x8 >>= count;
- mask_4x4 >>= count;
- mask_4x4_int >>= count;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-// This function ors into the current lfm structure, where to do loop
-// filters for the specific mi we are looking at. It uses information
-// including the block_size_type (32x16, 32x32, etc.), the transform size,
-// whether there were any coefficients encoded, and the loop filter strength
-// block we are currently looking at. Shift is used to position the
-// 1's we produce.
-// TODO(JBB) Need another function for different resolution color..
-static void build_masks(const loop_filter_info_n *const lfi_n,
- const MODE_INFO *mi, const int shift_y,
- const int shift_uv,
- LOOP_FILTER_MASK *lfm) {
- const MB_MODE_INFO *mbmi = &mi->mbmi;
- const BLOCK_SIZE block_size = mbmi->sb_type;
- const TX_SIZE tx_size_y = mbmi->tx_size;
- const TX_SIZE tx_size_uv = get_uv_tx_size_impl(tx_size_y, block_size, 1, 1);
- const int filter_level = get_filter_level(lfi_n, mbmi);
- uint64_t *const left_y = &lfm->left_y[tx_size_y];
- uint64_t *const above_y = &lfm->above_y[tx_size_y];
- uint64_t *const int_4x4_y = &lfm->int_4x4_y;
- uint16_t *const left_uv = &lfm->left_uv[tx_size_uv];
- uint16_t *const above_uv = &lfm->above_uv[tx_size_uv];
-#if CONFIG_MISC_FIXES
- uint16_t *const int_4x4_uv = &lfm->left_int_4x4_uv;
-#else
- uint16_t *const int_4x4_uv = &lfm->int_4x4_uv;
-#endif
- int i;
-
- // If filter level is 0 we don't loop filter.
- if (!filter_level) {
- return;
- } else {
- const int w = num_8x8_blocks_wide_lookup[block_size];
- const int h = num_8x8_blocks_high_lookup[block_size];
- int index = shift_y;
- for (i = 0; i < h; i++) {
- memset(&lfm->lfl_y[index], filter_level, w);
- index += 8;
- }
- }
-
- // These set 1 in the current block size for the block size edges.
- // For instance if the block size is 32x16, we'll set:
- // above = 1111
- // 0000
- // and
- // left = 1000
- // = 1000
- // NOTE : In this example the low bit is left most ( 1000 ) is stored as
- // 1, not 8...
- //
- // U and V set things on a 16 bit scale.
- //
- *above_y |= above_prediction_mask[block_size] << shift_y;
- *above_uv |= above_prediction_mask_uv[block_size] << shift_uv;
- *left_y |= left_prediction_mask[block_size] << shift_y;
- *left_uv |= left_prediction_mask_uv[block_size] << shift_uv;
-
- // If the block has no coefficients and is not intra we skip applying
- // the loop filter on block edges.
-#if CONFIG_MISC_FIXES
- if ((mbmi->skip || mbmi->has_no_coeffs) && is_inter_block(mbmi))
- return;
-#else
- if (mbmi->skip && is_inter_block(mbmi))
- return;
-#endif
-
- // Here we are adding a mask for the transform size. The transform
- // size mask is set to be correct for a 64x64 prediction block size. We
- // mask to match the size of the block we are working on and then shift it
- // into place..
- *above_y |= (size_mask[block_size] &
- above_64x64_txform_mask[tx_size_y]) << shift_y;
- *above_uv |= (size_mask_uv[block_size] &
- above_64x64_txform_mask_uv[tx_size_uv]) << shift_uv;
-
- *left_y |= (size_mask[block_size] &
- left_64x64_txform_mask[tx_size_y]) << shift_y;
- *left_uv |= (size_mask_uv[block_size] &
- left_64x64_txform_mask_uv[tx_size_uv]) << shift_uv;
-
- // Here we are trying to determine what to do with the internal 4x4 block
- // boundaries. These differ from the 4x4 boundaries on the outside edge of
- // an 8x8 in that the internal ones can be skipped and don't depend on
- // the prediction block size.
- if (tx_size_y == TX_4X4)
- *int_4x4_y |= (size_mask[block_size] & 0xffffffffffffffffULL) << shift_y;
-
- if (tx_size_uv == TX_4X4)
- *int_4x4_uv |= (size_mask_uv[block_size] & 0xffff) << shift_uv;
-}
-
-// This function does the same thing as the one above with the exception that
-// it only affects the y masks. It exists because for blocks < 16x16 in size,
-// we only update u and v masks on the first block.
-static void build_y_mask(const loop_filter_info_n *const lfi_n,
- const MODE_INFO *mi, const int shift_y,
- LOOP_FILTER_MASK *lfm) {
- const MB_MODE_INFO *mbmi = &mi->mbmi;
- const BLOCK_SIZE block_size = mbmi->sb_type;
- const TX_SIZE tx_size_y = mbmi->tx_size;
- const int filter_level = get_filter_level(lfi_n, mbmi);
- uint64_t *const left_y = &lfm->left_y[tx_size_y];
- uint64_t *const above_y = &lfm->above_y[tx_size_y];
- uint64_t *const int_4x4_y = &lfm->int_4x4_y;
- int i;
-
- if (!filter_level) {
- return;
- } else {
- const int w = num_8x8_blocks_wide_lookup[block_size];
- const int h = num_8x8_blocks_high_lookup[block_size];
- int index = shift_y;
- for (i = 0; i < h; i++) {
- memset(&lfm->lfl_y[index], filter_level, w);
- index += 8;
- }
- }
-
- *above_y |= above_prediction_mask[block_size] << shift_y;
- *left_y |= left_prediction_mask[block_size] << shift_y;
-
-#if CONFIG_MISC_FIXES
- if ((mbmi->skip || mbmi->has_no_coeffs) && is_inter_block(mbmi))
- return;
-#else
- if (mbmi->skip && is_inter_block(mbmi))
- return;
-#endif
-
- *above_y |= (size_mask[block_size] &
- above_64x64_txform_mask[tx_size_y]) << shift_y;
-
- *left_y |= (size_mask[block_size] &
- left_64x64_txform_mask[tx_size_y]) << shift_y;
-
- if (tx_size_y == TX_4X4)
- *int_4x4_y |= (size_mask[block_size] & 0xffffffffffffffffULL) << shift_y;
-}
-
-// This function sets up the bit masks for the entire 64x64 region represented
-// by mi_row, mi_col.
-// TODO(JBB): This function only works for yv12.
-void vp10_setup_mask(VP10_COMMON *const cm, const int mi_row, const int mi_col,
- MODE_INFO **mi, const int mode_info_stride,
- LOOP_FILTER_MASK *lfm) {
- int idx_32, idx_16, idx_8;
- const loop_filter_info_n *const lfi_n = &cm->lf_info;
- MODE_INFO **mip = mi;
- MODE_INFO **mip2 = mi;
-
- // These are offsets to the next mi in the 64x64 block. It is what gets
- // added to the mi ptr as we go through each loop. It helps us to avoid
- // setting up special row and column counters for each index. The last step
- // brings us out back to the starting position.
- const int offset_32[] = {4, (mode_info_stride << 2) - 4, 4,
- -(mode_info_stride << 2) - 4};
- const int offset_16[] = {2, (mode_info_stride << 1) - 2, 2,
- -(mode_info_stride << 1) - 2};
- const int offset[] = {1, mode_info_stride - 1, 1, -mode_info_stride - 1};
-
- // Following variables represent shifts to position the current block
- // mask over the appropriate block. A shift of 36 to the left will move
- // the bits for the final 32 by 32 block in the 64x64 up 4 rows and left
- // 4 rows to the appropriate spot.
- const int shift_32_y[] = {0, 4, 32, 36};
- const int shift_16_y[] = {0, 2, 16, 18};
- const int shift_8_y[] = {0, 1, 8, 9};
- const int shift_32_uv[] = {0, 2, 8, 10};
- const int shift_16_uv[] = {0, 1, 4, 5};
- int i;
- const int max_rows = (mi_row + MI_BLOCK_SIZE > cm->mi_rows ?
- cm->mi_rows - mi_row : MI_BLOCK_SIZE);
- const int max_cols = (mi_col + MI_BLOCK_SIZE > cm->mi_cols ?
- cm->mi_cols - mi_col : MI_BLOCK_SIZE);
-
- vp10_zero(*lfm);
- assert(mip[0] != NULL);
-
- // TODO(jimbankoski): Try moving most of the following code into decode
- // loop and storing lfm in the mbmi structure so that we don't have to go
- // through the recursive loop structure multiple times.
- switch (mip[0]->mbmi.sb_type) {
- case BLOCK_64X64:
- build_masks(lfi_n, mip[0] , 0, 0, lfm);
- break;
- case BLOCK_64X32:
- build_masks(lfi_n, mip[0], 0, 0, lfm);
- mip2 = mip + mode_info_stride * 4;
- if (4 >= max_rows)
- break;
- build_masks(lfi_n, mip2[0], 32, 8, lfm);
- break;
- case BLOCK_32X64:
- build_masks(lfi_n, mip[0], 0, 0, lfm);
- mip2 = mip + 4;
- if (4 >= max_cols)
- break;
- build_masks(lfi_n, mip2[0], 4, 2, lfm);
- break;
- default:
- for (idx_32 = 0; idx_32 < 4; mip += offset_32[idx_32], ++idx_32) {
- const int shift_y = shift_32_y[idx_32];
- const int shift_uv = shift_32_uv[idx_32];
- const int mi_32_col_offset = ((idx_32 & 1) << 2);
- const int mi_32_row_offset = ((idx_32 >> 1) << 2);
- if (mi_32_col_offset >= max_cols || mi_32_row_offset >= max_rows)
- continue;
- switch (mip[0]->mbmi.sb_type) {
- case BLOCK_32X32:
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- break;
- case BLOCK_32X16:
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- if (mi_32_row_offset + 2 >= max_rows)
- continue;
- mip2 = mip + mode_info_stride * 2;
- build_masks(lfi_n, mip2[0], shift_y + 16, shift_uv + 4, lfm);
- break;
- case BLOCK_16X32:
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- if (mi_32_col_offset + 2 >= max_cols)
- continue;
- mip2 = mip + 2;
- build_masks(lfi_n, mip2[0], shift_y + 2, shift_uv + 1, lfm);
- break;
- default:
- for (idx_16 = 0; idx_16 < 4; mip += offset_16[idx_16], ++idx_16) {
- const int shift_y = shift_32_y[idx_32] + shift_16_y[idx_16];
- const int shift_uv = shift_32_uv[idx_32] + shift_16_uv[idx_16];
- const int mi_16_col_offset = mi_32_col_offset +
- ((idx_16 & 1) << 1);
- const int mi_16_row_offset = mi_32_row_offset +
- ((idx_16 >> 1) << 1);
-
- if (mi_16_col_offset >= max_cols || mi_16_row_offset >= max_rows)
- continue;
-
- switch (mip[0]->mbmi.sb_type) {
- case BLOCK_16X16:
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- break;
- case BLOCK_16X8:
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- if (mi_16_row_offset + 1 >= max_rows)
- continue;
- mip2 = mip + mode_info_stride;
- build_y_mask(lfi_n, mip2[0], shift_y+8, lfm);
- break;
- case BLOCK_8X16:
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- if (mi_16_col_offset +1 >= max_cols)
- continue;
- mip2 = mip + 1;
- build_y_mask(lfi_n, mip2[0], shift_y+1, lfm);
- break;
- default: {
- const int shift_y = shift_32_y[idx_32] +
- shift_16_y[idx_16] +
- shift_8_y[0];
- build_masks(lfi_n, mip[0], shift_y, shift_uv, lfm);
- mip += offset[0];
- for (idx_8 = 1; idx_8 < 4; mip += offset[idx_8], ++idx_8) {
- const int shift_y = shift_32_y[idx_32] +
- shift_16_y[idx_16] +
- shift_8_y[idx_8];
- const int mi_8_col_offset = mi_16_col_offset +
- ((idx_8 & 1));
- const int mi_8_row_offset = mi_16_row_offset +
- ((idx_8 >> 1));
-
- if (mi_8_col_offset >= max_cols ||
- mi_8_row_offset >= max_rows)
- continue;
- build_y_mask(lfi_n, mip[0], shift_y, lfm);
- }
- break;
- }
- }
- }
- break;
- }
- }
- break;
- }
- // The largest loopfilter we have is 16x16 so we use the 16x16 mask
- // for 32x32 transforms also.
- lfm->left_y[TX_16X16] |= lfm->left_y[TX_32X32];
- lfm->above_y[TX_16X16] |= lfm->above_y[TX_32X32];
- lfm->left_uv[TX_16X16] |= lfm->left_uv[TX_32X32];
- lfm->above_uv[TX_16X16] |= lfm->above_uv[TX_32X32];
-
- // We do at least 8 tap filter on every 32x32 even if the transform size
- // is 4x4. So if the 4x4 is set on a border pixel add it to the 8x8 and
- // remove it from the 4x4.
- lfm->left_y[TX_8X8] |= lfm->left_y[TX_4X4] & left_border;
- lfm->left_y[TX_4X4] &= ~left_border;
- lfm->above_y[TX_8X8] |= lfm->above_y[TX_4X4] & above_border;
- lfm->above_y[TX_4X4] &= ~above_border;
- lfm->left_uv[TX_8X8] |= lfm->left_uv[TX_4X4] & left_border_uv;
- lfm->left_uv[TX_4X4] &= ~left_border_uv;
- lfm->above_uv[TX_8X8] |= lfm->above_uv[TX_4X4] & above_border_uv;
- lfm->above_uv[TX_4X4] &= ~above_border_uv;
-
- // We do some special edge handling.
- if (mi_row + MI_BLOCK_SIZE > cm->mi_rows) {
- const uint64_t rows = cm->mi_rows - mi_row;
-
- // Each pixel inside the border gets a 1,
- const uint64_t mask_y = (((uint64_t) 1 << (rows << 3)) - 1);
- const uint16_t mask_uv = (((uint16_t) 1 << (((rows + 1) >> 1) << 2)) - 1);
-
- // Remove values completely outside our border.
- for (i = 0; i < TX_32X32; i++) {
- lfm->left_y[i] &= mask_y;
- lfm->above_y[i] &= mask_y;
- lfm->left_uv[i] &= mask_uv;
- lfm->above_uv[i] &= mask_uv;
- }
- lfm->int_4x4_y &= mask_y;
-#if CONFIG_MISC_FIXES
- lfm->above_int_4x4_uv = lfm->left_int_4x4_uv & mask_uv;
-#else
- lfm->int_4x4_uv &= mask_uv;
-#endif
-
- // We don't apply a wide loop filter on the last uv block row. If set
- // apply the shorter one instead.
- if (rows == 1) {
- lfm->above_uv[TX_8X8] |= lfm->above_uv[TX_16X16];
- lfm->above_uv[TX_16X16] = 0;
- }
- if (rows == 5) {
- lfm->above_uv[TX_8X8] |= lfm->above_uv[TX_16X16] & 0xff00;
- lfm->above_uv[TX_16X16] &= ~(lfm->above_uv[TX_16X16] & 0xff00);
- }
- }
-
- if (mi_col + MI_BLOCK_SIZE > cm->mi_cols) {
- const uint64_t columns = cm->mi_cols - mi_col;
-
- // Each pixel inside the border gets a 1, the multiply copies the border
- // to where we need it.
- const uint64_t mask_y = (((1 << columns) - 1)) * 0x0101010101010101ULL;
- const uint16_t mask_uv = ((1 << ((columns + 1) >> 1)) - 1) * 0x1111;
-
- // Internal edges are not applied on the last column of the image so
- // we mask 1 more for the internal edges
- const uint16_t mask_uv_int = ((1 << (columns >> 1)) - 1) * 0x1111;
-
- // Remove the bits outside the image edge.
- for (i = 0; i < TX_32X32; i++) {
- lfm->left_y[i] &= mask_y;
- lfm->above_y[i] &= mask_y;
- lfm->left_uv[i] &= mask_uv;
- lfm->above_uv[i] &= mask_uv;
- }
- lfm->int_4x4_y &= mask_y;
-#if CONFIG_MISC_FIXES
- lfm->left_int_4x4_uv &= mask_uv_int;
-#else
- lfm->int_4x4_uv &= mask_uv_int;
-#endif
-
- // We don't apply a wide loop filter on the last uv column. If set
- // apply the shorter one instead.
- if (columns == 1) {
- lfm->left_uv[TX_8X8] |= lfm->left_uv[TX_16X16];
- lfm->left_uv[TX_16X16] = 0;
- }
- if (columns == 5) {
- lfm->left_uv[TX_8X8] |= (lfm->left_uv[TX_16X16] & 0xcccc);
- lfm->left_uv[TX_16X16] &= ~(lfm->left_uv[TX_16X16] & 0xcccc);
- }
- }
- // We don't apply a loop filter on the first column in the image, mask that
- // out.
- if (mi_col == 0) {
- for (i = 0; i < TX_32X32; i++) {
- lfm->left_y[i] &= 0xfefefefefefefefeULL;
- lfm->left_uv[i] &= 0xeeee;
- }
- }
-
- // Assert if we try to apply 2 different loop filters at the same position.
- assert(!(lfm->left_y[TX_16X16] & lfm->left_y[TX_8X8]));
- assert(!(lfm->left_y[TX_16X16] & lfm->left_y[TX_4X4]));
- assert(!(lfm->left_y[TX_8X8] & lfm->left_y[TX_4X4]));
- assert(!(lfm->int_4x4_y & lfm->left_y[TX_16X16]));
- assert(!(lfm->left_uv[TX_16X16]&lfm->left_uv[TX_8X8]));
- assert(!(lfm->left_uv[TX_16X16] & lfm->left_uv[TX_4X4]));
- assert(!(lfm->left_uv[TX_8X8] & lfm->left_uv[TX_4X4]));
-#if CONFIG_MISC_FIXES
- assert(!(lfm->left_int_4x4_uv & lfm->left_uv[TX_16X16]));
-#else
- assert(!(lfm->int_4x4_uv & lfm->left_uv[TX_16X16]));
-#endif
- assert(!(lfm->above_y[TX_16X16] & lfm->above_y[TX_8X8]));
- assert(!(lfm->above_y[TX_16X16] & lfm->above_y[TX_4X4]));
- assert(!(lfm->above_y[TX_8X8] & lfm->above_y[TX_4X4]));
- assert(!(lfm->int_4x4_y & lfm->above_y[TX_16X16]));
- assert(!(lfm->above_uv[TX_16X16] & lfm->above_uv[TX_8X8]));
- assert(!(lfm->above_uv[TX_16X16] & lfm->above_uv[TX_4X4]));
- assert(!(lfm->above_uv[TX_8X8] & lfm->above_uv[TX_4X4]));
-#if CONFIG_MISC_FIXES
- assert(!(lfm->above_int_4x4_uv & lfm->above_uv[TX_16X16]));
-#else
- assert(!(lfm->int_4x4_uv & lfm->above_uv[TX_16X16]));
-#endif
-}
-
-static void filter_selectively_vert(uint8_t *s, int pitch,
- unsigned int mask_16x16,
- unsigned int mask_8x8,
- unsigned int mask_4x4,
- unsigned int mask_4x4_int,
- const loop_filter_info_n *lfi_n,
- const uint8_t *lfl) {
- unsigned int mask;
-
- for (mask = mask_16x16 | mask_8x8 | mask_4x4 | mask_4x4_int;
- mask; mask >>= 1) {
- const loop_filter_thresh *lfi = lfi_n->lfthr + *lfl;
-
- if (mask & 1) {
- if (mask_16x16 & 1) {
- vpx_lpf_vertical_16(s, pitch, lfi->mblim, lfi->lim, lfi->hev_thr);
- } else if (mask_8x8 & 1) {
- vpx_lpf_vertical_8(s, pitch, lfi->mblim, lfi->lim, lfi->hev_thr, 1);
- } else if (mask_4x4 & 1) {
- vpx_lpf_vertical_4(s, pitch, lfi->mblim, lfi->lim, lfi->hev_thr, 1);
- }
- }
- if (mask_4x4_int & 1)
- vpx_lpf_vertical_4(s + 4, pitch, lfi->mblim, lfi->lim, lfi->hev_thr, 1);
- s += 8;
- lfl += 1;
- mask_16x16 >>= 1;
- mask_8x8 >>= 1;
- mask_4x4 >>= 1;
- mask_4x4_int >>= 1;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void highbd_filter_selectively_vert(uint16_t *s, int pitch,
- unsigned int mask_16x16,
- unsigned int mask_8x8,
- unsigned int mask_4x4,
- unsigned int mask_4x4_int,
- const loop_filter_info_n *lfi_n,
- const uint8_t *lfl, int bd) {
- unsigned int mask;
-
- for (mask = mask_16x16 | mask_8x8 | mask_4x4 | mask_4x4_int;
- mask; mask >>= 1) {
- const loop_filter_thresh *lfi = lfi_n->lfthr + *lfl;
-
- if (mask & 1) {
- if (mask_16x16 & 1) {
- vpx_highbd_lpf_vertical_16(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, bd);
- } else if (mask_8x8 & 1) {
- vpx_highbd_lpf_vertical_8(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
- } else if (mask_4x4 & 1) {
- vpx_highbd_lpf_vertical_4(s, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
- }
- }
- if (mask_4x4_int & 1)
- vpx_highbd_lpf_vertical_4(s + 4, pitch, lfi->mblim, lfi->lim,
- lfi->hev_thr, 1, bd);
- s += 8;
- lfl += 1;
- mask_16x16 >>= 1;
- mask_8x8 >>= 1;
- mask_4x4 >>= 1;
- mask_4x4_int >>= 1;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_filter_block_plane_non420(VP10_COMMON *cm,
- struct macroblockd_plane *plane,
- MODE_INFO **mi_8x8,
- int mi_row, int mi_col) {
- const int ss_x = plane->subsampling_x;
- const int ss_y = plane->subsampling_y;
- const int row_step = 1 << ss_y;
- const int col_step = 1 << ss_x;
- const int row_step_stride = cm->mi_stride * row_step;
- struct buf_2d *const dst = &plane->dst;
- uint8_t* const dst0 = dst->buf;
- unsigned int mask_16x16[MI_BLOCK_SIZE] = {0};
- unsigned int mask_8x8[MI_BLOCK_SIZE] = {0};
- unsigned int mask_4x4[MI_BLOCK_SIZE] = {0};
- unsigned int mask_4x4_int[MI_BLOCK_SIZE] = {0};
- uint8_t lfl[MI_BLOCK_SIZE * MI_BLOCK_SIZE];
- int r, c;
-
- for (r = 0; r < MI_BLOCK_SIZE && mi_row + r < cm->mi_rows; r += row_step) {
- unsigned int mask_16x16_c = 0;
- unsigned int mask_8x8_c = 0;
- unsigned int mask_4x4_c = 0;
- unsigned int border_mask;
-
- // Determine the vertical edges that need filtering
- for (c = 0; c < MI_BLOCK_SIZE && mi_col + c < cm->mi_cols; c += col_step) {
- const MODE_INFO *mi = mi_8x8[c];
- const BLOCK_SIZE sb_type = mi[0].mbmi.sb_type;
- const int skip_this = mi[0].mbmi.skip && is_inter_block(&mi[0].mbmi);
- // left edge of current unit is block/partition edge -> no skip
- const int block_edge_left = (num_4x4_blocks_wide_lookup[sb_type] > 1) ?
- !(c & (num_8x8_blocks_wide_lookup[sb_type] - 1)) : 1;
- const int skip_this_c = skip_this && !block_edge_left;
- // top edge of current unit is block/partition edge -> no skip
- const int block_edge_above = (num_4x4_blocks_high_lookup[sb_type] > 1) ?
- !(r & (num_8x8_blocks_high_lookup[sb_type] - 1)) : 1;
- const int skip_this_r = skip_this && !block_edge_above;
- const TX_SIZE tx_size = (plane->plane_type == PLANE_TYPE_UV)
- ? get_uv_tx_size(&mi[0].mbmi, plane)
- : mi[0].mbmi.tx_size;
- const int skip_border_4x4_c = ss_x && mi_col + c == cm->mi_cols - 1;
- const int skip_border_4x4_r = ss_y && mi_row + r == cm->mi_rows - 1;
-
- // Filter level can vary per MI
- if (!(lfl[(r << 3) + (c >> ss_x)] =
- get_filter_level(&cm->lf_info, &mi[0].mbmi)))
- continue;
-
- // Build masks based on the transform size of each block
- if (tx_size == TX_32X32) {
- if (!skip_this_c && ((c >> ss_x) & 3) == 0) {
- if (!skip_border_4x4_c)
- mask_16x16_c |= 1 << (c >> ss_x);
- else
- mask_8x8_c |= 1 << (c >> ss_x);
- }
- if (!skip_this_r && ((r >> ss_y) & 3) == 0) {
- if (!skip_border_4x4_r)
- mask_16x16[r] |= 1 << (c >> ss_x);
- else
- mask_8x8[r] |= 1 << (c >> ss_x);
- }
- } else if (tx_size == TX_16X16) {
- if (!skip_this_c && ((c >> ss_x) & 1) == 0) {
- if (!skip_border_4x4_c)
- mask_16x16_c |= 1 << (c >> ss_x);
- else
- mask_8x8_c |= 1 << (c >> ss_x);
- }
- if (!skip_this_r && ((r >> ss_y) & 1) == 0) {
- if (!skip_border_4x4_r)
- mask_16x16[r] |= 1 << (c >> ss_x);
- else
- mask_8x8[r] |= 1 << (c >> ss_x);
- }
- } else {
- // force 8x8 filtering on 32x32 boundaries
- if (!skip_this_c) {
- if (tx_size == TX_8X8 || ((c >> ss_x) & 3) == 0)
- mask_8x8_c |= 1 << (c >> ss_x);
- else
- mask_4x4_c |= 1 << (c >> ss_x);
- }
-
- if (!skip_this_r) {
- if (tx_size == TX_8X8 || ((r >> ss_y) & 3) == 0)
- mask_8x8[r] |= 1 << (c >> ss_x);
- else
- mask_4x4[r] |= 1 << (c >> ss_x);
- }
-
- if (!skip_this && tx_size < TX_8X8 && !skip_border_4x4_c)
- mask_4x4_int[r] |= 1 << (c >> ss_x);
- }
- }
-
- // Disable filtering on the leftmost column
- border_mask = ~(mi_col == 0);
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- highbd_filter_selectively_vert(CONVERT_TO_SHORTPTR(dst->buf),
- dst->stride,
- mask_16x16_c & border_mask,
- mask_8x8_c & border_mask,
- mask_4x4_c & border_mask,
- mask_4x4_int[r],
- &cm->lf_info, &lfl[r << 3],
- (int)cm->bit_depth);
- } else {
- filter_selectively_vert(dst->buf, dst->stride,
- mask_16x16_c & border_mask,
- mask_8x8_c & border_mask,
- mask_4x4_c & border_mask,
- mask_4x4_int[r],
- &cm->lf_info, &lfl[r << 3]);
- }
-#else
- filter_selectively_vert(dst->buf, dst->stride,
- mask_16x16_c & border_mask,
- mask_8x8_c & border_mask,
- mask_4x4_c & border_mask,
- mask_4x4_int[r],
- &cm->lf_info, &lfl[r << 3]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- dst->buf += 8 * dst->stride;
- mi_8x8 += row_step_stride;
- }
-
- // Now do horizontal pass
- dst->buf = dst0;
- for (r = 0; r < MI_BLOCK_SIZE && mi_row + r < cm->mi_rows; r += row_step) {
- const int skip_border_4x4_r = ss_y && mi_row + r == cm->mi_rows - 1;
- const unsigned int mask_4x4_int_r = skip_border_4x4_r ? 0 : mask_4x4_int[r];
-
- unsigned int mask_16x16_r;
- unsigned int mask_8x8_r;
- unsigned int mask_4x4_r;
-
- if (mi_row + r == 0) {
- mask_16x16_r = 0;
- mask_8x8_r = 0;
- mask_4x4_r = 0;
- } else {
- mask_16x16_r = mask_16x16[r];
- mask_8x8_r = mask_8x8[r];
- mask_4x4_r = mask_4x4[r];
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- highbd_filter_selectively_horiz(CONVERT_TO_SHORTPTR(dst->buf),
- dst->stride,
- mask_16x16_r,
- mask_8x8_r,
- mask_4x4_r,
- mask_4x4_int_r,
- &cm->lf_info, &lfl[r << 3],
- (int)cm->bit_depth);
- } else {
- filter_selectively_horiz(dst->buf, dst->stride,
- mask_16x16_r,
- mask_8x8_r,
- mask_4x4_r,
- mask_4x4_int_r,
- &cm->lf_info, &lfl[r << 3]);
- }
-#else
- filter_selectively_horiz(dst->buf, dst->stride,
- mask_16x16_r,
- mask_8x8_r,
- mask_4x4_r,
- mask_4x4_int_r,
- &cm->lf_info, &lfl[r << 3]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- dst->buf += 8 * dst->stride;
- }
-}
-
-void vp10_filter_block_plane_ss00(VP10_COMMON *const cm,
- struct macroblockd_plane *const plane,
- int mi_row,
- LOOP_FILTER_MASK *lfm) {
- struct buf_2d *const dst = &plane->dst;
- uint8_t *const dst0 = dst->buf;
- int r;
- uint64_t mask_16x16 = lfm->left_y[TX_16X16];
- uint64_t mask_8x8 = lfm->left_y[TX_8X8];
- uint64_t mask_4x4 = lfm->left_y[TX_4X4];
- uint64_t mask_4x4_int = lfm->int_4x4_y;
-
- assert(plane->subsampling_x == 0 && plane->subsampling_y == 0);
-
- // Vertical pass: do 2 rows at one time
- for (r = 0; r < MI_BLOCK_SIZE && mi_row + r < cm->mi_rows; r += 2) {
- unsigned int mask_16x16_l = mask_16x16 & 0xffff;
- unsigned int mask_8x8_l = mask_8x8 & 0xffff;
- unsigned int mask_4x4_l = mask_4x4 & 0xffff;
- unsigned int mask_4x4_int_l = mask_4x4_int & 0xffff;
-
-// Disable filtering on the leftmost column.
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- highbd_filter_selectively_vert_row2(
- plane->subsampling_x, CONVERT_TO_SHORTPTR(dst->buf), dst->stride,
- mask_16x16_l, mask_8x8_l, mask_4x4_l, mask_4x4_int_l, &cm->lf_info,
- &lfm->lfl_y[r << 3], (int)cm->bit_depth);
- } else {
- filter_selectively_vert_row2(
- plane->subsampling_x, dst->buf, dst->stride, mask_16x16_l, mask_8x8_l,
- mask_4x4_l, mask_4x4_int_l, &cm->lf_info, &lfm->lfl_y[r << 3]);
- }
-#else
- filter_selectively_vert_row2(
- plane->subsampling_x, dst->buf, dst->stride, mask_16x16_l, mask_8x8_l,
- mask_4x4_l, mask_4x4_int_l, &cm->lf_info, &lfm->lfl_y[r << 3]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- dst->buf += 16 * dst->stride;
- mask_16x16 >>= 16;
- mask_8x8 >>= 16;
- mask_4x4 >>= 16;
- mask_4x4_int >>= 16;
- }
-
- // Horizontal pass
- dst->buf = dst0;
- mask_16x16 = lfm->above_y[TX_16X16];
- mask_8x8 = lfm->above_y[TX_8X8];
- mask_4x4 = lfm->above_y[TX_4X4];
- mask_4x4_int = lfm->int_4x4_y;
-
- for (r = 0; r < MI_BLOCK_SIZE && mi_row + r < cm->mi_rows; r++) {
- unsigned int mask_16x16_r;
- unsigned int mask_8x8_r;
- unsigned int mask_4x4_r;
-
- if (mi_row + r == 0) {
- mask_16x16_r = 0;
- mask_8x8_r = 0;
- mask_4x4_r = 0;
- } else {
- mask_16x16_r = mask_16x16 & 0xff;
- mask_8x8_r = mask_8x8 & 0xff;
- mask_4x4_r = mask_4x4 & 0xff;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- highbd_filter_selectively_horiz(
- CONVERT_TO_SHORTPTR(dst->buf), dst->stride, mask_16x16_r, mask_8x8_r,
- mask_4x4_r, mask_4x4_int & 0xff, &cm->lf_info, &lfm->lfl_y[r << 3],
- (int)cm->bit_depth);
- } else {
- filter_selectively_horiz(dst->buf, dst->stride, mask_16x16_r, mask_8x8_r,
- mask_4x4_r, mask_4x4_int & 0xff, &cm->lf_info,
- &lfm->lfl_y[r << 3]);
- }
-#else
- filter_selectively_horiz(dst->buf, dst->stride, mask_16x16_r, mask_8x8_r,
- mask_4x4_r, mask_4x4_int & 0xff, &cm->lf_info,
- &lfm->lfl_y[r << 3]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- dst->buf += 8 * dst->stride;
- mask_16x16 >>= 8;
- mask_8x8 >>= 8;
- mask_4x4 >>= 8;
- mask_4x4_int >>= 8;
- }
-}
-
-void vp10_filter_block_plane_ss11(VP10_COMMON *const cm,
- struct macroblockd_plane *const plane,
- int mi_row,
- LOOP_FILTER_MASK *lfm) {
- struct buf_2d *const dst = &plane->dst;
- uint8_t *const dst0 = dst->buf;
- int r, c;
-
- uint16_t mask_16x16 = lfm->left_uv[TX_16X16];
- uint16_t mask_8x8 = lfm->left_uv[TX_8X8];
- uint16_t mask_4x4 = lfm->left_uv[TX_4X4];
-#if CONFIG_MISC_FIXES
- uint16_t mask_4x4_int = lfm->left_int_4x4_uv;
-#else
- uint16_t mask_4x4_int = lfm->int_4x4_uv;
-#endif
-
- assert(plane->subsampling_x == 1 && plane->subsampling_y == 1);
-
- // Vertical pass: do 2 rows at one time
- for (r = 0; r < MI_BLOCK_SIZE && mi_row + r < cm->mi_rows; r += 4) {
- if (plane->plane_type == 1) {
- for (c = 0; c < (MI_BLOCK_SIZE >> 1); c++) {
- lfm->lfl_uv[(r << 1) + c] = lfm->lfl_y[(r << 3) + (c << 1)];
- lfm->lfl_uv[((r + 2) << 1) + c] = lfm->lfl_y[((r + 2) << 3) + (c << 1)];
- }
- }
-
- {
- unsigned int mask_16x16_l = mask_16x16 & 0xff;
- unsigned int mask_8x8_l = mask_8x8 & 0xff;
- unsigned int mask_4x4_l = mask_4x4 & 0xff;
- unsigned int mask_4x4_int_l = mask_4x4_int & 0xff;
-
-// Disable filtering on the leftmost column.
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- highbd_filter_selectively_vert_row2(
- plane->subsampling_x, CONVERT_TO_SHORTPTR(dst->buf), dst->stride,
- mask_16x16_l, mask_8x8_l, mask_4x4_l, mask_4x4_int_l, &cm->lf_info,
- &lfm->lfl_uv[r << 1], (int)cm->bit_depth);
- } else {
- filter_selectively_vert_row2(
- plane->subsampling_x, dst->buf, dst->stride,
- mask_16x16_l, mask_8x8_l, mask_4x4_l, mask_4x4_int_l, &cm->lf_info,
- &lfm->lfl_uv[r << 1]);
- }
-#else
- filter_selectively_vert_row2(
- plane->subsampling_x, dst->buf, dst->stride,
- mask_16x16_l, mask_8x8_l, mask_4x4_l, mask_4x4_int_l, &cm->lf_info,
- &lfm->lfl_uv[r << 1]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- dst->buf += 16 * dst->stride;
- mask_16x16 >>= 8;
- mask_8x8 >>= 8;
- mask_4x4 >>= 8;
- mask_4x4_int >>= 8;
- }
- }
-
- // Horizontal pass
- dst->buf = dst0;
- mask_16x16 = lfm->above_uv[TX_16X16];
- mask_8x8 = lfm->above_uv[TX_8X8];
- mask_4x4 = lfm->above_uv[TX_4X4];
-#if CONFIG_MISC_FIXES
- mask_4x4_int = lfm->above_int_4x4_uv;
-#else
- mask_4x4_int = lfm->int_4x4_uv;
-#endif
-
- for (r = 0; r < MI_BLOCK_SIZE && mi_row + r < cm->mi_rows; r += 2) {
- const int skip_border_4x4_r = mi_row + r == cm->mi_rows - 1;
- const unsigned int mask_4x4_int_r =
- skip_border_4x4_r ? 0 : (mask_4x4_int & 0xf);
- unsigned int mask_16x16_r;
- unsigned int mask_8x8_r;
- unsigned int mask_4x4_r;
-
- if (mi_row + r == 0) {
- mask_16x16_r = 0;
- mask_8x8_r = 0;
- mask_4x4_r = 0;
- } else {
- mask_16x16_r = mask_16x16 & 0xf;
- mask_8x8_r = mask_8x8 & 0xf;
- mask_4x4_r = mask_4x4 & 0xf;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- highbd_filter_selectively_horiz(CONVERT_TO_SHORTPTR(dst->buf),
- dst->stride, mask_16x16_r, mask_8x8_r,
- mask_4x4_r, mask_4x4_int_r, &cm->lf_info,
- &lfm->lfl_uv[r << 1], (int)cm->bit_depth);
- } else {
- filter_selectively_horiz(dst->buf, dst->stride, mask_16x16_r, mask_8x8_r,
- mask_4x4_r, mask_4x4_int_r, &cm->lf_info,
- &lfm->lfl_uv[r << 1]);
- }
-#else
- filter_selectively_horiz(dst->buf, dst->stride, mask_16x16_r, mask_8x8_r,
- mask_4x4_r, mask_4x4_int_r, &cm->lf_info,
- &lfm->lfl_uv[r << 1]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- dst->buf += 8 * dst->stride;
- mask_16x16 >>= 4;
- mask_8x8 >>= 4;
- mask_4x4 >>= 4;
- mask_4x4_int >>= 4;
- }
-}
-
-void vp10_loop_filter_rows(YV12_BUFFER_CONFIG *frame_buffer,
- VP10_COMMON *cm,
- struct macroblockd_plane planes[MAX_MB_PLANE],
- int start, int stop, int y_only) {
- const int num_planes = y_only ? 1 : MAX_MB_PLANE;
- enum lf_path path;
- LOOP_FILTER_MASK lfm;
- int mi_row, mi_col;
-
- if (y_only)
- path = LF_PATH_444;
- else if (planes[1].subsampling_y == 1 && planes[1].subsampling_x == 1)
- path = LF_PATH_420;
- else if (planes[1].subsampling_y == 0 && planes[1].subsampling_x == 0)
- path = LF_PATH_444;
- else
- path = LF_PATH_SLOW;
-
- for (mi_row = start; mi_row < stop; mi_row += MI_BLOCK_SIZE) {
- MODE_INFO **mi = cm->mi_grid_visible + mi_row * cm->mi_stride;
-
- for (mi_col = 0; mi_col < cm->mi_cols; mi_col += MI_BLOCK_SIZE) {
- int plane;
-
- vp10_setup_dst_planes(planes, frame_buffer, mi_row, mi_col);
-
- // TODO(JBB): Make setup_mask work for non 420.
- vp10_setup_mask(cm, mi_row, mi_col, mi + mi_col, cm->mi_stride,
- &lfm);
-
- vp10_filter_block_plane_ss00(cm, &planes[0], mi_row, &lfm);
- for (plane = 1; plane < num_planes; ++plane) {
- switch (path) {
- case LF_PATH_420:
- vp10_filter_block_plane_ss11(cm, &planes[plane], mi_row, &lfm);
- break;
- case LF_PATH_444:
- vp10_filter_block_plane_ss00(cm, &planes[plane], mi_row, &lfm);
- break;
- case LF_PATH_SLOW:
- vp10_filter_block_plane_non420(cm, &planes[plane], mi + mi_col,
- mi_row, mi_col);
- break;
- }
- }
- }
- }
-}
-
-void vp10_loop_filter_frame(YV12_BUFFER_CONFIG *frame,
- VP10_COMMON *cm, MACROBLOCKD *xd,
- int frame_filter_level,
- int y_only, int partial_frame) {
- int start_mi_row, end_mi_row, mi_rows_to_filter;
- if (!frame_filter_level) return;
- start_mi_row = 0;
- mi_rows_to_filter = cm->mi_rows;
- if (partial_frame && cm->mi_rows > 8) {
- start_mi_row = cm->mi_rows >> 1;
- start_mi_row &= 0xfffffff8;
- mi_rows_to_filter = VPXMAX(cm->mi_rows / 8, 8);
- }
- end_mi_row = start_mi_row + mi_rows_to_filter;
- vp10_loop_filter_frame_init(cm, frame_filter_level);
- vp10_loop_filter_rows(frame, cm, xd->plane,
- start_mi_row, end_mi_row,
- y_only);
-}
-
-void vp10_loop_filter_data_reset(
- LFWorkerData *lf_data, YV12_BUFFER_CONFIG *frame_buffer,
- struct VP10Common *cm,
- const struct macroblockd_plane planes[MAX_MB_PLANE]) {
- lf_data->frame_buffer = frame_buffer;
- lf_data->cm = cm;
- lf_data->start = 0;
- lf_data->stop = 0;
- lf_data->y_only = 0;
- memcpy(lf_data->planes, planes, sizeof(lf_data->planes));
-}
-
-int vp10_loop_filter_worker(LFWorkerData *const lf_data, void *unused) {
- (void)unused;
- vp10_loop_filter_rows(lf_data->frame_buffer, lf_data->cm, lf_data->planes,
- lf_data->start, lf_data->stop, lf_data->y_only);
- return 1;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_LOOPFILTER_H_
-#define VP10_COMMON_LOOPFILTER_H_
-
-#include "vpx_ports/mem.h"
-#include "./vpx_config.h"
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/seg_common.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MAX_LOOP_FILTER 63
-#define MAX_SHARPNESS 7
-
-#define SIMD_WIDTH 16
-
-#define MAX_MODE_LF_DELTAS 2
-
-enum lf_path {
- LF_PATH_420,
- LF_PATH_444,
- LF_PATH_SLOW,
-};
-
-struct loopfilter {
- int filter_level;
-
- int sharpness_level;
- int last_sharpness_level;
-
- uint8_t mode_ref_delta_enabled;
- uint8_t mode_ref_delta_update;
-
- // 0 = Intra, Last, GF, ARF
- signed char ref_deltas[MAX_REF_FRAMES];
- signed char last_ref_deltas[MAX_REF_FRAMES];
-
- // 0 = ZERO_MV, MV
- signed char mode_deltas[MAX_MODE_LF_DELTAS];
- signed char last_mode_deltas[MAX_MODE_LF_DELTAS];
-};
-
-// Need to align this structure so when it is declared and
-// passed it can be loaded into vector registers.
-typedef struct {
- DECLARE_ALIGNED(SIMD_WIDTH, uint8_t, mblim[SIMD_WIDTH]);
- DECLARE_ALIGNED(SIMD_WIDTH, uint8_t, lim[SIMD_WIDTH]);
- DECLARE_ALIGNED(SIMD_WIDTH, uint8_t, hev_thr[SIMD_WIDTH]);
-} loop_filter_thresh;
-
-typedef struct {
- loop_filter_thresh lfthr[MAX_LOOP_FILTER + 1];
- uint8_t lvl[MAX_SEGMENTS][MAX_REF_FRAMES][MAX_MODE_LF_DELTAS];
-} loop_filter_info_n;
-
-// This structure holds bit masks for all 8x8 blocks in a 64x64 region.
-// Each 1 bit represents a position in which we want to apply the loop filter.
-// Left_ entries refer to whether we apply a filter on the border to the
-// left of the block. Above_ entries refer to whether or not to apply a
-// filter on the above border. Int_ entries refer to whether or not to
-// apply borders on the 4x4 edges within the 8x8 block that each bit
-// represents.
-// Since each transform is accompanied by a potentially different type of
-// loop filter there is a different entry in the array for each transform size.
-typedef struct {
- uint64_t left_y[TX_SIZES];
- uint64_t above_y[TX_SIZES];
- uint64_t int_4x4_y;
- uint16_t left_uv[TX_SIZES];
- uint16_t above_uv[TX_SIZES];
-#if CONFIG_MISC_FIXES
- uint16_t left_int_4x4_uv;
- uint16_t above_int_4x4_uv;
-#else
- uint16_t int_4x4_uv;
-#endif
- uint8_t lfl_y[64];
- uint8_t lfl_uv[16];
-} LOOP_FILTER_MASK;
-
-/* assorted loopfilter functions which get used elsewhere */
-struct VP10Common;
-struct macroblockd;
-struct VP9LfSyncData;
-
-// This function sets up the bit masks for the entire 64x64 region represented
-// by mi_row, mi_col.
-void vp10_setup_mask(struct VP10Common *const cm,
- const int mi_row, const int mi_col,
- MODE_INFO **mi_8x8, const int mode_info_stride,
- LOOP_FILTER_MASK *lfm);
-
-void vp10_filter_block_plane_ss00(struct VP10Common *const cm,
- struct macroblockd_plane *const plane,
- int mi_row,
- LOOP_FILTER_MASK *lfm);
-
-void vp10_filter_block_plane_ss11(struct VP10Common *const cm,
- struct macroblockd_plane *const plane,
- int mi_row,
- LOOP_FILTER_MASK *lfm);
-
-void vp10_filter_block_plane_non420(struct VP10Common *cm,
- struct macroblockd_plane *plane,
- MODE_INFO **mi_8x8,
- int mi_row, int mi_col);
-
-void vp10_loop_filter_init(struct VP10Common *cm);
-
-// Update the loop filter for the current frame.
-// This should be called before vp10_loop_filter_rows(), vp10_loop_filter_frame()
-// calls this function directly.
-void vp10_loop_filter_frame_init(struct VP10Common *cm, int default_filt_lvl);
-
-void vp10_loop_filter_frame(YV12_BUFFER_CONFIG *frame,
- struct VP10Common *cm,
- struct macroblockd *mbd,
- int filter_level,
- int y_only, int partial_frame);
-
-// Apply the loop filter to [start, stop) macro block rows in frame_buffer.
-void vp10_loop_filter_rows(YV12_BUFFER_CONFIG *frame_buffer,
- struct VP10Common *cm,
- struct macroblockd_plane planes[MAX_MB_PLANE],
- int start, int stop, int y_only);
-
-typedef struct LoopFilterWorkerData {
- YV12_BUFFER_CONFIG *frame_buffer;
- struct VP10Common *cm;
- struct macroblockd_plane planes[MAX_MB_PLANE];
-
- int start;
- int stop;
- int y_only;
-} LFWorkerData;
-
-void vp10_loop_filter_data_reset(
- LFWorkerData *lf_data, YV12_BUFFER_CONFIG *frame_buffer,
- struct VP10Common *cm, const struct macroblockd_plane planes[MAX_MB_PLANE]);
-
-// Operates on the rows described by 'lf_data'.
-int vp10_loop_filter_worker(LFWorkerData *const lf_data, void *unused);
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_LOOPFILTER_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_config.h"
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "./vpx_scale_rtcd.h"
-
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/postproc.h"
-
-// TODO(jackychen): Replace this function with SSE2 code. There is
-// one SSE2 implementation in vp8, so will consider how to share it
-// between vp8 and vp9.
-static void filter_by_weight(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- int block_size, int src_weight) {
- const int dst_weight = (1 << MFQE_PRECISION) - src_weight;
- const int rounding_bit = 1 << (MFQE_PRECISION - 1);
- int r, c;
-
- for (r = 0; r < block_size; r++) {
- for (c = 0; c < block_size; c++) {
- dst[c] = (src[c] * src_weight + dst[c] * dst_weight + rounding_bit)
- >> MFQE_PRECISION;
- }
- src += src_stride;
- dst += dst_stride;
- }
-}
-
-void vp10_filter_by_weight8x8_c(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride, int src_weight) {
- filter_by_weight(src, src_stride, dst, dst_stride, 8, src_weight);
-}
-
-void vp10_filter_by_weight16x16_c(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- int src_weight) {
- filter_by_weight(src, src_stride, dst, dst_stride, 16, src_weight);
-}
-
-static void filter_by_weight32x32(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride, int weight) {
- vp10_filter_by_weight16x16(src, src_stride, dst, dst_stride, weight);
- vp10_filter_by_weight16x16(src + 16, src_stride, dst + 16, dst_stride,
- weight);
- vp10_filter_by_weight16x16(src + src_stride * 16, src_stride,
- dst + dst_stride * 16, dst_stride, weight);
- vp10_filter_by_weight16x16(src + src_stride * 16 + 16, src_stride,
- dst + dst_stride * 16 + 16, dst_stride, weight);
-}
-
-static void filter_by_weight64x64(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride, int weight) {
- filter_by_weight32x32(src, src_stride, dst, dst_stride, weight);
- filter_by_weight32x32(src + 32, src_stride, dst + 32,
- dst_stride, weight);
- filter_by_weight32x32(src + src_stride * 32, src_stride,
- dst + dst_stride * 32, dst_stride, weight);
- filter_by_weight32x32(src + src_stride * 32 + 32, src_stride,
- dst + dst_stride * 32 + 32, dst_stride, weight);
-}
-
-static void apply_ifactor(const uint8_t *y, int y_stride, uint8_t *yd,
- int yd_stride, const uint8_t *u, const uint8_t *v,
- int uv_stride, uint8_t *ud, uint8_t *vd,
- int uvd_stride, BLOCK_SIZE block_size,
- int weight) {
- if (block_size == BLOCK_16X16) {
- vp10_filter_by_weight16x16(y, y_stride, yd, yd_stride, weight);
- vp10_filter_by_weight8x8(u, uv_stride, ud, uvd_stride, weight);
- vp10_filter_by_weight8x8(v, uv_stride, vd, uvd_stride, weight);
- } else if (block_size == BLOCK_32X32) {
- filter_by_weight32x32(y, y_stride, yd, yd_stride, weight);
- vp10_filter_by_weight16x16(u, uv_stride, ud, uvd_stride, weight);
- vp10_filter_by_weight16x16(v, uv_stride, vd, uvd_stride, weight);
- } else if (block_size == BLOCK_64X64) {
- filter_by_weight64x64(y, y_stride, yd, yd_stride, weight);
- filter_by_weight32x32(u, uv_stride, ud, uvd_stride, weight);
- filter_by_weight32x32(v, uv_stride, vd, uvd_stride, weight);
- }
-}
-
-// TODO(jackychen): Determine whether replace it with assembly code.
-static void copy_mem8x8(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride) {
- int r;
- for (r = 0; r < 8; r++) {
- memcpy(dst, src, 8);
- src += src_stride;
- dst += dst_stride;
- }
-}
-
-static void copy_mem16x16(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride) {
- int r;
- for (r = 0; r < 16; r++) {
- memcpy(dst, src, 16);
- src += src_stride;
- dst += dst_stride;
- }
-}
-
-static void copy_mem32x32(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride) {
- copy_mem16x16(src, src_stride, dst, dst_stride);
- copy_mem16x16(src + 16, src_stride, dst + 16, dst_stride);
- copy_mem16x16(src + src_stride * 16, src_stride,
- dst + dst_stride * 16, dst_stride);
- copy_mem16x16(src + src_stride * 16 + 16, src_stride,
- dst + dst_stride * 16 + 16, dst_stride);
-}
-
-void copy_mem64x64(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride) {
- copy_mem32x32(src, src_stride, dst, dst_stride);
- copy_mem32x32(src + 32, src_stride, dst + 32, dst_stride);
- copy_mem32x32(src + src_stride * 32, src_stride,
- dst + src_stride * 32, dst_stride);
- copy_mem32x32(src + src_stride * 32 + 32, src_stride,
- dst + src_stride * 32 + 32, dst_stride);
-}
-
-static void copy_block(const uint8_t *y, const uint8_t *u, const uint8_t *v,
- int y_stride, int uv_stride, uint8_t *yd, uint8_t *ud,
- uint8_t *vd, int yd_stride, int uvd_stride,
- BLOCK_SIZE bs) {
- if (bs == BLOCK_16X16) {
- copy_mem16x16(y, y_stride, yd, yd_stride);
- copy_mem8x8(u, uv_stride, ud, uvd_stride);
- copy_mem8x8(v, uv_stride, vd, uvd_stride);
- } else if (bs == BLOCK_32X32) {
- copy_mem32x32(y, y_stride, yd, yd_stride);
- copy_mem16x16(u, uv_stride, ud, uvd_stride);
- copy_mem16x16(v, uv_stride, vd, uvd_stride);
- } else {
- copy_mem64x64(y, y_stride, yd, yd_stride);
- copy_mem32x32(u, uv_stride, ud, uvd_stride);
- copy_mem32x32(v, uv_stride, vd, uvd_stride);
- }
-}
-
-static void get_thr(BLOCK_SIZE bs, int qdiff, int *sad_thr, int *vdiff_thr) {
- const int adj = qdiff >> MFQE_PRECISION;
- if (bs == BLOCK_16X16) {
- *sad_thr = 7 + adj;
- } else if (bs == BLOCK_32X32) {
- *sad_thr = 6 + adj;
- } else { // BLOCK_64X64
- *sad_thr = 5 + adj;
- }
- *vdiff_thr = 125 + qdiff;
-}
-
-static void mfqe_block(BLOCK_SIZE bs, const uint8_t *y, const uint8_t *u,
- const uint8_t *v, int y_stride, int uv_stride,
- uint8_t *yd, uint8_t *ud, uint8_t *vd, int yd_stride,
- int uvd_stride, int qdiff) {
- int sad, sad_thr, vdiff, vdiff_thr;
- uint32_t sse;
-
- get_thr(bs, qdiff, &sad_thr, &vdiff_thr);
-
- if (bs == BLOCK_16X16) {
- vdiff = (vpx_variance16x16(y, y_stride, yd, yd_stride, &sse) + 128) >> 8;
- sad = (vpx_sad16x16(y, y_stride, yd, yd_stride) + 128) >> 8;
- } else if (bs == BLOCK_32X32) {
- vdiff = (vpx_variance32x32(y, y_stride, yd, yd_stride, &sse) + 512) >> 10;
- sad = (vpx_sad32x32(y, y_stride, yd, yd_stride) + 512) >> 10;
- } else /* if (bs == BLOCK_64X64) */ {
- vdiff = (vpx_variance64x64(y, y_stride, yd, yd_stride, &sse) + 2048) >> 12;
- sad = (vpx_sad64x64(y, y_stride, yd, yd_stride) + 2048) >> 12;
- }
-
- // vdiff > sad * 3 means vdiff should not be too small, otherwise,
- // it might be a lighting change in smooth area. When there is a
- // lighting change in smooth area, it is dangerous to do MFQE.
- if (sad > 1 && vdiff > sad * 3) {
- const int weight = 1 << MFQE_PRECISION;
- int ifactor = weight * sad * vdiff / (sad_thr * vdiff_thr);
- // When ifactor equals weight, no MFQE is done.
- if (ifactor > weight) {
- ifactor = weight;
- }
- apply_ifactor(y, y_stride, yd, yd_stride, u, v, uv_stride, ud, vd,
- uvd_stride, bs, ifactor);
- } else {
- // Copy the block from current frame (i.e., no mfqe is done).
- copy_block(y, u, v, y_stride, uv_stride, yd, ud, vd,
- yd_stride, uvd_stride, bs);
- }
-}
-
-static int mfqe_decision(MODE_INFO *mi, BLOCK_SIZE cur_bs) {
- // Check the motion in current block(for inter frame),
- // or check the motion in the correlated block in last frame (for keyframe).
- const int mv_len_square = mi->mbmi.mv[0].as_mv.row *
- mi->mbmi.mv[0].as_mv.row +
- mi->mbmi.mv[0].as_mv.col *
- mi->mbmi.mv[0].as_mv.col;
- const int mv_threshold = 100;
- return mi->mbmi.mode >= NEARESTMV && // Not an intra block
- cur_bs >= BLOCK_16X16 &&
- mv_len_square <= mv_threshold;
-}
-
-// Process each partiton in a super block, recursively.
-static void mfqe_partition(VP10_COMMON *cm, MODE_INFO *mi, BLOCK_SIZE bs,
- const uint8_t *y, const uint8_t *u,
- const uint8_t *v, int y_stride, int uv_stride,
- uint8_t *yd, uint8_t *ud, uint8_t *vd,
- int yd_stride, int uvd_stride) {
- int mi_offset, y_offset, uv_offset;
- const BLOCK_SIZE cur_bs = mi->mbmi.sb_type;
- const int qdiff = cm->base_qindex - cm->postproc_state.last_base_qindex;
- const int bsl = b_width_log2_lookup[bs];
- PARTITION_TYPE partition = partition_lookup[bsl][cur_bs];
- const BLOCK_SIZE subsize = get_subsize(bs, partition);
-
- if (cur_bs < BLOCK_8X8) {
- // If there are blocks smaller than 8x8, it must be on the boundary.
- return;
- }
- // No MFQE on blocks smaller than 16x16
- if (bs == BLOCK_16X16) {
- partition = PARTITION_NONE;
- }
- if (bs == BLOCK_64X64) {
- mi_offset = 4;
- y_offset = 32;
- uv_offset = 16;
- } else {
- mi_offset = 2;
- y_offset = 16;
- uv_offset = 8;
- }
- switch (partition) {
- BLOCK_SIZE mfqe_bs, bs_tmp;
- case PARTITION_HORZ:
- if (bs == BLOCK_64X64) {
- mfqe_bs = BLOCK_64X32;
- bs_tmp = BLOCK_32X32;
- } else {
- mfqe_bs = BLOCK_32X16;
- bs_tmp = BLOCK_16X16;
- }
- if (mfqe_decision(mi, mfqe_bs)) {
- // Do mfqe on the first square partition.
- mfqe_block(bs_tmp, y, u, v, y_stride, uv_stride,
- yd, ud, vd, yd_stride, uvd_stride, qdiff);
- // Do mfqe on the second square partition.
- mfqe_block(bs_tmp, y + y_offset, u + uv_offset, v + uv_offset,
- y_stride, uv_stride, yd + y_offset, ud + uv_offset,
- vd + uv_offset, yd_stride, uvd_stride, qdiff);
- }
- if (mfqe_decision(mi + mi_offset * cm->mi_stride, mfqe_bs)) {
- // Do mfqe on the first square partition.
- mfqe_block(bs_tmp, y + y_offset * y_stride, u + uv_offset * uv_stride,
- v + uv_offset * uv_stride, y_stride, uv_stride,
- yd + y_offset * yd_stride, ud + uv_offset * uvd_stride,
- vd + uv_offset * uvd_stride, yd_stride, uvd_stride, qdiff);
- // Do mfqe on the second square partition.
- mfqe_block(bs_tmp, y + y_offset * y_stride + y_offset,
- u + uv_offset * uv_stride + uv_offset,
- v + uv_offset * uv_stride + uv_offset, y_stride,
- uv_stride, yd + y_offset * yd_stride + y_offset,
- ud + uv_offset * uvd_stride + uv_offset,
- vd + uv_offset * uvd_stride + uv_offset,
- yd_stride, uvd_stride, qdiff);
- }
- break;
- case PARTITION_VERT:
- if (bs == BLOCK_64X64) {
- mfqe_bs = BLOCK_32X64;
- bs_tmp = BLOCK_32X32;
- } else {
- mfqe_bs = BLOCK_16X32;
- bs_tmp = BLOCK_16X16;
- }
- if (mfqe_decision(mi, mfqe_bs)) {
- // Do mfqe on the first square partition.
- mfqe_block(bs_tmp, y, u, v, y_stride, uv_stride,
- yd, ud, vd, yd_stride, uvd_stride, qdiff);
- // Do mfqe on the second square partition.
- mfqe_block(bs_tmp, y + y_offset * y_stride, u + uv_offset * uv_stride,
- v + uv_offset * uv_stride, y_stride, uv_stride,
- yd + y_offset * yd_stride, ud + uv_offset * uvd_stride,
- vd + uv_offset * uvd_stride, yd_stride, uvd_stride, qdiff);
- }
- if (mfqe_decision(mi + mi_offset, mfqe_bs)) {
- // Do mfqe on the first square partition.
- mfqe_block(bs_tmp, y + y_offset, u + uv_offset, v + uv_offset,
- y_stride, uv_stride, yd + y_offset, ud + uv_offset,
- vd + uv_offset, yd_stride, uvd_stride, qdiff);
- // Do mfqe on the second square partition.
- mfqe_block(bs_tmp, y + y_offset * y_stride + y_offset,
- u + uv_offset * uv_stride + uv_offset,
- v + uv_offset * uv_stride + uv_offset, y_stride,
- uv_stride, yd + y_offset * yd_stride + y_offset,
- ud + uv_offset * uvd_stride + uv_offset,
- vd + uv_offset * uvd_stride + uv_offset,
- yd_stride, uvd_stride, qdiff);
- }
- break;
- case PARTITION_NONE:
- if (mfqe_decision(mi, cur_bs)) {
- // Do mfqe on this partition.
- mfqe_block(cur_bs, y, u, v, y_stride, uv_stride,
- yd, ud, vd, yd_stride, uvd_stride, qdiff);
- } else {
- // Copy the block from current frame(i.e., no mfqe is done).
- copy_block(y, u, v, y_stride, uv_stride, yd, ud, vd,
- yd_stride, uvd_stride, bs);
- }
- break;
- case PARTITION_SPLIT:
- // Recursion on four square partitions, e.g. if bs is 64X64,
- // then look into four 32X32 blocks in it.
- mfqe_partition(cm, mi, subsize, y, u, v, y_stride, uv_stride, yd, ud, vd,
- yd_stride, uvd_stride);
- mfqe_partition(cm, mi + mi_offset, subsize, y + y_offset, u + uv_offset,
- v + uv_offset, y_stride, uv_stride, yd + y_offset,
- ud + uv_offset, vd + uv_offset, yd_stride, uvd_stride);
- mfqe_partition(cm, mi + mi_offset * cm->mi_stride, subsize,
- y + y_offset * y_stride, u + uv_offset * uv_stride,
- v + uv_offset * uv_stride, y_stride, uv_stride,
- yd + y_offset * yd_stride, ud + uv_offset * uvd_stride,
- vd + uv_offset * uvd_stride, yd_stride, uvd_stride);
- mfqe_partition(cm, mi + mi_offset * cm->mi_stride + mi_offset,
- subsize, y + y_offset * y_stride + y_offset,
- u + uv_offset * uv_stride + uv_offset,
- v + uv_offset * uv_stride + uv_offset, y_stride,
- uv_stride, yd + y_offset * yd_stride + y_offset,
- ud + uv_offset * uvd_stride + uv_offset,
- vd + uv_offset * uvd_stride + uv_offset,
- yd_stride, uvd_stride);
- break;
- default:
- assert(0);
- }
-}
-
-void vp10_mfqe(VP10_COMMON *cm) {
- int mi_row, mi_col;
- // Current decoded frame.
- const YV12_BUFFER_CONFIG *show = cm->frame_to_show;
- // Last decoded frame and will store the MFQE result.
- YV12_BUFFER_CONFIG *dest = &cm->post_proc_buffer;
- // Loop through each super block.
- for (mi_row = 0; mi_row < cm->mi_rows; mi_row += MI_BLOCK_SIZE) {
- for (mi_col = 0; mi_col < cm->mi_cols; mi_col += MI_BLOCK_SIZE) {
- MODE_INFO *mi;
- MODE_INFO *mi_local = cm->mi + (mi_row * cm->mi_stride + mi_col);
- // Motion Info in last frame.
- MODE_INFO *mi_prev = cm->postproc_state.prev_mi +
- (mi_row * cm->mi_stride + mi_col);
- const uint32_t y_stride = show->y_stride;
- const uint32_t uv_stride = show->uv_stride;
- const uint32_t yd_stride = dest->y_stride;
- const uint32_t uvd_stride = dest->uv_stride;
- const uint32_t row_offset_y = mi_row << 3;
- const uint32_t row_offset_uv = mi_row << 2;
- const uint32_t col_offset_y = mi_col << 3;
- const uint32_t col_offset_uv = mi_col << 2;
- const uint8_t *y = show->y_buffer + row_offset_y * y_stride +
- col_offset_y;
- const uint8_t *u = show->u_buffer + row_offset_uv * uv_stride +
- col_offset_uv;
- const uint8_t *v = show->v_buffer + row_offset_uv * uv_stride +
- col_offset_uv;
- uint8_t *yd = dest->y_buffer + row_offset_y * yd_stride + col_offset_y;
- uint8_t *ud = dest->u_buffer + row_offset_uv * uvd_stride +
- col_offset_uv;
- uint8_t *vd = dest->v_buffer + row_offset_uv * uvd_stride +
- col_offset_uv;
- if (frame_is_intra_only(cm)) {
- mi = mi_prev;
- } else {
- mi = mi_local;
- }
- mfqe_partition(cm, mi, BLOCK_64X64, y, u, v, y_stride, uv_stride, yd, ud,
- vd, yd_stride, uvd_stride);
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_MFQE_H_
-#define VP10_COMMON_MFQE_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// Multiframe Quality Enhancement.
-// The aim for MFQE is to replace pixel blocks in the current frame with
-// the correlated pixel blocks (with higher quality) in the last frame.
-// The replacement can only be taken in stationary blocks by checking
-// the motion of the blocks and other conditions such as the SAD of
-// the current block and correlated block, the variance of the block
-// difference, etc.
-void vp10_mfqe(struct VP10Common *cm);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_MFQE_H_
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <stdio.h>
-
-#include "./vpx_config.h"
-#include "./vp10_rtcd.h"
-#include "vp10/common/common.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/idct.h"
-#include "vpx_dsp/mips/inv_txfm_dspr2.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_ports/mem.h"
-
-#if HAVE_DSPR2
-void vp10_iht16x16_256_add_dspr2(const int16_t *input, uint8_t *dest,
- int pitch, int tx_type) {
- int i, j;
- DECLARE_ALIGNED(32, int16_t, out[16 * 16]);
- int16_t *outptr = out;
- int16_t temp_out[16];
- uint32_t pos = 45;
-
- /* bit positon for extract from acc */
- __asm__ __volatile__ (
- "wrdsp %[pos], 1 \n\t"
- :
- : [pos] "r" (pos)
- );
-
- switch (tx_type) {
- case DCT_DCT: // DCT in both horizontal and vertical
- idct16_rows_dspr2(input, outptr, 16);
- idct16_cols_add_blk_dspr2(out, dest, pitch);
- break;
- case ADST_DCT: // ADST in vertical, DCT in horizontal
- idct16_rows_dspr2(input, outptr, 16);
-
- outptr = out;
-
- for (i = 0; i < 16; ++i) {
- iadst16_dspr2(outptr, temp_out);
-
- for (j = 0; j < 16; ++j)
- dest[j * pitch + i] =
- clip_pixel(ROUND_POWER_OF_TWO(temp_out[j], 6)
- + dest[j * pitch + i]);
- outptr += 16;
- }
- break;
- case DCT_ADST: // DCT in vertical, ADST in horizontal
- {
- int16_t temp_in[16 * 16];
-
- for (i = 0; i < 16; ++i) {
- /* prefetch row */
- prefetch_load((const uint8_t *)(input + 16));
-
- iadst16_dspr2(input, outptr);
- input += 16;
- outptr += 16;
- }
-
- for (i = 0; i < 16; ++i)
- for (j = 0; j < 16; ++j)
- temp_in[j * 16 + i] = out[i * 16 + j];
-
- idct16_cols_add_blk_dspr2(temp_in, dest, pitch);
- }
- break;
- case ADST_ADST: // ADST in both directions
- {
- int16_t temp_in[16];
-
- for (i = 0; i < 16; ++i) {
- /* prefetch row */
- prefetch_load((const uint8_t *)(input + 16));
-
- iadst16_dspr2(input, outptr);
- input += 16;
- outptr += 16;
- }
-
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- iadst16_dspr2(temp_in, temp_out);
- for (j = 0; j < 16; ++j)
- dest[j * pitch + i] =
- clip_pixel(ROUND_POWER_OF_TWO(temp_out[j], 6)
- + dest[j * pitch + i]);
- }
- }
- break;
- default:
- printf("vp10_short_iht16x16_add_dspr2 : Invalid tx_type\n");
- break;
- }
-}
-#endif // #if HAVE_DSPR2
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <stdio.h>
-
-#include "./vpx_config.h"
-#include "./vp10_rtcd.h"
-#include "vp10/common/common.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/idct.h"
-#include "vpx_dsp/mips/inv_txfm_dspr2.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_ports/mem.h"
-
-#if HAVE_DSPR2
-void vp10_iht4x4_16_add_dspr2(const int16_t *input, uint8_t *dest,
- int dest_stride, int tx_type) {
- int i, j;
- DECLARE_ALIGNED(32, int16_t, out[4 * 4]);
- int16_t *outptr = out;
- int16_t temp_in[4 * 4], temp_out[4];
- uint32_t pos = 45;
-
- /* bit positon for extract from acc */
- __asm__ __volatile__ (
- "wrdsp %[pos], 1 \n\t"
- :
- : [pos] "r" (pos)
- );
-
- switch (tx_type) {
- case DCT_DCT: // DCT in both horizontal and vertical
- vpx_idct4_rows_dspr2(input, outptr);
- vpx_idct4_columns_add_blk_dspr2(&out[0], dest, dest_stride);
- break;
- case ADST_DCT: // ADST in vertical, DCT in horizontal
- vpx_idct4_rows_dspr2(input, outptr);
-
- outptr = out;
-
- for (i = 0; i < 4; ++i) {
- iadst4_dspr2(outptr, temp_out);
-
- for (j = 0; j < 4; ++j)
- dest[j * dest_stride + i] =
- clip_pixel(ROUND_POWER_OF_TWO(temp_out[j], 4)
- + dest[j * dest_stride + i]);
-
- outptr += 4;
- }
- break;
- case DCT_ADST: // DCT in vertical, ADST in horizontal
- for (i = 0; i < 4; ++i) {
- iadst4_dspr2(input, outptr);
- input += 4;
- outptr += 4;
- }
-
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j) {
- temp_in[i * 4 + j] = out[j * 4 + i];
- }
- }
- vpx_idct4_columns_add_blk_dspr2(&temp_in[0], dest, dest_stride);
- break;
- case ADST_ADST: // ADST in both directions
- for (i = 0; i < 4; ++i) {
- iadst4_dspr2(input, outptr);
- input += 4;
- outptr += 4;
- }
-
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j * 4 + i];
- iadst4_dspr2(temp_in, temp_out);
-
- for (j = 0; j < 4; ++j)
- dest[j * dest_stride + i] =
- clip_pixel(ROUND_POWER_OF_TWO(temp_out[j], 4)
- + dest[j * dest_stride + i]);
- }
- break;
- default:
- printf("vp10_short_iht4x4_add_dspr2 : Invalid tx_type\n");
- break;
- }
-}
-#endif // #if HAVE_DSPR2
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <stdio.h>
-
-#include "./vpx_config.h"
-#include "./vp10_rtcd.h"
-#include "vp10/common/common.h"
-#include "vp10/common/blockd.h"
-#include "vpx_dsp/mips/inv_txfm_dspr2.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_ports/mem.h"
-
-#if HAVE_DSPR2
-void vp10_iht8x8_64_add_dspr2(const int16_t *input, uint8_t *dest,
- int dest_stride, int tx_type) {
- int i, j;
- DECLARE_ALIGNED(32, int16_t, out[8 * 8]);
- int16_t *outptr = out;
- int16_t temp_in[8 * 8], temp_out[8];
- uint32_t pos = 45;
-
- /* bit positon for extract from acc */
- __asm__ __volatile__ (
- "wrdsp %[pos], 1 \n\t"
- :
- : [pos] "r" (pos)
- );
-
- switch (tx_type) {
- case DCT_DCT: // DCT in both horizontal and vertical
- idct8_rows_dspr2(input, outptr, 8);
- idct8_columns_add_blk_dspr2(&out[0], dest, dest_stride);
- break;
- case ADST_DCT: // ADST in vertical, DCT in horizontal
- idct8_rows_dspr2(input, outptr, 8);
-
- for (i = 0; i < 8; ++i) {
- iadst8_dspr2(&out[i * 8], temp_out);
-
- for (j = 0; j < 8; ++j)
- dest[j * dest_stride + i] =
- clip_pixel(ROUND_POWER_OF_TWO(temp_out[j], 5)
- + dest[j * dest_stride + i]);
- }
- break;
- case DCT_ADST: // DCT in vertical, ADST in horizontal
- for (i = 0; i < 8; ++i) {
- iadst8_dspr2(input, outptr);
- input += 8;
- outptr += 8;
- }
-
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j) {
- temp_in[i * 8 + j] = out[j * 8 + i];
- }
- }
- idct8_columns_add_blk_dspr2(&temp_in[0], dest, dest_stride);
- break;
- case ADST_ADST: // ADST in both directions
- for (i = 0; i < 8; ++i) {
- iadst8_dspr2(input, outptr);
- input += 8;
- outptr += 8;
- }
-
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
-
- iadst8_dspr2(temp_in, temp_out);
-
- for (j = 0; j < 8; ++j)
- dest[j * dest_stride + i] =
- clip_pixel(ROUND_POWER_OF_TWO(temp_out[j], 5)
- + dest[j * dest_stride + i]);
- }
- break;
- default:
- printf("vp10_short_iht8x8_add_dspr2 : Invalid tx_type\n");
- break;
- }
-}
-#endif // #if HAVE_DSPR2
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/enums.h"
-#include "vpx_dsp/mips/inv_txfm_msa.h"
-
-void vp10_iht16x16_256_add_msa(const int16_t *input, uint8_t *dst,
- int32_t dst_stride, int32_t tx_type) {
- int32_t i;
- DECLARE_ALIGNED(32, int16_t, out[16 * 16]);
- int16_t *out_ptr = &out[0];
-
- switch (tx_type) {
- case DCT_DCT:
- /* transform rows */
- for (i = 0; i < 2; ++i) {
- /* process 16 * 8 block */
- vpx_idct16_1d_rows_msa((input + (i << 7)), (out_ptr + (i << 7)));
- }
-
- /* transform columns */
- for (i = 0; i < 2; ++i) {
- /* process 8 * 16 block */
- vpx_idct16_1d_columns_addblk_msa((out_ptr + (i << 3)), (dst + (i << 3)),
- dst_stride);
- }
- break;
- case ADST_DCT:
- /* transform rows */
- for (i = 0; i < 2; ++i) {
- /* process 16 * 8 block */
- vpx_idct16_1d_rows_msa((input + (i << 7)), (out_ptr + (i << 7)));
- }
-
- /* transform columns */
- for (i = 0; i < 2; ++i) {
- vpx_iadst16_1d_columns_addblk_msa((out_ptr + (i << 3)),
- (dst + (i << 3)), dst_stride);
- }
- break;
- case DCT_ADST:
- /* transform rows */
- for (i = 0; i < 2; ++i) {
- /* process 16 * 8 block */
- vpx_iadst16_1d_rows_msa((input + (i << 7)), (out_ptr + (i << 7)));
- }
-
- /* transform columns */
- for (i = 0; i < 2; ++i) {
- /* process 8 * 16 block */
- vpx_idct16_1d_columns_addblk_msa((out_ptr + (i << 3)), (dst + (i << 3)),
- dst_stride);
- }
- break;
- case ADST_ADST:
- /* transform rows */
- for (i = 0; i < 2; ++i) {
- /* process 16 * 8 block */
- vpx_iadst16_1d_rows_msa((input + (i << 7)), (out_ptr + (i << 7)));
- }
-
- /* transform columns */
- for (i = 0; i < 2; ++i) {
- vpx_iadst16_1d_columns_addblk_msa((out_ptr + (i << 3)),
- (dst + (i << 3)), dst_stride);
- }
- break;
- default:
- assert(0);
- break;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/enums.h"
-#include "vpx_dsp/mips/inv_txfm_msa.h"
-
-void vp10_iht4x4_16_add_msa(const int16_t *input, uint8_t *dst,
- int32_t dst_stride, int32_t tx_type) {
- v8i16 in0, in1, in2, in3;
-
- /* load vector elements of 4x4 block */
- LD4x4_SH(input, in0, in1, in2, in3);
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
-
- switch (tx_type) {
- case DCT_DCT:
- /* DCT in horizontal */
- VP9_IDCT4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- /* DCT in vertical */
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_IDCT4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- case ADST_DCT:
- /* DCT in horizontal */
- VP9_IDCT4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- /* ADST in vertical */
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_IADST4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- case DCT_ADST:
- /* ADST in horizontal */
- VP9_IADST4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- /* DCT in vertical */
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_IDCT4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- case ADST_ADST:
- /* ADST in horizontal */
- VP9_IADST4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- /* ADST in vertical */
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_IADST4x4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- default:
- assert(0);
- break;
- }
-
- /* final rounding (add 2^3, divide by 2^4) and shift */
- SRARI_H4_SH(in0, in1, in2, in3, 4);
- /* add block and store 4x4 */
- ADDBLK_ST4x4_UB(in0, in1, in2, in3, dst, dst_stride);
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/enums.h"
-#include "vpx_dsp/mips/inv_txfm_msa.h"
-
-void vp10_iht8x8_64_add_msa(const int16_t *input, uint8_t *dst,
- int32_t dst_stride, int32_t tx_type) {
- v8i16 in0, in1, in2, in3, in4, in5, in6, in7;
-
- /* load vector elements of 8x8 block */
- LD_SH8(input, 8, in0, in1, in2, in3, in4, in5, in6, in7);
-
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
-
- switch (tx_type) {
- case DCT_DCT:
- /* DCT in horizontal */
- VP9_IDCT8x8_1D(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- /* DCT in vertical */
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_IDCT8x8_1D(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- case ADST_DCT:
- /* DCT in horizontal */
- VP9_IDCT8x8_1D(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- /* ADST in vertical */
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- case DCT_ADST:
- /* ADST in horizontal */
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- /* DCT in vertical */
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_IDCT8x8_1D(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- case ADST_ADST:
- /* ADST in horizontal */
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- /* ADST in vertical */
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- default:
- assert(0);
- break;
- }
-
- /* final rounding (add 2^4, divide by 2^5) and shift */
- SRARI_H4_SH(in0, in1, in2, in3, 5);
- SRARI_H4_SH(in4, in5, in6, in7, 5);
-
- /* add block and store 8x8 */
- VP9_ADDBLK_ST8x4_UB(dst, dst_stride, in0, in1, in2, in3);
- dst += (4 * dst_stride);
- VP9_ADDBLK_ST8x4_UB(dst, dst_stride, in4, in5, in6, in7);
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vp10_rtcd.h"
-#include "vp10/common/onyxc_int.h"
-#include "vpx_dsp/mips/macros_msa.h"
-
-static void filter_by_weight8x8_msa(const uint8_t *src_ptr, int32_t src_stride,
- uint8_t *dst_ptr, int32_t dst_stride,
- int32_t src_weight) {
- int32_t dst_weight = (1 << MFQE_PRECISION) - src_weight;
- int32_t row;
- uint64_t src0_d, src1_d, dst0_d, dst1_d;
- v16i8 src0 = { 0 };
- v16i8 src1 = { 0 };
- v16i8 dst0 = { 0 };
- v16i8 dst1 = { 0 };
- v8i16 src_wt, dst_wt, res_h_r, res_h_l, src_r, src_l, dst_r, dst_l;
-
- src_wt = __msa_fill_h(src_weight);
- dst_wt = __msa_fill_h(dst_weight);
-
- for (row = 2; row--;) {
- LD2(src_ptr, src_stride, src0_d, src1_d);
- src_ptr += (2 * src_stride);
- LD2(dst_ptr, dst_stride, dst0_d, dst1_d);
- INSERT_D2_SB(src0_d, src1_d, src0);
- INSERT_D2_SB(dst0_d, dst1_d, dst0);
-
- LD2(src_ptr, src_stride, src0_d, src1_d);
- src_ptr += (2 * src_stride);
- LD2((dst_ptr + 2 * dst_stride), dst_stride, dst0_d, dst1_d);
- INSERT_D2_SB(src0_d, src1_d, src1);
- INSERT_D2_SB(dst0_d, dst1_d, dst1);
-
- UNPCK_UB_SH(src0, src_r, src_l);
- UNPCK_UB_SH(dst0, dst_r, dst_l);
- res_h_r = (src_r * src_wt);
- res_h_r += (dst_r * dst_wt);
- res_h_l = (src_l * src_wt);
- res_h_l += (dst_l * dst_wt);
- SRARI_H2_SH(res_h_r, res_h_l, MFQE_PRECISION);
- dst0 = (v16i8)__msa_pckev_b((v16i8)res_h_l, (v16i8)res_h_r);
- ST8x2_UB(dst0, dst_ptr, dst_stride);
- dst_ptr += (2 * dst_stride);
-
- UNPCK_UB_SH(src1, src_r, src_l);
- UNPCK_UB_SH(dst1, dst_r, dst_l);
- res_h_r = (src_r * src_wt);
- res_h_r += (dst_r * dst_wt);
- res_h_l = (src_l * src_wt);
- res_h_l += (dst_l * dst_wt);
- SRARI_H2_SH(res_h_r, res_h_l, MFQE_PRECISION);
- dst1 = (v16i8)__msa_pckev_b((v16i8)res_h_l, (v16i8)res_h_r);
- ST8x2_UB(dst1, dst_ptr, dst_stride);
- dst_ptr += (2 * dst_stride);
- }
-}
-
-static void filter_by_weight16x16_msa(const uint8_t *src_ptr,
- int32_t src_stride,
- uint8_t *dst_ptr,
- int32_t dst_stride,
- int32_t src_weight) {
- int32_t dst_weight = (1 << MFQE_PRECISION) - src_weight;
- int32_t row;
- v16i8 src0, src1, src2, src3, dst0, dst1, dst2, dst3;
- v8i16 src_wt, dst_wt, res_h_r, res_h_l, src_r, src_l, dst_r, dst_l;
-
- src_wt = __msa_fill_h(src_weight);
- dst_wt = __msa_fill_h(dst_weight);
-
- for (row = 4; row--;) {
- LD_SB4(src_ptr, src_stride, src0, src1, src2, src3);
- src_ptr += (4 * src_stride);
- LD_SB4(dst_ptr, dst_stride, dst0, dst1, dst2, dst3);
-
- UNPCK_UB_SH(src0, src_r, src_l);
- UNPCK_UB_SH(dst0, dst_r, dst_l);
- res_h_r = (src_r * src_wt);
- res_h_r += (dst_r * dst_wt);
- res_h_l = (src_l * src_wt);
- res_h_l += (dst_l * dst_wt);
- SRARI_H2_SH(res_h_r, res_h_l, MFQE_PRECISION);
- PCKEV_ST_SB(res_h_r, res_h_l, dst_ptr);
- dst_ptr += dst_stride;
-
- UNPCK_UB_SH(src1, src_r, src_l);
- UNPCK_UB_SH(dst1, dst_r, dst_l);
- res_h_r = (src_r * src_wt);
- res_h_r += (dst_r * dst_wt);
- res_h_l = (src_l * src_wt);
- res_h_l += (dst_l * dst_wt);
- SRARI_H2_SH(res_h_r, res_h_l, MFQE_PRECISION);
- PCKEV_ST_SB(res_h_r, res_h_l, dst_ptr);
- dst_ptr += dst_stride;
-
- UNPCK_UB_SH(src2, src_r, src_l);
- UNPCK_UB_SH(dst2, dst_r, dst_l);
- res_h_r = (src_r * src_wt);
- res_h_r += (dst_r * dst_wt);
- res_h_l = (src_l * src_wt);
- res_h_l += (dst_l * dst_wt);
- SRARI_H2_SH(res_h_r, res_h_l, MFQE_PRECISION);
- PCKEV_ST_SB(res_h_r, res_h_l, dst_ptr);
- dst_ptr += dst_stride;
-
- UNPCK_UB_SH(src3, src_r, src_l);
- UNPCK_UB_SH(dst3, dst_r, dst_l);
- res_h_r = (src_r * src_wt);
- res_h_r += (dst_r * dst_wt);
- res_h_l = (src_l * src_wt);
- res_h_l += (dst_l * dst_wt);
- SRARI_H2_SH(res_h_r, res_h_l, MFQE_PRECISION);
- PCKEV_ST_SB(res_h_r, res_h_l, dst_ptr);
- dst_ptr += dst_stride;
- }
-}
-
-void vp10_filter_by_weight8x8_msa(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- int src_weight) {
- filter_by_weight8x8_msa(src, src_stride, dst, dst_stride, src_weight);
-}
-
-void vp10_filter_by_weight16x16_msa(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- int src_weight) {
- filter_by_weight16x16_msa(src, src_stride, dst, dst_stride, src_weight);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_MV_H_
-#define VP10_COMMON_MV_H_
-
-#include "vpx/vpx_integer.h"
-
-#include "vp10/common/common.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct mv {
- int16_t row;
- int16_t col;
-} MV;
-
-typedef union int_mv {
- uint32_t as_int;
- MV as_mv;
-} int_mv; /* facilitates faster equality tests and copies */
-
-typedef struct mv32 {
- int32_t row;
- int32_t col;
-} MV32;
-
-static INLINE int is_zero_mv(const MV *mv) {
- return *((const uint32_t *)mv) == 0;
-}
-
-static INLINE int is_equal_mv(const MV *a, const MV *b) {
- return *((const uint32_t *)a) == *((const uint32_t *)b);
-}
-
-static INLINE void clamp_mv(MV *mv, int min_col, int max_col,
- int min_row, int max_row) {
- mv->col = clamp(mv->col, min_col, max_col);
- mv->row = clamp(mv->row, min_row, max_row);
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_MV_H_
+++ /dev/null
-
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/mvref_common.h"
-
-// This function searches the neighbourhood of a given MB/SB
-// to try and find candidate reference vectors.
-static void find_mv_refs_idx(const VP10_COMMON *cm, const MACROBLOCKD *xd,
- MODE_INFO *mi, MV_REFERENCE_FRAME ref_frame,
- int_mv *mv_ref_list,
- int block, int mi_row, int mi_col,
- find_mv_refs_sync sync, void *const data,
- uint8_t *mode_context) {
- const int *ref_sign_bias = cm->ref_frame_sign_bias;
- int i, refmv_count = 0;
- const POSITION *const mv_ref_search = mv_ref_blocks[mi->mbmi.sb_type];
- int different_ref_found = 0;
- int context_counter = 0;
- const MV_REF *const prev_frame_mvs = cm->use_prev_frame_mvs ?
- cm->prev_frame->mvs + mi_row * cm->mi_cols + mi_col : NULL;
- const TileInfo *const tile = &xd->tile;
- const int bw = num_8x8_blocks_wide_lookup[mi->mbmi.sb_type] << 3;
- const int bh = num_8x8_blocks_high_lookup[mi->mbmi.sb_type] << 3;
-
-#if !CONFIG_MISC_FIXES
- // Blank the reference vector list
- memset(mv_ref_list, 0, sizeof(*mv_ref_list) * MAX_MV_REF_CANDIDATES);
-#endif
-
- // The nearest 2 blocks are treated differently
- // if the size < 8x8 we get the mv from the bmi substructure,
- // and we also need to keep a mode count.
- for (i = 0; i < 2; ++i) {
- const POSITION *const mv_ref = &mv_ref_search[i];
- if (is_inside(tile, mi_col, mi_row, cm->mi_rows, mv_ref)) {
- const MODE_INFO *const candidate_mi = xd->mi[mv_ref->col + mv_ref->row *
- xd->mi_stride];
- const MB_MODE_INFO *const candidate = &candidate_mi->mbmi;
- // Keep counts for entropy encoding.
- context_counter += mode_2_counter[candidate->mode];
- different_ref_found = 1;
-
- if (candidate->ref_frame[0] == ref_frame)
- ADD_MV_REF_LIST(get_sub_block_mv(candidate_mi, 0, mv_ref->col, block),
- refmv_count, mv_ref_list, bw, bh, xd, Done);
- else if (candidate->ref_frame[1] == ref_frame)
- ADD_MV_REF_LIST(get_sub_block_mv(candidate_mi, 1, mv_ref->col, block),
- refmv_count, mv_ref_list, bw, bh, xd, Done);
- }
- }
-
- // Check the rest of the neighbors in much the same way
- // as before except we don't need to keep track of sub blocks or
- // mode counts.
- for (; i < MVREF_NEIGHBOURS; ++i) {
- const POSITION *const mv_ref = &mv_ref_search[i];
- if (is_inside(tile, mi_col, mi_row, cm->mi_rows, mv_ref)) {
- const MB_MODE_INFO *const candidate = &xd->mi[mv_ref->col + mv_ref->row *
- xd->mi_stride]->mbmi;
- different_ref_found = 1;
-
- if (candidate->ref_frame[0] == ref_frame)
- ADD_MV_REF_LIST(candidate->mv[0], refmv_count, mv_ref_list,
- bw, bh, xd, Done);
- else if (candidate->ref_frame[1] == ref_frame)
- ADD_MV_REF_LIST(candidate->mv[1], refmv_count, mv_ref_list,
- bw, bh, xd, Done);
- }
- }
-
- // TODO(hkuang): Remove this sync after fixing pthread_cond_broadcast
- // on windows platform. The sync here is unncessary if use_perv_frame_mvs
- // is 0. But after removing it, there will be hang in the unit test on windows
- // due to several threads waiting for a thread's signal.
-#if defined(_WIN32) && !HAVE_PTHREAD_H
- if (cm->frame_parallel_decode && sync != NULL) {
- sync(data, mi_row);
- }
-#endif
-
- // Check the last frame's mode and mv info.
- if (cm->use_prev_frame_mvs) {
- // Synchronize here for frame parallel decode if sync function is provided.
- if (cm->frame_parallel_decode && sync != NULL) {
- sync(data, mi_row);
- }
-
- if (prev_frame_mvs->ref_frame[0] == ref_frame) {
- ADD_MV_REF_LIST(prev_frame_mvs->mv[0], refmv_count, mv_ref_list,
- bw, bh, xd, Done);
- } else if (prev_frame_mvs->ref_frame[1] == ref_frame) {
- ADD_MV_REF_LIST(prev_frame_mvs->mv[1], refmv_count, mv_ref_list,
- bw, bh, xd, Done);
- }
- }
-
- // Since we couldn't find 2 mvs from the same reference frame
- // go back through the neighbors and find motion vectors from
- // different reference frames.
- if (different_ref_found) {
- for (i = 0; i < MVREF_NEIGHBOURS; ++i) {
- const POSITION *mv_ref = &mv_ref_search[i];
- if (is_inside(tile, mi_col, mi_row, cm->mi_rows, mv_ref)) {
- const MB_MODE_INFO *const candidate = &xd->mi[mv_ref->col + mv_ref->row
- * xd->mi_stride]->mbmi;
-
- // If the candidate is INTRA we don't want to consider its mv.
- IF_DIFF_REF_FRAME_ADD_MV(candidate, ref_frame, ref_sign_bias,
- refmv_count, mv_ref_list, bw, bh, xd, Done);
- }
- }
- }
-
- // Since we still don't have a candidate we'll try the last frame.
- if (cm->use_prev_frame_mvs) {
- if (prev_frame_mvs->ref_frame[0] != ref_frame &&
- prev_frame_mvs->ref_frame[0] > INTRA_FRAME) {
- int_mv mv = prev_frame_mvs->mv[0];
- if (ref_sign_bias[prev_frame_mvs->ref_frame[0]] !=
- ref_sign_bias[ref_frame]) {
- mv.as_mv.row *= -1;
- mv.as_mv.col *= -1;
- }
- ADD_MV_REF_LIST(mv, refmv_count, mv_ref_list, bw, bh, xd, Done);
- }
-
- if (prev_frame_mvs->ref_frame[1] > INTRA_FRAME &&
-#if !CONFIG_MISC_FIXES
- prev_frame_mvs->mv[1].as_int != prev_frame_mvs->mv[0].as_int &&
-#endif
- prev_frame_mvs->ref_frame[1] != ref_frame) {
- int_mv mv = prev_frame_mvs->mv[1];
- if (ref_sign_bias[prev_frame_mvs->ref_frame[1]] !=
- ref_sign_bias[ref_frame]) {
- mv.as_mv.row *= -1;
- mv.as_mv.col *= -1;
- }
- ADD_MV_REF_LIST(mv, refmv_count, mv_ref_list, bw, bh, xd, Done);
- }
- }
-
- Done:
-
- mode_context[ref_frame] = counter_to_context[context_counter];
-
-#if CONFIG_MISC_FIXES
- for (i = refmv_count; i < MAX_MV_REF_CANDIDATES; ++i)
- mv_ref_list[i].as_int = 0;
-#else
- // Clamp vectors
- for (i = 0; i < MAX_MV_REF_CANDIDATES; ++i)
- clamp_mv_ref(&mv_ref_list[i].as_mv, bw, bh, xd);
-#endif
-}
-
-void vp10_find_mv_refs(const VP10_COMMON *cm, const MACROBLOCKD *xd,
- MODE_INFO *mi, MV_REFERENCE_FRAME ref_frame,
- int_mv *mv_ref_list,
- int mi_row, int mi_col,
- find_mv_refs_sync sync, void *const data,
- uint8_t *mode_context) {
- find_mv_refs_idx(cm, xd, mi, ref_frame, mv_ref_list, -1,
- mi_row, mi_col, sync, data, mode_context);
-}
-
-static void lower_mv_precision(MV *mv, int allow_hp) {
- const int use_hp = allow_hp && vp10_use_mv_hp(mv);
- if (!use_hp) {
- if (mv->row & 1)
- mv->row += (mv->row > 0 ? -1 : 1);
- if (mv->col & 1)
- mv->col += (mv->col > 0 ? -1 : 1);
- }
-}
-
-void vp10_find_best_ref_mvs(int allow_hp,
- int_mv *mvlist, int_mv *nearest_mv,
- int_mv *near_mv) {
- int i;
- // Make sure all the candidates are properly clamped etc
- for (i = 0; i < MAX_MV_REF_CANDIDATES; ++i) {
- lower_mv_precision(&mvlist[i].as_mv, allow_hp);
- }
- *nearest_mv = mvlist[0];
- *near_mv = mvlist[1];
-}
-
-void vp10_append_sub8x8_mvs_for_idx(VP10_COMMON *cm, MACROBLOCKD *xd,
- int block, int ref, int mi_row, int mi_col,
- int_mv *nearest_mv, int_mv *near_mv,
- uint8_t *mode_context) {
- int_mv mv_list[MAX_MV_REF_CANDIDATES];
- MODE_INFO *const mi = xd->mi[0];
- b_mode_info *bmi = mi->bmi;
- int n;
-
- assert(MAX_MV_REF_CANDIDATES == 2);
-
- find_mv_refs_idx(cm, xd, mi, mi->mbmi.ref_frame[ref], mv_list, block,
- mi_row, mi_col, NULL, NULL, mode_context);
-
- near_mv->as_int = 0;
- switch (block) {
- case 0:
- nearest_mv->as_int = mv_list[0].as_int;
- near_mv->as_int = mv_list[1].as_int;
- break;
- case 1:
- case 2:
- nearest_mv->as_int = bmi[0].as_mv[ref].as_int;
- for (n = 0; n < MAX_MV_REF_CANDIDATES; ++n)
- if (nearest_mv->as_int != mv_list[n].as_int) {
- near_mv->as_int = mv_list[n].as_int;
- break;
- }
- break;
- case 3: {
- int_mv candidates[2 + MAX_MV_REF_CANDIDATES];
- candidates[0] = bmi[1].as_mv[ref];
- candidates[1] = bmi[0].as_mv[ref];
- candidates[2] = mv_list[0];
- candidates[3] = mv_list[1];
-
- nearest_mv->as_int = bmi[2].as_mv[ref].as_int;
- for (n = 0; n < 2 + MAX_MV_REF_CANDIDATES; ++n)
- if (nearest_mv->as_int != candidates[n].as_int) {
- near_mv->as_int = candidates[n].as_int;
- break;
- }
- break;
- }
- default:
- assert(0 && "Invalid block index.");
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#ifndef VP10_COMMON_MVREF_COMMON_H_
-#define VP10_COMMON_MVREF_COMMON_H_
-
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MVREF_NEIGHBOURS 8
-
-typedef struct position {
- int row;
- int col;
-} POSITION;
-
-typedef enum {
- BOTH_ZERO = 0,
- ZERO_PLUS_PREDICTED = 1,
- BOTH_PREDICTED = 2,
- NEW_PLUS_NON_INTRA = 3,
- BOTH_NEW = 4,
- INTRA_PLUS_NON_INTRA = 5,
- BOTH_INTRA = 6,
- INVALID_CASE = 9
-} motion_vector_context;
-
-// This is used to figure out a context for the ref blocks. The code flattens
-// an array that would have 3 possible counts (0, 1 & 2) for 3 choices by
-// adding 9 for each intra block, 3 for each zero mv and 1 for each new
-// motion vector. This single number is then converted into a context
-// with a single lookup ( counter_to_context ).
-static const int mode_2_counter[MB_MODE_COUNT] = {
- 9, // DC_PRED
- 9, // V_PRED
- 9, // H_PRED
- 9, // D45_PRED
- 9, // D135_PRED
- 9, // D117_PRED
- 9, // D153_PRED
- 9, // D207_PRED
- 9, // D63_PRED
- 9, // TM_PRED
- 0, // NEARESTMV
- 0, // NEARMV
- 3, // ZEROMV
- 1, // NEWMV
-};
-
-// There are 3^3 different combinations of 3 counts that can be either 0,1 or
-// 2. However the actual count can never be greater than 2 so the highest
-// counter we need is 18. 9 is an invalid counter that's never used.
-static const int counter_to_context[19] = {
- BOTH_PREDICTED, // 0
- NEW_PLUS_NON_INTRA, // 1
- BOTH_NEW, // 2
- ZERO_PLUS_PREDICTED, // 3
- NEW_PLUS_NON_INTRA, // 4
- INVALID_CASE, // 5
- BOTH_ZERO, // 6
- INVALID_CASE, // 7
- INVALID_CASE, // 8
- INTRA_PLUS_NON_INTRA, // 9
- INTRA_PLUS_NON_INTRA, // 10
- INVALID_CASE, // 11
- INTRA_PLUS_NON_INTRA, // 12
- INVALID_CASE, // 13
- INVALID_CASE, // 14
- INVALID_CASE, // 15
- INVALID_CASE, // 16
- INVALID_CASE, // 17
- BOTH_INTRA // 18
-};
-
-static const POSITION mv_ref_blocks[BLOCK_SIZES][MVREF_NEIGHBOURS] = {
- // 4X4
- {{-1, 0}, {0, -1}, {-1, -1}, {-2, 0}, {0, -2}, {-2, -1}, {-1, -2}, {-2, -2}},
- // 4X8
- {{-1, 0}, {0, -1}, {-1, -1}, {-2, 0}, {0, -2}, {-2, -1}, {-1, -2}, {-2, -2}},
- // 8X4
- {{-1, 0}, {0, -1}, {-1, -1}, {-2, 0}, {0, -2}, {-2, -1}, {-1, -2}, {-2, -2}},
- // 8X8
- {{-1, 0}, {0, -1}, {-1, -1}, {-2, 0}, {0, -2}, {-2, -1}, {-1, -2}, {-2, -2}},
- // 8X16
- {{0, -1}, {-1, 0}, {1, -1}, {-1, -1}, {0, -2}, {-2, 0}, {-2, -1}, {-1, -2}},
- // 16X8
- {{-1, 0}, {0, -1}, {-1, 1}, {-1, -1}, {-2, 0}, {0, -2}, {-1, -2}, {-2, -1}},
- // 16X16
- {{-1, 0}, {0, -1}, {-1, 1}, {1, -1}, {-1, -1}, {-3, 0}, {0, -3}, {-3, -3}},
- // 16X32
- {{0, -1}, {-1, 0}, {2, -1}, {-1, -1}, {-1, 1}, {0, -3}, {-3, 0}, {-3, -3}},
- // 32X16
- {{-1, 0}, {0, -1}, {-1, 2}, {-1, -1}, {1, -1}, {-3, 0}, {0, -3}, {-3, -3}},
- // 32X32
- {{-1, 1}, {1, -1}, {-1, 2}, {2, -1}, {-1, -1}, {-3, 0}, {0, -3}, {-3, -3}},
- // 32X64
- {{0, -1}, {-1, 0}, {4, -1}, {-1, 2}, {-1, -1}, {0, -3}, {-3, 0}, {2, -1}},
- // 64X32
- {{-1, 0}, {0, -1}, {-1, 4}, {2, -1}, {-1, -1}, {-3, 0}, {0, -3}, {-1, 2}},
- // 64X64
- {{-1, 3}, {3, -1}, {-1, 4}, {4, -1}, {-1, -1}, {-1, 0}, {0, -1}, {-1, 6}}
-};
-
-static const int idx_n_column_to_subblock[4][2] = {
- {1, 2},
- {1, 3},
- {3, 2},
- {3, 3}
-};
-
-// clamp_mv_ref
-#if CONFIG_MISC_FIXES
-#define MV_BORDER (8 << 3) // Allow 8 pels in 1/8th pel units
-#else
-#define MV_BORDER (16 << 3) // Allow 16 pels in 1/8th pel units
-#endif
-
-static INLINE void clamp_mv_ref(MV *mv, int bw, int bh, const MACROBLOCKD *xd) {
-#if CONFIG_MISC_FIXES
- clamp_mv(mv, xd->mb_to_left_edge - bw * 8 - MV_BORDER,
- xd->mb_to_right_edge + bw * 8 + MV_BORDER,
- xd->mb_to_top_edge - bh * 8 - MV_BORDER,
- xd->mb_to_bottom_edge + bh * 8 + MV_BORDER);
-#else
- (void) bw;
- (void) bh;
- clamp_mv(mv, xd->mb_to_left_edge - MV_BORDER,
- xd->mb_to_right_edge + MV_BORDER,
- xd->mb_to_top_edge - MV_BORDER,
- xd->mb_to_bottom_edge + MV_BORDER);
-#endif
-}
-
-// This function returns either the appropriate sub block or block's mv
-// on whether the block_size < 8x8 and we have check_sub_blocks set.
-static INLINE int_mv get_sub_block_mv(const MODE_INFO *candidate, int which_mv,
- int search_col, int block_idx) {
- return block_idx >= 0 && candidate->mbmi.sb_type < BLOCK_8X8
- ? candidate->bmi[idx_n_column_to_subblock[block_idx][search_col == 0]]
- .as_mv[which_mv]
- : candidate->mbmi.mv[which_mv];
-}
-
-
-// Performs mv sign inversion if indicated by the reference frame combination.
-static INLINE int_mv scale_mv(const MB_MODE_INFO *mbmi, int ref,
- const MV_REFERENCE_FRAME this_ref_frame,
- const int *ref_sign_bias) {
- int_mv mv = mbmi->mv[ref];
- if (ref_sign_bias[mbmi->ref_frame[ref]] != ref_sign_bias[this_ref_frame]) {
- mv.as_mv.row *= -1;
- mv.as_mv.col *= -1;
- }
- return mv;
-}
-
-#if CONFIG_MISC_FIXES
-#define CLIP_IN_ADD(mv, bw, bh, xd) clamp_mv_ref(mv, bw, bh, xd)
-#else
-#define CLIP_IN_ADD(mv, bw, bh, xd) do {} while (0)
-#endif
-
-// This macro is used to add a motion vector mv_ref list if it isn't
-// already in the list. If it's the second motion vector it will also
-// skip all additional processing and jump to done!
-#define ADD_MV_REF_LIST(mv, refmv_count, mv_ref_list, bw, bh, xd, Done) \
- do { \
- (mv_ref_list)[(refmv_count)] = (mv); \
- CLIP_IN_ADD(&(mv_ref_list)[(refmv_count)].as_mv, (bw), (bh), (xd)); \
- if (refmv_count && (mv_ref_list)[1].as_int != (mv_ref_list)[0].as_int) { \
- (refmv_count) = 2; \
- goto Done; \
- } \
- (refmv_count) = 1; \
- } while (0)
-
-// If either reference frame is different, not INTRA, and they
-// are different from each other scale and add the mv to our list.
-#define IF_DIFF_REF_FRAME_ADD_MV(mbmi, ref_frame, ref_sign_bias, refmv_count, \
- mv_ref_list, bw, bh, xd, Done) \
- do { \
- if (is_inter_block(mbmi)) { \
- if ((mbmi)->ref_frame[0] != ref_frame) \
- ADD_MV_REF_LIST(scale_mv((mbmi), 0, ref_frame, ref_sign_bias), \
- refmv_count, mv_ref_list, bw, bh, xd, Done); \
- if (has_second_ref(mbmi) && \
- (CONFIG_MISC_FIXES || \
- (mbmi)->mv[1].as_int != (mbmi)->mv[0].as_int) && \
- (mbmi)->ref_frame[1] != ref_frame) \
- ADD_MV_REF_LIST(scale_mv((mbmi), 1, ref_frame, ref_sign_bias), \
- refmv_count, mv_ref_list, bw, bh, xd, Done); \
- } \
- } while (0)
-
-
-// Checks that the given mi_row, mi_col and search point
-// are inside the borders of the tile.
-static INLINE int is_inside(const TileInfo *const tile,
- int mi_col, int mi_row, int mi_rows,
- const POSITION *mi_pos) {
- return !(mi_row + mi_pos->row < 0 ||
- mi_col + mi_pos->col < tile->mi_col_start ||
- mi_row + mi_pos->row >= mi_rows ||
- mi_col + mi_pos->col >= tile->mi_col_end);
-}
-
-typedef void (*find_mv_refs_sync)(void *const data, int mi_row);
-void vp10_find_mv_refs(const VP10_COMMON *cm, const MACROBLOCKD *xd,
- MODE_INFO *mi, MV_REFERENCE_FRAME ref_frame,
- int_mv *mv_ref_list, int mi_row, int mi_col,
- find_mv_refs_sync sync, void *const data,
- uint8_t *mode_context);
-
-// check a list of motion vectors by sad score using a number rows of pixels
-// above and a number cols of pixels in the left to select the one with best
-// score to use as ref motion vector
-void vp10_find_best_ref_mvs(int allow_hp,
- int_mv *mvlist, int_mv *nearest_mv, int_mv *near_mv);
-
-void vp10_append_sub8x8_mvs_for_idx(VP10_COMMON *cm, MACROBLOCKD *xd,
- int block, int ref, int mi_row, int mi_col,
- int_mv *nearest_mv, int_mv *near_mv,
- uint8_t *mode_context);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_MVREF_COMMON_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_ONYXC_INT_H_
-#define VP10_COMMON_ONYXC_INT_H_
-
-#include "./vpx_config.h"
-#include "vpx/internal/vpx_codec_internal.h"
-#include "vpx_util/vpx_thread.h"
-#include "./vp10_rtcd.h"
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/loopfilter.h"
-#include "vp10/common/entropymv.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/frame_buffers.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/tile_common.h"
-
-#if CONFIG_VP9_POSTPROC
-#include "vp10/common/postproc.h"
-#endif
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define REFS_PER_FRAME (ALTREF_FRAME - LAST_FRAME + 1)
-
-#define REF_FRAMES_LOG2 3
-#define REF_FRAMES (1 << REF_FRAMES_LOG2)
-
-// 4 scratch frames for the new frames to support a maximum of 4 cores decoding
-// in parallel, 3 for scaled references on the encoder.
-// TODO(hkuang): Add ondemand frame buffers instead of hardcoding the number
-// of framebuffers.
-// TODO(jkoleszar): These 3 extra references could probably come from the
-// normal reference pool.
-#define FRAME_BUFFERS (REF_FRAMES + 7)
-
-#define FRAME_CONTEXTS_LOG2 2
-#define FRAME_CONTEXTS (1 << FRAME_CONTEXTS_LOG2)
-
-#define NUM_PING_PONG_BUFFERS 2
-
-typedef enum {
- SINGLE_REFERENCE = 0,
- COMPOUND_REFERENCE = 1,
- REFERENCE_MODE_SELECT = 2,
- REFERENCE_MODES = 3,
-} REFERENCE_MODE;
-
-typedef enum {
- RESET_FRAME_CONTEXT_NONE = 0,
- RESET_FRAME_CONTEXT_CURRENT = 1,
- RESET_FRAME_CONTEXT_ALL = 2,
-} RESET_FRAME_CONTEXT_MODE;
-
-typedef enum {
- /**
- * Don't update frame context
- */
- REFRESH_FRAME_CONTEXT_OFF,
- /**
- * Update frame context to values resulting from forward probability
- * updates signaled in the frame header
- */
- REFRESH_FRAME_CONTEXT_FORWARD,
- /**
- * Update frame context to values resulting from backward probability
- * updates based on entropy/counts in the decoded frame
- */
- REFRESH_FRAME_CONTEXT_BACKWARD,
-} REFRESH_FRAME_CONTEXT_MODE;
-
-typedef struct {
- int_mv mv[2];
- MV_REFERENCE_FRAME ref_frame[2];
-} MV_REF;
-
-typedef struct {
- int ref_count;
- MV_REF *mvs;
- int mi_rows;
- int mi_cols;
- vpx_codec_frame_buffer_t raw_frame_buffer;
- YV12_BUFFER_CONFIG buf;
-
- // The Following variables will only be used in frame parallel decode.
-
- // frame_worker_owner indicates which FrameWorker owns this buffer. NULL means
- // that no FrameWorker owns, or is decoding, this buffer.
- VPxWorker *frame_worker_owner;
-
- // row and col indicate which position frame has been decoded to in real
- // pixel unit. They are reset to -1 when decoding begins and set to INT_MAX
- // when the frame is fully decoded.
- int row;
- int col;
-} RefCntBuffer;
-
-typedef struct BufferPool {
- // Protect BufferPool from being accessed by several FrameWorkers at
- // the same time during frame parallel decode.
- // TODO(hkuang): Try to use atomic variable instead of locking the whole pool.
-#if CONFIG_MULTITHREAD
- pthread_mutex_t pool_mutex;
-#endif
-
- // Private data associated with the frame buffer callbacks.
- void *cb_priv;
-
- vpx_get_frame_buffer_cb_fn_t get_fb_cb;
- vpx_release_frame_buffer_cb_fn_t release_fb_cb;
-
- RefCntBuffer frame_bufs[FRAME_BUFFERS];
-
- // Frame buffers allocated internally by the codec.
- InternalFrameBufferList int_frame_buffers;
-} BufferPool;
-
-typedef struct VP10Common {
- struct vpx_internal_error_info error;
- vpx_color_space_t color_space;
- int color_range;
- int width;
- int height;
- int render_width;
- int render_height;
- int last_width;
- int last_height;
-
- // TODO(jkoleszar): this implies chroma ss right now, but could vary per
- // plane. Revisit as part of the future change to YV12_BUFFER_CONFIG to
- // support additional planes.
- int subsampling_x;
- int subsampling_y;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth; // Marks if we need to use 16bit frame buffers.
-#endif
-
- YV12_BUFFER_CONFIG *frame_to_show;
- RefCntBuffer *prev_frame;
-
- // TODO(hkuang): Combine this with cur_buf in macroblockd.
- RefCntBuffer *cur_frame;
-
- int ref_frame_map[REF_FRAMES]; /* maps fb_idx to reference slot */
-
- // Prepare ref_frame_map for the next frame.
- // Only used in frame parallel decode.
- int next_ref_frame_map[REF_FRAMES];
-
- // TODO(jkoleszar): could expand active_ref_idx to 4, with 0 as intra, and
- // roll new_fb_idx into it.
-
- // Each frame can reference REFS_PER_FRAME buffers
- RefBuffer frame_refs[REFS_PER_FRAME];
-
- int new_fb_idx;
-
-#if CONFIG_VP9_POSTPROC
- YV12_BUFFER_CONFIG post_proc_buffer;
- YV12_BUFFER_CONFIG post_proc_buffer_int;
-#endif
-
- FRAME_TYPE last_frame_type; /* last frame's frame type for motion search.*/
- FRAME_TYPE frame_type;
-
- int show_frame;
- int last_show_frame;
- int show_existing_frame;
-
- // Flag signaling that the frame is encoded using only INTRA modes.
- uint8_t intra_only;
- uint8_t last_intra_only;
-
- int allow_high_precision_mv;
-
- int allow_screen_content_tools;
-
- // Flag signaling which frame contexts should be reset to default values.
- RESET_FRAME_CONTEXT_MODE reset_frame_context;
-
- // MBs, mb_rows/cols is in 16-pixel units; mi_rows/cols is in
- // MODE_INFO (8-pixel) units.
- int MBs;
- int mb_rows, mi_rows;
- int mb_cols, mi_cols;
- int mi_stride;
-
- /* profile settings */
- TX_MODE tx_mode;
-
- int base_qindex;
- int y_dc_delta_q;
- int uv_dc_delta_q;
- int uv_ac_delta_q;
- int16_t y_dequant[MAX_SEGMENTS][2];
- int16_t uv_dequant[MAX_SEGMENTS][2];
-
- /* We allocate a MODE_INFO struct for each macroblock, together with
- an extra row on top and column on the left to simplify prediction. */
- int mi_alloc_size;
- MODE_INFO *mip; /* Base of allocated array */
- MODE_INFO *mi; /* Corresponds to upper left visible macroblock */
-
- // TODO(agrange): Move prev_mi into encoder structure.
- // prev_mip and prev_mi will only be allocated in VP9 encoder.
- MODE_INFO *prev_mip; /* MODE_INFO array 'mip' from last decoded frame */
- MODE_INFO *prev_mi; /* 'mi' from last frame (points into prev_mip) */
-
- // Separate mi functions between encoder and decoder.
- int (*alloc_mi)(struct VP10Common *cm, int mi_size);
- void (*free_mi)(struct VP10Common *cm);
- void (*setup_mi)(struct VP10Common *cm);
-
- // Grid of pointers to 8x8 MODE_INFO structs. Any 8x8 not in the visible
- // area will be NULL.
- MODE_INFO **mi_grid_base;
- MODE_INFO **mi_grid_visible;
- MODE_INFO **prev_mi_grid_base;
- MODE_INFO **prev_mi_grid_visible;
-
- // Whether to use previous frame's motion vectors for prediction.
- int use_prev_frame_mvs;
-
- // Persistent mb segment id map used in prediction.
- int seg_map_idx;
- int prev_seg_map_idx;
-
- uint8_t *seg_map_array[NUM_PING_PONG_BUFFERS];
- uint8_t *last_frame_seg_map;
- uint8_t *current_frame_seg_map;
- int seg_map_alloc_size;
-
- INTERP_FILTER interp_filter;
-
- loop_filter_info_n lf_info;
-
- // Flag signaling how frame contexts should be updated at the end of
- // a frame decode
- REFRESH_FRAME_CONTEXT_MODE refresh_frame_context;
-
- int ref_frame_sign_bias[MAX_REF_FRAMES]; /* Two state 0, 1 */
-
- struct loopfilter lf;
- struct segmentation seg;
-#if !CONFIG_MISC_FIXES
- struct segmentation_probs segp;
-#endif
-
- int frame_parallel_decode; // frame-based threading.
-
- // Context probabilities for reference frame prediction
- MV_REFERENCE_FRAME comp_fixed_ref;
- MV_REFERENCE_FRAME comp_var_ref[2];
- REFERENCE_MODE reference_mode;
-
- FRAME_CONTEXT *fc; /* this frame entropy */
- FRAME_CONTEXT *frame_contexts; // FRAME_CONTEXTS
- unsigned int frame_context_idx; /* Context to use/update */
- FRAME_COUNTS counts;
-
- unsigned int current_video_frame;
- BITSTREAM_PROFILE profile;
-
- // VPX_BITS_8 in profile 0 or 1, VPX_BITS_10 or VPX_BITS_12 in profile 2 or 3.
- vpx_bit_depth_t bit_depth;
- vpx_bit_depth_t dequant_bit_depth; // bit_depth of current dequantizer
-
-#if CONFIG_VP9_POSTPROC
- struct postproc_state postproc_state;
-#endif
-
- int error_resilient_mode;
-
- int log2_tile_cols, log2_tile_rows;
- int tile_sz_mag;
- int byte_alignment;
- int skip_loop_filter;
-
- // Private data associated with the frame buffer callbacks.
- void *cb_priv;
- vpx_get_frame_buffer_cb_fn_t get_fb_cb;
- vpx_release_frame_buffer_cb_fn_t release_fb_cb;
-
- // Handles memory for the codec.
- InternalFrameBufferList int_frame_buffers;
-
- // External BufferPool passed from outside.
- BufferPool *buffer_pool;
-
- PARTITION_CONTEXT *above_seg_context;
- ENTROPY_CONTEXT *above_context;
- int above_context_alloc_cols;
-
- // scratch memory for intraonly/keyframe forward updates from default tables
- // - this is intentionally not placed in FRAME_CONTEXT since it's reset upon
- // each keyframe and not used afterwards
- vpx_prob kf_y_prob[INTRA_MODES][INTRA_MODES][INTRA_MODES - 1];
-} VP10_COMMON;
-
-// TODO(hkuang): Don't need to lock the whole pool after implementing atomic
-// frame reference count.
-static void lock_buffer_pool(BufferPool *const pool) {
-#if CONFIG_MULTITHREAD
- pthread_mutex_lock(&pool->pool_mutex);
-#else
- (void)pool;
-#endif
-}
-
-static void unlock_buffer_pool(BufferPool *const pool) {
-#if CONFIG_MULTITHREAD
- pthread_mutex_unlock(&pool->pool_mutex);
-#else
- (void)pool;
-#endif
-}
-
-static INLINE YV12_BUFFER_CONFIG *get_ref_frame(VP10_COMMON *cm, int index) {
- if (index < 0 || index >= REF_FRAMES)
- return NULL;
- if (cm->ref_frame_map[index] < 0)
- return NULL;
- assert(cm->ref_frame_map[index] < FRAME_BUFFERS);
- return &cm->buffer_pool->frame_bufs[cm->ref_frame_map[index]].buf;
-}
-
-static INLINE YV12_BUFFER_CONFIG *get_frame_new_buffer(VP10_COMMON *cm) {
- return &cm->buffer_pool->frame_bufs[cm->new_fb_idx].buf;
-}
-
-static INLINE int get_free_fb(VP10_COMMON *cm) {
- RefCntBuffer *const frame_bufs = cm->buffer_pool->frame_bufs;
- int i;
-
- lock_buffer_pool(cm->buffer_pool);
- for (i = 0; i < FRAME_BUFFERS; ++i)
- if (frame_bufs[i].ref_count == 0)
- break;
-
- if (i != FRAME_BUFFERS) {
- frame_bufs[i].ref_count = 1;
- } else {
- // Reset i to be INVALID_IDX to indicate no free buffer found.
- i = INVALID_IDX;
- }
-
- unlock_buffer_pool(cm->buffer_pool);
- return i;
-}
-
-static INLINE void ref_cnt_fb(RefCntBuffer *bufs, int *idx, int new_idx) {
- const int ref_index = *idx;
-
- if (ref_index >= 0 && bufs[ref_index].ref_count > 0)
- bufs[ref_index].ref_count--;
-
- *idx = new_idx;
-
- bufs[new_idx].ref_count++;
-}
-
-static INLINE int mi_cols_aligned_to_sb(int n_mis) {
- return ALIGN_POWER_OF_TWO(n_mis, MI_BLOCK_SIZE_LOG2);
-}
-
-static INLINE int frame_is_intra_only(const VP10_COMMON *const cm) {
- return cm->frame_type == KEY_FRAME || cm->intra_only;
-}
-
-static INLINE void vp10_init_macroblockd(VP10_COMMON *cm, MACROBLOCKD *xd,
- tran_low_t *dqcoeff) {
- int i;
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- xd->plane[i].dqcoeff = dqcoeff;
- xd->above_context[i] = cm->above_context +
- i * sizeof(*cm->above_context) * 2 * mi_cols_aligned_to_sb(cm->mi_cols);
-
- if (xd->plane[i].plane_type == PLANE_TYPE_Y) {
- memcpy(xd->plane[i].seg_dequant, cm->y_dequant, sizeof(cm->y_dequant));
- } else {
- memcpy(xd->plane[i].seg_dequant, cm->uv_dequant, sizeof(cm->uv_dequant));
- }
- xd->fc = cm->fc;
- }
-
- xd->above_seg_context = cm->above_seg_context;
- xd->mi_stride = cm->mi_stride;
- xd->error_info = &cm->error;
-}
-
-static INLINE void set_skip_context(MACROBLOCKD *xd, int mi_row, int mi_col) {
- const int above_idx = mi_col * 2;
- const int left_idx = (mi_row * 2) & 15;
- int i;
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- struct macroblockd_plane *const pd = &xd->plane[i];
- pd->above_context = &xd->above_context[i][above_idx >> pd->subsampling_x];
- pd->left_context = &xd->left_context[i][left_idx >> pd->subsampling_y];
- }
-}
-
-static INLINE int calc_mi_size(int len) {
- // len is in mi units.
- return len + MI_BLOCK_SIZE;
-}
-
-static INLINE void set_mi_row_col(MACROBLOCKD *xd, const TileInfo *const tile,
- int mi_row, int bh,
- int mi_col, int bw,
- int mi_rows, int mi_cols) {
- xd->mb_to_top_edge = -((mi_row * MI_SIZE) * 8);
- xd->mb_to_bottom_edge = ((mi_rows - bh - mi_row) * MI_SIZE) * 8;
- xd->mb_to_left_edge = -((mi_col * MI_SIZE) * 8);
- xd->mb_to_right_edge = ((mi_cols - bw - mi_col) * MI_SIZE) * 8;
-
- // Are edges available for intra prediction?
- xd->up_available = (mi_row != 0);
- xd->left_available = (mi_col > tile->mi_col_start);
- if (xd->up_available) {
- xd->above_mi = xd->mi[-xd->mi_stride];
- // above_mi may be NULL in VP9 encoder's first pass.
- xd->above_mbmi = xd->above_mi ? &xd->above_mi->mbmi : NULL;
- } else {
- xd->above_mi = NULL;
- xd->above_mbmi = NULL;
- }
-
- if (xd->left_available) {
- xd->left_mi = xd->mi[-1];
- // left_mi may be NULL in VP9 encoder's first pass.
- xd->left_mbmi = xd->left_mi ? &xd->left_mi->mbmi : NULL;
- } else {
- xd->left_mi = NULL;
- xd->left_mbmi = NULL;
- }
-}
-
-static INLINE const vpx_prob *get_y_mode_probs(const VP10_COMMON *cm,
- const MODE_INFO *mi,
- const MODE_INFO *above_mi,
- const MODE_INFO *left_mi,
- int block) {
- const PREDICTION_MODE above = vp10_above_block_mode(mi, above_mi, block);
- const PREDICTION_MODE left = vp10_left_block_mode(mi, left_mi, block);
- return cm->kf_y_prob[above][left];
-}
-
-static INLINE void update_partition_context(MACROBLOCKD *xd,
- int mi_row, int mi_col,
- BLOCK_SIZE subsize,
- BLOCK_SIZE bsize) {
- PARTITION_CONTEXT *const above_ctx = xd->above_seg_context + mi_col;
- PARTITION_CONTEXT *const left_ctx = xd->left_seg_context + (mi_row & MI_MASK);
-
- // num_4x4_blocks_wide_lookup[bsize] / 2
- const int bs = num_8x8_blocks_wide_lookup[bsize];
-
- // update the partition context at the end notes. set partition bits
- // of block sizes larger than the current one to be one, and partition
- // bits of smaller block sizes to be zero.
- memset(above_ctx, partition_context_lookup[subsize].above, bs);
- memset(left_ctx, partition_context_lookup[subsize].left, bs);
-}
-
-static INLINE int partition_plane_context(const MACROBLOCKD *xd,
- int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- const PARTITION_CONTEXT *above_ctx = xd->above_seg_context + mi_col;
- const PARTITION_CONTEXT *left_ctx = xd->left_seg_context + (mi_row & MI_MASK);
- const int bsl = mi_width_log2_lookup[bsize];
- int above = (*above_ctx >> bsl) & 1 , left = (*left_ctx >> bsl) & 1;
-
- assert(b_width_log2_lookup[bsize] == b_height_log2_lookup[bsize]);
- assert(bsl >= 0);
-
- return (left * 2 + above) + bsl * PARTITION_PLOFFSET;
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_ONYXC_INT_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-#include <stdlib.h>
-#include <stdio.h>
-
-#include "./vpx_config.h"
-#include "./vpx_scale_rtcd.h"
-#include "./vp10_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-#include "vpx_scale/vpx_scale.h"
-#include "vpx_scale/yv12config.h"
-
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/postproc.h"
-#include "vp10/common/textblit.h"
-
-#if CONFIG_VP9_POSTPROC
-static const short kernel5[] = {
- 1, 1, 4, 1, 1
-};
-
-const short vp10_rv[] = {
- 8, 5, 2, 2, 8, 12, 4, 9, 8, 3,
- 0, 3, 9, 0, 0, 0, 8, 3, 14, 4,
- 10, 1, 11, 14, 1, 14, 9, 6, 12, 11,
- 8, 6, 10, 0, 0, 8, 9, 0, 3, 14,
- 8, 11, 13, 4, 2, 9, 0, 3, 9, 6,
- 1, 2, 3, 14, 13, 1, 8, 2, 9, 7,
- 3, 3, 1, 13, 13, 6, 6, 5, 2, 7,
- 11, 9, 11, 8, 7, 3, 2, 0, 13, 13,
- 14, 4, 12, 5, 12, 10, 8, 10, 13, 10,
- 4, 14, 4, 10, 0, 8, 11, 1, 13, 7,
- 7, 14, 6, 14, 13, 2, 13, 5, 4, 4,
- 0, 10, 0, 5, 13, 2, 12, 7, 11, 13,
- 8, 0, 4, 10, 7, 2, 7, 2, 2, 5,
- 3, 4, 7, 3, 3, 14, 14, 5, 9, 13,
- 3, 14, 3, 6, 3, 0, 11, 8, 13, 1,
- 13, 1, 12, 0, 10, 9, 7, 6, 2, 8,
- 5, 2, 13, 7, 1, 13, 14, 7, 6, 7,
- 9, 6, 10, 11, 7, 8, 7, 5, 14, 8,
- 4, 4, 0, 8, 7, 10, 0, 8, 14, 11,
- 3, 12, 5, 7, 14, 3, 14, 5, 2, 6,
- 11, 12, 12, 8, 0, 11, 13, 1, 2, 0,
- 5, 10, 14, 7, 8, 0, 4, 11, 0, 8,
- 0, 3, 10, 5, 8, 0, 11, 6, 7, 8,
- 10, 7, 13, 9, 2, 5, 1, 5, 10, 2,
- 4, 3, 5, 6, 10, 8, 9, 4, 11, 14,
- 0, 10, 0, 5, 13, 2, 12, 7, 11, 13,
- 8, 0, 4, 10, 7, 2, 7, 2, 2, 5,
- 3, 4, 7, 3, 3, 14, 14, 5, 9, 13,
- 3, 14, 3, 6, 3, 0, 11, 8, 13, 1,
- 13, 1, 12, 0, 10, 9, 7, 6, 2, 8,
- 5, 2, 13, 7, 1, 13, 14, 7, 6, 7,
- 9, 6, 10, 11, 7, 8, 7, 5, 14, 8,
- 4, 4, 0, 8, 7, 10, 0, 8, 14, 11,
- 3, 12, 5, 7, 14, 3, 14, 5, 2, 6,
- 11, 12, 12, 8, 0, 11, 13, 1, 2, 0,
- 5, 10, 14, 7, 8, 0, 4, 11, 0, 8,
- 0, 3, 10, 5, 8, 0, 11, 6, 7, 8,
- 10, 7, 13, 9, 2, 5, 1, 5, 10, 2,
- 4, 3, 5, 6, 10, 8, 9, 4, 11, 14,
- 3, 8, 3, 7, 8, 5, 11, 4, 12, 3,
- 11, 9, 14, 8, 14, 13, 4, 3, 1, 2,
- 14, 6, 5, 4, 4, 11, 4, 6, 2, 1,
- 5, 8, 8, 12, 13, 5, 14, 10, 12, 13,
- 0, 9, 5, 5, 11, 10, 13, 9, 10, 13,
-};
-
-static const uint8_t q_diff_thresh = 20;
-static const uint8_t last_q_thresh = 170;
-
-void vp10_post_proc_down_and_across_c(const uint8_t *src_ptr,
- uint8_t *dst_ptr,
- int src_pixels_per_line,
- int dst_pixels_per_line,
- int rows,
- int cols,
- int flimit) {
- uint8_t const *p_src;
- uint8_t *p_dst;
- int row, col, i, v, kernel;
- int pitch = src_pixels_per_line;
- uint8_t d[8];
- (void)dst_pixels_per_line;
-
- for (row = 0; row < rows; row++) {
- /* post_proc_down for one row */
- p_src = src_ptr;
- p_dst = dst_ptr;
-
- for (col = 0; col < cols; col++) {
- kernel = 4;
- v = p_src[col];
-
- for (i = -2; i <= 2; i++) {
- if (abs(v - p_src[col + i * pitch]) > flimit)
- goto down_skip_convolve;
-
- kernel += kernel5[2 + i] * p_src[col + i * pitch];
- }
-
- v = (kernel >> 3);
- down_skip_convolve:
- p_dst[col] = v;
- }
-
- /* now post_proc_across */
- p_src = dst_ptr;
- p_dst = dst_ptr;
-
- for (i = 0; i < 8; i++)
- d[i] = p_src[i];
-
- for (col = 0; col < cols; col++) {
- kernel = 4;
- v = p_src[col];
-
- d[col & 7] = v;
-
- for (i = -2; i <= 2; i++) {
- if (abs(v - p_src[col + i]) > flimit)
- goto across_skip_convolve;
-
- kernel += kernel5[2 + i] * p_src[col + i];
- }
-
- d[col & 7] = (kernel >> 3);
- across_skip_convolve:
-
- if (col >= 2)
- p_dst[col - 2] = d[(col - 2) & 7];
- }
-
- /* handle the last two pixels */
- p_dst[col - 2] = d[(col - 2) & 7];
- p_dst[col - 1] = d[(col - 1) & 7];
-
-
- /* next row */
- src_ptr += pitch;
- dst_ptr += pitch;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_post_proc_down_and_across_c(const uint16_t *src_ptr,
- uint16_t *dst_ptr,
- int src_pixels_per_line,
- int dst_pixels_per_line,
- int rows,
- int cols,
- int flimit) {
- uint16_t const *p_src;
- uint16_t *p_dst;
- int row, col, i, v, kernel;
- int pitch = src_pixels_per_line;
- uint16_t d[8];
-
- for (row = 0; row < rows; row++) {
- // post_proc_down for one row.
- p_src = src_ptr;
- p_dst = dst_ptr;
-
- for (col = 0; col < cols; col++) {
- kernel = 4;
- v = p_src[col];
-
- for (i = -2; i <= 2; i++) {
- if (abs(v - p_src[col + i * pitch]) > flimit)
- goto down_skip_convolve;
-
- kernel += kernel5[2 + i] * p_src[col + i * pitch];
- }
-
- v = (kernel >> 3);
-
- down_skip_convolve:
- p_dst[col] = v;
- }
-
- /* now post_proc_across */
- p_src = dst_ptr;
- p_dst = dst_ptr;
-
- for (i = 0; i < 8; i++)
- d[i] = p_src[i];
-
- for (col = 0; col < cols; col++) {
- kernel = 4;
- v = p_src[col];
-
- d[col & 7] = v;
-
- for (i = -2; i <= 2; i++) {
- if (abs(v - p_src[col + i]) > flimit)
- goto across_skip_convolve;
-
- kernel += kernel5[2 + i] * p_src[col + i];
- }
-
- d[col & 7] = (kernel >> 3);
-
- across_skip_convolve:
- if (col >= 2)
- p_dst[col - 2] = d[(col - 2) & 7];
- }
-
- /* handle the last two pixels */
- p_dst[col - 2] = d[(col - 2) & 7];
- p_dst[col - 1] = d[(col - 1) & 7];
-
-
- /* next row */
- src_ptr += pitch;
- dst_ptr += dst_pixels_per_line;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static int q2mbl(int x) {
- if (x < 20) x = 20;
-
- x = 50 + (x - 50) * 10 / 8;
- return x * x / 3;
-}
-
-void vp10_mbpost_proc_across_ip_c(uint8_t *src, int pitch,
- int rows, int cols, int flimit) {
- int r, c, i;
- uint8_t *s = src;
- uint8_t d[16];
-
- for (r = 0; r < rows; r++) {
- int sumsq = 0;
- int sum = 0;
-
- for (i = -8; i <= 6; i++) {
- sumsq += s[i] * s[i];
- sum += s[i];
- d[i + 8] = 0;
- }
-
- for (c = 0; c < cols + 8; c++) {
- int x = s[c + 7] - s[c - 8];
- int y = s[c + 7] + s[c - 8];
-
- sum += x;
- sumsq += x * y;
-
- d[c & 15] = s[c];
-
- if (sumsq * 15 - sum * sum < flimit) {
- d[c & 15] = (8 + sum + s[c]) >> 4;
- }
-
- s[c - 8] = d[(c - 8) & 15];
- }
- s += pitch;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_mbpost_proc_across_ip_c(uint16_t *src, int pitch,
- int rows, int cols, int flimit) {
- int r, c, i;
-
- uint16_t *s = src;
- uint16_t d[16];
-
-
- for (r = 0; r < rows; r++) {
- int sumsq = 0;
- int sum = 0;
-
- for (i = -8; i <= 6; i++) {
- sumsq += s[i] * s[i];
- sum += s[i];
- d[i + 8] = 0;
- }
-
- for (c = 0; c < cols + 8; c++) {
- int x = s[c + 7] - s[c - 8];
- int y = s[c + 7] + s[c - 8];
-
- sum += x;
- sumsq += x * y;
-
- d[c & 15] = s[c];
-
- if (sumsq * 15 - sum * sum < flimit) {
- d[c & 15] = (8 + sum + s[c]) >> 4;
- }
-
- s[c - 8] = d[(c - 8) & 15];
- }
-
- s += pitch;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_mbpost_proc_down_c(uint8_t *dst, int pitch,
- int rows, int cols, int flimit) {
- int r, c, i;
- const short *rv3 = &vp10_rv[63 & rand()]; // NOLINT
-
- for (c = 0; c < cols; c++) {
- uint8_t *s = &dst[c];
- int sumsq = 0;
- int sum = 0;
- uint8_t d[16];
- const short *rv2 = rv3 + ((c * 17) & 127);
-
- for (i = -8; i <= 6; i++) {
- sumsq += s[i * pitch] * s[i * pitch];
- sum += s[i * pitch];
- }
-
- for (r = 0; r < rows + 8; r++) {
- sumsq += s[7 * pitch] * s[ 7 * pitch] - s[-8 * pitch] * s[-8 * pitch];
- sum += s[7 * pitch] - s[-8 * pitch];
- d[r & 15] = s[0];
-
- if (sumsq * 15 - sum * sum < flimit) {
- d[r & 15] = (rv2[r & 127] + sum + s[0]) >> 4;
- }
-
- s[-8 * pitch] = d[(r - 8) & 15];
- s += pitch;
- }
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_mbpost_proc_down_c(uint16_t *dst, int pitch,
- int rows, int cols, int flimit) {
- int r, c, i;
- const int16_t *rv3 = &vp10_rv[63 & rand()]; // NOLINT
-
- for (c = 0; c < cols; c++) {
- uint16_t *s = &dst[c];
- int sumsq = 0;
- int sum = 0;
- uint16_t d[16];
- const int16_t *rv2 = rv3 + ((c * 17) & 127);
-
- for (i = -8; i <= 6; i++) {
- sumsq += s[i * pitch] * s[i * pitch];
- sum += s[i * pitch];
- }
-
- for (r = 0; r < rows + 8; r++) {
- sumsq += s[7 * pitch] * s[ 7 * pitch] - s[-8 * pitch] * s[-8 * pitch];
- sum += s[7 * pitch] - s[-8 * pitch];
- d[r & 15] = s[0];
-
- if (sumsq * 15 - sum * sum < flimit) {
- d[r & 15] = (rv2[r & 127] + sum + s[0]) >> 4;
- }
-
- s[-8 * pitch] = d[(r - 8) & 15];
- s += pitch;
- }
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void deblock_and_de_macro_block(YV12_BUFFER_CONFIG *source,
- YV12_BUFFER_CONFIG *post,
- int q,
- int low_var_thresh,
- int flag) {
- double level = 6.0e-05 * q * q * q - .0067 * q * q + .306 * q + .0065;
- int ppl = (int)(level + .5);
- (void) low_var_thresh;
- (void) flag;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (source->flags & YV12_FLAG_HIGHBITDEPTH) {
- vp10_highbd_post_proc_down_and_across(CONVERT_TO_SHORTPTR(source->y_buffer),
- CONVERT_TO_SHORTPTR(post->y_buffer),
- source->y_stride, post->y_stride,
- source->y_height, source->y_width,
- ppl);
-
- vp10_highbd_mbpost_proc_across_ip(CONVERT_TO_SHORTPTR(post->y_buffer),
- post->y_stride, post->y_height,
- post->y_width, q2mbl(q));
-
- vp10_highbd_mbpost_proc_down(CONVERT_TO_SHORTPTR(post->y_buffer),
- post->y_stride, post->y_height,
- post->y_width, q2mbl(q));
-
- vp10_highbd_post_proc_down_and_across(CONVERT_TO_SHORTPTR(source->u_buffer),
- CONVERT_TO_SHORTPTR(post->u_buffer),
- source->uv_stride, post->uv_stride,
- source->uv_height, source->uv_width,
- ppl);
- vp10_highbd_post_proc_down_and_across(CONVERT_TO_SHORTPTR(source->v_buffer),
- CONVERT_TO_SHORTPTR(post->v_buffer),
- source->uv_stride, post->uv_stride,
- source->uv_height, source->uv_width,
- ppl);
- } else {
- vp10_post_proc_down_and_across(source->y_buffer, post->y_buffer,
- source->y_stride, post->y_stride,
- source->y_height, source->y_width, ppl);
-
- vp10_mbpost_proc_across_ip(post->y_buffer, post->y_stride, post->y_height,
- post->y_width, q2mbl(q));
-
- vp10_mbpost_proc_down(post->y_buffer, post->y_stride, post->y_height,
- post->y_width, q2mbl(q));
-
- vp10_post_proc_down_and_across(source->u_buffer, post->u_buffer,
- source->uv_stride, post->uv_stride,
- source->uv_height, source->uv_width, ppl);
- vp10_post_proc_down_and_across(source->v_buffer, post->v_buffer,
- source->uv_stride, post->uv_stride,
- source->uv_height, source->uv_width, ppl);
- }
-#else
- vp10_post_proc_down_and_across(source->y_buffer, post->y_buffer,
- source->y_stride, post->y_stride,
- source->y_height, source->y_width, ppl);
-
- vp10_mbpost_proc_across_ip(post->y_buffer, post->y_stride, post->y_height,
- post->y_width, q2mbl(q));
-
- vp10_mbpost_proc_down(post->y_buffer, post->y_stride, post->y_height,
- post->y_width, q2mbl(q));
-
- vp10_post_proc_down_and_across(source->u_buffer, post->u_buffer,
- source->uv_stride, post->uv_stride,
- source->uv_height, source->uv_width, ppl);
- vp10_post_proc_down_and_across(source->v_buffer, post->v_buffer,
- source->uv_stride, post->uv_stride,
- source->uv_height, source->uv_width, ppl);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-}
-
-void vp10_deblock(const YV12_BUFFER_CONFIG *src, YV12_BUFFER_CONFIG *dst,
- int q) {
- const int ppl = (int)(6.0e-05 * q * q * q - 0.0067 * q * q + 0.306 * q
- + 0.0065 + 0.5);
- int i;
-
- const uint8_t *const srcs[3] = {src->y_buffer, src->u_buffer, src->v_buffer};
- const int src_strides[3] = {src->y_stride, src->uv_stride, src->uv_stride};
- const int src_widths[3] = {src->y_width, src->uv_width, src->uv_width};
- const int src_heights[3] = {src->y_height, src->uv_height, src->uv_height};
-
- uint8_t *const dsts[3] = {dst->y_buffer, dst->u_buffer, dst->v_buffer};
- const int dst_strides[3] = {dst->y_stride, dst->uv_stride, dst->uv_stride};
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
-#if CONFIG_VP9_HIGHBITDEPTH
- assert((src->flags & YV12_FLAG_HIGHBITDEPTH) ==
- (dst->flags & YV12_FLAG_HIGHBITDEPTH));
- if (src->flags & YV12_FLAG_HIGHBITDEPTH) {
- vp10_highbd_post_proc_down_and_across(CONVERT_TO_SHORTPTR(srcs[i]),
- CONVERT_TO_SHORTPTR(dsts[i]),
- src_strides[i], dst_strides[i],
- src_heights[i], src_widths[i], ppl);
- } else {
- vp10_post_proc_down_and_across(srcs[i], dsts[i],
- src_strides[i], dst_strides[i],
- src_heights[i], src_widths[i], ppl);
- }
-#else
- vp10_post_proc_down_and_across(srcs[i], dsts[i],
- src_strides[i], dst_strides[i],
- src_heights[i], src_widths[i], ppl);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
-}
-
-void vp10_denoise(const YV12_BUFFER_CONFIG *src, YV12_BUFFER_CONFIG *dst,
- int q) {
- const int ppl = (int)(6.0e-05 * q * q * q - 0.0067 * q * q + 0.306 * q
- + 0.0065 + 0.5);
- int i;
-
- const uint8_t *const srcs[3] = {src->y_buffer, src->u_buffer, src->v_buffer};
- const int src_strides[3] = {src->y_stride, src->uv_stride, src->uv_stride};
- const int src_widths[3] = {src->y_width, src->uv_width, src->uv_width};
- const int src_heights[3] = {src->y_height, src->uv_height, src->uv_height};
-
- uint8_t *const dsts[3] = {dst->y_buffer, dst->u_buffer, dst->v_buffer};
- const int dst_strides[3] = {dst->y_stride, dst->uv_stride, dst->uv_stride};
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- const int src_stride = src_strides[i];
- const int src_width = src_widths[i] - 4;
- const int src_height = src_heights[i] - 4;
- const int dst_stride = dst_strides[i];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- assert((src->flags & YV12_FLAG_HIGHBITDEPTH) ==
- (dst->flags & YV12_FLAG_HIGHBITDEPTH));
- if (src->flags & YV12_FLAG_HIGHBITDEPTH) {
- const uint16_t *const src_plane = CONVERT_TO_SHORTPTR(
- srcs[i] + 2 * src_stride + 2);
- uint16_t *const dst_plane = CONVERT_TO_SHORTPTR(
- dsts[i] + 2 * dst_stride + 2);
- vp10_highbd_post_proc_down_and_across(src_plane, dst_plane, src_stride,
- dst_stride, src_height, src_width,
- ppl);
- } else {
- const uint8_t *const src_plane = srcs[i] + 2 * src_stride + 2;
- uint8_t *const dst_plane = dsts[i] + 2 * dst_stride + 2;
-
- vp10_post_proc_down_and_across(src_plane, dst_plane, src_stride,
- dst_stride, src_height, src_width, ppl);
- }
-#else
- const uint8_t *const src_plane = srcs[i] + 2 * src_stride + 2;
- uint8_t *const dst_plane = dsts[i] + 2 * dst_stride + 2;
- vp10_post_proc_down_and_across(src_plane, dst_plane, src_stride, dst_stride,
- src_height, src_width, ppl);
-#endif
- }
-}
-
-static double gaussian(double sigma, double mu, double x) {
- return 1 / (sigma * sqrt(2.0 * 3.14159265)) *
- (exp(-(x - mu) * (x - mu) / (2 * sigma * sigma)));
-}
-
-static void fillrd(struct postproc_state *state, int q, int a) {
- char char_dist[300];
-
- double sigma;
- int ai = a, qi = q, i;
-
- vpx_clear_system_state();
-
- sigma = ai + .5 + .6 * (63 - qi) / 63.0;
-
- /* set up a lookup table of 256 entries that matches
- * a gaussian distribution with sigma determined by q.
- */
- {
- int next, j;
-
- next = 0;
-
- for (i = -32; i < 32; i++) {
- int a_i = (int)(0.5 + 256 * gaussian(sigma, 0, i));
-
- if (a_i) {
- for (j = 0; j < a_i; j++) {
- char_dist[next + j] = (char) i;
- }
-
- next = next + j;
- }
- }
-
- for (; next < 256; next++)
- char_dist[next] = 0;
- }
-
- for (i = 0; i < 3072; i++) {
- state->noise[i] = char_dist[rand() & 0xff]; // NOLINT
- }
-
- for (i = 0; i < 16; i++) {
- state->blackclamp[i] = -char_dist[0];
- state->whiteclamp[i] = -char_dist[0];
- state->bothclamp[i] = -2 * char_dist[0];
- }
-
- state->last_q = q;
- state->last_noise = a;
-}
-
-void vp10_plane_add_noise_c(uint8_t *start, char *noise,
- char blackclamp[16],
- char whiteclamp[16],
- char bothclamp[16],
- unsigned int width, unsigned int height, int pitch) {
- unsigned int i, j;
-
- // TODO(jbb): why does simd code use both but c doesn't, normalize and
- // fix..
- (void) bothclamp;
- for (i = 0; i < height; i++) {
- uint8_t *pos = start + i * pitch;
- char *ref = (char *)(noise + (rand() & 0xff)); // NOLINT
-
- for (j = 0; j < width; j++) {
- if (pos[j] < blackclamp[0])
- pos[j] = blackclamp[0];
-
- if (pos[j] > 255 + whiteclamp[0])
- pos[j] = 255 + whiteclamp[0];
-
- pos[j] += ref[j];
- }
- }
-}
-
-static void swap_mi_and_prev_mi(VP10_COMMON *cm) {
- // Current mip will be the prev_mip for the next frame.
- MODE_INFO *temp = cm->postproc_state.prev_mip;
- cm->postproc_state.prev_mip = cm->mip;
- cm->mip = temp;
-
- // Update the upper left visible macroblock ptrs.
- cm->mi = cm->mip + cm->mi_stride + 1;
- cm->postproc_state.prev_mi = cm->postproc_state.prev_mip + cm->mi_stride + 1;
-}
-
-int vp10_post_proc_frame(struct VP10Common *cm,
- YV12_BUFFER_CONFIG *dest, vp10_ppflags_t *ppflags) {
- const int q = VPXMIN(105, cm->lf.filter_level * 2);
- const int flags = ppflags->post_proc_flag;
- YV12_BUFFER_CONFIG *const ppbuf = &cm->post_proc_buffer;
- struct postproc_state *const ppstate = &cm->postproc_state;
-
- if (!cm->frame_to_show)
- return -1;
-
- if (!flags) {
- *dest = *cm->frame_to_show;
- return 0;
- }
-
- vpx_clear_system_state();
-
- // Alloc memory for prev_mip in the first frame.
- if (cm->current_video_frame == 1) {
- cm->postproc_state.last_base_qindex = cm->base_qindex;
- cm->postproc_state.last_frame_valid = 1;
- ppstate->prev_mip = vpx_calloc(cm->mi_alloc_size, sizeof(*cm->mip));
- if (!ppstate->prev_mip) {
- return 1;
- }
- ppstate->prev_mi = ppstate->prev_mip + cm->mi_stride + 1;
- memset(ppstate->prev_mip, 0,
- cm->mi_stride * (cm->mi_rows + 1) * sizeof(*cm->mip));
- }
-
- // Allocate post_proc_buffer_int if needed.
- if ((flags & VP9D_MFQE) && !cm->post_proc_buffer_int.buffer_alloc) {
- if ((flags & VP9D_DEMACROBLOCK) || (flags & VP9D_DEBLOCK)) {
- const int width = ALIGN_POWER_OF_TWO(cm->width, 4);
- const int height = ALIGN_POWER_OF_TWO(cm->height, 4);
-
- if (vpx_alloc_frame_buffer(&cm->post_proc_buffer_int, width, height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif // CONFIG_VP9_HIGHBITDEPTH
- VP9_ENC_BORDER_IN_PIXELS,
- cm->byte_alignment) < 0) {
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate MFQE framebuffer");
- }
-
- // Ensure that postproc is set to all 0s so that post proc
- // doesn't pull random data in from edge.
- memset(cm->post_proc_buffer_int.buffer_alloc, 128,
- cm->post_proc_buffer.frame_size);
- }
- }
-
- if (vpx_realloc_frame_buffer(&cm->post_proc_buffer, cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_DEC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL) < 0)
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate post-processing buffer");
-
- if ((flags & VP9D_MFQE) && cm->current_video_frame >= 2 &&
- cm->postproc_state.last_frame_valid && cm->bit_depth == 8 &&
- cm->postproc_state.last_base_qindex <= last_q_thresh &&
- cm->base_qindex - cm->postproc_state.last_base_qindex >= q_diff_thresh) {
- vp10_mfqe(cm);
- // TODO(jackychen): Consider whether enable deblocking by default
- // if mfqe is enabled. Need to take both the quality and the speed
- // into consideration.
- if ((flags & VP9D_DEMACROBLOCK) || (flags & VP9D_DEBLOCK)) {
- vp8_yv12_copy_frame(ppbuf, &cm->post_proc_buffer_int);
- }
- if ((flags & VP9D_DEMACROBLOCK) && cm->post_proc_buffer_int.buffer_alloc) {
- deblock_and_de_macro_block(&cm->post_proc_buffer_int, ppbuf,
- q + (ppflags->deblocking_level - 5) * 10,
- 1, 0);
- } else if (flags & VP9D_DEBLOCK) {
- vp10_deblock(&cm->post_proc_buffer_int, ppbuf, q);
- } else {
- vp8_yv12_copy_frame(&cm->post_proc_buffer_int, ppbuf);
- }
- } else if (flags & VP9D_DEMACROBLOCK) {
- deblock_and_de_macro_block(cm->frame_to_show, ppbuf,
- q + (ppflags->deblocking_level - 5) * 10, 1, 0);
- } else if (flags & VP9D_DEBLOCK) {
- vp10_deblock(cm->frame_to_show, ppbuf, q);
- } else {
- vp8_yv12_copy_frame(cm->frame_to_show, ppbuf);
- }
-
- cm->postproc_state.last_base_qindex = cm->base_qindex;
- cm->postproc_state.last_frame_valid = 1;
-
- if (flags & VP9D_ADDNOISE) {
- const int noise_level = ppflags->noise_level;
- if (ppstate->last_q != q ||
- ppstate->last_noise != noise_level) {
- fillrd(ppstate, 63 - q, noise_level);
- }
-
- vp10_plane_add_noise(ppbuf->y_buffer, ppstate->noise, ppstate->blackclamp,
- ppstate->whiteclamp, ppstate->bothclamp,
- ppbuf->y_width, ppbuf->y_height, ppbuf->y_stride);
- }
-
- *dest = *ppbuf;
-
- /* handle problem with extending borders */
- dest->y_width = cm->width;
- dest->y_height = cm->height;
- dest->uv_width = dest->y_width >> cm->subsampling_x;
- dest->uv_height = dest->y_height >> cm->subsampling_y;
-
- swap_mi_and_prev_mi(cm);
- return 0;
-}
-#endif // CONFIG_VP9_POSTPROC
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_COMMON_POSTPROC_H_
-#define VP10_COMMON_POSTPROC_H_
-
-#include "vpx_ports/mem.h"
-#include "vpx_scale/yv12config.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/mfqe.h"
-#include "vp10/common/ppflags.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct postproc_state {
- int last_q;
- int last_noise;
- char noise[3072];
- int last_base_qindex;
- int last_frame_valid;
- MODE_INFO *prev_mip;
- MODE_INFO *prev_mi;
- DECLARE_ALIGNED(16, char, blackclamp[16]);
- DECLARE_ALIGNED(16, char, whiteclamp[16]);
- DECLARE_ALIGNED(16, char, bothclamp[16]);
-};
-
-struct VP10Common;
-
-#define MFQE_PRECISION 4
-
-int vp10_post_proc_frame(struct VP10Common *cm,
- YV12_BUFFER_CONFIG *dest, vp10_ppflags_t *flags);
-
-void vp10_denoise(const YV12_BUFFER_CONFIG *src, YV12_BUFFER_CONFIG *dst, int q);
-
-void vp10_deblock(const YV12_BUFFER_CONFIG *src, YV12_BUFFER_CONFIG *dst, int q);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_POSTPROC_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_PPFLAGS_H_
-#define VP10_COMMON_PPFLAGS_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-enum {
- VP9D_NOFILTERING = 0,
- VP9D_DEBLOCK = 1 << 0,
- VP9D_DEMACROBLOCK = 1 << 1,
- VP9D_ADDNOISE = 1 << 2,
- VP9D_DEBUG_TXT_FRAME_INFO = 1 << 3,
- VP9D_DEBUG_TXT_MBLK_MODES = 1 << 4,
- VP9D_DEBUG_TXT_DC_DIFF = 1 << 5,
- VP9D_DEBUG_TXT_RATE_INFO = 1 << 6,
- VP9D_DEBUG_DRAW_MV = 1 << 7,
- VP9D_DEBUG_CLR_BLK_MODES = 1 << 8,
- VP9D_DEBUG_CLR_FRM_REF_BLKS = 1 << 9,
- VP9D_MFQE = 1 << 10
-};
-
-typedef struct {
- int post_proc_flag;
- int deblocking_level;
- int noise_level;
-} vp10_ppflags_t;
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_PPFLAGS_H_
+++ /dev/null
-
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/seg_common.h"
-
-// Returns a context number for the given MB prediction signal
-int vp10_get_pred_context_switchable_interp(const MACROBLOCKD *xd) {
- // Note:
- // The mode info data structure has a one element border above and to the
- // left of the entries correpsonding to real macroblocks.
- // The prediction flags in these dummy entries are initialised to 0.
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int left_type = xd->left_available && is_inter_block(left_mbmi) ?
- left_mbmi->interp_filter : SWITCHABLE_FILTERS;
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const int above_type = xd->up_available && is_inter_block(above_mbmi) ?
- above_mbmi->interp_filter : SWITCHABLE_FILTERS;
-
- if (left_type == above_type)
- return left_type;
- else if (left_type == SWITCHABLE_FILTERS && above_type != SWITCHABLE_FILTERS)
- return above_type;
- else if (left_type != SWITCHABLE_FILTERS && above_type == SWITCHABLE_FILTERS)
- return left_type;
- else
- return SWITCHABLE_FILTERS;
-}
-
-// The mode info data structure has a one element border above and to the
-// left of the entries corresponding to real macroblocks.
-// The prediction flags in these dummy entries are initialized to 0.
-// 0 - inter/inter, inter/--, --/inter, --/--
-// 1 - intra/inter, inter/intra
-// 2 - intra/--, --/intra
-// 3 - intra/intra
-int vp10_get_intra_inter_context(const MACROBLOCKD *xd) {
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int has_above = xd->up_available;
- const int has_left = xd->left_available;
-
- if (has_above && has_left) { // both edges available
- const int above_intra = !is_inter_block(above_mbmi);
- const int left_intra = !is_inter_block(left_mbmi);
- return left_intra && above_intra ? 3
- : left_intra || above_intra;
- } else if (has_above || has_left) { // one edge available
- return 2 * !is_inter_block(has_above ? above_mbmi : left_mbmi);
- } else {
- return 0;
- }
-}
-
-int vp10_get_reference_mode_context(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- int ctx;
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int has_above = xd->up_available;
- const int has_left = xd->left_available;
- // Note:
- // The mode info data structure has a one element border above and to the
- // left of the entries correpsonding to real macroblocks.
- // The prediction flags in these dummy entries are initialised to 0.
- if (has_above && has_left) { // both edges available
- if (!has_second_ref(above_mbmi) && !has_second_ref(left_mbmi))
- // neither edge uses comp pred (0/1)
- ctx = (above_mbmi->ref_frame[0] == cm->comp_fixed_ref) ^
- (left_mbmi->ref_frame[0] == cm->comp_fixed_ref);
- else if (!has_second_ref(above_mbmi))
- // one of two edges uses comp pred (2/3)
- ctx = 2 + (above_mbmi->ref_frame[0] == cm->comp_fixed_ref ||
- !is_inter_block(above_mbmi));
- else if (!has_second_ref(left_mbmi))
- // one of two edges uses comp pred (2/3)
- ctx = 2 + (left_mbmi->ref_frame[0] == cm->comp_fixed_ref ||
- !is_inter_block(left_mbmi));
- else // both edges use comp pred (4)
- ctx = 4;
- } else if (has_above || has_left) { // one edge available
- const MB_MODE_INFO *edge_mbmi = has_above ? above_mbmi : left_mbmi;
-
- if (!has_second_ref(edge_mbmi))
- // edge does not use comp pred (0/1)
- ctx = edge_mbmi->ref_frame[0] == cm->comp_fixed_ref;
- else
- // edge uses comp pred (3)
- ctx = 3;
- } else { // no edges available (1)
- ctx = 1;
- }
- assert(ctx >= 0 && ctx < COMP_INTER_CONTEXTS);
- return ctx;
-}
-
-// Returns a context number for the given MB prediction signal
-int vp10_get_pred_context_comp_ref_p(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- int pred_context;
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int above_in_image = xd->up_available;
- const int left_in_image = xd->left_available;
-
- // Note:
- // The mode info data structure has a one element border above and to the
- // left of the entries correpsonding to real macroblocks.
- // The prediction flags in these dummy entries are initialised to 0.
- const int fix_ref_idx = cm->ref_frame_sign_bias[cm->comp_fixed_ref];
- const int var_ref_idx = !fix_ref_idx;
-
- if (above_in_image && left_in_image) { // both edges available
- const int above_intra = !is_inter_block(above_mbmi);
- const int left_intra = !is_inter_block(left_mbmi);
-
- if (above_intra && left_intra) { // intra/intra (2)
- pred_context = 2;
- } else if (above_intra || left_intra) { // intra/inter
- const MB_MODE_INFO *edge_mbmi = above_intra ? left_mbmi : above_mbmi;
-
- if (!has_second_ref(edge_mbmi)) // single pred (1/3)
- pred_context = 1 + 2 * (edge_mbmi->ref_frame[0] != cm->comp_var_ref[1]);
- else // comp pred (1/3)
- pred_context = 1 + 2 * (edge_mbmi->ref_frame[var_ref_idx]
- != cm->comp_var_ref[1]);
- } else { // inter/inter
- const int l_sg = !has_second_ref(left_mbmi);
- const int a_sg = !has_second_ref(above_mbmi);
- const MV_REFERENCE_FRAME vrfa = a_sg ? above_mbmi->ref_frame[0]
- : above_mbmi->ref_frame[var_ref_idx];
- const MV_REFERENCE_FRAME vrfl = l_sg ? left_mbmi->ref_frame[0]
- : left_mbmi->ref_frame[var_ref_idx];
-
- if (vrfa == vrfl && cm->comp_var_ref[1] == vrfa) {
- pred_context = 0;
- } else if (l_sg && a_sg) { // single/single
- if ((vrfa == cm->comp_fixed_ref && vrfl == cm->comp_var_ref[0]) ||
- (vrfl == cm->comp_fixed_ref && vrfa == cm->comp_var_ref[0]))
- pred_context = 4;
- else if (vrfa == vrfl)
- pred_context = 3;
- else
- pred_context = 1;
- } else if (l_sg || a_sg) { // single/comp
- const MV_REFERENCE_FRAME vrfc = l_sg ? vrfa : vrfl;
- const MV_REFERENCE_FRAME rfs = a_sg ? vrfa : vrfl;
- if (vrfc == cm->comp_var_ref[1] && rfs != cm->comp_var_ref[1])
- pred_context = 1;
- else if (rfs == cm->comp_var_ref[1] && vrfc != cm->comp_var_ref[1])
- pred_context = 2;
- else
- pred_context = 4;
- } else if (vrfa == vrfl) { // comp/comp
- pred_context = 4;
- } else {
- pred_context = 2;
- }
- }
- } else if (above_in_image || left_in_image) { // one edge available
- const MB_MODE_INFO *edge_mbmi = above_in_image ? above_mbmi : left_mbmi;
-
- if (!is_inter_block(edge_mbmi)) {
- pred_context = 2;
- } else {
- if (has_second_ref(edge_mbmi))
- pred_context = 4 * (edge_mbmi->ref_frame[var_ref_idx]
- != cm->comp_var_ref[1]);
- else
- pred_context = 3 * (edge_mbmi->ref_frame[0] != cm->comp_var_ref[1]);
- }
- } else { // no edges available (2)
- pred_context = 2;
- }
- assert(pred_context >= 0 && pred_context < REF_CONTEXTS);
-
- return pred_context;
-}
-
-int vp10_get_pred_context_single_ref_p1(const MACROBLOCKD *xd) {
- int pred_context;
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int has_above = xd->up_available;
- const int has_left = xd->left_available;
- // Note:
- // The mode info data structure has a one element border above and to the
- // left of the entries correpsonding to real macroblocks.
- // The prediction flags in these dummy entries are initialised to 0.
- if (has_above && has_left) { // both edges available
- const int above_intra = !is_inter_block(above_mbmi);
- const int left_intra = !is_inter_block(left_mbmi);
-
- if (above_intra && left_intra) { // intra/intra
- pred_context = 2;
- } else if (above_intra || left_intra) { // intra/inter or inter/intra
- const MB_MODE_INFO *edge_mbmi = above_intra ? left_mbmi : above_mbmi;
- if (!has_second_ref(edge_mbmi))
- pred_context = 4 * (edge_mbmi->ref_frame[0] == LAST_FRAME);
- else
- pred_context = 1 + (edge_mbmi->ref_frame[0] == LAST_FRAME ||
- edge_mbmi->ref_frame[1] == LAST_FRAME);
- } else { // inter/inter
- const int above_has_second = has_second_ref(above_mbmi);
- const int left_has_second = has_second_ref(left_mbmi);
- const MV_REFERENCE_FRAME above0 = above_mbmi->ref_frame[0];
- const MV_REFERENCE_FRAME above1 = above_mbmi->ref_frame[1];
- const MV_REFERENCE_FRAME left0 = left_mbmi->ref_frame[0];
- const MV_REFERENCE_FRAME left1 = left_mbmi->ref_frame[1];
-
- if (above_has_second && left_has_second) {
- pred_context = 1 + (above0 == LAST_FRAME || above1 == LAST_FRAME ||
- left0 == LAST_FRAME || left1 == LAST_FRAME);
- } else if (above_has_second || left_has_second) {
- const MV_REFERENCE_FRAME rfs = !above_has_second ? above0 : left0;
- const MV_REFERENCE_FRAME crf1 = above_has_second ? above0 : left0;
- const MV_REFERENCE_FRAME crf2 = above_has_second ? above1 : left1;
-
- if (rfs == LAST_FRAME)
- pred_context = 3 + (crf1 == LAST_FRAME || crf2 == LAST_FRAME);
- else
- pred_context = (crf1 == LAST_FRAME || crf2 == LAST_FRAME);
- } else {
- pred_context = 2 * (above0 == LAST_FRAME) + 2 * (left0 == LAST_FRAME);
- }
- }
- } else if (has_above || has_left) { // one edge available
- const MB_MODE_INFO *edge_mbmi = has_above ? above_mbmi : left_mbmi;
- if (!is_inter_block(edge_mbmi)) { // intra
- pred_context = 2;
- } else { // inter
- if (!has_second_ref(edge_mbmi))
- pred_context = 4 * (edge_mbmi->ref_frame[0] == LAST_FRAME);
- else
- pred_context = 1 + (edge_mbmi->ref_frame[0] == LAST_FRAME ||
- edge_mbmi->ref_frame[1] == LAST_FRAME);
- }
- } else { // no edges available
- pred_context = 2;
- }
-
- assert(pred_context >= 0 && pred_context < REF_CONTEXTS);
- return pred_context;
-}
-
-int vp10_get_pred_context_single_ref_p2(const MACROBLOCKD *xd) {
- int pred_context;
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int has_above = xd->up_available;
- const int has_left = xd->left_available;
-
- // Note:
- // The mode info data structure has a one element border above and to the
- // left of the entries correpsonding to real macroblocks.
- // The prediction flags in these dummy entries are initialised to 0.
- if (has_above && has_left) { // both edges available
- const int above_intra = !is_inter_block(above_mbmi);
- const int left_intra = !is_inter_block(left_mbmi);
-
- if (above_intra && left_intra) { // intra/intra
- pred_context = 2;
- } else if (above_intra || left_intra) { // intra/inter or inter/intra
- const MB_MODE_INFO *edge_mbmi = above_intra ? left_mbmi : above_mbmi;
- if (!has_second_ref(edge_mbmi)) {
- if (edge_mbmi->ref_frame[0] == LAST_FRAME)
- pred_context = 3;
- else
- pred_context = 4 * (edge_mbmi->ref_frame[0] == GOLDEN_FRAME);
- } else {
- pred_context = 1 + 2 * (edge_mbmi->ref_frame[0] == GOLDEN_FRAME ||
- edge_mbmi->ref_frame[1] == GOLDEN_FRAME);
- }
- } else { // inter/inter
- const int above_has_second = has_second_ref(above_mbmi);
- const int left_has_second = has_second_ref(left_mbmi);
- const MV_REFERENCE_FRAME above0 = above_mbmi->ref_frame[0];
- const MV_REFERENCE_FRAME above1 = above_mbmi->ref_frame[1];
- const MV_REFERENCE_FRAME left0 = left_mbmi->ref_frame[0];
- const MV_REFERENCE_FRAME left1 = left_mbmi->ref_frame[1];
-
- if (above_has_second && left_has_second) {
- if (above0 == left0 && above1 == left1)
- pred_context = 3 * (above0 == GOLDEN_FRAME ||
- above1 == GOLDEN_FRAME ||
- left0 == GOLDEN_FRAME ||
- left1 == GOLDEN_FRAME);
- else
- pred_context = 2;
- } else if (above_has_second || left_has_second) {
- const MV_REFERENCE_FRAME rfs = !above_has_second ? above0 : left0;
- const MV_REFERENCE_FRAME crf1 = above_has_second ? above0 : left0;
- const MV_REFERENCE_FRAME crf2 = above_has_second ? above1 : left1;
-
- if (rfs == GOLDEN_FRAME)
- pred_context = 3 + (crf1 == GOLDEN_FRAME || crf2 == GOLDEN_FRAME);
- else if (rfs == ALTREF_FRAME)
- pred_context = crf1 == GOLDEN_FRAME || crf2 == GOLDEN_FRAME;
- else
- pred_context = 1 + 2 * (crf1 == GOLDEN_FRAME || crf2 == GOLDEN_FRAME);
- } else {
- if (above0 == LAST_FRAME && left0 == LAST_FRAME) {
- pred_context = 3;
- } else if (above0 == LAST_FRAME || left0 == LAST_FRAME) {
- const MV_REFERENCE_FRAME edge0 = (above0 == LAST_FRAME) ? left0
- : above0;
- pred_context = 4 * (edge0 == GOLDEN_FRAME);
- } else {
- pred_context = 2 * (above0 == GOLDEN_FRAME) +
- 2 * (left0 == GOLDEN_FRAME);
- }
- }
- }
- } else if (has_above || has_left) { // one edge available
- const MB_MODE_INFO *edge_mbmi = has_above ? above_mbmi : left_mbmi;
-
- if (!is_inter_block(edge_mbmi) ||
- (edge_mbmi->ref_frame[0] == LAST_FRAME && !has_second_ref(edge_mbmi)))
- pred_context = 2;
- else if (!has_second_ref(edge_mbmi))
- pred_context = 4 * (edge_mbmi->ref_frame[0] == GOLDEN_FRAME);
- else
- pred_context = 3 * (edge_mbmi->ref_frame[0] == GOLDEN_FRAME ||
- edge_mbmi->ref_frame[1] == GOLDEN_FRAME);
- } else { // no edges available (2)
- pred_context = 2;
- }
- assert(pred_context >= 0 && pred_context < REF_CONTEXTS);
- return pred_context;
-}
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_PRED_COMMON_H_
-#define VP10_COMMON_PRED_COMMON_H_
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/onyxc_int.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-static INLINE int get_segment_id(const VP10_COMMON *cm,
- const uint8_t *segment_ids,
- BLOCK_SIZE bsize, int mi_row, int mi_col) {
- const int mi_offset = mi_row * cm->mi_cols + mi_col;
- const int bw = num_8x8_blocks_wide_lookup[bsize];
- const int bh = num_8x8_blocks_high_lookup[bsize];
- const int xmis = VPXMIN(cm->mi_cols - mi_col, bw);
- const int ymis = VPXMIN(cm->mi_rows - mi_row, bh);
- int x, y, segment_id = MAX_SEGMENTS;
-
- for (y = 0; y < ymis; ++y)
- for (x = 0; x < xmis; ++x)
- segment_id =
- VPXMIN(segment_id, segment_ids[mi_offset + y * cm->mi_cols + x]);
-
- assert(segment_id >= 0 && segment_id < MAX_SEGMENTS);
- return segment_id;
-}
-
-static INLINE int vp10_get_pred_context_seg_id(const MACROBLOCKD *xd) {
- const MODE_INFO *const above_mi = xd->above_mi;
- const MODE_INFO *const left_mi = xd->left_mi;
- const int above_sip = (above_mi != NULL) ?
- above_mi->mbmi.seg_id_predicted : 0;
- const int left_sip = (left_mi != NULL) ? left_mi->mbmi.seg_id_predicted : 0;
-
- return above_sip + left_sip;
-}
-
-static INLINE vpx_prob vp10_get_pred_prob_seg_id(
- const struct segmentation_probs *segp, const MACROBLOCKD *xd) {
- return segp->pred_probs[vp10_get_pred_context_seg_id(xd)];
-}
-
-static INLINE int vp10_get_skip_context(const MACROBLOCKD *xd) {
- const MODE_INFO *const above_mi = xd->above_mi;
- const MODE_INFO *const left_mi = xd->left_mi;
- const int above_skip = (above_mi != NULL) ? above_mi->mbmi.skip : 0;
- const int left_skip = (left_mi != NULL) ? left_mi->mbmi.skip : 0;
- return above_skip + left_skip;
-}
-
-static INLINE vpx_prob vp10_get_skip_prob(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- return cm->fc->skip_probs[vp10_get_skip_context(xd)];
-}
-
-int vp10_get_pred_context_switchable_interp(const MACROBLOCKD *xd);
-
-int vp10_get_intra_inter_context(const MACROBLOCKD *xd);
-
-static INLINE vpx_prob vp10_get_intra_inter_prob(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- return cm->fc->intra_inter_prob[vp10_get_intra_inter_context(xd)];
-}
-
-int vp10_get_reference_mode_context(const VP10_COMMON *cm,
- const MACROBLOCKD *xd);
-
-static INLINE vpx_prob vp10_get_reference_mode_prob(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- return cm->fc->comp_inter_prob[vp10_get_reference_mode_context(cm, xd)];
-}
-
-int vp10_get_pred_context_comp_ref_p(const VP10_COMMON *cm,
- const MACROBLOCKD *xd);
-
-static INLINE vpx_prob vp10_get_pred_prob_comp_ref_p(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- const int pred_context = vp10_get_pred_context_comp_ref_p(cm, xd);
- return cm->fc->comp_ref_prob[pred_context];
-}
-
-int vp10_get_pred_context_single_ref_p1(const MACROBLOCKD *xd);
-
-static INLINE vpx_prob vp10_get_pred_prob_single_ref_p1(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- return cm->fc->single_ref_prob[vp10_get_pred_context_single_ref_p1(xd)][0];
-}
-
-int vp10_get_pred_context_single_ref_p2(const MACROBLOCKD *xd);
-
-static INLINE vpx_prob vp10_get_pred_prob_single_ref_p2(const VP10_COMMON *cm,
- const MACROBLOCKD *xd) {
- return cm->fc->single_ref_prob[vp10_get_pred_context_single_ref_p2(xd)][1];
-}
-
-// Returns a context number for the given MB prediction signal
-// The mode info data structure has a one element border above and to the
-// left of the entries corresponding to real blocks.
-// The prediction flags in these dummy entries are initialized to 0.
-static INLINE int get_tx_size_context(const MACROBLOCKD *xd) {
- const int max_tx_size = max_txsize_lookup[xd->mi[0]->mbmi.sb_type];
- const MB_MODE_INFO *const above_mbmi = xd->above_mbmi;
- const MB_MODE_INFO *const left_mbmi = xd->left_mbmi;
- const int has_above = xd->up_available;
- const int has_left = xd->left_available;
- int above_ctx = (has_above && !above_mbmi->skip) ? (int)above_mbmi->tx_size
- : max_tx_size;
- int left_ctx = (has_left && !left_mbmi->skip) ? (int)left_mbmi->tx_size
- : max_tx_size;
- if (!has_left)
- left_ctx = above_ctx;
-
- if (!has_above)
- above_ctx = left_ctx;
-
- return (above_ctx + left_ctx) > max_tx_size;
-}
-
-static INLINE const vpx_prob *get_tx_probs(TX_SIZE max_tx_size, int ctx,
- const struct tx_probs *tx_probs) {
- switch (max_tx_size) {
- case TX_8X8:
- return tx_probs->p8x8[ctx];
- case TX_16X16:
- return tx_probs->p16x16[ctx];
- case TX_32X32:
- return tx_probs->p32x32[ctx];
- default:
- assert(0 && "Invalid max_tx_size.");
- return NULL;
- }
-}
-
-static INLINE const vpx_prob *get_tx_probs2(TX_SIZE max_tx_size,
- const MACROBLOCKD *xd,
- const struct tx_probs *tx_probs) {
- return get_tx_probs(max_tx_size, get_tx_size_context(xd), tx_probs);
-}
-
-static INLINE unsigned int *get_tx_counts(TX_SIZE max_tx_size, int ctx,
- struct tx_counts *tx_counts) {
- switch (max_tx_size) {
- case TX_8X8:
- return tx_counts->p8x8[ctx];
- case TX_16X16:
- return tx_counts->p16x16[ctx];
- case TX_32X32:
- return tx_counts->p32x32[ctx];
- default:
- assert(0 && "Invalid max_tx_size.");
- return NULL;
- }
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_PRED_COMMON_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/common.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/seg_common.h"
-
-static const int16_t dc_qlookup[QINDEX_RANGE] = {
- 4, 8, 8, 9, 10, 11, 12, 12,
- 13, 14, 15, 16, 17, 18, 19, 19,
- 20, 21, 22, 23, 24, 25, 26, 26,
- 27, 28, 29, 30, 31, 32, 32, 33,
- 34, 35, 36, 37, 38, 38, 39, 40,
- 41, 42, 43, 43, 44, 45, 46, 47,
- 48, 48, 49, 50, 51, 52, 53, 53,
- 54, 55, 56, 57, 57, 58, 59, 60,
- 61, 62, 62, 63, 64, 65, 66, 66,
- 67, 68, 69, 70, 70, 71, 72, 73,
- 74, 74, 75, 76, 77, 78, 78, 79,
- 80, 81, 81, 82, 83, 84, 85, 85,
- 87, 88, 90, 92, 93, 95, 96, 98,
- 99, 101, 102, 104, 105, 107, 108, 110,
- 111, 113, 114, 116, 117, 118, 120, 121,
- 123, 125, 127, 129, 131, 134, 136, 138,
- 140, 142, 144, 146, 148, 150, 152, 154,
- 156, 158, 161, 164, 166, 169, 172, 174,
- 177, 180, 182, 185, 187, 190, 192, 195,
- 199, 202, 205, 208, 211, 214, 217, 220,
- 223, 226, 230, 233, 237, 240, 243, 247,
- 250, 253, 257, 261, 265, 269, 272, 276,
- 280, 284, 288, 292, 296, 300, 304, 309,
- 313, 317, 322, 326, 330, 335, 340, 344,
- 349, 354, 359, 364, 369, 374, 379, 384,
- 389, 395, 400, 406, 411, 417, 423, 429,
- 435, 441, 447, 454, 461, 467, 475, 482,
- 489, 497, 505, 513, 522, 530, 539, 549,
- 559, 569, 579, 590, 602, 614, 626, 640,
- 654, 668, 684, 700, 717, 736, 755, 775,
- 796, 819, 843, 869, 896, 925, 955, 988,
- 1022, 1058, 1098, 1139, 1184, 1232, 1282, 1336,
-};
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static const int16_t dc_qlookup_10[QINDEX_RANGE] = {
- 4, 9, 10, 13, 15, 17, 20, 22,
- 25, 28, 31, 34, 37, 40, 43, 47,
- 50, 53, 57, 60, 64, 68, 71, 75,
- 78, 82, 86, 90, 93, 97, 101, 105,
- 109, 113, 116, 120, 124, 128, 132, 136,
- 140, 143, 147, 151, 155, 159, 163, 166,
- 170, 174, 178, 182, 185, 189, 193, 197,
- 200, 204, 208, 212, 215, 219, 223, 226,
- 230, 233, 237, 241, 244, 248, 251, 255,
- 259, 262, 266, 269, 273, 276, 280, 283,
- 287, 290, 293, 297, 300, 304, 307, 310,
- 314, 317, 321, 324, 327, 331, 334, 337,
- 343, 350, 356, 362, 369, 375, 381, 387,
- 394, 400, 406, 412, 418, 424, 430, 436,
- 442, 448, 454, 460, 466, 472, 478, 484,
- 490, 499, 507, 516, 525, 533, 542, 550,
- 559, 567, 576, 584, 592, 601, 609, 617,
- 625, 634, 644, 655, 666, 676, 687, 698,
- 708, 718, 729, 739, 749, 759, 770, 782,
- 795, 807, 819, 831, 844, 856, 868, 880,
- 891, 906, 920, 933, 947, 961, 975, 988,
- 1001, 1015, 1030, 1045, 1061, 1076, 1090, 1105,
- 1120, 1137, 1153, 1170, 1186, 1202, 1218, 1236,
- 1253, 1271, 1288, 1306, 1323, 1342, 1361, 1379,
- 1398, 1416, 1436, 1456, 1476, 1496, 1516, 1537,
- 1559, 1580, 1601, 1624, 1647, 1670, 1692, 1717,
- 1741, 1766, 1791, 1817, 1844, 1871, 1900, 1929,
- 1958, 1990, 2021, 2054, 2088, 2123, 2159, 2197,
- 2236, 2276, 2319, 2363, 2410, 2458, 2508, 2561,
- 2616, 2675, 2737, 2802, 2871, 2944, 3020, 3102,
- 3188, 3280, 3375, 3478, 3586, 3702, 3823, 3953,
- 4089, 4236, 4394, 4559, 4737, 4929, 5130, 5347,
-};
-
-static const int16_t dc_qlookup_12[QINDEX_RANGE] = {
- 4, 12, 18, 25, 33, 41, 50, 60,
- 70, 80, 91, 103, 115, 127, 140, 153,
- 166, 180, 194, 208, 222, 237, 251, 266,
- 281, 296, 312, 327, 343, 358, 374, 390,
- 405, 421, 437, 453, 469, 484, 500, 516,
- 532, 548, 564, 580, 596, 611, 627, 643,
- 659, 674, 690, 706, 721, 737, 752, 768,
- 783, 798, 814, 829, 844, 859, 874, 889,
- 904, 919, 934, 949, 964, 978, 993, 1008,
- 1022, 1037, 1051, 1065, 1080, 1094, 1108, 1122,
- 1136, 1151, 1165, 1179, 1192, 1206, 1220, 1234,
- 1248, 1261, 1275, 1288, 1302, 1315, 1329, 1342,
- 1368, 1393, 1419, 1444, 1469, 1494, 1519, 1544,
- 1569, 1594, 1618, 1643, 1668, 1692, 1717, 1741,
- 1765, 1789, 1814, 1838, 1862, 1885, 1909, 1933,
- 1957, 1992, 2027, 2061, 2096, 2130, 2165, 2199,
- 2233, 2267, 2300, 2334, 2367, 2400, 2434, 2467,
- 2499, 2532, 2575, 2618, 2661, 2704, 2746, 2788,
- 2830, 2872, 2913, 2954, 2995, 3036, 3076, 3127,
- 3177, 3226, 3275, 3324, 3373, 3421, 3469, 3517,
- 3565, 3621, 3677, 3733, 3788, 3843, 3897, 3951,
- 4005, 4058, 4119, 4181, 4241, 4301, 4361, 4420,
- 4479, 4546, 4612, 4677, 4742, 4807, 4871, 4942,
- 5013, 5083, 5153, 5222, 5291, 5367, 5442, 5517,
- 5591, 5665, 5745, 5825, 5905, 5984, 6063, 6149,
- 6234, 6319, 6404, 6495, 6587, 6678, 6769, 6867,
- 6966, 7064, 7163, 7269, 7376, 7483, 7599, 7715,
- 7832, 7958, 8085, 8214, 8352, 8492, 8635, 8788,
- 8945, 9104, 9275, 9450, 9639, 9832, 10031, 10245,
- 10465, 10702, 10946, 11210, 11482, 11776, 12081, 12409,
- 12750, 13118, 13501, 13913, 14343, 14807, 15290, 15812,
- 16356, 16943, 17575, 18237, 18949, 19718, 20521, 21387,
-};
-#endif
-
-static const int16_t ac_qlookup[QINDEX_RANGE] = {
- 4, 8, 9, 10, 11, 12, 13, 14,
- 15, 16, 17, 18, 19, 20, 21, 22,
- 23, 24, 25, 26, 27, 28, 29, 30,
- 31, 32, 33, 34, 35, 36, 37, 38,
- 39, 40, 41, 42, 43, 44, 45, 46,
- 47, 48, 49, 50, 51, 52, 53, 54,
- 55, 56, 57, 58, 59, 60, 61, 62,
- 63, 64, 65, 66, 67, 68, 69, 70,
- 71, 72, 73, 74, 75, 76, 77, 78,
- 79, 80, 81, 82, 83, 84, 85, 86,
- 87, 88, 89, 90, 91, 92, 93, 94,
- 95, 96, 97, 98, 99, 100, 101, 102,
- 104, 106, 108, 110, 112, 114, 116, 118,
- 120, 122, 124, 126, 128, 130, 132, 134,
- 136, 138, 140, 142, 144, 146, 148, 150,
- 152, 155, 158, 161, 164, 167, 170, 173,
- 176, 179, 182, 185, 188, 191, 194, 197,
- 200, 203, 207, 211, 215, 219, 223, 227,
- 231, 235, 239, 243, 247, 251, 255, 260,
- 265, 270, 275, 280, 285, 290, 295, 300,
- 305, 311, 317, 323, 329, 335, 341, 347,
- 353, 359, 366, 373, 380, 387, 394, 401,
- 408, 416, 424, 432, 440, 448, 456, 465,
- 474, 483, 492, 501, 510, 520, 530, 540,
- 550, 560, 571, 582, 593, 604, 615, 627,
- 639, 651, 663, 676, 689, 702, 715, 729,
- 743, 757, 771, 786, 801, 816, 832, 848,
- 864, 881, 898, 915, 933, 951, 969, 988,
- 1007, 1026, 1046, 1066, 1087, 1108, 1129, 1151,
- 1173, 1196, 1219, 1243, 1267, 1292, 1317, 1343,
- 1369, 1396, 1423, 1451, 1479, 1508, 1537, 1567,
- 1597, 1628, 1660, 1692, 1725, 1759, 1793, 1828,
-};
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static const int16_t ac_qlookup_10[QINDEX_RANGE] = {
- 4, 9, 11, 13, 16, 18, 21, 24,
- 27, 30, 33, 37, 40, 44, 48, 51,
- 55, 59, 63, 67, 71, 75, 79, 83,
- 88, 92, 96, 100, 105, 109, 114, 118,
- 122, 127, 131, 136, 140, 145, 149, 154,
- 158, 163, 168, 172, 177, 181, 186, 190,
- 195, 199, 204, 208, 213, 217, 222, 226,
- 231, 235, 240, 244, 249, 253, 258, 262,
- 267, 271, 275, 280, 284, 289, 293, 297,
- 302, 306, 311, 315, 319, 324, 328, 332,
- 337, 341, 345, 349, 354, 358, 362, 367,
- 371, 375, 379, 384, 388, 392, 396, 401,
- 409, 417, 425, 433, 441, 449, 458, 466,
- 474, 482, 490, 498, 506, 514, 523, 531,
- 539, 547, 555, 563, 571, 579, 588, 596,
- 604, 616, 628, 640, 652, 664, 676, 688,
- 700, 713, 725, 737, 749, 761, 773, 785,
- 797, 809, 825, 841, 857, 873, 889, 905,
- 922, 938, 954, 970, 986, 1002, 1018, 1038,
- 1058, 1078, 1098, 1118, 1138, 1158, 1178, 1198,
- 1218, 1242, 1266, 1290, 1314, 1338, 1362, 1386,
- 1411, 1435, 1463, 1491, 1519, 1547, 1575, 1603,
- 1631, 1663, 1695, 1727, 1759, 1791, 1823, 1859,
- 1895, 1931, 1967, 2003, 2039, 2079, 2119, 2159,
- 2199, 2239, 2283, 2327, 2371, 2415, 2459, 2507,
- 2555, 2603, 2651, 2703, 2755, 2807, 2859, 2915,
- 2971, 3027, 3083, 3143, 3203, 3263, 3327, 3391,
- 3455, 3523, 3591, 3659, 3731, 3803, 3876, 3952,
- 4028, 4104, 4184, 4264, 4348, 4432, 4516, 4604,
- 4692, 4784, 4876, 4972, 5068, 5168, 5268, 5372,
- 5476, 5584, 5692, 5804, 5916, 6032, 6148, 6268,
- 6388, 6512, 6640, 6768, 6900, 7036, 7172, 7312,
-};
-
-static const int16_t ac_qlookup_12[QINDEX_RANGE] = {
- 4, 13, 19, 27, 35, 44, 54, 64,
- 75, 87, 99, 112, 126, 139, 154, 168,
- 183, 199, 214, 230, 247, 263, 280, 297,
- 314, 331, 349, 366, 384, 402, 420, 438,
- 456, 475, 493, 511, 530, 548, 567, 586,
- 604, 623, 642, 660, 679, 698, 716, 735,
- 753, 772, 791, 809, 828, 846, 865, 884,
- 902, 920, 939, 957, 976, 994, 1012, 1030,
- 1049, 1067, 1085, 1103, 1121, 1139, 1157, 1175,
- 1193, 1211, 1229, 1246, 1264, 1282, 1299, 1317,
- 1335, 1352, 1370, 1387, 1405, 1422, 1440, 1457,
- 1474, 1491, 1509, 1526, 1543, 1560, 1577, 1595,
- 1627, 1660, 1693, 1725, 1758, 1791, 1824, 1856,
- 1889, 1922, 1954, 1987, 2020, 2052, 2085, 2118,
- 2150, 2183, 2216, 2248, 2281, 2313, 2346, 2378,
- 2411, 2459, 2508, 2556, 2605, 2653, 2701, 2750,
- 2798, 2847, 2895, 2943, 2992, 3040, 3088, 3137,
- 3185, 3234, 3298, 3362, 3426, 3491, 3555, 3619,
- 3684, 3748, 3812, 3876, 3941, 4005, 4069, 4149,
- 4230, 4310, 4390, 4470, 4550, 4631, 4711, 4791,
- 4871, 4967, 5064, 5160, 5256, 5352, 5448, 5544,
- 5641, 5737, 5849, 5961, 6073, 6185, 6297, 6410,
- 6522, 6650, 6778, 6906, 7034, 7162, 7290, 7435,
- 7579, 7723, 7867, 8011, 8155, 8315, 8475, 8635,
- 8795, 8956, 9132, 9308, 9484, 9660, 9836, 10028,
- 10220, 10412, 10604, 10812, 11020, 11228, 11437, 11661,
- 11885, 12109, 12333, 12573, 12813, 13053, 13309, 13565,
- 13821, 14093, 14365, 14637, 14925, 15213, 15502, 15806,
- 16110, 16414, 16734, 17054, 17390, 17726, 18062, 18414,
- 18766, 19134, 19502, 19886, 20270, 20670, 21070, 21486,
- 21902, 22334, 22766, 23214, 23662, 24126, 24590, 25070,
- 25551, 26047, 26559, 27071, 27599, 28143, 28687, 29247,
-};
-#endif
-
-int16_t vp10_dc_quant(int qindex, int delta, vpx_bit_depth_t bit_depth) {
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (bit_depth) {
- case VPX_BITS_8:
- return dc_qlookup[clamp(qindex + delta, 0, MAXQ)];
- case VPX_BITS_10:
- return dc_qlookup_10[clamp(qindex + delta, 0, MAXQ)];
- case VPX_BITS_12:
- return dc_qlookup_12[clamp(qindex + delta, 0, MAXQ)];
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1;
- }
-#else
- (void) bit_depth;
- return dc_qlookup[clamp(qindex + delta, 0, MAXQ)];
-#endif
-}
-
-int16_t vp10_ac_quant(int qindex, int delta, vpx_bit_depth_t bit_depth) {
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (bit_depth) {
- case VPX_BITS_8:
- return ac_qlookup[clamp(qindex + delta, 0, MAXQ)];
- case VPX_BITS_10:
- return ac_qlookup_10[clamp(qindex + delta, 0, MAXQ)];
- case VPX_BITS_12:
- return ac_qlookup_12[clamp(qindex + delta, 0, MAXQ)];
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1;
- }
-#else
- (void) bit_depth;
- return ac_qlookup[clamp(qindex + delta, 0, MAXQ)];
-#endif
-}
-
-int vp10_get_qindex(const struct segmentation *seg, int segment_id,
- int base_qindex) {
- if (segfeature_active(seg, segment_id, SEG_LVL_ALT_Q)) {
- const int data = get_segdata(seg, segment_id, SEG_LVL_ALT_Q);
- const int seg_qindex = seg->abs_delta == SEGMENT_ABSDATA ?
- data : base_qindex + data;
- return clamp(seg_qindex, 0, MAXQ);
- } else {
- return base_qindex;
- }
-}
-
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_QUANT_COMMON_H_
-#define VP10_COMMON_QUANT_COMMON_H_
-
-#include "vpx/vpx_codec.h"
-#include "vp10/common/seg_common.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MINQ 0
-#define MAXQ 255
-#define QINDEX_RANGE (MAXQ - MINQ + 1)
-#define QINDEX_BITS 8
-
-int16_t vp10_dc_quant(int qindex, int delta, vpx_bit_depth_t bit_depth);
-int16_t vp10_ac_quant(int qindex, int delta, vpx_bit_depth_t bit_depth);
-
-int vp10_get_qindex(const struct segmentation *seg, int segment_id,
- int base_qindex);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_QUANT_COMMON_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "./vpx_scale_rtcd.h"
-#include "./vpx_config.h"
-
-#include "vpx/vpx_integer.h"
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/reconintra.h"
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_build_inter_predictor(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- const MV *src_mv,
- const struct scale_factors *sf,
- int w, int h, int ref,
- const InterpKernel *kernel,
- enum mv_precision precision,
- int x, int y, int bd) {
- const int is_q4 = precision == MV_PRECISION_Q4;
- const MV mv_q4 = { is_q4 ? src_mv->row : src_mv->row * 2,
- is_q4 ? src_mv->col : src_mv->col * 2 };
- MV32 mv = vp10_scale_mv(&mv_q4, x, y, sf);
- const int subpel_x = mv.col & SUBPEL_MASK;
- const int subpel_y = mv.row & SUBPEL_MASK;
-
- src += (mv.row >> SUBPEL_BITS) * src_stride + (mv.col >> SUBPEL_BITS);
-
- high_inter_predictor(src, src_stride, dst, dst_stride, subpel_x, subpel_y,
- sf, w, h, ref, kernel, sf->x_step_q4, sf->y_step_q4, bd);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_build_inter_predictor(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- const MV *src_mv,
- const struct scale_factors *sf,
- int w, int h, int ref,
- const InterpKernel *kernel,
- enum mv_precision precision,
- int x, int y) {
- const int is_q4 = precision == MV_PRECISION_Q4;
- const MV mv_q4 = { is_q4 ? src_mv->row : src_mv->row * 2,
- is_q4 ? src_mv->col : src_mv->col * 2 };
- MV32 mv = vp10_scale_mv(&mv_q4, x, y, sf);
- const int subpel_x = mv.col & SUBPEL_MASK;
- const int subpel_y = mv.row & SUBPEL_MASK;
-
- src += (mv.row >> SUBPEL_BITS) * src_stride + (mv.col >> SUBPEL_BITS);
-
- inter_predictor(src, src_stride, dst, dst_stride, subpel_x, subpel_y,
- sf, w, h, ref, kernel, sf->x_step_q4, sf->y_step_q4);
-}
-
-void build_inter_predictors(MACROBLOCKD *xd, int plane, int block,
- int bw, int bh,
- int x, int y, int w, int h,
- int mi_x, int mi_y) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- const MODE_INFO *mi = xd->mi[0];
- const int is_compound = has_second_ref(&mi->mbmi);
- const InterpKernel *kernel = vp10_filter_kernels[mi->mbmi.interp_filter];
- int ref;
-
- for (ref = 0; ref < 1 + is_compound; ++ref) {
- const struct scale_factors *const sf = &xd->block_refs[ref]->sf;
- struct buf_2d *const pre_buf = &pd->pre[ref];
- struct buf_2d *const dst_buf = &pd->dst;
- uint8_t *const dst = dst_buf->buf + dst_buf->stride * y + x;
- const MV mv = mi->mbmi.sb_type < BLOCK_8X8
- ? average_split_mvs(pd, mi, ref, block)
- : mi->mbmi.mv[ref].as_mv;
-
- // TODO(jkoleszar): This clamping is done in the incorrect place for the
- // scaling case. It needs to be done on the scaled MV, not the pre-scaling
- // MV. Note however that it performs the subsampling aware scaling so
- // that the result is always q4.
- // mv_precision precision is MV_PRECISION_Q4.
- const MV mv_q4 = clamp_mv_to_umv_border_sb(xd, &mv, bw, bh,
- pd->subsampling_x,
- pd->subsampling_y);
-
- uint8_t *pre;
- MV32 scaled_mv;
- int xs, ys, subpel_x, subpel_y;
- const int is_scaled = vp10_is_scaled(sf);
-
- if (is_scaled) {
- pre = pre_buf->buf + scaled_buffer_offset(x, y, pre_buf->stride, sf);
- scaled_mv = vp10_scale_mv(&mv_q4, mi_x + x, mi_y + y, sf);
- xs = sf->x_step_q4;
- ys = sf->y_step_q4;
- } else {
- pre = pre_buf->buf + (y * pre_buf->stride + x);
- scaled_mv.row = mv_q4.row;
- scaled_mv.col = mv_q4.col;
- xs = ys = 16;
- }
- subpel_x = scaled_mv.col & SUBPEL_MASK;
- subpel_y = scaled_mv.row & SUBPEL_MASK;
- pre += (scaled_mv.row >> SUBPEL_BITS) * pre_buf->stride
- + (scaled_mv.col >> SUBPEL_BITS);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- high_inter_predictor(pre, pre_buf->stride, dst, dst_buf->stride,
- subpel_x, subpel_y, sf, w, h, ref, kernel, xs, ys,
- xd->bd);
- } else {
- inter_predictor(pre, pre_buf->stride, dst, dst_buf->stride,
- subpel_x, subpel_y, sf, w, h, ref, kernel, xs, ys);
- }
-#else
- inter_predictor(pre, pre_buf->stride, dst, dst_buf->stride,
- subpel_x, subpel_y, sf, w, h, ref, kernel, xs, ys);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
-}
-
-void vp10_build_inter_predictor_sub8x8(MACROBLOCKD *xd, int plane,
- int i, int ir, int ic,
- int mi_row, int mi_col) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- MODE_INFO *const mi = xd->mi[0];
- const BLOCK_SIZE plane_bsize = get_plane_block_size(mi->mbmi.sb_type, pd);
- const int width = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- const int height = 4 * num_4x4_blocks_high_lookup[plane_bsize];
-
- uint8_t *const dst = &pd->dst.buf[(ir * pd->dst.stride + ic) << 2];
- int ref;
- const int is_compound = has_second_ref(&mi->mbmi);
- const InterpKernel *kernel = vp10_filter_kernels[mi->mbmi.interp_filter];
-
- for (ref = 0; ref < 1 + is_compound; ++ref) {
- const uint8_t *pre =
- &pd->pre[ref].buf[(ir * pd->pre[ref].stride + ic) << 2];
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- vp10_highbd_build_inter_predictor(pre, pd->pre[ref].stride,
- dst, pd->dst.stride,
- &mi->bmi[i].as_mv[ref].as_mv,
- &xd->block_refs[ref]->sf, width, height,
- ref, kernel, MV_PRECISION_Q3,
- mi_col * MI_SIZE + 4 * ic,
- mi_row * MI_SIZE + 4 * ir, xd->bd);
- } else {
- vp10_build_inter_predictor(pre, pd->pre[ref].stride,
- dst, pd->dst.stride,
- &mi->bmi[i].as_mv[ref].as_mv,
- &xd->block_refs[ref]->sf, width, height, ref,
- kernel, MV_PRECISION_Q3,
- mi_col * MI_SIZE + 4 * ic,
- mi_row * MI_SIZE + 4 * ir);
- }
-#else
- vp10_build_inter_predictor(pre, pd->pre[ref].stride,
- dst, pd->dst.stride,
- &mi->bmi[i].as_mv[ref].as_mv,
- &xd->block_refs[ref]->sf, width, height, ref,
- kernel, MV_PRECISION_Q3,
- mi_col * MI_SIZE + 4 * ic,
- mi_row * MI_SIZE + 4 * ir);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
-}
-
-static void build_inter_predictors_for_planes(MACROBLOCKD *xd, BLOCK_SIZE bsize,
- int mi_row, int mi_col,
- int plane_from, int plane_to) {
- int plane;
- const int mi_x = mi_col * MI_SIZE;
- const int mi_y = mi_row * MI_SIZE;
- for (plane = plane_from; plane <= plane_to; ++plane) {
- const struct macroblockd_plane *pd = &xd->plane[plane];
- const int bw = 4 * num_4x4_blocks_wide_lookup[bsize] >> pd->subsampling_x;
- const int bh = 4 * num_4x4_blocks_high_lookup[bsize] >> pd->subsampling_y;
-
- if (xd->mi[0]->mbmi.sb_type < BLOCK_8X8) {
- const PARTITION_TYPE bp = bsize - xd->mi[0]->mbmi.sb_type;
- const int have_vsplit = bp != PARTITION_HORZ;
- const int have_hsplit = bp != PARTITION_VERT;
- const int num_4x4_w = 2 >> ((!have_vsplit) | pd->subsampling_x);
- const int num_4x4_h = 2 >> ((!have_hsplit) | pd->subsampling_y);
- const int pw = 8 >> (have_vsplit | pd->subsampling_x);
- const int ph = 8 >> (have_hsplit | pd->subsampling_y);
- int x, y;
- assert(bp != PARTITION_NONE && bp < PARTITION_TYPES);
- assert(bsize == BLOCK_8X8);
- assert(pw * num_4x4_w == bw && ph * num_4x4_h == bh);
- for (y = 0; y < num_4x4_h; ++y)
- for (x = 0; x < num_4x4_w; ++x)
- build_inter_predictors(xd, plane, y * 2 + x, bw, bh,
- 4 * x, 4 * y, pw, ph, mi_x, mi_y);
- } else {
- build_inter_predictors(xd, plane, 0, bw, bh,
- 0, 0, bw, bh, mi_x, mi_y);
- }
- }
-}
-
-void vp10_build_inter_predictors_sby(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- build_inter_predictors_for_planes(xd, bsize, mi_row, mi_col, 0, 0);
-}
-
-void vp10_build_inter_predictors_sbp(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize, int plane) {
- build_inter_predictors_for_planes(xd, bsize, mi_row, mi_col, plane, plane);
-}
-
-void vp10_build_inter_predictors_sbuv(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- build_inter_predictors_for_planes(xd, bsize, mi_row, mi_col, 1,
- MAX_MB_PLANE - 1);
-}
-
-void vp10_build_inter_predictors_sb(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- build_inter_predictors_for_planes(xd, bsize, mi_row, mi_col, 0,
- MAX_MB_PLANE - 1);
-}
-
-void vp10_setup_dst_planes(struct macroblockd_plane planes[MAX_MB_PLANE],
- const YV12_BUFFER_CONFIG *src,
- int mi_row, int mi_col) {
- uint8_t *const buffers[MAX_MB_PLANE] = { src->y_buffer, src->u_buffer,
- src->v_buffer};
- const int strides[MAX_MB_PLANE] = { src->y_stride, src->uv_stride,
- src->uv_stride};
- int i;
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- struct macroblockd_plane *const pd = &planes[i];
- setup_pred_plane(&pd->dst, buffers[i], strides[i], mi_row, mi_col, NULL,
- pd->subsampling_x, pd->subsampling_y);
- }
-}
-
-void vp10_setup_pre_planes(MACROBLOCKD *xd, int idx,
- const YV12_BUFFER_CONFIG *src,
- int mi_row, int mi_col,
- const struct scale_factors *sf) {
- if (src != NULL) {
- int i;
- uint8_t *const buffers[MAX_MB_PLANE] = { src->y_buffer, src->u_buffer,
- src->v_buffer};
- const int strides[MAX_MB_PLANE] = { src->y_stride, src->uv_stride,
- src->uv_stride};
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- struct macroblockd_plane *const pd = &xd->plane[i];
- setup_pred_plane(&pd->pre[idx], buffers[i], strides[i], mi_row, mi_col,
- sf, pd->subsampling_x, pd->subsampling_y);
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_RECONINTER_H_
-#define VP10_COMMON_RECONINTER_H_
-
-#include "vp10/common/filter.h"
-#include "vp10/common/onyxc_int.h"
-#include "vpx/vpx_integer.h"
-#include "vpx_dsp/vpx_filter.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-static INLINE void inter_predictor(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- const int subpel_x,
- const int subpel_y,
- const struct scale_factors *sf,
- int w, int h, int ref,
- const InterpKernel *kernel,
- int xs, int ys) {
- sf->predict[subpel_x != 0][subpel_y != 0][ref](
- src, src_stride, dst, dst_stride,
- kernel[subpel_x], xs, kernel[subpel_y], ys, w, h);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static INLINE void high_inter_predictor(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- const int subpel_x,
- const int subpel_y,
- const struct scale_factors *sf,
- int w, int h, int ref,
- const InterpKernel *kernel,
- int xs, int ys, int bd) {
- sf->highbd_predict[subpel_x != 0][subpel_y != 0][ref](
- src, src_stride, dst, dst_stride,
- kernel[subpel_x], xs, kernel[subpel_y], ys, w, h, bd);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static INLINE int round_mv_comp_q4(int value) {
- return (value < 0 ? value - 2 : value + 2) / 4;
-}
-
-static MV mi_mv_pred_q4(const MODE_INFO *mi, int idx) {
- MV res = { round_mv_comp_q4(mi->bmi[0].as_mv[idx].as_mv.row +
- mi->bmi[1].as_mv[idx].as_mv.row +
- mi->bmi[2].as_mv[idx].as_mv.row +
- mi->bmi[3].as_mv[idx].as_mv.row),
- round_mv_comp_q4(mi->bmi[0].as_mv[idx].as_mv.col +
- mi->bmi[1].as_mv[idx].as_mv.col +
- mi->bmi[2].as_mv[idx].as_mv.col +
- mi->bmi[3].as_mv[idx].as_mv.col) };
- return res;
-}
-
-static INLINE int round_mv_comp_q2(int value) {
- return (value < 0 ? value - 1 : value + 1) / 2;
-}
-
-static MV mi_mv_pred_q2(const MODE_INFO *mi, int idx, int block0, int block1) {
- MV res = { round_mv_comp_q2(mi->bmi[block0].as_mv[idx].as_mv.row +
- mi->bmi[block1].as_mv[idx].as_mv.row),
- round_mv_comp_q2(mi->bmi[block0].as_mv[idx].as_mv.col +
- mi->bmi[block1].as_mv[idx].as_mv.col) };
- return res;
-}
-
-// TODO(jkoleszar): yet another mv clamping function :-(
-static INLINE MV clamp_mv_to_umv_border_sb(const MACROBLOCKD *xd,
- const MV *src_mv,
- int bw, int bh, int ss_x, int ss_y) {
- // If the MV points so far into the UMV border that no visible pixels
- // are used for reconstruction, the subpel part of the MV can be
- // discarded and the MV limited to 16 pixels with equivalent results.
- const int spel_left = (VP9_INTERP_EXTEND + bw) << SUBPEL_BITS;
- const int spel_right = spel_left - SUBPEL_SHIFTS;
- const int spel_top = (VP9_INTERP_EXTEND + bh) << SUBPEL_BITS;
- const int spel_bottom = spel_top - SUBPEL_SHIFTS;
- MV clamped_mv = {
- src_mv->row * (1 << (1 - ss_y)),
- src_mv->col * (1 << (1 - ss_x))
- };
- assert(ss_x <= 1);
- assert(ss_y <= 1);
-
- clamp_mv(&clamped_mv,
- xd->mb_to_left_edge * (1 << (1 - ss_x)) - spel_left,
- xd->mb_to_right_edge * (1 << (1 - ss_x)) + spel_right,
- xd->mb_to_top_edge * (1 << (1 - ss_y)) - spel_top,
- xd->mb_to_bottom_edge * (1 << (1 - ss_y)) + spel_bottom);
-
- return clamped_mv;
-}
-
-static INLINE MV average_split_mvs(const struct macroblockd_plane *pd,
- const MODE_INFO *mi, int ref, int block) {
- const int ss_idx = ((pd->subsampling_x > 0) << 1) | (pd->subsampling_y > 0);
- MV res = {0, 0};
- switch (ss_idx) {
- case 0:
- res = mi->bmi[block].as_mv[ref].as_mv;
- break;
- case 1:
- res = mi_mv_pred_q2(mi, ref, block, block + 2);
- break;
- case 2:
- res = mi_mv_pred_q2(mi, ref, block, block + 1);
- break;
- case 3:
- res = mi_mv_pred_q4(mi, ref);
- break;
- default:
- assert(ss_idx <= 3 && ss_idx >= 0);
- }
- return res;
-}
-
-void build_inter_predictors(MACROBLOCKD *xd, int plane, int block,
- int bw, int bh,
- int x, int y, int w, int h,
- int mi_x, int mi_y);
-
-void vp10_build_inter_predictor_sub8x8(MACROBLOCKD *xd, int plane,
- int i, int ir, int ic,
- int mi_row, int mi_col);
-
-void vp10_build_inter_predictors_sby(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize);
-
-void vp10_build_inter_predictors_sbp(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize, int plane);
-
-void vp10_build_inter_predictors_sbuv(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize);
-
-void vp10_build_inter_predictors_sb(MACROBLOCKD *xd, int mi_row, int mi_col,
- BLOCK_SIZE bsize);
-
-void vp10_build_inter_predictor(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- const MV *mv_q3,
- const struct scale_factors *sf,
- int w, int h, int do_avg,
- const InterpKernel *kernel,
- enum mv_precision precision,
- int x, int y);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_build_inter_predictor(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- const MV *mv_q3,
- const struct scale_factors *sf,
- int w, int h, int do_avg,
- const InterpKernel *kernel,
- enum mv_precision precision,
- int x, int y, int bd);
-#endif
-
-static INLINE int scaled_buffer_offset(int x_offset, int y_offset, int stride,
- const struct scale_factors *sf) {
- const int x = sf ? sf->scale_value_x(x_offset, sf) : x_offset;
- const int y = sf ? sf->scale_value_y(y_offset, sf) : y_offset;
- return y * stride + x;
-}
-
-static INLINE void setup_pred_plane(struct buf_2d *dst,
- uint8_t *src, int stride,
- int mi_row, int mi_col,
- const struct scale_factors *scale,
- int subsampling_x, int subsampling_y) {
- const int x = (MI_SIZE * mi_col) >> subsampling_x;
- const int y = (MI_SIZE * mi_row) >> subsampling_y;
- dst->buf = src + scaled_buffer_offset(x, y, stride, scale);
- dst->stride = stride;
-}
-
-void vp10_setup_dst_planes(struct macroblockd_plane planes[MAX_MB_PLANE],
- const YV12_BUFFER_CONFIG *src,
- int mi_row, int mi_col);
-
-void vp10_setup_pre_planes(MACROBLOCKD *xd, int idx,
- const YV12_BUFFER_CONFIG *src, int mi_row, int mi_col,
- const struct scale_factors *sf);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_RECONINTER_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-
-#if CONFIG_VP9_HIGHBITDEPTH
-#include "vpx_dsp/vpx_dsp_common.h"
-#endif // CONFIG_VP9_HIGHBITDEPTH
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/vpx_once.h"
-
-#include "vp10/common/reconintra.h"
-#include "vp10/common/onyxc_int.h"
-
-#if CONFIG_MISC_FIXES
-enum {
- NEED_LEFT = 1 << 1,
- NEED_ABOVE = 1 << 2,
- NEED_ABOVERIGHT = 1 << 3,
- NEED_ABOVELEFT = 1 << 4,
- NEED_BOTTOMLEFT = 1 << 5,
-};
-
-static const uint8_t extend_modes[INTRA_MODES] = {
- NEED_ABOVE | NEED_LEFT, // DC
- NEED_ABOVE, // V
- NEED_LEFT, // H
- NEED_ABOVE | NEED_ABOVERIGHT, // D45
- NEED_LEFT | NEED_ABOVE | NEED_ABOVELEFT, // D135
- NEED_LEFT | NEED_ABOVE | NEED_ABOVELEFT, // D117
- NEED_LEFT | NEED_ABOVE | NEED_ABOVELEFT, // D153
- NEED_LEFT | NEED_BOTTOMLEFT, // D207
- NEED_ABOVE | NEED_ABOVERIGHT, // D63
- NEED_LEFT | NEED_ABOVE | NEED_ABOVELEFT, // TM
-};
-#else
-enum {
- NEED_LEFT = 1 << 1,
- NEED_ABOVE = 1 << 2,
- NEED_ABOVERIGHT = 1 << 3,
-};
-
-static const uint8_t extend_modes[INTRA_MODES] = {
- NEED_ABOVE | NEED_LEFT, // DC
- NEED_ABOVE, // V
- NEED_LEFT, // H
- NEED_ABOVERIGHT, // D45
- NEED_LEFT | NEED_ABOVE, // D135
- NEED_LEFT | NEED_ABOVE, // D117
- NEED_LEFT | NEED_ABOVE, // D153
- NEED_LEFT, // D207
- NEED_ABOVERIGHT, // D63
- NEED_LEFT | NEED_ABOVE, // TM
-};
-#endif
-
-#if CONFIG_MISC_FIXES
-static const uint8_t orders_64x64[1] = { 0 };
-static const uint8_t orders_64x32[2] = { 0, 1 };
-static const uint8_t orders_32x64[2] = { 0, 1 };
-static const uint8_t orders_32x32[4] = {
- 0, 1,
- 2, 3,
-};
-static const uint8_t orders_32x16[8] = {
- 0, 2,
- 1, 3,
- 4, 6,
- 5, 7,
-};
-static const uint8_t orders_16x32[8] = {
- 0, 1, 2, 3,
- 4, 5, 6, 7,
-};
-static const uint8_t orders_16x16[16] = {
- 0, 1, 4, 5,
- 2, 3, 6, 7,
- 8, 9, 12, 13,
- 10, 11, 14, 15,
-};
-static const uint8_t orders_16x8[32] = {
- 0, 2, 8, 10,
- 1, 3, 9, 11,
- 4, 6, 12, 14,
- 5, 7, 13, 15,
- 16, 18, 24, 26,
- 17, 19, 25, 27,
- 20, 22, 28, 30,
- 21, 23, 29, 31,
-};
-static const uint8_t orders_8x16[32] = {
- 0, 1, 2, 3, 8, 9, 10, 11,
- 4, 5, 6, 7, 12, 13, 14, 15,
- 16, 17, 18, 19, 24, 25, 26, 27,
- 20, 21, 22, 23, 28, 29, 30, 31,
-};
-static const uint8_t orders_8x8[64] = {
- 0, 1, 4, 5, 16, 17, 20, 21,
- 2, 3, 6, 7, 18, 19, 22, 23,
- 8, 9, 12, 13, 24, 25, 28, 29,
- 10, 11, 14, 15, 26, 27, 30, 31,
- 32, 33, 36, 37, 48, 49, 52, 53,
- 34, 35, 38, 39, 50, 51, 54, 55,
- 40, 41, 44, 45, 56, 57, 60, 61,
- 42, 43, 46, 47, 58, 59, 62, 63,
-};
-static const uint8_t *const orders[BLOCK_SIZES] = {
- orders_8x8, orders_8x8, orders_8x8, orders_8x8,
- orders_8x16, orders_16x8, orders_16x16,
- orders_16x32, orders_32x16, orders_32x32,
- orders_32x64, orders_64x32, orders_64x64,
-};
-
-static int vp10_has_right(BLOCK_SIZE bsize, int mi_row, int mi_col,
- int right_available,
- TX_SIZE txsz, int y, int x, int ss_x) {
- if (y == 0) {
- int wl = mi_width_log2_lookup[bsize];
- int hl = mi_height_log2_lookup[bsize];
- int w = 1 << (wl + 1 - ss_x);
- int step = 1 << txsz;
- const uint8_t *order = orders[bsize];
- int my_order, tr_order;
-
- if (x + step < w)
- return 1;
-
- mi_row = (mi_row & 7) >> hl;
- mi_col = (mi_col & 7) >> wl;
-
- if (mi_row == 0)
- return right_available;
-
- if (((mi_col + 1) << wl) >= 8)
- return 0;
-
- my_order = order[((mi_row + 0) << (3 - wl)) + mi_col + 0];
- tr_order = order[((mi_row - 1) << (3 - wl)) + mi_col + 1];
-
- return my_order > tr_order && right_available;
- } else {
- int wl = mi_width_log2_lookup[bsize];
- int w = 1 << (wl + 1 - ss_x);
- int step = 1 << txsz;
-
- return x + step < w;
- }
-}
-
-static int vp10_has_bottom(BLOCK_SIZE bsize, int mi_row, int mi_col,
- int bottom_available, TX_SIZE txsz,
- int y, int x, int ss_y) {
- if (x == 0) {
- int wl = mi_width_log2_lookup[bsize];
- int hl = mi_height_log2_lookup[bsize];
- int h = 1 << (hl + 1 - ss_y);
- int step = 1 << txsz;
- const uint8_t *order = orders[bsize];
- int my_order, bl_order;
-
- mi_row = (mi_row & 7) >> hl;
- mi_col = (mi_col & 7) >> wl;
-
- if (mi_col == 0)
- return bottom_available &&
- (mi_row << (hl + !ss_y)) + y + step < (8 << !ss_y);
-
- if (((mi_row + 1) << hl) >= 8)
- return 0;
-
- if (y + step < h)
- return 1;
-
- my_order = order[((mi_row + 0) << (3 - wl)) + mi_col + 0];
- bl_order = order[((mi_row + 1) << (3 - wl)) + mi_col - 1];
-
- return bl_order < my_order && bottom_available;
- } else {
- return 0;
- }
-}
-#endif
-
-typedef void (*intra_pred_fn)(uint8_t *dst, ptrdiff_t stride,
- const uint8_t *above, const uint8_t *left);
-
-static intra_pred_fn pred[INTRA_MODES][TX_SIZES];
-static intra_pred_fn dc_pred[2][2][TX_SIZES];
-
-#if CONFIG_VP9_HIGHBITDEPTH
-typedef void (*intra_high_pred_fn)(uint16_t *dst, ptrdiff_t stride,
- const uint16_t *above, const uint16_t *left,
- int bd);
-static intra_high_pred_fn pred_high[INTRA_MODES][4];
-static intra_high_pred_fn dc_pred_high[2][2][4];
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void vp10_init_intra_predictors_internal(void) {
-#define INIT_NO_4X4(p, type) \
- p[TX_8X8] = vpx_##type##_predictor_8x8; \
- p[TX_16X16] = vpx_##type##_predictor_16x16; \
- p[TX_32X32] = vpx_##type##_predictor_32x32
-
-#define INIT_ALL_SIZES(p, type) \
- p[TX_4X4] = vpx_##type##_predictor_4x4; \
- INIT_NO_4X4(p, type)
-
- INIT_ALL_SIZES(pred[V_PRED], v);
- INIT_ALL_SIZES(pred[H_PRED], h);
-#if CONFIG_MISC_FIXES
- INIT_ALL_SIZES(pred[D207_PRED], d207e);
- INIT_ALL_SIZES(pred[D45_PRED], d45e);
- INIT_ALL_SIZES(pred[D63_PRED], d63e);
-#else
- INIT_ALL_SIZES(pred[D207_PRED], d207);
- INIT_ALL_SIZES(pred[D45_PRED], d45);
- INIT_ALL_SIZES(pred[D63_PRED], d63);
-#endif
- INIT_ALL_SIZES(pred[D117_PRED], d117);
- INIT_ALL_SIZES(pred[D135_PRED], d135);
- INIT_ALL_SIZES(pred[D153_PRED], d153);
- INIT_ALL_SIZES(pred[TM_PRED], tm);
-
- INIT_ALL_SIZES(dc_pred[0][0], dc_128);
- INIT_ALL_SIZES(dc_pred[0][1], dc_top);
- INIT_ALL_SIZES(dc_pred[1][0], dc_left);
- INIT_ALL_SIZES(dc_pred[1][1], dc);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- INIT_ALL_SIZES(pred_high[V_PRED], highbd_v);
- INIT_ALL_SIZES(pred_high[H_PRED], highbd_h);
-#if CONFIG_MISC_FIXES
- INIT_ALL_SIZES(pred_high[D207_PRED], highbd_d207e);
- INIT_ALL_SIZES(pred_high[D45_PRED], highbd_d45e);
- INIT_ALL_SIZES(pred_high[D63_PRED], highbd_d63);
-#else
- INIT_ALL_SIZES(pred_high[D207_PRED], highbd_d207);
- INIT_ALL_SIZES(pred_high[D45_PRED], highbd_d45);
- INIT_ALL_SIZES(pred_high[D63_PRED], highbd_d63);
-#endif
- INIT_ALL_SIZES(pred_high[D117_PRED], highbd_d117);
- INIT_ALL_SIZES(pred_high[D135_PRED], highbd_d135);
- INIT_ALL_SIZES(pred_high[D153_PRED], highbd_d153);
- INIT_ALL_SIZES(pred_high[TM_PRED], highbd_tm);
-
- INIT_ALL_SIZES(dc_pred_high[0][0], highbd_dc_128);
- INIT_ALL_SIZES(dc_pred_high[0][1], highbd_dc_top);
- INIT_ALL_SIZES(dc_pred_high[1][0], highbd_dc_left);
- INIT_ALL_SIZES(dc_pred_high[1][1], highbd_dc);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#undef intra_pred_allsizes
-}
-
-#if CONFIG_MISC_FIXES
-static inline void memset16(uint16_t *dst, int val, int n) {
- while (n--)
- *dst++ = val;
-}
-#endif
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void build_intra_predictors_high(const MACROBLOCKD *xd,
- const uint8_t *ref8,
- int ref_stride,
- uint8_t *dst8,
- int dst_stride,
- PREDICTION_MODE mode,
- TX_SIZE tx_size,
-#if CONFIG_MISC_FIXES
- int n_top_px, int n_topright_px,
- int n_left_px, int n_bottomleft_px,
-#else
- int up_available,
- int left_available,
- int right_available,
-#endif
- int x, int y,
- int plane, int bd) {
- int i;
- uint16_t *dst = CONVERT_TO_SHORTPTR(dst8);
- uint16_t *ref = CONVERT_TO_SHORTPTR(ref8);
-#if CONFIG_MISC_FIXES
- DECLARE_ALIGNED(16, uint16_t, left_col[32]);
-#else
- DECLARE_ALIGNED(16, uint16_t, left_col[64]);
-#endif
- DECLARE_ALIGNED(16, uint16_t, above_data[64 + 16]);
- uint16_t *above_row = above_data + 16;
- const uint16_t *const_above_row = above_row;
- const int bs = 4 << tx_size;
-#if CONFIG_MISC_FIXES
- const uint16_t *above_ref = ref - ref_stride;
-#else
- int frame_width, frame_height;
- int x0, y0;
- const struct macroblockd_plane *const pd = &xd->plane[plane];
-#endif
- int base = 128 << (bd - 8);
- // 127 127 127 .. 127 127 127 127 127 127
- // 129 A B .. Y Z
- // 129 C D .. W X
- // 129 E F .. U V
- // 129 G H .. S T T T T T
-
-#if CONFIG_MISC_FIXES
- (void) x;
- (void) y;
- (void) plane;
-
- // NEED_LEFT
- if (extend_modes[mode] & NEED_LEFT) {
- const int need_bottom = !!(extend_modes[mode] & NEED_BOTTOMLEFT);
- i = 0;
- if (n_left_px > 0) {
- for (; i < n_left_px; i++)
- left_col[i] = ref[i * ref_stride - 1];
- if (need_bottom && n_bottomleft_px > 0) {
- assert(i == bs);
- for (; i < bs + n_bottomleft_px; i++)
- left_col[i] = ref[i * ref_stride - 1];
- }
- if (i < (bs << need_bottom))
- memset16(&left_col[i], left_col[i - 1], (bs << need_bottom) - i);
- } else {
- memset16(left_col, base + 1, bs << need_bottom);
- }
- }
-
- // NEED_ABOVE
- if (extend_modes[mode] & NEED_ABOVE) {
- const int need_right = !!(extend_modes[mode] & NEED_ABOVERIGHT);
- if (n_top_px > 0) {
- memcpy(above_row, above_ref, n_top_px * 2);
- i = n_top_px;
- if (need_right && n_topright_px > 0) {
- assert(n_top_px == bs);
- memcpy(above_row + bs, above_ref + bs, n_topright_px * 2);
- i += n_topright_px;
- }
- if (i < (bs << need_right))
- memset16(&above_row[i], above_row[i - 1], (bs << need_right) - i);
- } else {
- memset16(above_row, base - 1, bs << need_right);
- }
- }
-
- if (extend_modes[mode] & NEED_ABOVELEFT) {
- above_row[-1] = n_top_px > 0 ?
- (n_left_px > 0 ? above_ref[-1] : base + 1) : base - 1;
- }
-#else
- // Get current frame pointer, width and height.
- if (plane == 0) {
- frame_width = xd->cur_buf->y_width;
- frame_height = xd->cur_buf->y_height;
- } else {
- frame_width = xd->cur_buf->uv_width;
- frame_height = xd->cur_buf->uv_height;
- }
-
- // Get block position in current frame.
- x0 = (-xd->mb_to_left_edge >> (3 + pd->subsampling_x)) + x;
- y0 = (-xd->mb_to_top_edge >> (3 + pd->subsampling_y)) + y;
-
- // left
- if (left_available) {
- if (xd->mb_to_bottom_edge < 0) {
- /* slower path if the block needs border extension */
- if (y0 + bs <= frame_height) {
- for (i = 0; i < bs; ++i)
- left_col[i] = ref[i * ref_stride - 1];
- } else {
- const int extend_bottom = frame_height - y0;
- for (i = 0; i < extend_bottom; ++i)
- left_col[i] = ref[i * ref_stride - 1];
- for (; i < bs; ++i)
- left_col[i] = ref[(extend_bottom - 1) * ref_stride - 1];
- }
- } else {
- /* faster path if the block does not need extension */
- for (i = 0; i < bs; ++i)
- left_col[i] = ref[i * ref_stride - 1];
- }
- } else {
- // TODO(Peter): this value should probably change for high bitdepth
- vpx_memset16(left_col, base + 1, bs);
- }
-
- // TODO(hkuang) do not extend 2*bs pixels for all modes.
- // above
- if (up_available) {
- const uint16_t *above_ref = ref - ref_stride;
- if (xd->mb_to_right_edge < 0) {
- /* slower path if the block needs border extension */
- if (x0 + 2 * bs <= frame_width) {
- if (right_available && bs == 4) {
- memcpy(above_row, above_ref, 2 * bs * sizeof(above_row[0]));
- } else {
- memcpy(above_row, above_ref, bs * sizeof(above_row[0]));
- vpx_memset16(above_row + bs, above_row[bs - 1], bs);
- }
- } else if (x0 + bs <= frame_width) {
- const int r = frame_width - x0;
- if (right_available && bs == 4) {
- memcpy(above_row, above_ref, r * sizeof(above_row[0]));
- vpx_memset16(above_row + r, above_row[r - 1],
- x0 + 2 * bs - frame_width);
- } else {
- memcpy(above_row, above_ref, bs * sizeof(above_row[0]));
- vpx_memset16(above_row + bs, above_row[bs - 1], bs);
- }
- } else if (x0 <= frame_width) {
- const int r = frame_width - x0;
- memcpy(above_row, above_ref, r * sizeof(above_row[0]));
- vpx_memset16(above_row + r, above_row[r - 1],
- x0 + 2 * bs - frame_width);
- }
- // TODO(Peter) this value should probably change for high bitdepth
- above_row[-1] = left_available ? above_ref[-1] : (base+1);
- } else {
- /* faster path if the block does not need extension */
- if (bs == 4 && right_available && left_available) {
- const_above_row = above_ref;
- } else {
- memcpy(above_row, above_ref, bs * sizeof(above_row[0]));
- if (bs == 4 && right_available)
- memcpy(above_row + bs, above_ref + bs, bs * sizeof(above_row[0]));
- else
- vpx_memset16(above_row + bs, above_row[bs - 1], bs);
- // TODO(Peter): this value should probably change for high bitdepth
- above_row[-1] = left_available ? above_ref[-1] : (base+1);
- }
- }
- } else {
- vpx_memset16(above_row, base - 1, bs * 2);
- // TODO(Peter): this value should probably change for high bitdepth
- above_row[-1] = base - 1;
- }
-#endif
-
- // predict
- if (mode == DC_PRED) {
-#if CONFIG_MISC_FIXES
- dc_pred_high[n_left_px > 0][n_top_px > 0][tx_size](dst, dst_stride,
- const_above_row,
- left_col, xd->bd);
-#else
- dc_pred_high[left_available][up_available][tx_size](dst, dst_stride,
- const_above_row,
- left_col, xd->bd);
-#endif
- } else {
- pred_high[mode][tx_size](dst, dst_stride, const_above_row, left_col,
- xd->bd);
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void build_intra_predictors(const MACROBLOCKD *xd, const uint8_t *ref,
- int ref_stride, uint8_t *dst, int dst_stride,
- PREDICTION_MODE mode, TX_SIZE tx_size,
-#if CONFIG_MISC_FIXES
- int n_top_px, int n_topright_px,
- int n_left_px, int n_bottomleft_px,
-#else
- int up_available, int left_available,
- int right_available,
-#endif
- int x, int y, int plane) {
- int i;
-#if CONFIG_MISC_FIXES
- DECLARE_ALIGNED(16, uint8_t, left_col[64]);
- const uint8_t *above_ref = ref - ref_stride;
-#else
- DECLARE_ALIGNED(16, uint8_t, left_col[32]);
- int frame_width, frame_height;
- int x0, y0;
- const struct macroblockd_plane *const pd = &xd->plane[plane];
-#endif
- DECLARE_ALIGNED(16, uint8_t, above_data[64 + 16]);
- uint8_t *above_row = above_data + 16;
- const uint8_t *const_above_row = above_row;
- const int bs = 4 << tx_size;
-
- // 127 127 127 .. 127 127 127 127 127 127
- // 129 A B .. Y Z
- // 129 C D .. W X
- // 129 E F .. U V
- // 129 G H .. S T T T T T
- // ..
-
-#if CONFIG_MISC_FIXES
- (void) xd;
- (void) x;
- (void) y;
- (void) plane;
- assert(n_top_px >= 0);
- assert(n_topright_px >= 0);
- assert(n_left_px >= 0);
- assert(n_bottomleft_px >= 0);
-#else
- // Get current frame pointer, width and height.
- if (plane == 0) {
- frame_width = xd->cur_buf->y_width;
- frame_height = xd->cur_buf->y_height;
- } else {
- frame_width = xd->cur_buf->uv_width;
- frame_height = xd->cur_buf->uv_height;
- }
-
- // Get block position in current frame.
- x0 = (-xd->mb_to_left_edge >> (3 + pd->subsampling_x)) + x;
- y0 = (-xd->mb_to_top_edge >> (3 + pd->subsampling_y)) + y;
-#endif
-
- // NEED_LEFT
- if (extend_modes[mode] & NEED_LEFT) {
-#if CONFIG_MISC_FIXES
- const int need_bottom = !!(extend_modes[mode] & NEED_BOTTOMLEFT);
- i = 0;
- if (n_left_px > 0) {
- for (; i < n_left_px; i++)
- left_col[i] = ref[i * ref_stride - 1];
- if (need_bottom && n_bottomleft_px > 0) {
- assert(i == bs);
- for (; i < bs + n_bottomleft_px; i++)
- left_col[i] = ref[i * ref_stride - 1];
- }
- if (i < (bs << need_bottom))
- memset(&left_col[i], left_col[i - 1], (bs << need_bottom) - i);
- } else {
- memset(left_col, 129, bs << need_bottom);
- }
-#else
- if (left_available) {
- if (xd->mb_to_bottom_edge < 0) {
- /* slower path if the block needs border extension */
- if (y0 + bs <= frame_height) {
- for (i = 0; i < bs; ++i)
- left_col[i] = ref[i * ref_stride - 1];
- } else {
- const int extend_bottom = frame_height - y0;
- for (i = 0; i < extend_bottom; ++i)
- left_col[i] = ref[i * ref_stride - 1];
- for (; i < bs; ++i)
- left_col[i] = ref[(extend_bottom - 1) * ref_stride - 1];
- }
- } else {
- /* faster path if the block does not need extension */
- for (i = 0; i < bs; ++i)
- left_col[i] = ref[i * ref_stride - 1];
- }
- } else {
- memset(left_col, 129, bs);
- }
-#endif
- }
-
- // NEED_ABOVE
- if (extend_modes[mode] & NEED_ABOVE) {
-#if CONFIG_MISC_FIXES
- const int need_right = !!(extend_modes[mode] & NEED_ABOVERIGHT);
- if (n_top_px > 0) {
- memcpy(above_row, above_ref, n_top_px);
- i = n_top_px;
- if (need_right && n_topright_px > 0) {
- assert(n_top_px == bs);
- memcpy(above_row + bs, above_ref + bs, n_topright_px);
- i += n_topright_px;
- }
- if (i < (bs << need_right))
- memset(&above_row[i], above_row[i - 1], (bs << need_right) - i);
- } else {
- memset(above_row, 127, bs << need_right);
- }
-#else
- if (up_available) {
- const uint8_t *above_ref = ref - ref_stride;
- if (xd->mb_to_right_edge < 0) {
- /* slower path if the block needs border extension */
- if (x0 + bs <= frame_width) {
- memcpy(above_row, above_ref, bs);
- } else if (x0 <= frame_width) {
- const int r = frame_width - x0;
- memcpy(above_row, above_ref, r);
- memset(above_row + r, above_row[r - 1], x0 + bs - frame_width);
- }
- } else {
- /* faster path if the block does not need extension */
- if (bs == 4 && right_available && left_available) {
- const_above_row = above_ref;
- } else {
- memcpy(above_row, above_ref, bs);
- }
- }
- above_row[-1] = left_available ? above_ref[-1] : 129;
- } else {
- memset(above_row, 127, bs);
- above_row[-1] = 127;
- }
-#endif
- }
-
-#if CONFIG_MISC_FIXES
- if (extend_modes[mode] & NEED_ABOVELEFT) {
- above_row[-1] = n_top_px > 0 ? (n_left_px > 0 ? above_ref[-1] : 129) : 127;
- }
-#else
- // NEED_ABOVERIGHT
- if (extend_modes[mode] & NEED_ABOVERIGHT) {
- if (up_available) {
- const uint8_t *above_ref = ref - ref_stride;
- if (xd->mb_to_right_edge < 0) {
- /* slower path if the block needs border extension */
- if (x0 + 2 * bs <= frame_width) {
- if (right_available && bs == 4) {
- memcpy(above_row, above_ref, 2 * bs);
- } else {
- memcpy(above_row, above_ref, bs);
- memset(above_row + bs, above_row[bs - 1], bs);
- }
- } else if (x0 + bs <= frame_width) {
- const int r = frame_width - x0;
- if (right_available && bs == 4) {
- memcpy(above_row, above_ref, r);
- memset(above_row + r, above_row[r - 1], x0 + 2 * bs - frame_width);
- } else {
- memcpy(above_row, above_ref, bs);
- memset(above_row + bs, above_row[bs - 1], bs);
- }
- } else if (x0 <= frame_width) {
- const int r = frame_width - x0;
- memcpy(above_row, above_ref, r);
- memset(above_row + r, above_row[r - 1], x0 + 2 * bs - frame_width);
- }
- } else {
- /* faster path if the block does not need extension */
- if (bs == 4 && right_available && left_available) {
- const_above_row = above_ref;
- } else {
- memcpy(above_row, above_ref, bs);
- if (bs == 4 && right_available)
- memcpy(above_row + bs, above_ref + bs, bs);
- else
- memset(above_row + bs, above_row[bs - 1], bs);
- }
- }
- above_row[-1] = left_available ? above_ref[-1] : 129;
- } else {
- memset(above_row, 127, bs * 2);
- above_row[-1] = 127;
- }
- }
-#endif
-
- // predict
- if (mode == DC_PRED) {
-#if CONFIG_MISC_FIXES
- dc_pred[n_left_px > 0][n_top_px > 0][tx_size](dst, dst_stride,
- const_above_row, left_col);
-#else
- dc_pred[left_available][up_available][tx_size](dst, dst_stride,
- const_above_row, left_col);
-#endif
- } else {
- pred[mode][tx_size](dst, dst_stride, const_above_row, left_col);
- }
-}
-
-void vp10_predict_intra_block(const MACROBLOCKD *xd, int bwl_in, int bhl_in,
- TX_SIZE tx_size, PREDICTION_MODE mode,
- const uint8_t *ref, int ref_stride,
- uint8_t *dst, int dst_stride,
- int aoff, int loff, int plane) {
- const int txw = (1 << tx_size);
- const int have_top = loff || xd->up_available;
- const int have_left = aoff || xd->left_available;
-#if !CONFIG_MISC_FIXES
- const int bw = (1 << bwl_in);
- const int have_right = (aoff + txw) < bw;
-#endif
- const int x = aoff * 4;
- const int y = loff * 4;
-#if CONFIG_MISC_FIXES
- const int bw = VPXMAX(2, 1 << bwl_in);
- const int bh = VPXMAX(2, 1 << bhl_in);
- const int mi_row = -xd->mb_to_top_edge >> 6;
- const int mi_col = -xd->mb_to_left_edge >> 6;
- const BLOCK_SIZE bsize = xd->mi[0]->mbmi.sb_type;
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- const int right_available =
- mi_col + (bw >> !pd->subsampling_x) < xd->tile.mi_col_end;
- const int have_right = vp10_has_right(bsize, mi_row, mi_col,
- right_available,
- tx_size, loff, aoff,
- pd->subsampling_x);
- const int have_bottom = vp10_has_bottom(bsize, mi_row, mi_col,
- xd->mb_to_bottom_edge > 0,
- tx_size, loff, aoff,
- pd->subsampling_y);
- const int wpx = 4 * bw;
- const int hpx = 4 * bh;
- const int txpx = 4 * txw;
-
- int xr = (xd->mb_to_right_edge >> (3 + pd->subsampling_x)) + (wpx - x - txpx);
- int yd =
- (xd->mb_to_bottom_edge >> (3 + pd->subsampling_y)) + (hpx - y - txpx);
-
- if (xd->mi[0]->mbmi.palette_mode_info.palette_size[plane != 0] > 0) {
- const int bs = 4 * (1 << tx_size);
- const int stride = 4 * (1 << bwl_in);
- int r, c;
- uint8_t *map = NULL;
-#if CONFIG_VP9_HIGHBITDEPTH
- uint16_t *palette = xd->mi[0]->mbmi.palette_mode_info.palette_colors +
- plane * PALETTE_MAX_SIZE;
-#else
- uint8_t *palette = xd->mi[0]->mbmi.palette_mode_info.palette_colors +
- plane * PALETTE_MAX_SIZE;
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- map = xd->plane[plane != 0].color_index_map;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- uint16_t *dst16 = CONVERT_TO_SHORTPTR(dst);
- for (r = 0; r < bs; ++r)
- for (c = 0; c < bs; ++c)
- dst16[r * dst_stride + c] =
- palette[map[(r + y) * stride + c + x]];
- } else {
-#endif // CONFIG_VP9_HIGHBITDEPTH
- for (r = 0; r < bs; ++r)
- for (c = 0; c < bs; ++c)
- dst[r * dst_stride + c] = palette[map[(r + y) * stride + c + x]];
-#if CONFIG_VP9_HIGHBITDEPTH
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- return;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- build_intra_predictors_high(xd, ref, ref_stride, dst, dst_stride, mode,
- tx_size,
- have_top ? VPXMIN(txpx, xr + txpx) : 0,
- have_top && have_right ? VPXMIN(txpx, xr) : 0,
- have_left ? VPXMIN(txpx, yd + txpx) : 0,
- have_bottom && have_left ? VPXMIN(txpx, yd) : 0,
- x, y, plane, xd->bd);
- return;
- }
-#endif
- build_intra_predictors(xd, ref, ref_stride, dst, dst_stride, mode,
- tx_size,
- have_top ? VPXMIN(txpx, xr + txpx) : 0,
- have_top && have_right ? VPXMIN(txpx, xr) : 0,
- have_left ? VPXMIN(txpx, yd + txpx) : 0,
- have_bottom && have_left ? VPXMIN(txpx, yd) : 0,
- x, y, plane);
-#else
- (void) bhl_in;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- build_intra_predictors_high(xd, ref, ref_stride, dst, dst_stride, mode,
- tx_size, have_top, have_left, have_right,
- x, y, plane, xd->bd);
- return;
- }
-#endif
- build_intra_predictors(xd, ref, ref_stride, dst, dst_stride, mode, tx_size,
- have_top, have_left, have_right, x, y, plane);
-#endif
-}
-
-void vp10_init_intra_predictors(void) {
- once(vp10_init_intra_predictors_internal);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_RECONINTRA_H_
-#define VP10_COMMON_RECONINTRA_H_
-
-#include "vpx/vpx_integer.h"
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_init_intra_predictors(void);
-
-void vp10_predict_intra_block(const MACROBLOCKD *xd, int bwl_in, int bhl_in,
- TX_SIZE tx_size, PREDICTION_MODE mode,
- const uint8_t *ref, int ref_stride,
- uint8_t *dst, int dst_stride,
- int aoff, int loff, int plane);
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_RECONINTRA_H_
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_dsp_rtcd.h"
-#include "vp10/common/filter.h"
-#include "vp10/common/scale.h"
-#include "vpx_dsp/vpx_filter.h"
-
-static INLINE int scaled_x(int val, const struct scale_factors *sf) {
- return (int)((int64_t)val * sf->x_scale_fp >> REF_SCALE_SHIFT);
-}
-
-static INLINE int scaled_y(int val, const struct scale_factors *sf) {
- return (int)((int64_t)val * sf->y_scale_fp >> REF_SCALE_SHIFT);
-}
-
-static int unscaled_value(int val, const struct scale_factors *sf) {
- (void) sf;
- return val;
-}
-
-static int get_fixed_point_scale_factor(int other_size, int this_size) {
- // Calculate scaling factor once for each reference frame
- // and use fixed point scaling factors in decoding and encoding routines.
- // Hardware implementations can calculate scale factor in device driver
- // and use multiplication and shifting on hardware instead of division.
- return (other_size << REF_SCALE_SHIFT) / this_size;
-}
-
-MV32 vp10_scale_mv(const MV *mv, int x, int y, const struct scale_factors *sf) {
- const int x_off_q4 = scaled_x(x << SUBPEL_BITS, sf) & SUBPEL_MASK;
- const int y_off_q4 = scaled_y(y << SUBPEL_BITS, sf) & SUBPEL_MASK;
- const MV32 res = {
- scaled_y(mv->row, sf) + y_off_q4,
- scaled_x(mv->col, sf) + x_off_q4
- };
- return res;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_setup_scale_factors_for_frame(struct scale_factors *sf,
- int other_w, int other_h,
- int this_w, int this_h,
- int use_highbd) {
-#else
-void vp10_setup_scale_factors_for_frame(struct scale_factors *sf,
- int other_w, int other_h,
- int this_w, int this_h) {
-#endif
- if (!valid_ref_frame_size(other_w, other_h, this_w, this_h)) {
- sf->x_scale_fp = REF_INVALID_SCALE;
- sf->y_scale_fp = REF_INVALID_SCALE;
- return;
- }
-
- sf->x_scale_fp = get_fixed_point_scale_factor(other_w, this_w);
- sf->y_scale_fp = get_fixed_point_scale_factor(other_h, this_h);
- sf->x_step_q4 = scaled_x(16, sf);
- sf->y_step_q4 = scaled_y(16, sf);
-
- if (vp10_is_scaled(sf)) {
- sf->scale_value_x = scaled_x;
- sf->scale_value_y = scaled_y;
- } else {
- sf->scale_value_x = unscaled_value;
- sf->scale_value_y = unscaled_value;
- }
-
- // TODO(agrange): Investigate the best choice of functions to use here
- // for EIGHTTAP_SMOOTH. Since it is not interpolating, need to choose what
- // to do at full-pel offsets. The current selection, where the filter is
- // applied in one direction only, and not at all for 0,0, seems to give the
- // best quality, but it may be worth trying an additional mode that does
- // do the filtering on full-pel.
- if (sf->x_step_q4 == 16) {
- if (sf->y_step_q4 == 16) {
- // No scaling in either direction.
- sf->predict[0][0][0] = vpx_convolve_copy;
- sf->predict[0][0][1] = vpx_convolve_avg;
- sf->predict[0][1][0] = vpx_convolve8_vert;
- sf->predict[0][1][1] = vpx_convolve8_avg_vert;
- sf->predict[1][0][0] = vpx_convolve8_horiz;
- sf->predict[1][0][1] = vpx_convolve8_avg_horiz;
- } else {
- // No scaling in x direction. Must always scale in the y direction.
- sf->predict[0][0][0] = vpx_convolve8_vert;
- sf->predict[0][0][1] = vpx_convolve8_avg_vert;
- sf->predict[0][1][0] = vpx_convolve8_vert;
- sf->predict[0][1][1] = vpx_convolve8_avg_vert;
- sf->predict[1][0][0] = vpx_convolve8;
- sf->predict[1][0][1] = vpx_convolve8_avg;
- }
- } else {
- if (sf->y_step_q4 == 16) {
- // No scaling in the y direction. Must always scale in the x direction.
- sf->predict[0][0][0] = vpx_convolve8_horiz;
- sf->predict[0][0][1] = vpx_convolve8_avg_horiz;
- sf->predict[0][1][0] = vpx_convolve8;
- sf->predict[0][1][1] = vpx_convolve8_avg;
- sf->predict[1][0][0] = vpx_convolve8_horiz;
- sf->predict[1][0][1] = vpx_convolve8_avg_horiz;
- } else {
- // Must always scale in both directions.
- sf->predict[0][0][0] = vpx_convolve8;
- sf->predict[0][0][1] = vpx_convolve8_avg;
- sf->predict[0][1][0] = vpx_convolve8;
- sf->predict[0][1][1] = vpx_convolve8_avg;
- sf->predict[1][0][0] = vpx_convolve8;
- sf->predict[1][0][1] = vpx_convolve8_avg;
- }
- }
- // 2D subpel motion always gets filtered in both directions
- sf->predict[1][1][0] = vpx_convolve8;
- sf->predict[1][1][1] = vpx_convolve8_avg;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (use_highbd) {
- if (sf->x_step_q4 == 16) {
- if (sf->y_step_q4 == 16) {
- // No scaling in either direction.
- sf->highbd_predict[0][0][0] = vpx_highbd_convolve_copy;
- sf->highbd_predict[0][0][1] = vpx_highbd_convolve_avg;
- sf->highbd_predict[0][1][0] = vpx_highbd_convolve8_vert;
- sf->highbd_predict[0][1][1] = vpx_highbd_convolve8_avg_vert;
- sf->highbd_predict[1][0][0] = vpx_highbd_convolve8_horiz;
- sf->highbd_predict[1][0][1] = vpx_highbd_convolve8_avg_horiz;
- } else {
- // No scaling in x direction. Must always scale in the y direction.
- sf->highbd_predict[0][0][0] = vpx_highbd_convolve8_vert;
- sf->highbd_predict[0][0][1] = vpx_highbd_convolve8_avg_vert;
- sf->highbd_predict[0][1][0] = vpx_highbd_convolve8_vert;
- sf->highbd_predict[0][1][1] = vpx_highbd_convolve8_avg_vert;
- sf->highbd_predict[1][0][0] = vpx_highbd_convolve8;
- sf->highbd_predict[1][0][1] = vpx_highbd_convolve8_avg;
- }
- } else {
- if (sf->y_step_q4 == 16) {
- // No scaling in the y direction. Must always scale in the x direction.
- sf->highbd_predict[0][0][0] = vpx_highbd_convolve8_horiz;
- sf->highbd_predict[0][0][1] = vpx_highbd_convolve8_avg_horiz;
- sf->highbd_predict[0][1][0] = vpx_highbd_convolve8;
- sf->highbd_predict[0][1][1] = vpx_highbd_convolve8_avg;
- sf->highbd_predict[1][0][0] = vpx_highbd_convolve8_horiz;
- sf->highbd_predict[1][0][1] = vpx_highbd_convolve8_avg_horiz;
- } else {
- // Must always scale in both directions.
- sf->highbd_predict[0][0][0] = vpx_highbd_convolve8;
- sf->highbd_predict[0][0][1] = vpx_highbd_convolve8_avg;
- sf->highbd_predict[0][1][0] = vpx_highbd_convolve8;
- sf->highbd_predict[0][1][1] = vpx_highbd_convolve8_avg;
- sf->highbd_predict[1][0][0] = vpx_highbd_convolve8;
- sf->highbd_predict[1][0][1] = vpx_highbd_convolve8_avg;
- }
- }
- // 2D subpel motion always gets filtered in both directions.
- sf->highbd_predict[1][1][0] = vpx_highbd_convolve8;
- sf->highbd_predict[1][1][1] = vpx_highbd_convolve8_avg;
- }
-#endif
-}
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_SCALE_H_
-#define VP10_COMMON_SCALE_H_
-
-#include "vp10/common/mv.h"
-#include "vpx_dsp/vpx_convolve.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define REF_SCALE_SHIFT 14
-#define REF_NO_SCALE (1 << REF_SCALE_SHIFT)
-#define REF_INVALID_SCALE -1
-
-struct scale_factors {
- int x_scale_fp; // horizontal fixed point scale factor
- int y_scale_fp; // vertical fixed point scale factor
- int x_step_q4;
- int y_step_q4;
-
- int (*scale_value_x)(int val, const struct scale_factors *sf);
- int (*scale_value_y)(int val, const struct scale_factors *sf);
-
- convolve_fn_t predict[2][2][2]; // horiz, vert, avg
-#if CONFIG_VP9_HIGHBITDEPTH
- highbd_convolve_fn_t highbd_predict[2][2][2]; // horiz, vert, avg
-#endif
-};
-
-MV32 vp10_scale_mv(const MV *mv, int x, int y, const struct scale_factors *sf);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_setup_scale_factors_for_frame(struct scale_factors *sf,
- int other_w, int other_h,
- int this_w, int this_h,
- int use_high);
-#else
-void vp10_setup_scale_factors_for_frame(struct scale_factors *sf,
- int other_w, int other_h,
- int this_w, int this_h);
-#endif
-
-static INLINE int vp10_is_valid_scale(const struct scale_factors *sf) {
- return sf->x_scale_fp != REF_INVALID_SCALE &&
- sf->y_scale_fp != REF_INVALID_SCALE;
-}
-
-static INLINE int vp10_is_scaled(const struct scale_factors *sf) {
- return vp10_is_valid_scale(sf) &&
- (sf->x_scale_fp != REF_NO_SCALE || sf->y_scale_fp != REF_NO_SCALE);
-}
-
-static INLINE int valid_ref_frame_size(int ref_width, int ref_height,
- int this_width, int this_height) {
- return 2 * this_width >= ref_width &&
- 2 * this_height >= ref_height &&
- this_width <= 16 * ref_width &&
- this_height <= 16 * ref_height;
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_SCALE_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/scan.h"
-
-DECLARE_ALIGNED(16, static const int16_t, default_scan_4x4[16]) = {
- 0, 4, 1, 5,
- 8, 2, 12, 9,
- 3, 6, 13, 10,
- 7, 14, 11, 15,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, col_scan_4x4[16]) = {
- 0, 4, 8, 1,
- 12, 5, 9, 2,
- 13, 6, 10, 3,
- 7, 14, 11, 15,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, row_scan_4x4[16]) = {
- 0, 1, 4, 2,
- 5, 3, 6, 8,
- 9, 7, 12, 10,
- 13, 11, 14, 15,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, default_scan_8x8[64]) = {
- 0, 8, 1, 16, 9, 2, 17, 24,
- 10, 3, 18, 25, 32, 11, 4, 26,
- 33, 19, 40, 12, 34, 27, 5, 41,
- 20, 48, 13, 35, 42, 28, 21, 6,
- 49, 56, 36, 43, 29, 7, 14, 50,
- 57, 44, 22, 37, 15, 51, 58, 30,
- 45, 23, 52, 59, 38, 31, 60, 53,
- 46, 39, 61, 54, 47, 62, 55, 63,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, col_scan_8x8[64]) = {
- 0, 8, 16, 1, 24, 9, 32, 17,
- 2, 40, 25, 10, 33, 18, 48, 3,
- 26, 41, 11, 56, 19, 34, 4, 49,
- 27, 42, 12, 35, 20, 57, 50, 28,
- 5, 43, 13, 36, 58, 51, 21, 44,
- 6, 29, 59, 37, 14, 52, 22, 7,
- 45, 60, 30, 15, 38, 53, 23, 46,
- 31, 61, 39, 54, 47, 62, 55, 63,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, row_scan_8x8[64]) = {
- 0, 1, 2, 8, 9, 3, 16, 10,
- 4, 17, 11, 24, 5, 18, 25, 12,
- 19, 26, 32, 6, 13, 20, 33, 27,
- 7, 34, 40, 21, 28, 41, 14, 35,
- 48, 42, 29, 36, 49, 22, 43, 15,
- 56, 37, 50, 44, 30, 57, 23, 51,
- 58, 45, 38, 52, 31, 59, 53, 46,
- 60, 39, 61, 47, 54, 55, 62, 63,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, default_scan_16x16[256]) = {
- 0, 16, 1, 32, 17, 2, 48, 33, 18, 3, 64, 34, 49, 19, 65, 80,
- 50, 4, 35, 66, 20, 81, 96, 51, 5, 36, 82, 97, 67, 112, 21, 52,
- 98, 37, 83, 113, 6, 68, 128, 53, 22, 99, 114, 84, 7, 129, 38, 69,
- 100, 115, 144, 130, 85, 54, 23, 8, 145, 39, 70, 116, 101, 131, 160, 146,
- 55, 86, 24, 71, 132, 117, 161, 40, 9, 102, 147, 176, 162, 87, 56, 25,
- 133, 118, 177, 148, 72, 103, 41, 163, 10, 192, 178, 88, 57, 134, 149, 119,
- 26, 164, 73, 104, 193, 42, 179, 208, 11, 135, 89, 165, 120, 150, 58, 194,
- 180, 27, 74, 209, 105, 151, 136, 43, 90, 224, 166, 195, 181, 121, 210, 59,
- 12, 152, 106, 167, 196, 75, 137, 225, 211, 240, 182, 122, 91, 28, 197, 13,
- 226, 168, 183, 153, 44, 212, 138, 107, 241, 60, 29, 123, 198, 184, 227, 169,
- 242, 76, 213, 154, 45, 92, 14, 199, 139, 61, 228, 214, 170, 185, 243, 108,
- 77, 155, 30, 15, 200, 229, 124, 215, 244, 93, 46, 186, 171, 201, 109, 140,
- 230, 62, 216, 245, 31, 125, 78, 156, 231, 47, 187, 202, 217, 94, 246, 141,
- 63, 232, 172, 110, 247, 157, 79, 218, 203, 126, 233, 188, 248, 95, 173, 142,
- 219, 111, 249, 234, 158, 127, 189, 204, 250, 235, 143, 174, 220, 205, 159,
- 251,
- 190, 221, 175, 236, 237, 191, 206, 252, 222, 253, 207, 238, 223, 254, 239,
- 255,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, col_scan_16x16[256]) = {
- 0, 16, 32, 48, 1, 64, 17, 80, 33, 96, 49, 2, 65, 112, 18, 81,
- 34, 128, 50, 97, 3, 66, 144, 19, 113, 35, 82, 160, 98, 51, 129, 4,
- 67, 176, 20, 114, 145, 83, 36, 99, 130, 52, 192, 5, 161, 68, 115, 21,
- 146, 84, 208, 177, 37, 131, 100, 53, 162, 224, 69, 6, 116, 193, 147, 85,
- 22, 240, 132, 38, 178, 101, 163, 54, 209, 117, 70, 7, 148, 194, 86, 179,
- 225, 23, 133, 39, 164, 8, 102, 210, 241, 55, 195, 118, 149, 71, 180, 24,
- 87, 226, 134, 165, 211, 40, 103, 56, 72, 150, 196, 242, 119, 9, 181, 227,
- 88, 166, 25, 135, 41, 104, 212, 57, 151, 197, 120, 73, 243, 182, 136, 167,
- 213, 89, 10, 228, 105, 152, 198, 26, 42, 121, 183, 244, 168, 58, 137, 229,
- 74, 214, 90, 153, 199, 184, 11, 106, 245, 27, 122, 230, 169, 43, 215, 59,
- 200, 138, 185, 246, 75, 12, 91, 154, 216, 231, 107, 28, 44, 201, 123, 170,
- 60, 247, 232, 76, 139, 13, 92, 217, 186, 248, 155, 108, 29, 124, 45, 202,
- 233, 171, 61, 14, 77, 140, 15, 249, 93, 30, 187, 156, 218, 46, 109, 125,
- 62, 172, 78, 203, 31, 141, 234, 94, 47, 188, 63, 157, 110, 250, 219, 79,
- 126, 204, 173, 142, 95, 189, 111, 235, 158, 220, 251, 127, 174, 143, 205,
- 236,
- 159, 190, 221, 252, 175, 206, 237, 191, 253, 222, 238, 207, 254, 223, 239,
- 255,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, row_scan_16x16[256]) = {
- 0, 1, 2, 16, 3, 17, 4, 18, 32, 5, 33, 19, 6, 34, 48, 20,
- 49, 7, 35, 21, 50, 64, 8, 36, 65, 22, 51, 37, 80, 9, 66, 52,
- 23, 38, 81, 67, 10, 53, 24, 82, 68, 96, 39, 11, 54, 83, 97, 69,
- 25, 98, 84, 40, 112, 55, 12, 70, 99, 113, 85, 26, 41, 56, 114, 100,
- 13, 71, 128, 86, 27, 115, 101, 129, 42, 57, 72, 116, 14, 87, 130, 102,
- 144, 73, 131, 117, 28, 58, 15, 88, 43, 145, 103, 132, 146, 118, 74, 160,
- 89, 133, 104, 29, 59, 147, 119, 44, 161, 148, 90, 105, 134, 162, 120, 176,
- 75, 135, 149, 30, 60, 163, 177, 45, 121, 91, 106, 164, 178, 150, 192, 136,
- 165, 179, 31, 151, 193, 76, 122, 61, 137, 194, 107, 152, 180, 208, 46, 166,
- 167, 195, 92, 181, 138, 209, 123, 153, 224, 196, 77, 168, 210, 182, 240, 108,
- 197, 62, 154, 225, 183, 169, 211, 47, 139, 93, 184, 226, 212, 241, 198, 170,
- 124, 155, 199, 78, 213, 185, 109, 227, 200, 63, 228, 242, 140, 214, 171, 186,
- 156, 229, 243, 125, 94, 201, 244, 215, 216, 230, 141, 187, 202, 79, 172, 110,
- 157, 245, 217, 231, 95, 246, 232, 126, 203, 247, 233, 173, 218, 142, 111,
- 158,
- 188, 248, 127, 234, 219, 249, 189, 204, 143, 174, 159, 250, 235, 205, 220,
- 175,
- 190, 251, 221, 191, 206, 236, 207, 237, 252, 222, 253, 223, 238, 239, 254,
- 255,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, default_scan_32x32[1024]) = {
- 0, 32, 1, 64, 33, 2, 96, 65, 34, 128, 3, 97, 66, 160,
- 129, 35, 98, 4, 67, 130, 161, 192, 36, 99, 224, 5, 162, 193,
- 68, 131, 37, 100,
- 225, 194, 256, 163, 69, 132, 6, 226, 257, 288, 195, 101, 164, 38,
- 258, 7, 227, 289, 133, 320, 70, 196, 165, 290, 259, 228, 39, 321,
- 102, 352, 8, 197,
- 71, 134, 322, 291, 260, 353, 384, 229, 166, 103, 40, 354, 323, 292,
- 135, 385, 198, 261, 72, 9, 416, 167, 386, 355, 230, 324, 104, 293,
- 41, 417, 199, 136,
- 262, 387, 448, 325, 356, 10, 73, 418, 231, 168, 449, 294, 388, 105,
- 419, 263, 42, 200, 357, 450, 137, 480, 74, 326, 232, 11, 389, 169,
- 295, 420, 106, 451,
- 481, 358, 264, 327, 201, 43, 138, 512, 482, 390, 296, 233, 170, 421,
- 75, 452, 359, 12, 513, 265, 483, 328, 107, 202, 514, 544, 422, 391,
- 453, 139, 44, 234,
- 484, 297, 360, 171, 76, 515, 545, 266, 329, 454, 13, 423, 203, 108,
- 546, 485, 576, 298, 235, 140, 361, 330, 172, 547, 45, 455, 267, 577,
- 486, 77, 204, 362,
- 608, 14, 299, 578, 109, 236, 487, 609, 331, 141, 579, 46, 15, 173,
- 610, 363, 78, 205, 16, 110, 237, 611, 142, 47, 174, 79, 206, 17,
- 111, 238, 48, 143,
- 80, 175, 112, 207, 49, 18, 239, 81, 113, 19, 50, 82, 114, 51,
- 83, 115, 640, 516, 392, 268, 144, 20, 672, 641, 548, 517, 424,
- 393, 300, 269, 176, 145,
- 52, 21, 704, 673, 642, 580, 549, 518, 456, 425, 394, 332, 301,
- 270, 208, 177, 146, 84, 53, 22, 736, 705, 674, 643, 612, 581,
- 550, 519, 488, 457, 426, 395,
- 364, 333, 302, 271, 240, 209, 178, 147, 116, 85, 54, 23, 737,
- 706, 675, 613, 582, 551, 489, 458, 427, 365, 334, 303, 241,
- 210, 179, 117, 86, 55, 738, 707,
- 614, 583, 490, 459, 366, 335, 242, 211, 118, 87, 739, 615, 491,
- 367, 243, 119, 768, 644, 520, 396, 272, 148, 24, 800, 769, 676,
- 645, 552, 521, 428, 397, 304,
- 273, 180, 149, 56, 25, 832, 801, 770, 708, 677, 646, 584, 553,
- 522, 460, 429, 398, 336, 305, 274, 212, 181, 150, 88, 57, 26,
- 864, 833, 802, 771, 740, 709,
- 678, 647, 616, 585, 554, 523, 492, 461, 430, 399, 368, 337, 306,
- 275, 244, 213, 182, 151, 120, 89, 58, 27, 865, 834, 803, 741,
- 710, 679, 617, 586, 555, 493,
- 462, 431, 369, 338, 307, 245, 214, 183, 121, 90, 59, 866, 835,
- 742, 711, 618, 587, 494, 463, 370, 339, 246, 215, 122, 91, 867,
- 743, 619, 495, 371, 247, 123,
- 896, 772, 648, 524, 400, 276, 152, 28, 928, 897, 804, 773, 680,
- 649, 556, 525, 432, 401, 308, 277, 184, 153, 60, 29, 960, 929,
- 898, 836, 805, 774, 712, 681,
- 650, 588, 557, 526, 464, 433, 402, 340, 309, 278, 216, 185, 154,
- 92, 61, 30, 992, 961, 930, 899, 868, 837, 806, 775, 744, 713, 682,
- 651, 620, 589, 558, 527,
- 496, 465, 434, 403, 372, 341, 310, 279, 248, 217, 186, 155, 124,
- 93, 62, 31, 993, 962, 931, 869, 838, 807, 745, 714, 683, 621, 590,
- 559, 497, 466, 435, 373,
- 342, 311, 249, 218, 187, 125, 94, 63, 994, 963, 870, 839, 746, 715,
- 622, 591, 498, 467, 374, 343, 250, 219, 126, 95, 995, 871, 747, 623,
- 499, 375, 251, 127,
- 900, 776, 652, 528, 404, 280, 156, 932, 901, 808, 777, 684, 653, 560,
- 529, 436, 405, 312, 281, 188, 157, 964, 933, 902, 840, 809, 778, 716,
- 685, 654, 592, 561,
- 530, 468, 437, 406, 344, 313, 282, 220, 189, 158, 996, 965, 934, 903,
- 872, 841, 810, 779, 748, 717, 686, 655, 624, 593, 562, 531, 500, 469,
- 438, 407, 376, 345,
- 314, 283, 252, 221, 190, 159, 997, 966, 935, 873, 842, 811, 749, 718,
- 687, 625, 594, 563, 501, 470, 439, 377, 346, 315, 253, 222, 191, 998,
- 967, 874, 843, 750,
- 719, 626, 595, 502, 471, 378, 347, 254, 223, 999, 875, 751, 627, 503,
- 379, 255, 904, 780, 656, 532, 408, 284, 936, 905, 812, 781, 688, 657,
- 564, 533, 440, 409,
- 316, 285, 968, 937, 906, 844, 813, 782, 720, 689, 658, 596, 565, 534,
- 472, 441, 410, 348, 317, 286, 1000, 969, 938, 907, 876, 845, 814, 783,
- 752, 721, 690, 659,
- 628, 597, 566, 535, 504, 473, 442, 411, 380, 349, 318, 287, 1001, 970,
- 939, 877, 846, 815, 753, 722, 691, 629, 598, 567, 505, 474, 443, 381,
- 350, 319, 1002, 971,
- 878, 847, 754, 723, 630, 599, 506, 475, 382, 351, 1003, 879, 755, 631,
- 507, 383, 908, 784, 660, 536, 412, 940, 909, 816, 785, 692, 661, 568,
- 537, 444, 413, 972,
- 941, 910, 848, 817, 786, 724, 693, 662, 600, 569, 538, 476, 445, 414,
- 1004, 973, 942, 911, 880, 849, 818, 787, 756, 725, 694, 663, 632, 601,
- 570, 539, 508, 477,
- 446, 415, 1005, 974, 943, 881, 850, 819, 757, 726, 695, 633, 602, 571,
- 509, 478, 447, 1006, 975, 882, 851, 758, 727, 634, 603, 510, 479,
- 1007, 883, 759, 635, 511,
- 912, 788, 664, 540, 944, 913, 820, 789, 696, 665, 572, 541, 976, 945,
- 914, 852, 821, 790, 728, 697, 666, 604, 573, 542, 1008, 977, 946, 915,
- 884, 853, 822, 791,
- 760, 729, 698, 667, 636, 605, 574, 543, 1009, 978, 947, 885, 854, 823,
- 761, 730, 699, 637, 606, 575, 1010, 979, 886, 855, 762, 731, 638, 607,
- 1011, 887, 763, 639,
- 916, 792, 668, 948, 917, 824, 793, 700, 669, 980, 949, 918, 856, 825,
- 794, 732, 701, 670, 1012, 981, 950, 919, 888, 857, 826, 795, 764, 733,
- 702, 671, 1013, 982,
- 951, 889, 858, 827, 765, 734, 703, 1014, 983, 890, 859, 766, 735, 1015,
- 891, 767, 920, 796, 952, 921, 828, 797, 984, 953, 922, 860, 829, 798,
- 1016, 985, 954, 923,
- 892, 861, 830, 799, 1017, 986, 955, 893, 862, 831, 1018, 987, 894, 863,
- 1019, 895, 924, 956, 925, 988, 957, 926, 1020, 989, 958, 927, 1021,
- 990, 959, 1022, 991, 1023,
-};
-
-// Neighborhood 5-tuples for various scans and blocksizes,
-// in {top, left, topleft, topright, bottomleft} order
-// for each position in raster scan order.
-// -1 indicates the neighbor does not exist.
-DECLARE_ALIGNED(16, static const int16_t,
- default_scan_4x4_neighbors[17 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 0, 0, 1, 4, 4, 4, 1, 1, 8, 8, 5, 8, 2, 2, 2, 5, 9, 12, 6, 9,
- 3, 6, 10, 13, 7, 10, 11, 14, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- col_scan_4x4_neighbors[17 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 4, 4, 0, 0, 8, 8, 1, 1, 5, 5, 1, 1, 9, 9, 2, 2, 6, 6, 2, 2, 3,
- 3, 10, 10, 7, 7, 11, 11, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- row_scan_4x4_neighbors[17 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 0, 0, 1, 1, 4, 4, 2, 2, 5, 5, 4, 4, 8, 8, 6, 6, 8, 8, 9, 9, 12,
- 12, 10, 10, 13, 13, 14, 14, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- col_scan_8x8_neighbors[65 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 8, 8, 0, 0, 16, 16, 1, 1, 24, 24, 9, 9, 1, 1, 32, 32, 17, 17, 2,
- 2, 25, 25, 10, 10, 40, 40, 2, 2, 18, 18, 33, 33, 3, 3, 48, 48, 11, 11, 26,
- 26, 3, 3, 41, 41, 19, 19, 34, 34, 4, 4, 27, 27, 12, 12, 49, 49, 42, 42, 20,
- 20, 4, 4, 35, 35, 5, 5, 28, 28, 50, 50, 43, 43, 13, 13, 36, 36, 5, 5, 21, 21,
- 51, 51, 29, 29, 6, 6, 44, 44, 14, 14, 6, 6, 37, 37, 52, 52, 22, 22, 7, 7, 30,
- 30, 45, 45, 15, 15, 38, 38, 23, 23, 53, 53, 31, 31, 46, 46, 39, 39, 54, 54,
- 47, 47, 55, 55, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- row_scan_8x8_neighbors[65 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 1, 1, 0, 0, 8, 8, 2, 2, 8, 8, 9, 9, 3, 3, 16, 16, 10, 10, 16, 16,
- 4, 4, 17, 17, 24, 24, 11, 11, 18, 18, 25, 25, 24, 24, 5, 5, 12, 12, 19, 19,
- 32, 32, 26, 26, 6, 6, 33, 33, 32, 32, 20, 20, 27, 27, 40, 40, 13, 13, 34, 34,
- 40, 40, 41, 41, 28, 28, 35, 35, 48, 48, 21, 21, 42, 42, 14, 14, 48, 48, 36,
- 36, 49, 49, 43, 43, 29, 29, 56, 56, 22, 22, 50, 50, 57, 57, 44, 44, 37, 37,
- 51, 51, 30, 30, 58, 58, 52, 52, 45, 45, 59, 59, 38, 38, 60, 60, 46, 46, 53,
- 53, 54, 54, 61, 61, 62, 62, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- default_scan_8x8_neighbors[65 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 0, 0, 8, 8, 1, 8, 1, 1, 9, 16, 16, 16, 2, 9, 2, 2, 10, 17, 17,
- 24, 24, 24, 3, 10, 3, 3, 18, 25, 25, 32, 11, 18, 32, 32, 4, 11, 26, 33, 19,
- 26, 4, 4, 33, 40, 12, 19, 40, 40, 5, 12, 27, 34, 34, 41, 20, 27, 13, 20, 5,
- 5, 41, 48, 48, 48, 28, 35, 35, 42, 21, 28, 6, 6, 6, 13, 42, 49, 49, 56, 36,
- 43, 14, 21, 29, 36, 7, 14, 43, 50, 50, 57, 22, 29, 37, 44, 15, 22, 44, 51,
- 51, 58, 30, 37, 23, 30, 52, 59, 45, 52, 38, 45, 31, 38, 53, 60, 46, 53, 39,
- 46, 54, 61, 47, 54, 55, 62, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- col_scan_16x16_neighbors[257 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 16, 16, 32, 32, 0, 0, 48, 48, 1, 1, 64, 64,
- 17, 17, 80, 80, 33, 33, 1, 1, 49, 49, 96, 96, 2, 2, 65, 65,
- 18, 18, 112, 112, 34, 34, 81, 81, 2, 2, 50, 50, 128, 128, 3, 3,
- 97, 97, 19, 19, 66, 66, 144, 144, 82, 82, 35, 35, 113, 113, 3, 3,
- 51, 51, 160, 160, 4, 4, 98, 98, 129, 129, 67, 67, 20, 20, 83, 83,
- 114, 114, 36, 36, 176, 176, 4, 4, 145, 145, 52, 52, 99, 99, 5, 5,
- 130, 130, 68, 68, 192, 192, 161, 161, 21, 21, 115, 115, 84, 84, 37, 37,
- 146, 146, 208, 208, 53, 53, 5, 5, 100, 100, 177, 177, 131, 131, 69, 69,
- 6, 6, 224, 224, 116, 116, 22, 22, 162, 162, 85, 85, 147, 147, 38, 38,
- 193, 193, 101, 101, 54, 54, 6, 6, 132, 132, 178, 178, 70, 70, 163, 163,
- 209, 209, 7, 7, 117, 117, 23, 23, 148, 148, 7, 7, 86, 86, 194, 194,
- 225, 225, 39, 39, 179, 179, 102, 102, 133, 133, 55, 55, 164, 164, 8, 8,
- 71, 71, 210, 210, 118, 118, 149, 149, 195, 195, 24, 24, 87, 87, 40, 40,
- 56, 56, 134, 134, 180, 180, 226, 226, 103, 103, 8, 8, 165, 165, 211, 211,
- 72, 72, 150, 150, 9, 9, 119, 119, 25, 25, 88, 88, 196, 196, 41, 41,
- 135, 135, 181, 181, 104, 104, 57, 57, 227, 227, 166, 166, 120, 120, 151, 151,
- 197, 197, 73, 73, 9, 9, 212, 212, 89, 89, 136, 136, 182, 182, 10, 10,
- 26, 26, 105, 105, 167, 167, 228, 228, 152, 152, 42, 42, 121, 121, 213, 213,
- 58, 58, 198, 198, 74, 74, 137, 137, 183, 183, 168, 168, 10, 10, 90, 90,
- 229, 229, 11, 11, 106, 106, 214, 214, 153, 153, 27, 27, 199, 199, 43, 43,
- 184, 184, 122, 122, 169, 169, 230, 230, 59, 59, 11, 11, 75, 75, 138, 138,
- 200, 200, 215, 215, 91, 91, 12, 12, 28, 28, 185, 185, 107, 107, 154, 154,
- 44, 44, 231, 231, 216, 216, 60, 60, 123, 123, 12, 12, 76, 76, 201, 201,
- 170, 170, 232, 232, 139, 139, 92, 92, 13, 13, 108, 108, 29, 29, 186, 186,
- 217, 217, 155, 155, 45, 45, 13, 13, 61, 61, 124, 124, 14, 14, 233, 233,
- 77, 77, 14, 14, 171, 171, 140, 140, 202, 202, 30, 30, 93, 93, 109, 109,
- 46, 46, 156, 156, 62, 62, 187, 187, 15, 15, 125, 125, 218, 218, 78, 78,
- 31, 31, 172, 172, 47, 47, 141, 141, 94, 94, 234, 234, 203, 203, 63, 63,
- 110, 110, 188, 188, 157, 157, 126, 126, 79, 79, 173, 173, 95, 95, 219, 219,
- 142, 142, 204, 204, 235, 235, 111, 111, 158, 158, 127, 127, 189, 189, 220,
- 220, 143, 143, 174, 174, 205, 205, 236, 236, 159, 159, 190, 190, 221, 221,
- 175, 175, 237, 237, 206, 206, 222, 222, 191, 191, 238, 238, 207, 207, 223,
- 223, 239, 239, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- row_scan_16x16_neighbors[257 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 1, 1, 0, 0, 2, 2, 16, 16, 3, 3, 17, 17,
- 16, 16, 4, 4, 32, 32, 18, 18, 5, 5, 33, 33, 32, 32, 19, 19,
- 48, 48, 6, 6, 34, 34, 20, 20, 49, 49, 48, 48, 7, 7, 35, 35,
- 64, 64, 21, 21, 50, 50, 36, 36, 64, 64, 8, 8, 65, 65, 51, 51,
- 22, 22, 37, 37, 80, 80, 66, 66, 9, 9, 52, 52, 23, 23, 81, 81,
- 67, 67, 80, 80, 38, 38, 10, 10, 53, 53, 82, 82, 96, 96, 68, 68,
- 24, 24, 97, 97, 83, 83, 39, 39, 96, 96, 54, 54, 11, 11, 69, 69,
- 98, 98, 112, 112, 84, 84, 25, 25, 40, 40, 55, 55, 113, 113, 99, 99,
- 12, 12, 70, 70, 112, 112, 85, 85, 26, 26, 114, 114, 100, 100, 128, 128,
- 41, 41, 56, 56, 71, 71, 115, 115, 13, 13, 86, 86, 129, 129, 101, 101,
- 128, 128, 72, 72, 130, 130, 116, 116, 27, 27, 57, 57, 14, 14, 87, 87,
- 42, 42, 144, 144, 102, 102, 131, 131, 145, 145, 117, 117, 73, 73, 144, 144,
- 88, 88, 132, 132, 103, 103, 28, 28, 58, 58, 146, 146, 118, 118, 43, 43,
- 160, 160, 147, 147, 89, 89, 104, 104, 133, 133, 161, 161, 119, 119, 160, 160,
- 74, 74, 134, 134, 148, 148, 29, 29, 59, 59, 162, 162, 176, 176, 44, 44,
- 120, 120, 90, 90, 105, 105, 163, 163, 177, 177, 149, 149, 176, 176, 135, 135,
- 164, 164, 178, 178, 30, 30, 150, 150, 192, 192, 75, 75, 121, 121, 60, 60,
- 136, 136, 193, 193, 106, 106, 151, 151, 179, 179, 192, 192, 45, 45, 165, 165,
- 166, 166, 194, 194, 91, 91, 180, 180, 137, 137, 208, 208, 122, 122, 152, 152,
- 208, 208, 195, 195, 76, 76, 167, 167, 209, 209, 181, 181, 224, 224, 107, 107,
- 196, 196, 61, 61, 153, 153, 224, 224, 182, 182, 168, 168, 210, 210, 46, 46,
- 138, 138, 92, 92, 183, 183, 225, 225, 211, 211, 240, 240, 197, 197, 169, 169,
- 123, 123, 154, 154, 198, 198, 77, 77, 212, 212, 184, 184, 108, 108, 226, 226,
- 199, 199, 62, 62, 227, 227, 241, 241, 139, 139, 213, 213, 170, 170, 185, 185,
- 155, 155, 228, 228, 242, 242, 124, 124, 93, 93, 200, 200, 243, 243, 214, 214,
- 215, 215, 229, 229, 140, 140, 186, 186, 201, 201, 78, 78, 171, 171, 109, 109,
- 156, 156, 244, 244, 216, 216, 230, 230, 94, 94, 245, 245, 231, 231, 125, 125,
- 202, 202, 246, 246, 232, 232, 172, 172, 217, 217, 141, 141, 110, 110, 157,
- 157, 187, 187, 247, 247, 126, 126, 233, 233, 218, 218, 248, 248, 188, 188,
- 203, 203, 142, 142, 173, 173, 158, 158, 249, 249, 234, 234, 204, 204, 219,
- 219, 174, 174, 189, 189, 250, 250, 220, 220, 190, 190, 205, 205, 235, 235,
- 206, 206, 236, 236, 251, 251, 221, 221, 252, 252, 222, 222, 237, 237, 238,
- 238, 253, 253, 254, 254, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- default_scan_16x16_neighbors[257 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 0, 0, 16, 16, 1, 16, 1, 1, 32, 32, 17, 32,
- 2, 17, 2, 2, 48, 48, 18, 33, 33, 48, 3, 18, 49, 64, 64, 64,
- 34, 49, 3, 3, 19, 34, 50, 65, 4, 19, 65, 80, 80, 80, 35, 50,
- 4, 4, 20, 35, 66, 81, 81, 96, 51, 66, 96, 96, 5, 20, 36, 51,
- 82, 97, 21, 36, 67, 82, 97, 112, 5, 5, 52, 67, 112, 112, 37, 52,
- 6, 21, 83, 98, 98, 113, 68, 83, 6, 6, 113, 128, 22, 37, 53, 68,
- 84, 99, 99, 114, 128, 128, 114, 129, 69, 84, 38, 53, 7, 22, 7, 7,
- 129, 144, 23, 38, 54, 69, 100, 115, 85, 100, 115, 130, 144, 144, 130, 145,
- 39, 54, 70, 85, 8, 23, 55, 70, 116, 131, 101, 116, 145, 160, 24, 39,
- 8, 8, 86, 101, 131, 146, 160, 160, 146, 161, 71, 86, 40, 55, 9, 24,
- 117, 132, 102, 117, 161, 176, 132, 147, 56, 71, 87, 102, 25, 40, 147, 162,
- 9, 9, 176, 176, 162, 177, 72, 87, 41, 56, 118, 133, 133, 148, 103, 118,
- 10, 25, 148, 163, 57, 72, 88, 103, 177, 192, 26, 41, 163, 178, 192, 192,
- 10, 10, 119, 134, 73, 88, 149, 164, 104, 119, 134, 149, 42, 57, 178, 193,
- 164, 179, 11, 26, 58, 73, 193, 208, 89, 104, 135, 150, 120, 135, 27, 42,
- 74, 89, 208, 208, 150, 165, 179, 194, 165, 180, 105, 120, 194, 209, 43, 58,
- 11, 11, 136, 151, 90, 105, 151, 166, 180, 195, 59, 74, 121, 136, 209, 224,
- 195, 210, 224, 224, 166, 181, 106, 121, 75, 90, 12, 27, 181, 196, 12, 12,
- 210, 225, 152, 167, 167, 182, 137, 152, 28, 43, 196, 211, 122, 137, 91, 106,
- 225, 240, 44, 59, 13, 28, 107, 122, 182, 197, 168, 183, 211, 226, 153, 168,
- 226, 241, 60, 75, 197, 212, 138, 153, 29, 44, 76, 91, 13, 13, 183, 198,
- 123, 138, 45, 60, 212, 227, 198, 213, 154, 169, 169, 184, 227, 242, 92, 107,
- 61, 76, 139, 154, 14, 29, 14, 14, 184, 199, 213, 228, 108, 123, 199, 214,
- 228, 243, 77, 92, 30, 45, 170, 185, 155, 170, 185, 200, 93, 108, 124, 139,
- 214, 229, 46, 61, 200, 215, 229, 244, 15, 30, 109, 124, 62, 77, 140, 155,
- 215, 230, 31, 46, 171, 186, 186, 201, 201, 216, 78, 93, 230, 245, 125, 140,
- 47, 62, 216, 231, 156, 171, 94, 109, 231, 246, 141, 156, 63, 78, 202, 217,
- 187, 202, 110, 125, 217, 232, 172, 187, 232, 247, 79, 94, 157, 172, 126, 141,
- 203, 218, 95, 110, 233, 248, 218, 233, 142, 157, 111, 126, 173, 188, 188, 203,
- 234, 249, 219, 234, 127, 142, 158, 173, 204, 219, 189, 204, 143, 158, 235,
- 250, 174, 189, 205, 220, 159, 174, 220, 235, 221, 236, 175, 190, 190, 205,
- 236, 251, 206, 221, 237, 252, 191, 206, 222, 237, 207, 222, 238, 253, 223,
- 238, 239, 254, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t,
- default_scan_32x32_neighbors[1025 * MAX_NEIGHBORS]) = {
- 0, 0, 0, 0, 0, 0, 32, 32, 1, 32, 1, 1, 64, 64, 33, 64,
- 2, 33, 96, 96, 2, 2, 65, 96, 34, 65, 128, 128, 97, 128, 3, 34,
- 66, 97, 3, 3, 35, 66, 98, 129, 129, 160, 160, 160, 4, 35, 67, 98,
- 192, 192, 4, 4, 130, 161, 161, 192, 36, 67, 99, 130, 5, 36, 68, 99,
- 193, 224, 162, 193, 224, 224, 131, 162, 37, 68, 100, 131, 5, 5, 194, 225,
- 225, 256, 256, 256, 163, 194, 69, 100, 132, 163, 6, 37, 226, 257, 6, 6,
- 195, 226, 257, 288, 101, 132, 288, 288, 38, 69, 164, 195, 133, 164, 258, 289,
- 227, 258, 196, 227, 7, 38, 289, 320, 70, 101, 320, 320, 7, 7, 165, 196,
- 39, 70, 102, 133, 290, 321, 259, 290, 228, 259, 321, 352, 352, 352, 197, 228,
- 134, 165, 71, 102, 8, 39, 322, 353, 291, 322, 260, 291, 103, 134, 353, 384,
- 166, 197, 229, 260, 40, 71, 8, 8, 384, 384, 135, 166, 354, 385, 323, 354,
- 198, 229, 292, 323, 72, 103, 261, 292, 9, 40, 385, 416, 167, 198, 104, 135,
- 230, 261, 355, 386, 416, 416, 293, 324, 324, 355, 9, 9, 41, 72, 386, 417,
- 199, 230, 136, 167, 417, 448, 262, 293, 356, 387, 73, 104, 387, 418, 231, 262,
- 10, 41, 168, 199, 325, 356, 418, 449, 105, 136, 448, 448, 42, 73, 294, 325,
- 200, 231, 10, 10, 357, 388, 137, 168, 263, 294, 388, 419, 74, 105, 419, 450,
- 449, 480, 326, 357, 232, 263, 295, 326, 169, 200, 11, 42, 106, 137, 480, 480,
- 450, 481, 358, 389, 264, 295, 201, 232, 138, 169, 389, 420, 43, 74, 420, 451,
- 327, 358, 11, 11, 481, 512, 233, 264, 451, 482, 296, 327, 75, 106, 170, 201,
- 482, 513, 512, 512, 390, 421, 359, 390, 421, 452, 107, 138, 12, 43, 202, 233,
- 452, 483, 265, 296, 328, 359, 139, 170, 44, 75, 483, 514, 513, 544, 234, 265,
- 297, 328, 422, 453, 12, 12, 391, 422, 171, 202, 76, 107, 514, 545, 453, 484,
- 544, 544, 266, 297, 203, 234, 108, 139, 329, 360, 298, 329, 140, 171, 515,
- 546, 13, 44, 423, 454, 235, 266, 545, 576, 454, 485, 45, 76, 172, 203, 330,
- 361, 576, 576, 13, 13, 267, 298, 546, 577, 77, 108, 204, 235, 455, 486, 577,
- 608, 299, 330, 109, 140, 547, 578, 14, 45, 14, 14, 141, 172, 578, 609, 331,
- 362, 46, 77, 173, 204, 15, 15, 78, 109, 205, 236, 579, 610, 110, 141, 15, 46,
- 142, 173, 47, 78, 174, 205, 16, 16, 79, 110, 206, 237, 16, 47, 111, 142,
- 48, 79, 143, 174, 80, 111, 175, 206, 17, 48, 17, 17, 207, 238, 49, 80,
- 81, 112, 18, 18, 18, 49, 50, 81, 82, 113, 19, 50, 51, 82, 83, 114, 608, 608,
- 484, 515, 360, 391, 236, 267, 112, 143, 19, 19, 640, 640, 609, 640, 516, 547,
- 485, 516, 392, 423, 361, 392, 268, 299, 237, 268, 144, 175, 113, 144, 20, 51,
- 20, 20, 672, 672, 641, 672, 610, 641, 548, 579, 517, 548, 486, 517, 424, 455,
- 393, 424, 362, 393, 300, 331, 269, 300, 238, 269, 176, 207, 145, 176, 114,
- 145, 52, 83, 21, 52, 21, 21, 704, 704, 673, 704, 642, 673, 611, 642, 580,
- 611, 549, 580, 518, 549, 487, 518, 456, 487, 425, 456, 394, 425, 363, 394,
- 332, 363, 301, 332, 270, 301, 239, 270, 208, 239, 177, 208, 146, 177, 115,
- 146, 84, 115, 53, 84, 22, 53, 22, 22, 705, 736, 674, 705, 643, 674, 581, 612,
- 550, 581, 519, 550, 457, 488, 426, 457, 395, 426, 333, 364, 302, 333, 271,
- 302, 209, 240, 178, 209, 147, 178, 85, 116, 54, 85, 23, 54, 706, 737, 675,
- 706, 582, 613, 551, 582, 458, 489, 427, 458, 334, 365, 303, 334, 210, 241,
- 179, 210, 86, 117, 55, 86, 707, 738, 583, 614, 459, 490, 335, 366, 211, 242,
- 87, 118, 736, 736, 612, 643, 488, 519, 364, 395, 240, 271, 116, 147, 23, 23,
- 768, 768, 737, 768, 644, 675, 613, 644, 520, 551, 489, 520, 396, 427, 365,
- 396, 272, 303, 241, 272, 148, 179, 117, 148, 24, 55, 24, 24, 800, 800, 769,
- 800, 738, 769, 676, 707, 645, 676, 614, 645, 552, 583, 521, 552, 490, 521,
- 428, 459, 397, 428, 366, 397, 304, 335, 273, 304, 242, 273, 180, 211, 149,
- 180, 118, 149, 56, 87, 25, 56, 25, 25, 832, 832, 801, 832, 770, 801, 739,
- 770, 708, 739, 677, 708, 646, 677, 615, 646, 584, 615, 553, 584, 522, 553,
- 491, 522, 460, 491, 429, 460, 398, 429, 367, 398, 336, 367, 305, 336, 274,
- 305, 243, 274, 212, 243, 181, 212, 150, 181, 119, 150, 88, 119, 57, 88, 26,
- 57, 26, 26, 833, 864, 802, 833, 771, 802, 709, 740, 678, 709, 647, 678, 585,
- 616, 554, 585, 523, 554, 461, 492, 430, 461, 399, 430, 337, 368, 306, 337,
- 275, 306, 213, 244, 182, 213, 151, 182, 89, 120, 58, 89, 27, 58, 834, 865,
- 803, 834, 710, 741, 679, 710, 586, 617, 555, 586, 462, 493, 431, 462, 338,
- 369, 307, 338, 214, 245, 183, 214, 90, 121, 59, 90, 835, 866, 711, 742, 587,
- 618, 463, 494, 339, 370, 215, 246, 91, 122, 864, 864, 740, 771, 616, 647,
- 492, 523, 368, 399, 244, 275, 120, 151, 27, 27, 896, 896, 865, 896, 772, 803,
- 741, 772, 648, 679, 617, 648, 524, 555, 493, 524, 400, 431, 369, 400, 276,
- 307, 245, 276, 152, 183, 121, 152, 28, 59, 28, 28, 928, 928, 897, 928, 866,
- 897, 804, 835, 773, 804, 742, 773, 680, 711, 649, 680, 618, 649, 556, 587,
- 525, 556, 494, 525, 432, 463, 401, 432, 370, 401, 308, 339, 277, 308, 246,
- 277, 184, 215, 153, 184, 122, 153, 60, 91, 29, 60, 29, 29, 960, 960, 929,
- 960, 898, 929, 867, 898, 836, 867, 805, 836, 774, 805, 743, 774, 712, 743,
- 681, 712, 650, 681, 619, 650, 588, 619, 557, 588, 526, 557, 495, 526, 464,
- 495, 433, 464, 402, 433, 371, 402, 340, 371, 309, 340, 278, 309, 247, 278,
- 216, 247, 185, 216, 154, 185, 123, 154, 92, 123, 61, 92, 30, 61, 30, 30,
- 961, 992, 930, 961, 899, 930, 837, 868, 806, 837, 775, 806, 713, 744, 682,
- 713, 651, 682, 589, 620, 558, 589, 527, 558, 465, 496, 434, 465, 403, 434,
- 341, 372, 310, 341, 279, 310, 217, 248, 186, 217, 155, 186, 93, 124, 62, 93,
- 31, 62, 962, 993, 931, 962, 838, 869, 807, 838, 714, 745, 683, 714, 590, 621,
- 559, 590, 466, 497, 435, 466, 342, 373, 311, 342, 218, 249, 187, 218, 94,
- 125, 63, 94, 963, 994, 839, 870, 715, 746, 591, 622, 467, 498, 343, 374, 219,
- 250, 95, 126, 868, 899, 744, 775, 620, 651, 496, 527, 372, 403, 248, 279,
- 124, 155, 900, 931, 869, 900, 776, 807, 745, 776, 652, 683, 621, 652, 528,
- 559, 497, 528, 404, 435, 373, 404, 280, 311, 249, 280, 156, 187, 125, 156,
- 932, 963, 901, 932, 870, 901, 808, 839, 777, 808, 746, 777, 684, 715, 653,
- 684, 622, 653, 560, 591, 529, 560, 498, 529, 436, 467, 405, 436, 374, 405,
- 312, 343, 281, 312, 250, 281, 188, 219, 157, 188, 126, 157, 964, 995, 933,
- 964, 902, 933, 871, 902, 840, 871, 809, 840, 778, 809, 747, 778, 716, 747,
- 685, 716, 654, 685, 623, 654, 592, 623, 561, 592, 530, 561, 499, 530, 468,
- 499, 437, 468, 406, 437, 375, 406, 344, 375, 313, 344, 282, 313, 251, 282,
- 220, 251, 189, 220, 158, 189, 127, 158, 965, 996, 934, 965, 903, 934, 841,
- 872, 810, 841, 779, 810, 717, 748, 686, 717, 655, 686, 593, 624, 562, 593,
- 531, 562, 469, 500, 438, 469, 407, 438, 345, 376, 314, 345, 283, 314, 221,
- 252, 190, 221, 159, 190, 966, 997, 935, 966, 842, 873, 811, 842, 718, 749,
- 687, 718, 594, 625, 563, 594, 470, 501, 439, 470, 346, 377, 315, 346, 222,
- 253, 191, 222, 967, 998, 843, 874, 719, 750, 595, 626, 471, 502, 347, 378,
- 223, 254, 872, 903, 748, 779, 624, 655, 500, 531, 376, 407, 252, 283, 904,
- 935, 873, 904, 780, 811, 749, 780, 656, 687, 625, 656, 532, 563, 501, 532,
- 408, 439, 377, 408, 284, 315, 253, 284, 936, 967, 905, 936, 874, 905, 812,
- 843, 781, 812, 750, 781, 688, 719, 657, 688, 626, 657, 564, 595, 533, 564,
- 502, 533, 440, 471, 409, 440, 378, 409, 316, 347, 285, 316, 254, 285, 968,
- 999, 937, 968, 906, 937, 875, 906, 844, 875, 813, 844, 782, 813, 751, 782,
- 720, 751, 689, 720, 658, 689, 627, 658, 596, 627, 565, 596, 534, 565, 503,
- 534, 472, 503, 441, 472, 410, 441, 379, 410, 348, 379, 317, 348, 286, 317,
- 255, 286, 969, 1000, 938, 969, 907, 938, 845, 876, 814, 845, 783, 814, 721,
- 752, 690, 721, 659, 690, 597, 628, 566, 597, 535, 566, 473, 504, 442, 473,
- 411, 442, 349, 380, 318, 349, 287, 318, 970, 1001, 939, 970, 846, 877, 815,
- 846, 722, 753, 691, 722, 598, 629, 567, 598, 474, 505, 443, 474, 350, 381,
- 319, 350, 971, 1002, 847, 878, 723, 754, 599, 630, 475, 506, 351, 382, 876,
- 907, 752, 783, 628, 659, 504, 535, 380, 411, 908, 939, 877, 908, 784, 815,
- 753, 784, 660, 691, 629, 660, 536, 567, 505, 536, 412, 443, 381, 412, 940,
- 971, 909, 940, 878, 909, 816, 847, 785, 816, 754, 785, 692, 723, 661, 692,
- 630, 661, 568, 599, 537, 568, 506, 537, 444, 475, 413, 444, 382, 413, 972,
- 1003, 941, 972, 910, 941, 879, 910, 848, 879, 817, 848, 786, 817, 755, 786,
- 724, 755, 693, 724, 662, 693, 631, 662, 600, 631, 569, 600, 538, 569, 507,
- 538, 476, 507, 445, 476, 414, 445, 383, 414, 973, 1004, 942, 973, 911, 942,
- 849, 880, 818, 849, 787, 818, 725, 756, 694, 725, 663, 694, 601, 632, 570,
- 601, 539, 570, 477, 508, 446, 477, 415, 446, 974, 1005, 943, 974, 850, 881,
- 819, 850, 726, 757, 695, 726, 602, 633, 571, 602, 478, 509, 447, 478, 975,
- 1006, 851, 882, 727, 758, 603, 634, 479, 510, 880, 911, 756, 787, 632, 663,
- 508, 539, 912, 943, 881, 912, 788, 819, 757, 788, 664, 695, 633, 664, 540,
- 571, 509, 540, 944, 975, 913, 944, 882, 913, 820, 851, 789, 820, 758, 789,
- 696, 727, 665, 696, 634, 665, 572, 603, 541, 572, 510, 541, 976, 1007, 945,
- 976, 914, 945, 883, 914, 852, 883, 821, 852, 790, 821, 759, 790, 728, 759,
- 697, 728, 666, 697, 635, 666, 604, 635, 573, 604, 542, 573, 511, 542, 977,
- 1008, 946, 977, 915, 946, 853, 884, 822, 853, 791, 822, 729, 760, 698, 729,
- 667, 698, 605, 636, 574, 605, 543, 574, 978, 1009, 947, 978, 854, 885, 823,
- 854, 730, 761, 699, 730, 606, 637, 575, 606, 979, 1010, 855, 886, 731, 762,
- 607, 638, 884, 915, 760, 791, 636, 667, 916, 947, 885, 916, 792, 823, 761,
- 792, 668, 699, 637, 668, 948, 979, 917, 948, 886, 917, 824, 855, 793, 824,
- 762, 793, 700, 731, 669, 700, 638, 669, 980, 1011, 949, 980, 918, 949, 887,
- 918, 856, 887, 825, 856, 794, 825, 763, 794, 732, 763, 701, 732, 670, 701,
- 639, 670, 981, 1012, 950, 981, 919, 950, 857, 888, 826, 857, 795, 826, 733,
- 764, 702, 733, 671, 702, 982, 1013, 951, 982, 858, 889, 827, 858, 734, 765,
- 703, 734, 983, 1014, 859, 890, 735, 766, 888, 919, 764, 795, 920, 951, 889,
- 920, 796, 827, 765, 796, 952, 983, 921, 952, 890, 921, 828, 859, 797, 828,
- 766, 797, 984, 1015, 953, 984, 922, 953, 891, 922, 860, 891, 829, 860, 798,
- 829, 767, 798, 985, 1016, 954, 985, 923, 954, 861, 892, 830, 861, 799, 830,
- 986, 1017, 955, 986, 862, 893, 831, 862, 987, 1018, 863, 894, 892, 923, 924,
- 955, 893, 924, 956, 987, 925, 956, 894, 925, 988, 1019, 957, 988, 926, 957,
- 895, 926, 989, 1020, 958, 989, 927, 958, 990, 1021, 959, 990, 991, 1022, 0, 0,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_default_iscan_4x4[16]) = {
- 0, 2, 5, 8, 1, 3, 9, 12, 4, 7, 11, 14, 6, 10, 13, 15,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_col_iscan_4x4[16]) = {
- 0, 3, 7, 11, 1, 5, 9, 12, 2, 6, 10, 14, 4, 8, 13, 15,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_row_iscan_4x4[16]) = {
- 0, 1, 3, 5, 2, 4, 6, 9, 7, 8, 11, 13, 10, 12, 14, 15,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_col_iscan_8x8[64]) = {
- 0, 3, 8, 15, 22, 32, 40, 47, 1, 5, 11, 18, 26, 34, 44, 51,
- 2, 7, 13, 20, 28, 38, 46, 54, 4, 10, 16, 24, 31, 41, 50, 56,
- 6, 12, 21, 27, 35, 43, 52, 58, 9, 17, 25, 33, 39, 48, 55, 60,
- 14, 23, 30, 37, 45, 53, 59, 62, 19, 29, 36, 42, 49, 57, 61, 63,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_row_iscan_8x8[64]) = {
- 0, 1, 2, 5, 8, 12, 19, 24, 3, 4, 7, 10, 15, 20, 30, 39,
- 6, 9, 13, 16, 21, 27, 37, 46, 11, 14, 17, 23, 28, 34, 44, 52,
- 18, 22, 25, 31, 35, 41, 50, 57, 26, 29, 33, 38, 43, 49, 55, 59,
- 32, 36, 42, 47, 51, 54, 60, 61, 40, 45, 48, 53, 56, 58, 62, 63,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_default_iscan_8x8[64]) = {
- 0, 2, 5, 9, 14, 22, 31, 37, 1, 4, 8, 13, 19, 26, 38, 44,
- 3, 6, 10, 17, 24, 30, 42, 49, 7, 11, 15, 21, 29, 36, 47, 53,
- 12, 16, 20, 27, 34, 43, 52, 57, 18, 23, 28, 35, 41, 48, 56, 60,
- 25, 32, 39, 45, 50, 55, 59, 62, 33, 40, 46, 51, 54, 58, 61, 63,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_col_iscan_16x16[256]) = {
- 0, 4, 11, 20, 31, 43, 59, 75, 85, 109, 130, 150, 165, 181, 195, 198,
- 1, 6, 14, 23, 34, 47, 64, 81, 95, 114, 135, 153, 171, 188, 201, 212,
- 2, 8, 16, 25, 38, 52, 67, 83, 101, 116, 136, 157, 172, 190, 205, 216,
- 3, 10, 18, 29, 41, 55, 71, 89, 103, 119, 141, 159, 176, 194, 208, 218,
- 5, 12, 21, 32, 45, 58, 74, 93, 104, 123, 144, 164, 179, 196, 210, 223,
- 7, 15, 26, 37, 49, 63, 78, 96, 112, 129, 146, 166, 182, 200, 215, 228,
- 9, 19, 28, 39, 54, 69, 86, 102, 117, 132, 151, 170, 187, 206, 220, 230,
- 13, 24, 35, 46, 60, 73, 91, 108, 122, 137, 154, 174, 189, 207, 224, 235,
- 17, 30, 40, 53, 66, 82, 98, 115, 126, 142, 161, 180, 197, 213, 227, 237,
- 22, 36, 48, 62, 76, 92, 105, 120, 133, 147, 167, 186, 203, 219, 232, 240,
- 27, 44, 56, 70, 84, 99, 113, 127, 140, 156, 175, 193, 209, 226, 236, 244,
- 33, 51, 68, 79, 94, 110, 125, 138, 149, 162, 184, 202, 217, 229, 241, 247,
- 42, 61, 77, 90, 106, 121, 134, 148, 160, 173, 191, 211, 225, 238, 245, 251,
- 50, 72, 87, 100, 118, 128, 145, 158, 168, 183, 204, 222, 233, 242, 249, 253,
- 57, 80, 97, 111, 131, 143, 155, 169, 178, 192, 214, 231, 239, 246, 250, 254,
- 65, 88, 107, 124, 139, 152, 163, 177, 185, 199, 221, 234, 243, 248, 252, 255,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_row_iscan_16x16[256]) = {
- 0, 1, 2, 4, 6, 9, 12, 17, 22, 29, 36, 43, 54, 64, 76, 86,
- 3, 5, 7, 11, 15, 19, 25, 32, 38, 48, 59, 68, 84, 99, 115, 130,
- 8, 10, 13, 18, 23, 27, 33, 42, 51, 60, 72, 88, 103, 119, 142, 167,
- 14, 16, 20, 26, 31, 37, 44, 53, 61, 73, 85, 100, 116, 135, 161, 185,
- 21, 24, 30, 35, 40, 47, 55, 65, 74, 81, 94, 112, 133, 154, 179, 205,
- 28, 34, 39, 45, 50, 58, 67, 77, 87, 96, 106, 121, 146, 169, 196, 212,
- 41, 46, 49, 56, 63, 70, 79, 90, 98, 107, 122, 138, 159, 182, 207, 222,
- 52, 57, 62, 69, 75, 83, 93, 102, 110, 120, 134, 150, 176, 195, 215, 226,
- 66, 71, 78, 82, 91, 97, 108, 113, 127, 136, 148, 168, 188, 202, 221, 232,
- 80, 89, 92, 101, 105, 114, 125, 131, 139, 151, 162, 177, 192, 208, 223, 234,
- 95, 104, 109, 117, 123, 128, 143, 144, 155, 165, 175, 190, 206, 219, 233, 239,
- 111, 118, 124, 129, 140, 147, 157, 164, 170, 181, 191, 203, 224, 230, 240,
- 243, 126, 132, 137, 145, 153, 160, 174, 178, 184, 197, 204, 216, 231, 237,
- 244, 246, 141, 149, 156, 166, 172, 180, 189, 199, 200, 210, 220, 228, 238,
- 242, 249, 251, 152, 163, 171, 183, 186, 193, 201, 211, 214, 218, 227, 236,
- 245, 247, 252, 253, 158, 173, 187, 194, 198, 209, 213, 217, 225, 229, 235,
- 241, 248, 250, 254, 255,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_default_iscan_16x16[256]) = {
- 0, 2, 5, 9, 17, 24, 36, 44, 55, 72, 88, 104, 128, 143, 166, 179,
- 1, 4, 8, 13, 20, 30, 40, 54, 66, 79, 96, 113, 141, 154, 178, 196,
- 3, 7, 11, 18, 25, 33, 46, 57, 71, 86, 101, 119, 148, 164, 186, 201,
- 6, 12, 16, 23, 31, 39, 53, 64, 78, 92, 110, 127, 153, 169, 193, 208,
- 10, 14, 19, 28, 37, 47, 58, 67, 84, 98, 114, 133, 161, 176, 198, 214,
- 15, 21, 26, 34, 43, 52, 65, 77, 91, 106, 120, 140, 165, 185, 205, 221,
- 22, 27, 32, 41, 48, 60, 73, 85, 99, 116, 130, 151, 175, 190, 211, 225,
- 29, 35, 42, 49, 59, 69, 81, 95, 108, 125, 139, 155, 182, 197, 217, 229,
- 38, 45, 51, 61, 68, 80, 93, 105, 118, 134, 150, 168, 191, 207, 223, 234,
- 50, 56, 63, 74, 83, 94, 109, 117, 129, 147, 163, 177, 199, 213, 228, 238,
- 62, 70, 76, 87, 97, 107, 122, 131, 145, 159, 172, 188, 210, 222, 235, 242,
- 75, 82, 90, 102, 112, 124, 138, 146, 157, 173, 187, 202, 219, 230, 240, 245,
- 89, 100, 111, 123, 132, 142, 156, 167, 180, 189, 203, 216, 231, 237, 246, 250,
- 103, 115, 126, 136, 149, 162, 171, 183, 194, 204, 215, 224, 236, 241, 248,
- 252, 121, 135, 144, 158, 170, 181, 192, 200, 209, 218, 227, 233, 243, 244,
- 251, 254, 137, 152, 160, 174, 184, 195, 206, 212, 220, 226, 232, 239, 247,
- 249, 253, 255,
-};
-
-DECLARE_ALIGNED(16, static const int16_t, vp10_default_iscan_32x32[1024]) = {
- 0, 2, 5, 10, 17, 25, 38, 47, 62, 83, 101, 121, 145, 170, 193, 204,
- 210, 219, 229, 233, 245, 257, 275, 299, 342, 356, 377, 405, 455, 471, 495,
- 527, 1, 4, 8, 15, 22, 30, 45, 58, 74, 92, 112, 133, 158, 184, 203, 215, 222,
- 228, 234, 237, 256, 274, 298, 317, 355, 376, 404, 426, 470, 494, 526, 551,
- 3, 7, 12, 18, 28, 36, 52, 64, 82, 102, 118, 142, 164, 189, 208, 217, 224,
- 231, 235, 238, 273, 297, 316, 329, 375, 403, 425, 440, 493, 525, 550, 567,
- 6, 11, 16, 23, 31, 43, 60, 73, 90, 109, 126, 150, 173, 196, 211, 220, 226,
- 232, 236, 239, 296, 315, 328, 335, 402, 424, 439, 447, 524, 549, 566, 575,
- 9, 14, 19, 29, 37, 50, 65, 78, 95, 116, 134, 157, 179, 201, 214, 223, 244,
- 255, 272, 295, 341, 354, 374, 401, 454, 469, 492, 523, 582, 596, 617, 645,
- 13, 20, 26, 35, 44, 54, 72, 85, 105, 123, 140, 163, 182, 205, 216, 225,
- 254, 271, 294, 314, 353, 373, 400, 423, 468, 491, 522, 548, 595, 616, 644,
- 666, 21, 27, 33, 42, 53, 63, 80, 94, 113, 132, 151, 172, 190, 209, 218, 227,
- 270, 293, 313, 327, 372, 399, 422, 438, 490, 521, 547, 565, 615, 643, 665,
- 680, 24, 32, 39, 48, 57, 71, 88, 104, 120, 139, 159, 178, 197, 212, 221, 230,
- 292, 312, 326, 334, 398, 421, 437, 446, 520, 546, 564, 574, 642, 664, 679,
- 687, 34, 40, 46, 56, 68, 81, 96, 111, 130, 147, 167, 186, 243, 253, 269, 291,
- 340, 352, 371, 397, 453, 467, 489, 519, 581, 594, 614, 641, 693, 705, 723,
- 747, 41, 49, 55, 67, 77, 91, 107, 124, 138, 161, 177, 194, 252, 268, 290,
- 311, 351, 370, 396, 420, 466, 488, 518, 545, 593, 613, 640, 663, 704, 722,
- 746, 765, 51, 59, 66, 76, 89, 99, 119, 131, 149, 168, 181, 200, 267, 289,
- 310, 325, 369, 395, 419, 436, 487, 517, 544, 563, 612, 639, 662, 678, 721,
- 745, 764, 777, 61, 69, 75, 87, 100, 114, 129, 144, 162, 180, 191, 207, 288,
- 309, 324, 333, 394, 418, 435, 445, 516, 543, 562, 573, 638, 661, 677, 686,
- 744, 763, 776, 783, 70, 79, 86, 97, 108, 122, 137, 155, 242, 251, 266, 287,
- 339, 350, 368, 393, 452, 465, 486, 515, 580, 592, 611, 637, 692, 703, 720,
- 743, 788, 798, 813, 833, 84, 93, 103, 110, 125, 141, 154, 171, 250, 265, 286,
- 308, 349, 367, 392, 417, 464, 485, 514, 542, 591, 610, 636, 660, 702, 719,
- 742, 762, 797, 812, 832, 848, 98, 106, 115, 127, 143, 156, 169, 185, 264,
- 285, 307, 323, 366, 391, 416, 434, 484, 513, 541, 561, 609, 635, 659, 676,
- 718, 741, 761, 775, 811, 831, 847, 858, 117, 128, 136, 148, 160, 175, 188,
- 198, 284, 306, 322, 332, 390, 415, 433, 444, 512, 540, 560, 572, 634, 658,
- 675, 685, 740, 760, 774, 782, 830, 846, 857, 863, 135, 146, 152, 165, 241,
- 249, 263, 283, 338, 348, 365, 389, 451, 463, 483, 511, 579, 590, 608, 633,
- 691, 701, 717, 739, 787, 796, 810, 829, 867, 875, 887, 903, 153, 166, 174,
- 183, 248, 262, 282, 305, 347, 364, 388, 414, 462, 482, 510, 539, 589, 607,
- 632, 657, 700, 716, 738, 759, 795, 809, 828, 845, 874, 886, 902, 915, 176,
- 187, 195, 202, 261, 281, 304, 321, 363, 387, 413, 432, 481, 509, 538, 559,
- 606, 631, 656, 674, 715, 737, 758, 773, 808, 827, 844, 856, 885, 901, 914,
- 923, 192, 199, 206, 213, 280, 303, 320, 331, 386, 412, 431, 443, 508, 537,
- 558, 571, 630, 655, 673, 684, 736, 757, 772, 781, 826, 843, 855, 862, 900,
- 913, 922, 927, 240, 247, 260, 279, 337, 346, 362, 385, 450, 461, 480, 507,
- 578, 588, 605, 629, 690, 699, 714, 735, 786, 794, 807, 825, 866, 873, 884,
- 899, 930, 936, 945, 957, 246, 259, 278, 302, 345, 361, 384, 411, 460, 479,
- 506, 536, 587, 604, 628, 654, 698, 713, 734, 756, 793, 806, 824, 842, 872,
- 883, 898, 912, 935, 944, 956, 966, 258, 277, 301, 319, 360, 383, 410, 430,
- 478, 505, 535, 557, 603, 627, 653, 672, 712, 733, 755, 771, 805, 823, 841,
- 854, 882, 897, 911, 921, 943, 955, 965, 972, 276, 300, 318, 330, 382, 409,
- 429, 442, 504, 534, 556, 570, 626, 652, 671, 683, 732, 754, 770, 780, 822,
- 840, 853, 861, 896, 910, 920, 926, 954, 964, 971, 975, 336, 344, 359, 381,
- 449, 459, 477, 503, 577, 586, 602, 625, 689, 697, 711, 731, 785, 792, 804,
- 821, 865, 871, 881, 895, 929, 934, 942, 953, 977, 981, 987, 995, 343, 358,
- 380, 408, 458, 476, 502, 533, 585, 601, 624, 651, 696, 710, 730, 753, 791,
- 803, 820, 839, 870, 880, 894, 909, 933, 941, 952, 963, 980, 986, 994, 1001,
- 357, 379, 407, 428, 475, 501, 532, 555, 600, 623, 650, 670, 709, 729, 752,
- 769, 802, 819, 838, 852, 879, 893, 908, 919, 940, 951, 962, 970, 985, 993,
- 1000, 1005, 378, 406, 427, 441, 500, 531, 554, 569, 622, 649, 669, 682, 728,
- 751, 768, 779, 818, 837, 851, 860, 892, 907, 918, 925, 950, 961, 969, 974,
- 992, 999, 1004, 1007, 448, 457, 474, 499, 576, 584, 599, 621, 688, 695, 708,
- 727, 784, 790, 801, 817, 864, 869, 878, 891, 928, 932, 939, 949, 976, 979,
- 984, 991, 1008, 1010, 1013, 1017, 456, 473, 498, 530, 583, 598, 620, 648,
- 694, 707, 726, 750, 789, 800, 816, 836, 868, 877, 890, 906, 931, 938, 948,
- 960, 978, 983, 990, 998, 1009, 1012, 1016, 1020, 472, 497, 529, 553, 597,
- 619, 647, 668, 706, 725, 749, 767, 799, 815, 835, 850, 876, 889, 905, 917,
- 937, 947, 959, 968, 982, 989, 997, 1003, 1011, 1015, 1019, 1022, 496, 528,
- 552, 568, 618, 646, 667, 681, 724, 748, 766, 778, 814, 834, 849, 859, 888,
- 904, 916, 924, 946, 958, 967, 973, 988, 996, 1002, 1006, 1014, 1018, 1021,
- 1023,
-};
-
-const scan_order vp10_default_scan_orders[TX_SIZES] = {
- {default_scan_4x4, vp10_default_iscan_4x4, default_scan_4x4_neighbors},
- {default_scan_8x8, vp10_default_iscan_8x8, default_scan_8x8_neighbors},
- {default_scan_16x16, vp10_default_iscan_16x16, default_scan_16x16_neighbors},
- {default_scan_32x32, vp10_default_iscan_32x32, default_scan_32x32_neighbors},
-};
-
-const scan_order vp10_scan_orders[TX_SIZES][TX_TYPES] = {
- { // TX_4X4
- {default_scan_4x4, vp10_default_iscan_4x4, default_scan_4x4_neighbors},
- {row_scan_4x4, vp10_row_iscan_4x4, row_scan_4x4_neighbors},
- {col_scan_4x4, vp10_col_iscan_4x4, col_scan_4x4_neighbors},
- {default_scan_4x4, vp10_default_iscan_4x4, default_scan_4x4_neighbors}
- }, { // TX_8X8
- {default_scan_8x8, vp10_default_iscan_8x8, default_scan_8x8_neighbors},
- {row_scan_8x8, vp10_row_iscan_8x8, row_scan_8x8_neighbors},
- {col_scan_8x8, vp10_col_iscan_8x8, col_scan_8x8_neighbors},
- {default_scan_8x8, vp10_default_iscan_8x8, default_scan_8x8_neighbors}
- }, { // TX_16X16
- {default_scan_16x16, vp10_default_iscan_16x16, default_scan_16x16_neighbors},
- {row_scan_16x16, vp10_row_iscan_16x16, row_scan_16x16_neighbors},
- {col_scan_16x16, vp10_col_iscan_16x16, col_scan_16x16_neighbors},
- {default_scan_16x16, vp10_default_iscan_16x16, default_scan_16x16_neighbors}
- }, { // TX_32X32
- {default_scan_32x32, vp10_default_iscan_32x32, default_scan_32x32_neighbors},
- {default_scan_32x32, vp10_default_iscan_32x32, default_scan_32x32_neighbors},
- {default_scan_32x32, vp10_default_iscan_32x32, default_scan_32x32_neighbors},
- {default_scan_32x32, vp10_default_iscan_32x32, default_scan_32x32_neighbors},
- }
-};
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_SCAN_H_
-#define VP10_COMMON_SCAN_H_
-
-#include "vpx/vpx_integer.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/enums.h"
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MAX_NEIGHBORS 2
-
-typedef struct {
- const int16_t *scan;
- const int16_t *iscan;
- const int16_t *neighbors;
-} scan_order;
-
-extern const scan_order vp10_default_scan_orders[TX_SIZES];
-extern const scan_order vp10_scan_orders[TX_SIZES][TX_TYPES];
-
-static INLINE int get_coef_context(const int16_t *neighbors,
- const uint8_t *token_cache, int c) {
- return (1 + token_cache[neighbors[MAX_NEIGHBORS * c + 0]] +
- token_cache[neighbors[MAX_NEIGHBORS * c + 1]]) >> 1;
-}
-
-static INLINE const scan_order *get_scan(TX_SIZE tx_size, TX_TYPE tx_type) {
- return &vp10_scan_orders[tx_size][tx_type];
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_SCAN_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/loopfilter.h"
-#include "vp10/common/seg_common.h"
-#include "vp10/common/quant_common.h"
-
-static const int seg_feature_data_signed[SEG_LVL_MAX] = { 1, 1, 0, 0 };
-
-static const int seg_feature_data_max[SEG_LVL_MAX] = {
- MAXQ, MAX_LOOP_FILTER, 3, 0 };
-
-// These functions provide access to new segment level features.
-// Eventually these function may be "optimized out" but for the moment,
-// the coding mechanism is still subject to change so these provide a
-// convenient single point of change.
-
-void vp10_clearall_segfeatures(struct segmentation *seg) {
- vp10_zero(seg->feature_data);
- vp10_zero(seg->feature_mask);
-}
-
-void vp10_enable_segfeature(struct segmentation *seg, int segment_id,
- SEG_LVL_FEATURES feature_id) {
- seg->feature_mask[segment_id] |= 1 << feature_id;
-}
-
-int vp10_seg_feature_data_max(SEG_LVL_FEATURES feature_id) {
- return seg_feature_data_max[feature_id];
-}
-
-int vp10_is_segfeature_signed(SEG_LVL_FEATURES feature_id) {
- return seg_feature_data_signed[feature_id];
-}
-
-void vp10_set_segdata(struct segmentation *seg, int segment_id,
- SEG_LVL_FEATURES feature_id, int seg_data) {
- assert(seg_data <= seg_feature_data_max[feature_id]);
- if (seg_data < 0) {
- assert(seg_feature_data_signed[feature_id]);
- assert(-seg_data <= seg_feature_data_max[feature_id]);
- }
-
- seg->feature_data[segment_id][feature_id] = seg_data;
-}
-
-const vpx_tree_index vp10_segment_tree[TREE_SIZE(MAX_SEGMENTS)] = {
- 2, 4, 6, 8, 10, 12,
- 0, -1, -2, -3, -4, -5, -6, -7
-};
-
-
-// TBD? Functions to read and write segment data with range / validity checking
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_SEG_COMMON_H_
-#define VP10_COMMON_SEG_COMMON_H_
-
-#include "vpx_dsp/prob.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define SEGMENT_DELTADATA 0
-#define SEGMENT_ABSDATA 1
-
-#define MAX_SEGMENTS 8
-#define SEG_TREE_PROBS (MAX_SEGMENTS-1)
-
-#define PREDICTION_PROBS 3
-
-// Segment level features.
-typedef enum {
- SEG_LVL_ALT_Q = 0, // Use alternate Quantizer ....
- SEG_LVL_ALT_LF = 1, // Use alternate loop filter value...
- SEG_LVL_REF_FRAME = 2, // Optional Segment reference frame
- SEG_LVL_SKIP = 3, // Optional Segment (0,0) + skip mode
- SEG_LVL_MAX = 4 // Number of features supported
-} SEG_LVL_FEATURES;
-
-
-struct segmentation {
- uint8_t enabled;
- uint8_t update_map;
- uint8_t update_data;
- uint8_t abs_delta;
- uint8_t temporal_update;
-
- int16_t feature_data[MAX_SEGMENTS][SEG_LVL_MAX];
- unsigned int feature_mask[MAX_SEGMENTS];
-};
-
-struct segmentation_probs {
- vpx_prob tree_probs[SEG_TREE_PROBS];
- vpx_prob pred_probs[PREDICTION_PROBS];
-};
-
-static INLINE int segfeature_active(const struct segmentation *seg,
- int segment_id,
- SEG_LVL_FEATURES feature_id) {
- return seg->enabled &&
- (seg->feature_mask[segment_id] & (1 << feature_id));
-}
-
-void vp10_clearall_segfeatures(struct segmentation *seg);
-
-void vp10_enable_segfeature(struct segmentation *seg,
- int segment_id,
- SEG_LVL_FEATURES feature_id);
-
-int vp10_seg_feature_data_max(SEG_LVL_FEATURES feature_id);
-
-int vp10_is_segfeature_signed(SEG_LVL_FEATURES feature_id);
-
-void vp10_set_segdata(struct segmentation *seg,
- int segment_id,
- SEG_LVL_FEATURES feature_id,
- int seg_data);
-
-static INLINE int get_segdata(const struct segmentation *seg, int segment_id,
- SEG_LVL_FEATURES feature_id) {
- return seg->feature_data[segment_id][feature_id];
-}
-
-extern const vpx_tree_index vp10_segment_tree[TREE_SIZE(MAX_SEGMENTS)];
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_SEG_COMMON_H_
-
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <stdlib.h>
-
-#include "vp10/common/textblit.h"
-
-static const int font[] = {
- 0x0, 0x5C00, 0x8020, 0xAFABEA, 0xD7EC0, 0x1111111, 0x1855740, 0x18000,
- 0x45C0, 0x74400, 0x51140, 0x23880, 0xC4000, 0x21080, 0x80000, 0x111110,
- 0xE9D72E, 0x87E40, 0x12AD732, 0xAAD62A, 0x4F94C4, 0x4D6B7, 0x456AA,
- 0x3E8423, 0xAAD6AA, 0xAAD6A2, 0x2800, 0x2A00, 0x8A880, 0x52940, 0x22A20,
- 0x15422, 0x6AD62E, 0x1E4A53E, 0xAAD6BF, 0x8C62E, 0xE8C63F, 0x118D6BF,
- 0x1094BF, 0xCAC62E, 0x1F2109F, 0x118FE31, 0xF8C628, 0x8A89F, 0x108421F,
- 0x1F1105F, 0x1F4105F, 0xE8C62E, 0x2294BF, 0x164C62E, 0x12694BF, 0x8AD6A2,
- 0x10FC21, 0x1F8421F, 0x744107, 0xF8220F, 0x1151151, 0x117041, 0x119D731,
- 0x47E0, 0x1041041, 0xFC400, 0x10440, 0x1084210, 0x820
-};
-
-static void plot(int x, int y, unsigned char *image, int pitch) {
- image[x + y * pitch] ^= 255;
-}
-
-void vp10_blit_text(const char *msg, unsigned char *address, const int pitch) {
- int letter_bitmap;
- unsigned char *output_pos = address;
- int colpos = 0;
-
- while (msg[colpos] != 0) {
- char letter = msg[colpos];
- int fontcol, fontrow;
-
- if (letter <= 'Z' && letter >= ' ')
- letter_bitmap = font[letter - ' '];
- else if (letter <= 'z' && letter >= 'a')
- letter_bitmap = font[letter - 'a' + 'A' - ' '];
- else
- letter_bitmap = font[0];
-
- for (fontcol = 6; fontcol >= 0; fontcol--)
- for (fontrow = 0; fontrow < 5; fontrow++)
- output_pos[fontrow * pitch + fontcol] =
- ((letter_bitmap >> (fontcol * 5)) & (1 << fontrow) ? 255 : 0);
-
- output_pos += 7;
- colpos++;
- }
-}
-
-
-
-/* Bresenham line algorithm */
-void vp10_blit_line(int x0, int x1, int y0, int y1, unsigned char *image,
- int pitch) {
- int steep = abs(y1 - y0) > abs(x1 - x0);
- int deltax, deltay;
- int error, ystep, y, x;
-
- if (steep) {
- int t;
- t = x0;
- x0 = y0;
- y0 = t;
-
- t = x1;
- x1 = y1;
- y1 = t;
- }
-
- if (x0 > x1) {
- int t;
- t = x0;
- x0 = x1;
- x1 = t;
-
- t = y0;
- y0 = y1;
- y1 = t;
- }
-
- deltax = x1 - x0;
- deltay = abs(y1 - y0);
- error = deltax / 2;
-
- y = y0;
-
- if (y0 < y1)
- ystep = 1;
- else
- ystep = -1;
-
- if (steep) {
- for (x = x0; x <= x1; x++) {
- plot(y, x, image, pitch);
-
- error = error - deltay;
- if (error < 0) {
- y = y + ystep;
- error = error + deltax;
- }
- }
- } else {
- for (x = x0; x <= x1; x++) {
- plot(x, y, image, pitch);
-
- error = error - deltay;
- if (error < 0) {
- y = y + ystep;
- error = error + deltax;
- }
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_TEXTBLIT_H_
-#define VP10_COMMON_TEXTBLIT_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_blit_text(const char *msg, unsigned char *address, int pitch);
-
-void vp10_blit_line(int x0, int x1, int y0, int y1, unsigned char *image,
- int pitch);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_TEXTBLIT_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_config.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/thread_common.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/loopfilter.h"
-
-#if CONFIG_MULTITHREAD
-static INLINE void mutex_lock(pthread_mutex_t *const mutex) {
- const int kMaxTryLocks = 4000;
- int locked = 0;
- int i;
-
- for (i = 0; i < kMaxTryLocks; ++i) {
- if (!pthread_mutex_trylock(mutex)) {
- locked = 1;
- break;
- }
- }
-
- if (!locked)
- pthread_mutex_lock(mutex);
-}
-#endif // CONFIG_MULTITHREAD
-
-static INLINE void sync_read(VP9LfSync *const lf_sync, int r, int c) {
-#if CONFIG_MULTITHREAD
- const int nsync = lf_sync->sync_range;
-
- if (r && !(c & (nsync - 1))) {
- pthread_mutex_t *const mutex = &lf_sync->mutex_[r - 1];
- mutex_lock(mutex);
-
- while (c > lf_sync->cur_sb_col[r - 1] - nsync) {
- pthread_cond_wait(&lf_sync->cond_[r - 1], mutex);
- }
- pthread_mutex_unlock(mutex);
- }
-#else
- (void)lf_sync;
- (void)r;
- (void)c;
-#endif // CONFIG_MULTITHREAD
-}
-
-static INLINE void sync_write(VP9LfSync *const lf_sync, int r, int c,
- const int sb_cols) {
-#if CONFIG_MULTITHREAD
- const int nsync = lf_sync->sync_range;
- int cur;
- // Only signal when there are enough filtered SB for next row to run.
- int sig = 1;
-
- if (c < sb_cols - 1) {
- cur = c;
- if (c % nsync)
- sig = 0;
- } else {
- cur = sb_cols + nsync;
- }
-
- if (sig) {
- mutex_lock(&lf_sync->mutex_[r]);
-
- lf_sync->cur_sb_col[r] = cur;
-
- pthread_cond_signal(&lf_sync->cond_[r]);
- pthread_mutex_unlock(&lf_sync->mutex_[r]);
- }
-#else
- (void)lf_sync;
- (void)r;
- (void)c;
- (void)sb_cols;
-#endif // CONFIG_MULTITHREAD
-}
-
-// Implement row loopfiltering for each thread.
-static INLINE
-void thread_loop_filter_rows(const YV12_BUFFER_CONFIG *const frame_buffer,
- VP10_COMMON *const cm,
- struct macroblockd_plane planes[MAX_MB_PLANE],
- int start, int stop, int y_only,
- VP9LfSync *const lf_sync) {
- const int num_planes = y_only ? 1 : MAX_MB_PLANE;
- const int sb_cols = mi_cols_aligned_to_sb(cm->mi_cols) >> MI_BLOCK_SIZE_LOG2;
- int mi_row, mi_col;
- enum lf_path path;
- if (y_only)
- path = LF_PATH_444;
- else if (planes[1].subsampling_y == 1 && planes[1].subsampling_x == 1)
- path = LF_PATH_420;
- else if (planes[1].subsampling_y == 0 && planes[1].subsampling_x == 0)
- path = LF_PATH_444;
- else
- path = LF_PATH_SLOW;
-
- for (mi_row = start; mi_row < stop;
- mi_row += lf_sync->num_workers * MI_BLOCK_SIZE) {
- MODE_INFO **const mi = cm->mi_grid_visible + mi_row * cm->mi_stride;
-
- for (mi_col = 0; mi_col < cm->mi_cols; mi_col += MI_BLOCK_SIZE) {
- const int r = mi_row >> MI_BLOCK_SIZE_LOG2;
- const int c = mi_col >> MI_BLOCK_SIZE_LOG2;
- LOOP_FILTER_MASK lfm;
- int plane;
-
- sync_read(lf_sync, r, c);
-
- vp10_setup_dst_planes(planes, frame_buffer, mi_row, mi_col);
-
- // TODO(JBB): Make setup_mask work for non 420.
- vp10_setup_mask(cm, mi_row, mi_col, mi + mi_col, cm->mi_stride,
- &lfm);
-
- vp10_filter_block_plane_ss00(cm, &planes[0], mi_row, &lfm);
- for (plane = 1; plane < num_planes; ++plane) {
- switch (path) {
- case LF_PATH_420:
- vp10_filter_block_plane_ss11(cm, &planes[plane], mi_row, &lfm);
- break;
- case LF_PATH_444:
- vp10_filter_block_plane_ss00(cm, &planes[plane], mi_row, &lfm);
- break;
- case LF_PATH_SLOW:
- vp10_filter_block_plane_non420(cm, &planes[plane], mi + mi_col,
- mi_row, mi_col);
- break;
- }
- }
-
- sync_write(lf_sync, r, c, sb_cols);
- }
- }
-}
-
-// Row-based multi-threaded loopfilter hook
-static int loop_filter_row_worker(VP9LfSync *const lf_sync,
- LFWorkerData *const lf_data) {
- thread_loop_filter_rows(lf_data->frame_buffer, lf_data->cm, lf_data->planes,
- lf_data->start, lf_data->stop, lf_data->y_only,
- lf_sync);
- return 1;
-}
-
-static void loop_filter_rows_mt(YV12_BUFFER_CONFIG *frame,
- VP10_COMMON *cm,
- struct macroblockd_plane planes[MAX_MB_PLANE],
- int start, int stop, int y_only,
- VPxWorker *workers, int nworkers,
- VP9LfSync *lf_sync) {
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- // Number of superblock rows and cols
- const int sb_rows = mi_cols_aligned_to_sb(cm->mi_rows) >> MI_BLOCK_SIZE_LOG2;
- // Decoder may allocate more threads than number of tiles based on user's
- // input.
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int num_workers = VPXMIN(nworkers, tile_cols);
- int i;
-
- if (!lf_sync->sync_range || sb_rows != lf_sync->rows ||
- num_workers > lf_sync->num_workers) {
- vp10_loop_filter_dealloc(lf_sync);
- vp10_loop_filter_alloc(lf_sync, cm, sb_rows, cm->width, num_workers);
- }
-
- // Initialize cur_sb_col to -1 for all SB rows.
- memset(lf_sync->cur_sb_col, -1, sizeof(*lf_sync->cur_sb_col) * sb_rows);
-
- // Set up loopfilter thread data.
- // The decoder is capping num_workers because it has been observed that using
- // more threads on the loopfilter than there are cores will hurt performance
- // on Android. This is because the system will only schedule the tile decode
- // workers on cores equal to the number of tile columns. Then if the decoder
- // tries to use more threads for the loopfilter, it will hurt performance
- // because of contention. If the multithreading code changes in the future
- // then the number of workers used by the loopfilter should be revisited.
- for (i = 0; i < num_workers; ++i) {
- VPxWorker *const worker = &workers[i];
- LFWorkerData *const lf_data = &lf_sync->lfdata[i];
-
- worker->hook = (VPxWorkerHook)loop_filter_row_worker;
- worker->data1 = lf_sync;
- worker->data2 = lf_data;
-
- // Loopfilter data
- vp10_loop_filter_data_reset(lf_data, frame, cm, planes);
- lf_data->start = start + i * MI_BLOCK_SIZE;
- lf_data->stop = stop;
- lf_data->y_only = y_only;
-
- // Start loopfiltering
- if (i == num_workers - 1) {
- winterface->execute(worker);
- } else {
- winterface->launch(worker);
- }
- }
-
- // Wait till all rows are finished
- for (i = 0; i < num_workers; ++i) {
- winterface->sync(&workers[i]);
- }
-}
-
-void vp10_loop_filter_frame_mt(YV12_BUFFER_CONFIG *frame,
- VP10_COMMON *cm,
- struct macroblockd_plane planes[MAX_MB_PLANE],
- int frame_filter_level,
- int y_only, int partial_frame,
- VPxWorker *workers, int num_workers,
- VP9LfSync *lf_sync) {
- int start_mi_row, end_mi_row, mi_rows_to_filter;
-
- if (!frame_filter_level) return;
-
- start_mi_row = 0;
- mi_rows_to_filter = cm->mi_rows;
- if (partial_frame && cm->mi_rows > 8) {
- start_mi_row = cm->mi_rows >> 1;
- start_mi_row &= 0xfffffff8;
- mi_rows_to_filter = VPXMAX(cm->mi_rows / 8, 8);
- }
- end_mi_row = start_mi_row + mi_rows_to_filter;
- vp10_loop_filter_frame_init(cm, frame_filter_level);
-
- loop_filter_rows_mt(frame, cm, planes, start_mi_row, end_mi_row,
- y_only, workers, num_workers, lf_sync);
-}
-
-// Set up nsync by width.
-static INLINE int get_sync_range(int width) {
- // nsync numbers are picked by testing. For example, for 4k
- // video, using 4 gives best performance.
- if (width < 640)
- return 1;
- else if (width <= 1280)
- return 2;
- else if (width <= 4096)
- return 4;
- else
- return 8;
-}
-
-// Allocate memory for lf row synchronization
-void vp10_loop_filter_alloc(VP9LfSync *lf_sync, VP10_COMMON *cm, int rows,
- int width, int num_workers) {
- lf_sync->rows = rows;
-#if CONFIG_MULTITHREAD
- {
- int i;
-
- CHECK_MEM_ERROR(cm, lf_sync->mutex_,
- vpx_malloc(sizeof(*lf_sync->mutex_) * rows));
- if (lf_sync->mutex_) {
- for (i = 0; i < rows; ++i) {
- pthread_mutex_init(&lf_sync->mutex_[i], NULL);
- }
- }
-
- CHECK_MEM_ERROR(cm, lf_sync->cond_,
- vpx_malloc(sizeof(*lf_sync->cond_) * rows));
- if (lf_sync->cond_) {
- for (i = 0; i < rows; ++i) {
- pthread_cond_init(&lf_sync->cond_[i], NULL);
- }
- }
- }
-#endif // CONFIG_MULTITHREAD
-
- CHECK_MEM_ERROR(cm, lf_sync->lfdata,
- vpx_malloc(num_workers * sizeof(*lf_sync->lfdata)));
- lf_sync->num_workers = num_workers;
-
- CHECK_MEM_ERROR(cm, lf_sync->cur_sb_col,
- vpx_malloc(sizeof(*lf_sync->cur_sb_col) * rows));
-
- // Set up nsync.
- lf_sync->sync_range = get_sync_range(width);
-}
-
-// Deallocate lf synchronization related mutex and data
-void vp10_loop_filter_dealloc(VP9LfSync *lf_sync) {
- if (lf_sync != NULL) {
-#if CONFIG_MULTITHREAD
- int i;
-
- if (lf_sync->mutex_ != NULL) {
- for (i = 0; i < lf_sync->rows; ++i) {
- pthread_mutex_destroy(&lf_sync->mutex_[i]);
- }
- vpx_free(lf_sync->mutex_);
- }
- if (lf_sync->cond_ != NULL) {
- for (i = 0; i < lf_sync->rows; ++i) {
- pthread_cond_destroy(&lf_sync->cond_[i]);
- }
- vpx_free(lf_sync->cond_);
- }
-#endif // CONFIG_MULTITHREAD
- vpx_free(lf_sync->lfdata);
- vpx_free(lf_sync->cur_sb_col);
- // clear the structure as the source of this call may be a resize in which
- // case this call will be followed by an _alloc() which may fail.
- vp10_zero(*lf_sync);
- }
-}
-
-// Accumulate frame counts.
-void vp10_accumulate_frame_counts(VP10_COMMON *cm, FRAME_COUNTS *counts,
- int is_dec) {
- int i, j, k, l, m;
-
- for (i = 0; i < BLOCK_SIZE_GROUPS; i++)
- for (j = 0; j < INTRA_MODES; j++)
- cm->counts.y_mode[i][j] += counts->y_mode[i][j];
-
- for (i = 0; i < INTRA_MODES; i++)
- for (j = 0; j < INTRA_MODES; j++)
- cm->counts.uv_mode[i][j] += counts->uv_mode[i][j];
-
- for (i = 0; i < PARTITION_CONTEXTS; i++)
- for (j = 0; j < PARTITION_TYPES; j++)
- cm->counts.partition[i][j] += counts->partition[i][j];
-
- if (is_dec) {
- int n;
- for (i = 0; i < TX_SIZES; i++)
- for (j = 0; j < PLANE_TYPES; j++)
- for (k = 0; k < REF_TYPES; k++)
- for (l = 0; l < COEF_BANDS; l++)
- for (m = 0; m < COEFF_CONTEXTS; m++) {
- cm->counts.eob_branch[i][j][k][l][m] +=
- counts->eob_branch[i][j][k][l][m];
- for (n = 0; n < UNCONSTRAINED_NODES + 1; n++)
- cm->counts.coef[i][j][k][l][m][n] +=
- counts->coef[i][j][k][l][m][n];
- }
- } else {
- for (i = 0; i < TX_SIZES; i++)
- for (j = 0; j < PLANE_TYPES; j++)
- for (k = 0; k < REF_TYPES; k++)
- for (l = 0; l < COEF_BANDS; l++)
- for (m = 0; m < COEFF_CONTEXTS; m++)
- cm->counts.eob_branch[i][j][k][l][m] +=
- counts->eob_branch[i][j][k][l][m];
- // In the encoder, cm->counts.coef is only updated at frame
- // level, so not need to accumulate it here.
- // for (n = 0; n < UNCONSTRAINED_NODES + 1; n++)
- // cm->counts.coef[i][j][k][l][m][n] +=
- // counts->coef[i][j][k][l][m][n];
- }
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- for (j = 0; j < SWITCHABLE_FILTERS; j++)
- cm->counts.switchable_interp[i][j] += counts->switchable_interp[i][j];
-
- for (i = 0; i < INTER_MODE_CONTEXTS; i++)
- for (j = 0; j < INTER_MODES; j++)
- cm->counts.inter_mode[i][j] += counts->inter_mode[i][j];
-
- for (i = 0; i < INTRA_INTER_CONTEXTS; i++)
- for (j = 0; j < 2; j++)
- cm->counts.intra_inter[i][j] += counts->intra_inter[i][j];
-
- for (i = 0; i < COMP_INTER_CONTEXTS; i++)
- for (j = 0; j < 2; j++)
- cm->counts.comp_inter[i][j] += counts->comp_inter[i][j];
-
- for (i = 0; i < REF_CONTEXTS; i++)
- for (j = 0; j < 2; j++)
- for (k = 0; k < 2; k++)
- cm->counts.single_ref[i][j][k] += counts->single_ref[i][j][k];
-
- for (i = 0; i < REF_CONTEXTS; i++)
- for (j = 0; j < 2; j++)
- cm->counts.comp_ref[i][j] += counts->comp_ref[i][j];
-
- for (i = 0; i < TX_SIZE_CONTEXTS; i++) {
- for (j = 0; j < TX_SIZES; j++)
- cm->counts.tx.p32x32[i][j] += counts->tx.p32x32[i][j];
-
- for (j = 0; j < TX_SIZES - 1; j++)
- cm->counts.tx.p16x16[i][j] += counts->tx.p16x16[i][j];
-
- for (j = 0; j < TX_SIZES - 2; j++)
- cm->counts.tx.p8x8[i][j] += counts->tx.p8x8[i][j];
- }
-
- for (i = 0; i < TX_SIZES; i++)
- cm->counts.tx.tx_totals[i] += counts->tx.tx_totals[i];
-
- for (i = 0; i < SKIP_CONTEXTS; i++)
- for (j = 0; j < 2; j++)
- cm->counts.skip[i][j] += counts->skip[i][j];
-
- for (i = 0; i < MV_JOINTS; i++)
- cm->counts.mv.joints[i] += counts->mv.joints[i];
-
- for (k = 0; k < 2; k++) {
- nmv_component_counts *comps = &cm->counts.mv.comps[k];
- nmv_component_counts *comps_t = &counts->mv.comps[k];
-
- for (i = 0; i < 2; i++) {
- comps->sign[i] += comps_t->sign[i];
- comps->class0_hp[i] += comps_t->class0_hp[i];
- comps->hp[i] += comps_t->hp[i];
- }
-
- for (i = 0; i < MV_CLASSES; i++)
- comps->classes[i] += comps_t->classes[i];
-
- for (i = 0; i < CLASS0_SIZE; i++) {
- comps->class0[i] += comps_t->class0[i];
- for (j = 0; j < MV_FP_SIZE; j++)
- comps->class0_fp[i][j] += comps_t->class0_fp[i][j];
- }
-
- for (i = 0; i < MV_OFFSET_BITS; i++)
- for (j = 0; j < 2; j++)
- comps->bits[i][j] += comps_t->bits[i][j];
-
- for (i = 0; i < MV_FP_SIZE; i++)
- comps->fp[i] += comps_t->fp[i];
- }
-
-#if CONFIG_MISC_FIXES
- for (i = 0; i < PREDICTION_PROBS; i++)
- for (j = 0; j < 2; j++)
- cm->counts.seg.pred[i][j] += counts->seg.pred[i][j];
-
- for (i = 0; i < MAX_SEGMENTS; i++) {
- cm->counts.seg.tree_total[i] += counts->seg.tree_total[i];
- cm->counts.seg.tree_mispred[i] += counts->seg.tree_mispred[i];
- }
-#endif
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_LOOPFILTER_THREAD_H_
-#define VP10_COMMON_LOOPFILTER_THREAD_H_
-#include "./vpx_config.h"
-#include "vp10/common/loopfilter.h"
-#include "vpx_util/vpx_thread.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10Common;
-struct FRAME_COUNTS;
-
-// Loopfilter row synchronization
-typedef struct VP9LfSyncData {
-#if CONFIG_MULTITHREAD
- pthread_mutex_t *mutex_;
- pthread_cond_t *cond_;
-#endif
- // Allocate memory to store the loop-filtered superblock index in each row.
- int *cur_sb_col;
- // The optimal sync_range for different resolution and platform should be
- // determined by testing. Currently, it is chosen to be a power-of-2 number.
- int sync_range;
- int rows;
-
- // Row-based parallel loopfilter data
- LFWorkerData *lfdata;
- int num_workers;
-} VP9LfSync;
-
-// Allocate memory for loopfilter row synchronization.
-void vp10_loop_filter_alloc(VP9LfSync *lf_sync, struct VP10Common *cm, int rows,
- int width, int num_workers);
-
-// Deallocate loopfilter synchronization related mutex and data.
-void vp10_loop_filter_dealloc(VP9LfSync *lf_sync);
-
-// Multi-threaded loopfilter that uses the tile threads.
-void vp10_loop_filter_frame_mt(YV12_BUFFER_CONFIG *frame,
- struct VP10Common *cm,
- struct macroblockd_plane planes[MAX_MB_PLANE],
- int frame_filter_level,
- int y_only, int partial_frame,
- VPxWorker *workers, int num_workers,
- VP9LfSync *lf_sync);
-
-void vp10_accumulate_frame_counts(struct VP10Common *cm,
- struct FRAME_COUNTS *counts, int is_dec);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_LOOPFILTER_THREAD_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/tile_common.h"
-#include "vp10/common/onyxc_int.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-
-#define MIN_TILE_WIDTH_B64 4
-#define MAX_TILE_WIDTH_B64 64
-
-static int get_tile_offset(int idx, int mis, int log2) {
- const int sb_cols = mi_cols_aligned_to_sb(mis) >> MI_BLOCK_SIZE_LOG2;
- const int offset = ((idx * sb_cols) >> log2) << MI_BLOCK_SIZE_LOG2;
- return VPXMIN(offset, mis);
-}
-
-void vp10_tile_set_row(TileInfo *tile, const VP10_COMMON *cm, int row) {
- tile->mi_row_start = get_tile_offset(row, cm->mi_rows, cm->log2_tile_rows);
- tile->mi_row_end = get_tile_offset(row + 1, cm->mi_rows, cm->log2_tile_rows);
-}
-
-void vp10_tile_set_col(TileInfo *tile, const VP10_COMMON *cm, int col) {
- tile->mi_col_start = get_tile_offset(col, cm->mi_cols, cm->log2_tile_cols);
- tile->mi_col_end = get_tile_offset(col + 1, cm->mi_cols, cm->log2_tile_cols);
-}
-
-void vp10_tile_init(TileInfo *tile, const VP10_COMMON *cm, int row, int col) {
- vp10_tile_set_row(tile, cm, row);
- vp10_tile_set_col(tile, cm, col);
-}
-
-static int get_min_log2_tile_cols(const int sb64_cols) {
- int min_log2 = 0;
- while ((MAX_TILE_WIDTH_B64 << min_log2) < sb64_cols)
- ++min_log2;
- return min_log2;
-}
-
-static int get_max_log2_tile_cols(const int sb64_cols) {
- int max_log2 = 1;
- while ((sb64_cols >> max_log2) >= MIN_TILE_WIDTH_B64)
- ++max_log2;
- return max_log2 - 1;
-}
-
-void vp10_get_tile_n_bits(int mi_cols,
- int *min_log2_tile_cols, int *max_log2_tile_cols) {
- const int sb64_cols = mi_cols_aligned_to_sb(mi_cols) >> MI_BLOCK_SIZE_LOG2;
- *min_log2_tile_cols = get_min_log2_tile_cols(sb64_cols);
- *max_log2_tile_cols = get_max_log2_tile_cols(sb64_cols);
- assert(*min_log2_tile_cols <= *max_log2_tile_cols);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_TILE_COMMON_H_
-#define VP10_COMMON_TILE_COMMON_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10Common;
-
-typedef struct TileInfo {
- int mi_row_start, mi_row_end;
- int mi_col_start, mi_col_end;
-} TileInfo;
-
-// initializes 'tile->mi_(row|col)_(start|end)' for (row, col) based on
-// 'cm->log2_tile_(rows|cols)' & 'cm->mi_(rows|cols)'
-void vp10_tile_init(TileInfo *tile, const struct VP10Common *cm,
- int row, int col);
-
-void vp10_tile_set_row(TileInfo *tile, const struct VP10Common *cm, int row);
-void vp10_tile_set_col(TileInfo *tile, const struct VP10Common *cm, int col);
-
-void vp10_get_tile_n_bits(int mi_cols,
- int *min_log2_tile_cols, int *max_log2_tile_cols);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_COMMON_TILE_COMMON_H_
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/common/vp10_fwd_txfm.h"
-
-void vp10_fdct4x4_c(const int16_t *input, tran_low_t *output, int stride) {
- // The 2D transform is done with two passes which are actually pretty
- // similar. In the first one, we transform the columns and transpose
- // the results. In the second one, we transform the rows. To achieve that,
- // as the first pass results are transposed, we transpose the columns (that
- // is the transposed rows) and transpose the results (so that it goes back
- // in normal/row positions).
- int pass;
- // We need an intermediate buffer between passes.
- tran_low_t intermediate[4 * 4];
- const int16_t *in_pass0 = input;
- const tran_low_t *in = NULL;
- tran_low_t *out = intermediate;
- // Do the two transform/transpose passes
- for (pass = 0; pass < 2; ++pass) {
- tran_high_t input[4]; // canbe16
- tran_high_t step[4]; // canbe16
- tran_high_t temp1, temp2; // needs32
- int i;
- for (i = 0; i < 4; ++i) {
- // Load inputs.
- if (0 == pass) {
- input[0] = in_pass0[0 * stride] * 16;
- input[1] = in_pass0[1 * stride] * 16;
- input[2] = in_pass0[2 * stride] * 16;
- input[3] = in_pass0[3 * stride] * 16;
- if (i == 0 && input[0]) {
- input[0] += 1;
- }
- } else {
- input[0] = in[0 * 4];
- input[1] = in[1 * 4];
- input[2] = in[2 * 4];
- input[3] = in[3 * 4];
- }
- // Transform.
- step[0] = input[0] + input[3];
- step[1] = input[1] + input[2];
- step[2] = input[1] - input[2];
- step[3] = input[0] - input[3];
- temp1 = (step[0] + step[1]) * cospi_16_64;
- temp2 = (step[0] - step[1]) * cospi_16_64;
- out[0] = (tran_low_t)fdct_round_shift(temp1);
- out[2] = (tran_low_t)fdct_round_shift(temp2);
- temp1 = step[2] * cospi_24_64 + step[3] * cospi_8_64;
- temp2 = -step[2] * cospi_8_64 + step[3] * cospi_24_64;
- out[1] = (tran_low_t)fdct_round_shift(temp1);
- out[3] = (tran_low_t)fdct_round_shift(temp2);
- // Do next column (which is a transposed row in second/horizontal pass)
- in_pass0++;
- in++;
- out += 4;
- }
- // Setup in/out for next pass.
- in = intermediate;
- out = output;
- }
-
- {
- int i, j;
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- output[j + i * 4] = (output[j + i * 4] + 1) >> 2;
- }
- }
-}
-
-void vp10_fdct4x4_1_c(const int16_t *input, tran_low_t *output, int stride) {
- int r, c;
- tran_low_t sum = 0;
- for (r = 0; r < 4; ++r)
- for (c = 0; c < 4; ++c)
- sum += input[r * stride + c];
-
- output[0] = sum << 1;
- output[1] = 0;
-}
-
-void vp10_fdct8x8_c(const int16_t *input,
- tran_low_t *final_output, int stride) {
- int i, j;
- tran_low_t intermediate[64];
- int pass;
- tran_low_t *output = intermediate;
- const tran_low_t *in = NULL;
-
- // Transform columns
- for (pass = 0; pass < 2; ++pass) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7; // canbe16
- tran_high_t t0, t1, t2, t3; // needs32
- tran_high_t x0, x1, x2, x3; // canbe16
-
- int i;
- for (i = 0; i < 8; i++) {
- // stage 1
- if (pass == 0) {
- s0 = (input[0 * stride] + input[7 * stride]) * 4;
- s1 = (input[1 * stride] + input[6 * stride]) * 4;
- s2 = (input[2 * stride] + input[5 * stride]) * 4;
- s3 = (input[3 * stride] + input[4 * stride]) * 4;
- s4 = (input[3 * stride] - input[4 * stride]) * 4;
- s5 = (input[2 * stride] - input[5 * stride]) * 4;
- s6 = (input[1 * stride] - input[6 * stride]) * 4;
- s7 = (input[0 * stride] - input[7 * stride]) * 4;
- ++input;
- } else {
- s0 = in[0 * 8] + in[7 * 8];
- s1 = in[1 * 8] + in[6 * 8];
- s2 = in[2 * 8] + in[5 * 8];
- s3 = in[3 * 8] + in[4 * 8];
- s4 = in[3 * 8] - in[4 * 8];
- s5 = in[2 * 8] - in[5 * 8];
- s6 = in[1 * 8] - in[6 * 8];
- s7 = in[0 * 8] - in[7 * 8];
- ++in;
- }
-
- // fdct4(step, step);
- x0 = s0 + s3;
- x1 = s1 + s2;
- x2 = s1 - s2;
- x3 = s0 - s3;
- t0 = (x0 + x1) * cospi_16_64;
- t1 = (x0 - x1) * cospi_16_64;
- t2 = x2 * cospi_24_64 + x3 * cospi_8_64;
- t3 = -x2 * cospi_8_64 + x3 * cospi_24_64;
- output[0] = (tran_low_t)fdct_round_shift(t0);
- output[2] = (tran_low_t)fdct_round_shift(t2);
- output[4] = (tran_low_t)fdct_round_shift(t1);
- output[6] = (tran_low_t)fdct_round_shift(t3);
-
- // Stage 2
- t0 = (s6 - s5) * cospi_16_64;
- t1 = (s6 + s5) * cospi_16_64;
- t2 = fdct_round_shift(t0);
- t3 = fdct_round_shift(t1);
-
- // Stage 3
- x0 = s4 + t2;
- x1 = s4 - t2;
- x2 = s7 - t3;
- x3 = s7 + t3;
-
- // Stage 4
- t0 = x0 * cospi_28_64 + x3 * cospi_4_64;
- t1 = x1 * cospi_12_64 + x2 * cospi_20_64;
- t2 = x2 * cospi_12_64 + x1 * -cospi_20_64;
- t3 = x3 * cospi_28_64 + x0 * -cospi_4_64;
- output[1] = (tran_low_t)fdct_round_shift(t0);
- output[3] = (tran_low_t)fdct_round_shift(t2);
- output[5] = (tran_low_t)fdct_round_shift(t1);
- output[7] = (tran_low_t)fdct_round_shift(t3);
- output += 8;
- }
- in = intermediate;
- output = final_output;
- }
-
- // Rows
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- final_output[j + i * 8] /= 2;
- }
-}
-
-void vp10_fdct8x8_1_c(const int16_t *input, tran_low_t *output, int stride) {
- int r, c;
- tran_low_t sum = 0;
- for (r = 0; r < 8; ++r)
- for (c = 0; c < 8; ++c)
- sum += input[r * stride + c];
-
- output[0] = sum;
- output[1] = 0;
-}
-
-void vp10_fdct16x16_c(const int16_t *input, tran_low_t *output, int stride) {
- // The 2D transform is done with two passes which are actually pretty
- // similar. In the first one, we transform the columns and transpose
- // the results. In the second one, we transform the rows. To achieve that,
- // as the first pass results are transposed, we transpose the columns (that
- // is the transposed rows) and transpose the results (so that it goes back
- // in normal/row positions).
- int pass;
- // We need an intermediate buffer between passes.
- tran_low_t intermediate[256];
- const int16_t *in_pass0 = input;
- const tran_low_t *in = NULL;
- tran_low_t *out = intermediate;
- // Do the two transform/transpose passes
- for (pass = 0; pass < 2; ++pass) {
- tran_high_t step1[8]; // canbe16
- tran_high_t step2[8]; // canbe16
- tran_high_t step3[8]; // canbe16
- tran_high_t input[8]; // canbe16
- tran_high_t temp1, temp2; // needs32
- int i;
- for (i = 0; i < 16; i++) {
- if (0 == pass) {
- // Calculate input for the first 8 results.
- input[0] = (in_pass0[0 * stride] + in_pass0[15 * stride]) * 4;
- input[1] = (in_pass0[1 * stride] + in_pass0[14 * stride]) * 4;
- input[2] = (in_pass0[2 * stride] + in_pass0[13 * stride]) * 4;
- input[3] = (in_pass0[3 * stride] + in_pass0[12 * stride]) * 4;
- input[4] = (in_pass0[4 * stride] + in_pass0[11 * stride]) * 4;
- input[5] = (in_pass0[5 * stride] + in_pass0[10 * stride]) * 4;
- input[6] = (in_pass0[6 * stride] + in_pass0[ 9 * stride]) * 4;
- input[7] = (in_pass0[7 * stride] + in_pass0[ 8 * stride]) * 4;
- // Calculate input for the next 8 results.
- step1[0] = (in_pass0[7 * stride] - in_pass0[ 8 * stride]) * 4;
- step1[1] = (in_pass0[6 * stride] - in_pass0[ 9 * stride]) * 4;
- step1[2] = (in_pass0[5 * stride] - in_pass0[10 * stride]) * 4;
- step1[3] = (in_pass0[4 * stride] - in_pass0[11 * stride]) * 4;
- step1[4] = (in_pass0[3 * stride] - in_pass0[12 * stride]) * 4;
- step1[5] = (in_pass0[2 * stride] - in_pass0[13 * stride]) * 4;
- step1[6] = (in_pass0[1 * stride] - in_pass0[14 * stride]) * 4;
- step1[7] = (in_pass0[0 * stride] - in_pass0[15 * stride]) * 4;
- } else {
- // Calculate input for the first 8 results.
- input[0] = ((in[0 * 16] + 1) >> 2) + ((in[15 * 16] + 1) >> 2);
- input[1] = ((in[1 * 16] + 1) >> 2) + ((in[14 * 16] + 1) >> 2);
- input[2] = ((in[2 * 16] + 1) >> 2) + ((in[13 * 16] + 1) >> 2);
- input[3] = ((in[3 * 16] + 1) >> 2) + ((in[12 * 16] + 1) >> 2);
- input[4] = ((in[4 * 16] + 1) >> 2) + ((in[11 * 16] + 1) >> 2);
- input[5] = ((in[5 * 16] + 1) >> 2) + ((in[10 * 16] + 1) >> 2);
- input[6] = ((in[6 * 16] + 1) >> 2) + ((in[ 9 * 16] + 1) >> 2);
- input[7] = ((in[7 * 16] + 1) >> 2) + ((in[ 8 * 16] + 1) >> 2);
- // Calculate input for the next 8 results.
- step1[0] = ((in[7 * 16] + 1) >> 2) - ((in[ 8 * 16] + 1) >> 2);
- step1[1] = ((in[6 * 16] + 1) >> 2) - ((in[ 9 * 16] + 1) >> 2);
- step1[2] = ((in[5 * 16] + 1) >> 2) - ((in[10 * 16] + 1) >> 2);
- step1[3] = ((in[4 * 16] + 1) >> 2) - ((in[11 * 16] + 1) >> 2);
- step1[4] = ((in[3 * 16] + 1) >> 2) - ((in[12 * 16] + 1) >> 2);
- step1[5] = ((in[2 * 16] + 1) >> 2) - ((in[13 * 16] + 1) >> 2);
- step1[6] = ((in[1 * 16] + 1) >> 2) - ((in[14 * 16] + 1) >> 2);
- step1[7] = ((in[0 * 16] + 1) >> 2) - ((in[15 * 16] + 1) >> 2);
- }
- // Work on the first eight values; fdct8(input, even_results);
- {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7; // canbe16
- tran_high_t t0, t1, t2, t3; // needs32
- tran_high_t x0, x1, x2, x3; // canbe16
-
- // stage 1
- s0 = input[0] + input[7];
- s1 = input[1] + input[6];
- s2 = input[2] + input[5];
- s3 = input[3] + input[4];
- s4 = input[3] - input[4];
- s5 = input[2] - input[5];
- s6 = input[1] - input[6];
- s7 = input[0] - input[7];
-
- // fdct4(step, step);
- x0 = s0 + s3;
- x1 = s1 + s2;
- x2 = s1 - s2;
- x3 = s0 - s3;
- t0 = (x0 + x1) * cospi_16_64;
- t1 = (x0 - x1) * cospi_16_64;
- t2 = x3 * cospi_8_64 + x2 * cospi_24_64;
- t3 = x3 * cospi_24_64 - x2 * cospi_8_64;
- out[0] = (tran_low_t)fdct_round_shift(t0);
- out[4] = (tran_low_t)fdct_round_shift(t2);
- out[8] = (tran_low_t)fdct_round_shift(t1);
- out[12] = (tran_low_t)fdct_round_shift(t3);
-
- // Stage 2
- t0 = (s6 - s5) * cospi_16_64;
- t1 = (s6 + s5) * cospi_16_64;
- t2 = fdct_round_shift(t0);
- t3 = fdct_round_shift(t1);
-
- // Stage 3
- x0 = s4 + t2;
- x1 = s4 - t2;
- x2 = s7 - t3;
- x3 = s7 + t3;
-
- // Stage 4
- t0 = x0 * cospi_28_64 + x3 * cospi_4_64;
- t1 = x1 * cospi_12_64 + x2 * cospi_20_64;
- t2 = x2 * cospi_12_64 + x1 * -cospi_20_64;
- t3 = x3 * cospi_28_64 + x0 * -cospi_4_64;
- out[2] = (tran_low_t)fdct_round_shift(t0);
- out[6] = (tran_low_t)fdct_round_shift(t2);
- out[10] = (tran_low_t)fdct_round_shift(t1);
- out[14] = (tran_low_t)fdct_round_shift(t3);
- }
- // Work on the next eight values; step1 -> odd_results
- {
- // step 2
- temp1 = (step1[5] - step1[2]) * cospi_16_64;
- temp2 = (step1[4] - step1[3]) * cospi_16_64;
- step2[2] = fdct_round_shift(temp1);
- step2[3] = fdct_round_shift(temp2);
- temp1 = (step1[4] + step1[3]) * cospi_16_64;
- temp2 = (step1[5] + step1[2]) * cospi_16_64;
- step2[4] = fdct_round_shift(temp1);
- step2[5] = fdct_round_shift(temp2);
- // step 3
- step3[0] = step1[0] + step2[3];
- step3[1] = step1[1] + step2[2];
- step3[2] = step1[1] - step2[2];
- step3[3] = step1[0] - step2[3];
- step3[4] = step1[7] - step2[4];
- step3[5] = step1[6] - step2[5];
- step3[6] = step1[6] + step2[5];
- step3[7] = step1[7] + step2[4];
- // step 4
- temp1 = step3[1] * -cospi_8_64 + step3[6] * cospi_24_64;
- temp2 = step3[2] * cospi_24_64 + step3[5] * cospi_8_64;
- step2[1] = fdct_round_shift(temp1);
- step2[2] = fdct_round_shift(temp2);
- temp1 = step3[2] * cospi_8_64 - step3[5] * cospi_24_64;
- temp2 = step3[1] * cospi_24_64 + step3[6] * cospi_8_64;
- step2[5] = fdct_round_shift(temp1);
- step2[6] = fdct_round_shift(temp2);
- // step 5
- step1[0] = step3[0] + step2[1];
- step1[1] = step3[0] - step2[1];
- step1[2] = step3[3] + step2[2];
- step1[3] = step3[3] - step2[2];
- step1[4] = step3[4] - step2[5];
- step1[5] = step3[4] + step2[5];
- step1[6] = step3[7] - step2[6];
- step1[7] = step3[7] + step2[6];
- // step 6
- temp1 = step1[0] * cospi_30_64 + step1[7] * cospi_2_64;
- temp2 = step1[1] * cospi_14_64 + step1[6] * cospi_18_64;
- out[1] = (tran_low_t)fdct_round_shift(temp1);
- out[9] = (tran_low_t)fdct_round_shift(temp2);
- temp1 = step1[2] * cospi_22_64 + step1[5] * cospi_10_64;
- temp2 = step1[3] * cospi_6_64 + step1[4] * cospi_26_64;
- out[5] = (tran_low_t)fdct_round_shift(temp1);
- out[13] = (tran_low_t)fdct_round_shift(temp2);
- temp1 = step1[3] * -cospi_26_64 + step1[4] * cospi_6_64;
- temp2 = step1[2] * -cospi_10_64 + step1[5] * cospi_22_64;
- out[3] = (tran_low_t)fdct_round_shift(temp1);
- out[11] = (tran_low_t)fdct_round_shift(temp2);
- temp1 = step1[1] * -cospi_18_64 + step1[6] * cospi_14_64;
- temp2 = step1[0] * -cospi_2_64 + step1[7] * cospi_30_64;
- out[7] = (tran_low_t)fdct_round_shift(temp1);
- out[15] = (tran_low_t)fdct_round_shift(temp2);
- }
- // Do next column (which is a transposed row in second/horizontal pass)
- in++;
- in_pass0++;
- out += 16;
- }
- // Setup in/out for next pass.
- in = intermediate;
- out = output;
- }
-}
-
-void vp10_fdct16x16_1_c(const int16_t *input, tran_low_t *output, int stride) {
- int r, c;
- tran_low_t sum = 0;
- for (r = 0; r < 16; ++r)
- for (c = 0; c < 16; ++c)
- sum += input[r * stride + c];
-
- output[0] = sum >> 1;
- output[1] = 0;
-}
-
-static INLINE tran_high_t dct_32_round(tran_high_t input) {
- tran_high_t rv = ROUND_POWER_OF_TWO(input, DCT_CONST_BITS);
- // TODO(debargha, peter.derivaz): Find new bounds for this assert,
- // and make the bounds consts.
- // assert(-131072 <= rv && rv <= 131071);
- return rv;
-}
-
-static INLINE tran_high_t half_round_shift(tran_high_t input) {
- tran_high_t rv = (input + 1 + (input < 0)) >> 2;
- return rv;
-}
-
-void vp10_fdct32(const tran_high_t *input, tran_high_t *output, int round) {
- tran_high_t step[32];
- // Stage 1
- step[0] = input[0] + input[(32 - 1)];
- step[1] = input[1] + input[(32 - 2)];
- step[2] = input[2] + input[(32 - 3)];
- step[3] = input[3] + input[(32 - 4)];
- step[4] = input[4] + input[(32 - 5)];
- step[5] = input[5] + input[(32 - 6)];
- step[6] = input[6] + input[(32 - 7)];
- step[7] = input[7] + input[(32 - 8)];
- step[8] = input[8] + input[(32 - 9)];
- step[9] = input[9] + input[(32 - 10)];
- step[10] = input[10] + input[(32 - 11)];
- step[11] = input[11] + input[(32 - 12)];
- step[12] = input[12] + input[(32 - 13)];
- step[13] = input[13] + input[(32 - 14)];
- step[14] = input[14] + input[(32 - 15)];
- step[15] = input[15] + input[(32 - 16)];
- step[16] = -input[16] + input[(32 - 17)];
- step[17] = -input[17] + input[(32 - 18)];
- step[18] = -input[18] + input[(32 - 19)];
- step[19] = -input[19] + input[(32 - 20)];
- step[20] = -input[20] + input[(32 - 21)];
- step[21] = -input[21] + input[(32 - 22)];
- step[22] = -input[22] + input[(32 - 23)];
- step[23] = -input[23] + input[(32 - 24)];
- step[24] = -input[24] + input[(32 - 25)];
- step[25] = -input[25] + input[(32 - 26)];
- step[26] = -input[26] + input[(32 - 27)];
- step[27] = -input[27] + input[(32 - 28)];
- step[28] = -input[28] + input[(32 - 29)];
- step[29] = -input[29] + input[(32 - 30)];
- step[30] = -input[30] + input[(32 - 31)];
- step[31] = -input[31] + input[(32 - 32)];
-
- // Stage 2
- output[0] = step[0] + step[16 - 1];
- output[1] = step[1] + step[16 - 2];
- output[2] = step[2] + step[16 - 3];
- output[3] = step[3] + step[16 - 4];
- output[4] = step[4] + step[16 - 5];
- output[5] = step[5] + step[16 - 6];
- output[6] = step[6] + step[16 - 7];
- output[7] = step[7] + step[16 - 8];
- output[8] = -step[8] + step[16 - 9];
- output[9] = -step[9] + step[16 - 10];
- output[10] = -step[10] + step[16 - 11];
- output[11] = -step[11] + step[16 - 12];
- output[12] = -step[12] + step[16 - 13];
- output[13] = -step[13] + step[16 - 14];
- output[14] = -step[14] + step[16 - 15];
- output[15] = -step[15] + step[16 - 16];
-
- output[16] = step[16];
- output[17] = step[17];
- output[18] = step[18];
- output[19] = step[19];
-
- output[20] = dct_32_round((-step[20] + step[27]) * cospi_16_64);
- output[21] = dct_32_round((-step[21] + step[26]) * cospi_16_64);
- output[22] = dct_32_round((-step[22] + step[25]) * cospi_16_64);
- output[23] = dct_32_round((-step[23] + step[24]) * cospi_16_64);
-
- output[24] = dct_32_round((step[24] + step[23]) * cospi_16_64);
- output[25] = dct_32_round((step[25] + step[22]) * cospi_16_64);
- output[26] = dct_32_round((step[26] + step[21]) * cospi_16_64);
- output[27] = dct_32_round((step[27] + step[20]) * cospi_16_64);
-
- output[28] = step[28];
- output[29] = step[29];
- output[30] = step[30];
- output[31] = step[31];
-
- // dump the magnitude by 4, hence the intermediate values are within
- // the range of 16 bits.
- if (round) {
- output[0] = half_round_shift(output[0]);
- output[1] = half_round_shift(output[1]);
- output[2] = half_round_shift(output[2]);
- output[3] = half_round_shift(output[3]);
- output[4] = half_round_shift(output[4]);
- output[5] = half_round_shift(output[5]);
- output[6] = half_round_shift(output[6]);
- output[7] = half_round_shift(output[7]);
- output[8] = half_round_shift(output[8]);
- output[9] = half_round_shift(output[9]);
- output[10] = half_round_shift(output[10]);
- output[11] = half_round_shift(output[11]);
- output[12] = half_round_shift(output[12]);
- output[13] = half_round_shift(output[13]);
- output[14] = half_round_shift(output[14]);
- output[15] = half_round_shift(output[15]);
-
- output[16] = half_round_shift(output[16]);
- output[17] = half_round_shift(output[17]);
- output[18] = half_round_shift(output[18]);
- output[19] = half_round_shift(output[19]);
- output[20] = half_round_shift(output[20]);
- output[21] = half_round_shift(output[21]);
- output[22] = half_round_shift(output[22]);
- output[23] = half_round_shift(output[23]);
- output[24] = half_round_shift(output[24]);
- output[25] = half_round_shift(output[25]);
- output[26] = half_round_shift(output[26]);
- output[27] = half_round_shift(output[27]);
- output[28] = half_round_shift(output[28]);
- output[29] = half_round_shift(output[29]);
- output[30] = half_round_shift(output[30]);
- output[31] = half_round_shift(output[31]);
- }
-
- // Stage 3
- step[0] = output[0] + output[(8 - 1)];
- step[1] = output[1] + output[(8 - 2)];
- step[2] = output[2] + output[(8 - 3)];
- step[3] = output[3] + output[(8 - 4)];
- step[4] = -output[4] + output[(8 - 5)];
- step[5] = -output[5] + output[(8 - 6)];
- step[6] = -output[6] + output[(8 - 7)];
- step[7] = -output[7] + output[(8 - 8)];
- step[8] = output[8];
- step[9] = output[9];
- step[10] = dct_32_round((-output[10] + output[13]) * cospi_16_64);
- step[11] = dct_32_round((-output[11] + output[12]) * cospi_16_64);
- step[12] = dct_32_round((output[12] + output[11]) * cospi_16_64);
- step[13] = dct_32_round((output[13] + output[10]) * cospi_16_64);
- step[14] = output[14];
- step[15] = output[15];
-
- step[16] = output[16] + output[23];
- step[17] = output[17] + output[22];
- step[18] = output[18] + output[21];
- step[19] = output[19] + output[20];
- step[20] = -output[20] + output[19];
- step[21] = -output[21] + output[18];
- step[22] = -output[22] + output[17];
- step[23] = -output[23] + output[16];
- step[24] = -output[24] + output[31];
- step[25] = -output[25] + output[30];
- step[26] = -output[26] + output[29];
- step[27] = -output[27] + output[28];
- step[28] = output[28] + output[27];
- step[29] = output[29] + output[26];
- step[30] = output[30] + output[25];
- step[31] = output[31] + output[24];
-
- // Stage 4
- output[0] = step[0] + step[3];
- output[1] = step[1] + step[2];
- output[2] = -step[2] + step[1];
- output[3] = -step[3] + step[0];
- output[4] = step[4];
- output[5] = dct_32_round((-step[5] + step[6]) * cospi_16_64);
- output[6] = dct_32_round((step[6] + step[5]) * cospi_16_64);
- output[7] = step[7];
- output[8] = step[8] + step[11];
- output[9] = step[9] + step[10];
- output[10] = -step[10] + step[9];
- output[11] = -step[11] + step[8];
- output[12] = -step[12] + step[15];
- output[13] = -step[13] + step[14];
- output[14] = step[14] + step[13];
- output[15] = step[15] + step[12];
-
- output[16] = step[16];
- output[17] = step[17];
- output[18] = dct_32_round(step[18] * -cospi_8_64 + step[29] * cospi_24_64);
- output[19] = dct_32_round(step[19] * -cospi_8_64 + step[28] * cospi_24_64);
- output[20] = dct_32_round(step[20] * -cospi_24_64 + step[27] * -cospi_8_64);
- output[21] = dct_32_round(step[21] * -cospi_24_64 + step[26] * -cospi_8_64);
- output[22] = step[22];
- output[23] = step[23];
- output[24] = step[24];
- output[25] = step[25];
- output[26] = dct_32_round(step[26] * cospi_24_64 + step[21] * -cospi_8_64);
- output[27] = dct_32_round(step[27] * cospi_24_64 + step[20] * -cospi_8_64);
- output[28] = dct_32_round(step[28] * cospi_8_64 + step[19] * cospi_24_64);
- output[29] = dct_32_round(step[29] * cospi_8_64 + step[18] * cospi_24_64);
- output[30] = step[30];
- output[31] = step[31];
-
- // Stage 5
- step[0] = dct_32_round((output[0] + output[1]) * cospi_16_64);
- step[1] = dct_32_round((-output[1] + output[0]) * cospi_16_64);
- step[2] = dct_32_round(output[2] * cospi_24_64 + output[3] * cospi_8_64);
- step[3] = dct_32_round(output[3] * cospi_24_64 - output[2] * cospi_8_64);
- step[4] = output[4] + output[5];
- step[5] = -output[5] + output[4];
- step[6] = -output[6] + output[7];
- step[7] = output[7] + output[6];
- step[8] = output[8];
- step[9] = dct_32_round(output[9] * -cospi_8_64 + output[14] * cospi_24_64);
- step[10] = dct_32_round(output[10] * -cospi_24_64 + output[13] * -cospi_8_64);
- step[11] = output[11];
- step[12] = output[12];
- step[13] = dct_32_round(output[13] * cospi_24_64 + output[10] * -cospi_8_64);
- step[14] = dct_32_round(output[14] * cospi_8_64 + output[9] * cospi_24_64);
- step[15] = output[15];
-
- step[16] = output[16] + output[19];
- step[17] = output[17] + output[18];
- step[18] = -output[18] + output[17];
- step[19] = -output[19] + output[16];
- step[20] = -output[20] + output[23];
- step[21] = -output[21] + output[22];
- step[22] = output[22] + output[21];
- step[23] = output[23] + output[20];
- step[24] = output[24] + output[27];
- step[25] = output[25] + output[26];
- step[26] = -output[26] + output[25];
- step[27] = -output[27] + output[24];
- step[28] = -output[28] + output[31];
- step[29] = -output[29] + output[30];
- step[30] = output[30] + output[29];
- step[31] = output[31] + output[28];
-
- // Stage 6
- output[0] = step[0];
- output[1] = step[1];
- output[2] = step[2];
- output[3] = step[3];
- output[4] = dct_32_round(step[4] * cospi_28_64 + step[7] * cospi_4_64);
- output[5] = dct_32_round(step[5] * cospi_12_64 + step[6] * cospi_20_64);
- output[6] = dct_32_round(step[6] * cospi_12_64 + step[5] * -cospi_20_64);
- output[7] = dct_32_round(step[7] * cospi_28_64 + step[4] * -cospi_4_64);
- output[8] = step[8] + step[9];
- output[9] = -step[9] + step[8];
- output[10] = -step[10] + step[11];
- output[11] = step[11] + step[10];
- output[12] = step[12] + step[13];
- output[13] = -step[13] + step[12];
- output[14] = -step[14] + step[15];
- output[15] = step[15] + step[14];
-
- output[16] = step[16];
- output[17] = dct_32_round(step[17] * -cospi_4_64 + step[30] * cospi_28_64);
- output[18] = dct_32_round(step[18] * -cospi_28_64 + step[29] * -cospi_4_64);
- output[19] = step[19];
- output[20] = step[20];
- output[21] = dct_32_round(step[21] * -cospi_20_64 + step[26] * cospi_12_64);
- output[22] = dct_32_round(step[22] * -cospi_12_64 + step[25] * -cospi_20_64);
- output[23] = step[23];
- output[24] = step[24];
- output[25] = dct_32_round(step[25] * cospi_12_64 + step[22] * -cospi_20_64);
- output[26] = dct_32_round(step[26] * cospi_20_64 + step[21] * cospi_12_64);
- output[27] = step[27];
- output[28] = step[28];
- output[29] = dct_32_round(step[29] * cospi_28_64 + step[18] * -cospi_4_64);
- output[30] = dct_32_round(step[30] * cospi_4_64 + step[17] * cospi_28_64);
- output[31] = step[31];
-
- // Stage 7
- step[0] = output[0];
- step[1] = output[1];
- step[2] = output[2];
- step[3] = output[3];
- step[4] = output[4];
- step[5] = output[5];
- step[6] = output[6];
- step[7] = output[7];
- step[8] = dct_32_round(output[8] * cospi_30_64 + output[15] * cospi_2_64);
- step[9] = dct_32_round(output[9] * cospi_14_64 + output[14] * cospi_18_64);
- step[10] = dct_32_round(output[10] * cospi_22_64 + output[13] * cospi_10_64);
- step[11] = dct_32_round(output[11] * cospi_6_64 + output[12] * cospi_26_64);
- step[12] = dct_32_round(output[12] * cospi_6_64 + output[11] * -cospi_26_64);
- step[13] = dct_32_round(output[13] * cospi_22_64 + output[10] * -cospi_10_64);
- step[14] = dct_32_round(output[14] * cospi_14_64 + output[9] * -cospi_18_64);
- step[15] = dct_32_round(output[15] * cospi_30_64 + output[8] * -cospi_2_64);
-
- step[16] = output[16] + output[17];
- step[17] = -output[17] + output[16];
- step[18] = -output[18] + output[19];
- step[19] = output[19] + output[18];
- step[20] = output[20] + output[21];
- step[21] = -output[21] + output[20];
- step[22] = -output[22] + output[23];
- step[23] = output[23] + output[22];
- step[24] = output[24] + output[25];
- step[25] = -output[25] + output[24];
- step[26] = -output[26] + output[27];
- step[27] = output[27] + output[26];
- step[28] = output[28] + output[29];
- step[29] = -output[29] + output[28];
- step[30] = -output[30] + output[31];
- step[31] = output[31] + output[30];
-
- // Final stage --- outputs indices are bit-reversed.
- output[0] = step[0];
- output[16] = step[1];
- output[8] = step[2];
- output[24] = step[3];
- output[4] = step[4];
- output[20] = step[5];
- output[12] = step[6];
- output[28] = step[7];
- output[2] = step[8];
- output[18] = step[9];
- output[10] = step[10];
- output[26] = step[11];
- output[6] = step[12];
- output[22] = step[13];
- output[14] = step[14];
- output[30] = step[15];
-
- output[1] = dct_32_round(step[16] * cospi_31_64 + step[31] * cospi_1_64);
- output[17] = dct_32_round(step[17] * cospi_15_64 + step[30] * cospi_17_64);
- output[9] = dct_32_round(step[18] * cospi_23_64 + step[29] * cospi_9_64);
- output[25] = dct_32_round(step[19] * cospi_7_64 + step[28] * cospi_25_64);
- output[5] = dct_32_round(step[20] * cospi_27_64 + step[27] * cospi_5_64);
- output[21] = dct_32_round(step[21] * cospi_11_64 + step[26] * cospi_21_64);
- output[13] = dct_32_round(step[22] * cospi_19_64 + step[25] * cospi_13_64);
- output[29] = dct_32_round(step[23] * cospi_3_64 + step[24] * cospi_29_64);
- output[3] = dct_32_round(step[24] * cospi_3_64 + step[23] * -cospi_29_64);
- output[19] = dct_32_round(step[25] * cospi_19_64 + step[22] * -cospi_13_64);
- output[11] = dct_32_round(step[26] * cospi_11_64 + step[21] * -cospi_21_64);
- output[27] = dct_32_round(step[27] * cospi_27_64 + step[20] * -cospi_5_64);
- output[7] = dct_32_round(step[28] * cospi_7_64 + step[19] * -cospi_25_64);
- output[23] = dct_32_round(step[29] * cospi_23_64 + step[18] * -cospi_9_64);
- output[15] = dct_32_round(step[30] * cospi_15_64 + step[17] * -cospi_17_64);
- output[31] = dct_32_round(step[31] * cospi_31_64 + step[16] * -cospi_1_64);
-}
-
-void vp10_fdct32x32_c(const int16_t *input, tran_low_t *out, int stride) {
- int i, j;
- tran_high_t output[32 * 32];
-
- // Columns
- for (i = 0; i < 32; ++i) {
- tran_high_t temp_in[32], temp_out[32];
- for (j = 0; j < 32; ++j)
- temp_in[j] = input[j * stride + i] * 4;
- vp10_fdct32(temp_in, temp_out, 0);
- for (j = 0; j < 32; ++j)
- output[j * 32 + i] = (temp_out[j] + 1 + (temp_out[j] > 0)) >> 2;
- }
-
- // Rows
- for (i = 0; i < 32; ++i) {
- tran_high_t temp_in[32], temp_out[32];
- for (j = 0; j < 32; ++j)
- temp_in[j] = output[j + i * 32];
- vp10_fdct32(temp_in, temp_out, 0);
- for (j = 0; j < 32; ++j)
- out[j + i * 32] =
- (tran_low_t)((temp_out[j] + 1 + (temp_out[j] < 0)) >> 2);
- }
-}
-
-// Note that although we use dct_32_round in dct32 computation flow,
-// this 2d fdct32x32 for rate-distortion optimization loop is operating
-// within 16 bits precision.
-void vp10_fdct32x32_rd_c(const int16_t *input, tran_low_t *out, int stride) {
- int i, j;
- tran_high_t output[32 * 32];
-
- // Columns
- for (i = 0; i < 32; ++i) {
- tran_high_t temp_in[32], temp_out[32];
- for (j = 0; j < 32; ++j)
- temp_in[j] = input[j * stride + i] * 4;
- vp10_fdct32(temp_in, temp_out, 0);
- for (j = 0; j < 32; ++j)
- // TODO(cd): see quality impact of only doing
- // output[j * 32 + i] = (temp_out[j] + 1) >> 2;
- // PS: also change code in vp10_dsp/x86/vp10_dct_sse2.c
- output[j * 32 + i] = (temp_out[j] + 1 + (temp_out[j] > 0)) >> 2;
- }
-
- // Rows
- for (i = 0; i < 32; ++i) {
- tran_high_t temp_in[32], temp_out[32];
- for (j = 0; j < 32; ++j)
- temp_in[j] = output[j + i * 32];
- vp10_fdct32(temp_in, temp_out, 1);
- for (j = 0; j < 32; ++j)
- out[j + i * 32] = (tran_low_t)temp_out[j];
- }
-}
-
-void vp10_fdct32x32_1_c(const int16_t *input, tran_low_t *output, int stride) {
- int r, c;
- tran_low_t sum = 0;
- for (r = 0; r < 32; ++r)
- for (c = 0; c < 32; ++c)
- sum += input[r * stride + c];
-
- output[0] = sum >> 3;
- output[1] = 0;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_fdct4x4_c(const int16_t *input, tran_low_t *output,
- int stride) {
- vp10_fdct4x4_c(input, output, stride);
-}
-
-void vp10_highbd_fdct8x8_c(const int16_t *input, tran_low_t *final_output,
- int stride) {
- vp10_fdct8x8_c(input, final_output, stride);
-}
-
-void vp10_highbd_fdct8x8_1_c(const int16_t *input, tran_low_t *final_output,
- int stride) {
- vp10_fdct8x8_1_c(input, final_output, stride);
-}
-
-void vp10_highbd_fdct16x16_c(const int16_t *input, tran_low_t *output,
- int stride) {
- vp10_fdct16x16_c(input, output, stride);
-}
-
-void vp10_highbd_fdct16x16_1_c(const int16_t *input, tran_low_t *output,
- int stride) {
- vp10_fdct16x16_1_c(input, output, stride);
-}
-
-void vp10_highbd_fdct32x32_c(const int16_t *input,
- tran_low_t *out, int stride) {
- vp10_fdct32x32_c(input, out, stride);
-}
-
-void vp10_highbd_fdct32x32_rd_c(const int16_t *input, tran_low_t *out,
- int stride) {
- vp10_fdct32x32_rd_c(input, out, stride);
-}
-
-void vp10_highbd_fdct32x32_1_c(const int16_t *input,
- tran_low_t *out, int stride) {
- vp10_fdct32x32_1_c(input, out, stride);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_COMMON_VP10_FWD_TXFM_H_
-#define VP10_COMMON_VP10_FWD_TXFM_H_
-
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_dsp/fwd_txfm.h"
-
-void vp10_fdct32(const tran_high_t *input, tran_high_t *output, int round);
-#endif // VP10_COMMON_VP10_FWD_TXFM_H_
+++ /dev/null
-/*
- *
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-#include <string.h>
-
-#include "vp10/common/vp10_inv_txfm.h"
-
-void vp10_iwht4x4_16_add_c(const tran_low_t *input, uint8_t *dest, int stride) {
-/* 4-point reversible, orthonormal inverse Walsh-Hadamard in 3.5 adds,
- 0.5 shifts per pixel. */
- int i;
- tran_low_t output[16];
- tran_high_t a1, b1, c1, d1, e1;
- const tran_low_t *ip = input;
- tran_low_t *op = output;
-
- for (i = 0; i < 4; i++) {
- a1 = ip[0] >> UNIT_QUANT_SHIFT;
- c1 = ip[1] >> UNIT_QUANT_SHIFT;
- d1 = ip[2] >> UNIT_QUANT_SHIFT;
- b1 = ip[3] >> UNIT_QUANT_SHIFT;
- a1 += c1;
- d1 -= b1;
- e1 = (a1 - d1) >> 1;
- b1 = e1 - b1;
- c1 = e1 - c1;
- a1 -= b1;
- d1 += c1;
- op[0] = WRAPLOW(a1, 8);
- op[1] = WRAPLOW(b1, 8);
- op[2] = WRAPLOW(c1, 8);
- op[3] = WRAPLOW(d1, 8);
- ip += 4;
- op += 4;
- }
-
- ip = output;
- for (i = 0; i < 4; i++) {
- a1 = ip[4 * 0];
- c1 = ip[4 * 1];
- d1 = ip[4 * 2];
- b1 = ip[4 * 3];
- a1 += c1;
- d1 -= b1;
- e1 = (a1 - d1) >> 1;
- b1 = e1 - b1;
- c1 = e1 - c1;
- a1 -= b1;
- d1 += c1;
- dest[stride * 0] = clip_pixel_add(dest[stride * 0], a1);
- dest[stride * 1] = clip_pixel_add(dest[stride * 1], b1);
- dest[stride * 2] = clip_pixel_add(dest[stride * 2], c1);
- dest[stride * 3] = clip_pixel_add(dest[stride * 3], d1);
-
- ip++;
- dest++;
- }
-}
-
-void vp10_iwht4x4_1_add_c(const tran_low_t *in,
- uint8_t *dest,
- int dest_stride) {
- int i;
- tran_high_t a1, e1;
- tran_low_t tmp[4];
- const tran_low_t *ip = in;
- tran_low_t *op = tmp;
-
- a1 = ip[0] >> UNIT_QUANT_SHIFT;
- e1 = a1 >> 1;
- a1 -= e1;
- op[0] = WRAPLOW(a1, 8);
- op[1] = op[2] = op[3] = WRAPLOW(e1, 8);
-
- ip = tmp;
- for (i = 0; i < 4; i++) {
- e1 = ip[0] >> 1;
- a1 = ip[0] - e1;
- dest[dest_stride * 0] = clip_pixel_add(dest[dest_stride * 0], a1);
- dest[dest_stride * 1] = clip_pixel_add(dest[dest_stride * 1], e1);
- dest[dest_stride * 2] = clip_pixel_add(dest[dest_stride * 2], e1);
- dest[dest_stride * 3] = clip_pixel_add(dest[dest_stride * 3], e1);
- ip++;
- dest++;
- }
-}
-
-void vp10_idct4_c(const tran_low_t *input, tran_low_t *output) {
- tran_low_t step[4];
- tran_high_t temp1, temp2;
- // stage 1
- temp1 = (input[0] + input[2]) * cospi_16_64;
- temp2 = (input[0] - input[2]) * cospi_16_64;
- step[0] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step[1] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = input[1] * cospi_24_64 - input[3] * cospi_8_64;
- temp2 = input[1] * cospi_8_64 + input[3] * cospi_24_64;
- step[2] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step[3] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- // stage 2
- output[0] = WRAPLOW(step[0] + step[3], 8);
- output[1] = WRAPLOW(step[1] + step[2], 8);
- output[2] = WRAPLOW(step[1] - step[2], 8);
- output[3] = WRAPLOW(step[0] - step[3], 8);
-}
-
-void vp10_idct4x4_16_add_c(const tran_low_t *input, uint8_t *dest, int stride) {
- tran_low_t out[4 * 4];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[4], temp_out[4];
-
- // Rows
- for (i = 0; i < 4; ++i) {
- vp10_idct4_c(input, outptr);
- input += 4;
- outptr += 4;
- }
-
- // Columns
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j * 4 + i];
- vp10_idct4_c(temp_in, temp_out);
- for (j = 0; j < 4; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 4));
- }
- }
-}
-
-void vp10_idct4x4_1_add_c(const tran_low_t *input, uint8_t *dest,
- int dest_stride) {
- int i;
- tran_high_t a1;
- tran_low_t out = WRAPLOW(dct_const_round_shift(input[0] * cospi_16_64), 8);
- out = WRAPLOW(dct_const_round_shift(out * cospi_16_64), 8);
- a1 = ROUND_POWER_OF_TWO(out, 4);
-
- for (i = 0; i < 4; i++) {
- dest[0] = clip_pixel_add(dest[0], a1);
- dest[1] = clip_pixel_add(dest[1], a1);
- dest[2] = clip_pixel_add(dest[2], a1);
- dest[3] = clip_pixel_add(dest[3], a1);
- dest += dest_stride;
- }
-}
-
-void vp10_idct8_c(const tran_low_t *input, tran_low_t *output) {
- tran_low_t step1[8], step2[8];
- tran_high_t temp1, temp2;
- // stage 1
- step1[0] = input[0];
- step1[2] = input[4];
- step1[1] = input[2];
- step1[3] = input[6];
- temp1 = input[1] * cospi_28_64 - input[7] * cospi_4_64;
- temp2 = input[1] * cospi_4_64 + input[7] * cospi_28_64;
- step1[4] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[7] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = input[5] * cospi_12_64 - input[3] * cospi_20_64;
- temp2 = input[5] * cospi_20_64 + input[3] * cospi_12_64;
- step1[5] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[6] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- // stage 2
- temp1 = (step1[0] + step1[2]) * cospi_16_64;
- temp2 = (step1[0] - step1[2]) * cospi_16_64;
- step2[0] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[1] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = step1[1] * cospi_24_64 - step1[3] * cospi_8_64;
- temp2 = step1[1] * cospi_8_64 + step1[3] * cospi_24_64;
- step2[2] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[3] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[4] = WRAPLOW(step1[4] + step1[5], 8);
- step2[5] = WRAPLOW(step1[4] - step1[5], 8);
- step2[6] = WRAPLOW(-step1[6] + step1[7], 8);
- step2[7] = WRAPLOW(step1[6] + step1[7], 8);
-
- // stage 3
- step1[0] = WRAPLOW(step2[0] + step2[3], 8);
- step1[1] = WRAPLOW(step2[1] + step2[2], 8);
- step1[2] = WRAPLOW(step2[1] - step2[2], 8);
- step1[3] = WRAPLOW(step2[0] - step2[3], 8);
- step1[4] = step2[4];
- temp1 = (step2[6] - step2[5]) * cospi_16_64;
- temp2 = (step2[5] + step2[6]) * cospi_16_64;
- step1[5] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[6] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[7] = step2[7];
-
- // stage 4
- output[0] = WRAPLOW(step1[0] + step1[7], 8);
- output[1] = WRAPLOW(step1[1] + step1[6], 8);
- output[2] = WRAPLOW(step1[2] + step1[5], 8);
- output[3] = WRAPLOW(step1[3] + step1[4], 8);
- output[4] = WRAPLOW(step1[3] - step1[4], 8);
- output[5] = WRAPLOW(step1[2] - step1[5], 8);
- output[6] = WRAPLOW(step1[1] - step1[6], 8);
- output[7] = WRAPLOW(step1[0] - step1[7], 8);
-}
-
-void vp10_idct8x8_64_add_c(const tran_low_t *input, uint8_t *dest, int stride) {
- tran_low_t out[8 * 8];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[8], temp_out[8];
-
- // First transform rows
- for (i = 0; i < 8; ++i) {
- vp10_idct8_c(input, outptr);
- input += 8;
- outptr += 8;
- }
-
- // Then transform columns
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- vp10_idct8_c(temp_in, temp_out);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 5));
- }
- }
-}
-
-void vp10_idct8x8_1_add_c(const tran_low_t *input, uint8_t *dest, int stride) {
- int i, j;
- tran_high_t a1;
- tran_low_t out = WRAPLOW(dct_const_round_shift(input[0] * cospi_16_64), 8);
- out = WRAPLOW(dct_const_round_shift(out * cospi_16_64), 8);
- a1 = ROUND_POWER_OF_TWO(out, 5);
- for (j = 0; j < 8; ++j) {
- for (i = 0; i < 8; ++i)
- dest[i] = clip_pixel_add(dest[i], a1);
- dest += stride;
- }
-}
-
-void vp10_iadst4_c(const tran_low_t *input, tran_low_t *output) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7;
-
- tran_low_t x0 = input[0];
- tran_low_t x1 = input[1];
- tran_low_t x2 = input[2];
- tran_low_t x3 = input[3];
-
- if (!(x0 | x1 | x2 | x3)) {
- output[0] = output[1] = output[2] = output[3] = 0;
- return;
- }
-
- s0 = sinpi_1_9 * x0;
- s1 = sinpi_2_9 * x0;
- s2 = sinpi_3_9 * x1;
- s3 = sinpi_4_9 * x2;
- s4 = sinpi_1_9 * x2;
- s5 = sinpi_2_9 * x3;
- s6 = sinpi_4_9 * x3;
- s7 = x0 - x2 + x3;
-
- s0 = s0 + s3 + s5;
- s1 = s1 - s4 - s6;
- s3 = s2;
- s2 = sinpi_3_9 * s7;
-
- // 1-D transform scaling factor is sqrt(2).
- // The overall dynamic range is 14b (input) + 14b (multiplication scaling)
- // + 1b (addition) = 29b.
- // Hence the output bit depth is 15b.
- output[0] = WRAPLOW(dct_const_round_shift(s0 + s3), 8);
- output[1] = WRAPLOW(dct_const_round_shift(s1 + s3), 8);
- output[2] = WRAPLOW(dct_const_round_shift(s2), 8);
- output[3] = WRAPLOW(dct_const_round_shift(s0 + s1 - s3), 8);
-}
-
-void vp10_iadst8_c(const tran_low_t *input, tran_low_t *output) {
- int s0, s1, s2, s3, s4, s5, s6, s7;
-
- tran_high_t x0 = input[7];
- tran_high_t x1 = input[0];
- tran_high_t x2 = input[5];
- tran_high_t x3 = input[2];
- tran_high_t x4 = input[3];
- tran_high_t x5 = input[4];
- tran_high_t x6 = input[1];
- tran_high_t x7 = input[6];
-
- if (!(x0 | x1 | x2 | x3 | x4 | x5 | x6 | x7)) {
- output[0] = output[1] = output[2] = output[3] = output[4]
- = output[5] = output[6] = output[7] = 0;
- return;
- }
-
- // stage 1
- s0 = (int)(cospi_2_64 * x0 + cospi_30_64 * x1);
- s1 = (int)(cospi_30_64 * x0 - cospi_2_64 * x1);
- s2 = (int)(cospi_10_64 * x2 + cospi_22_64 * x3);
- s3 = (int)(cospi_22_64 * x2 - cospi_10_64 * x3);
- s4 = (int)(cospi_18_64 * x4 + cospi_14_64 * x5);
- s5 = (int)(cospi_14_64 * x4 - cospi_18_64 * x5);
- s6 = (int)(cospi_26_64 * x6 + cospi_6_64 * x7);
- s7 = (int)(cospi_6_64 * x6 - cospi_26_64 * x7);
-
- x0 = WRAPLOW(dct_const_round_shift(s0 + s4), 8);
- x1 = WRAPLOW(dct_const_round_shift(s1 + s5), 8);
- x2 = WRAPLOW(dct_const_round_shift(s2 + s6), 8);
- x3 = WRAPLOW(dct_const_round_shift(s3 + s7), 8);
- x4 = WRAPLOW(dct_const_round_shift(s0 - s4), 8);
- x5 = WRAPLOW(dct_const_round_shift(s1 - s5), 8);
- x6 = WRAPLOW(dct_const_round_shift(s2 - s6), 8);
- x7 = WRAPLOW(dct_const_round_shift(s3 - s7), 8);
-
- // stage 2
- s0 = (int)x0;
- s1 = (int)x1;
- s2 = (int)x2;
- s3 = (int)x3;
- s4 = (int)(cospi_8_64 * x4 + cospi_24_64 * x5);
- s5 = (int)(cospi_24_64 * x4 - cospi_8_64 * x5);
- s6 = (int)(-cospi_24_64 * x6 + cospi_8_64 * x7);
- s7 = (int)(cospi_8_64 * x6 + cospi_24_64 * x7);
-
- x0 = WRAPLOW(s0 + s2, 8);
- x1 = WRAPLOW(s1 + s3, 8);
- x2 = WRAPLOW(s0 - s2, 8);
- x3 = WRAPLOW(s1 - s3, 8);
- x4 = WRAPLOW(dct_const_round_shift(s4 + s6), 8);
- x5 = WRAPLOW(dct_const_round_shift(s5 + s7), 8);
- x6 = WRAPLOW(dct_const_round_shift(s4 - s6), 8);
- x7 = WRAPLOW(dct_const_round_shift(s5 - s7), 8);
-
- // stage 3
- s2 = (int)(cospi_16_64 * (x2 + x3));
- s3 = (int)(cospi_16_64 * (x2 - x3));
- s6 = (int)(cospi_16_64 * (x6 + x7));
- s7 = (int)(cospi_16_64 * (x6 - x7));
-
- x2 = WRAPLOW(dct_const_round_shift(s2), 8);
- x3 = WRAPLOW(dct_const_round_shift(s3), 8);
- x6 = WRAPLOW(dct_const_round_shift(s6), 8);
- x7 = WRAPLOW(dct_const_round_shift(s7), 8);
-
- output[0] = WRAPLOW(x0, 8);
- output[1] = WRAPLOW(-x4, 8);
- output[2] = WRAPLOW(x6, 8);
- output[3] = WRAPLOW(-x2, 8);
- output[4] = WRAPLOW(x3, 8);
- output[5] = WRAPLOW(-x7, 8);
- output[6] = WRAPLOW(x5, 8);
- output[7] = WRAPLOW(-x1, 8);
-}
-
-void vp10_idct8x8_12_add_c(const tran_low_t *input, uint8_t *dest, int stride) {
- tran_low_t out[8 * 8] = { 0 };
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[8], temp_out[8];
-
- // First transform rows
- // only first 4 row has non-zero coefs
- for (i = 0; i < 4; ++i) {
- vp10_idct8_c(input, outptr);
- input += 8;
- outptr += 8;
- }
-
- // Then transform columns
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- vp10_idct8_c(temp_in, temp_out);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 5));
- }
- }
-}
-
-void vp10_idct16_c(const tran_low_t *input, tran_low_t *output) {
- tran_low_t step1[16], step2[16];
- tran_high_t temp1, temp2;
-
- // stage 1
- step1[0] = input[0/2];
- step1[1] = input[16/2];
- step1[2] = input[8/2];
- step1[3] = input[24/2];
- step1[4] = input[4/2];
- step1[5] = input[20/2];
- step1[6] = input[12/2];
- step1[7] = input[28/2];
- step1[8] = input[2/2];
- step1[9] = input[18/2];
- step1[10] = input[10/2];
- step1[11] = input[26/2];
- step1[12] = input[6/2];
- step1[13] = input[22/2];
- step1[14] = input[14/2];
- step1[15] = input[30/2];
-
- // stage 2
- step2[0] = step1[0];
- step2[1] = step1[1];
- step2[2] = step1[2];
- step2[3] = step1[3];
- step2[4] = step1[4];
- step2[5] = step1[5];
- step2[6] = step1[6];
- step2[7] = step1[7];
-
- temp1 = step1[8] * cospi_30_64 - step1[15] * cospi_2_64;
- temp2 = step1[8] * cospi_2_64 + step1[15] * cospi_30_64;
- step2[8] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[15] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = step1[9] * cospi_14_64 - step1[14] * cospi_18_64;
- temp2 = step1[9] * cospi_18_64 + step1[14] * cospi_14_64;
- step2[9] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[14] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = step1[10] * cospi_22_64 - step1[13] * cospi_10_64;
- temp2 = step1[10] * cospi_10_64 + step1[13] * cospi_22_64;
- step2[10] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[13] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = step1[11] * cospi_6_64 - step1[12] * cospi_26_64;
- temp2 = step1[11] * cospi_26_64 + step1[12] * cospi_6_64;
- step2[11] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[12] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- // stage 3
- step1[0] = step2[0];
- step1[1] = step2[1];
- step1[2] = step2[2];
- step1[3] = step2[3];
-
- temp1 = step2[4] * cospi_28_64 - step2[7] * cospi_4_64;
- temp2 = step2[4] * cospi_4_64 + step2[7] * cospi_28_64;
- step1[4] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[7] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = step2[5] * cospi_12_64 - step2[6] * cospi_20_64;
- temp2 = step2[5] * cospi_20_64 + step2[6] * cospi_12_64;
- step1[5] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[6] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- step1[8] = WRAPLOW(step2[8] + step2[9], 8);
- step1[9] = WRAPLOW(step2[8] - step2[9], 8);
- step1[10] = WRAPLOW(-step2[10] + step2[11], 8);
- step1[11] = WRAPLOW(step2[10] + step2[11], 8);
- step1[12] = WRAPLOW(step2[12] + step2[13], 8);
- step1[13] = WRAPLOW(step2[12] - step2[13], 8);
- step1[14] = WRAPLOW(-step2[14] + step2[15], 8);
- step1[15] = WRAPLOW(step2[14] + step2[15], 8);
-
- // stage 4
- temp1 = (step1[0] + step1[1]) * cospi_16_64;
- temp2 = (step1[0] - step1[1]) * cospi_16_64;
- step2[0] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[1] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = step1[2] * cospi_24_64 - step1[3] * cospi_8_64;
- temp2 = step1[2] * cospi_8_64 + step1[3] * cospi_24_64;
- step2[2] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[3] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[4] = WRAPLOW(step1[4] + step1[5], 8);
- step2[5] = WRAPLOW(step1[4] - step1[5], 8);
- step2[6] = WRAPLOW(-step1[6] + step1[7], 8);
- step2[7] = WRAPLOW(step1[6] + step1[7], 8);
-
- step2[8] = step1[8];
- step2[15] = step1[15];
- temp1 = -step1[9] * cospi_8_64 + step1[14] * cospi_24_64;
- temp2 = step1[9] * cospi_24_64 + step1[14] * cospi_8_64;
- step2[9] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[14] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step1[10] * cospi_24_64 - step1[13] * cospi_8_64;
- temp2 = -step1[10] * cospi_8_64 + step1[13] * cospi_24_64;
- step2[10] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[13] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[11] = step1[11];
- step2[12] = step1[12];
-
- // stage 5
- step1[0] = WRAPLOW(step2[0] + step2[3], 8);
- step1[1] = WRAPLOW(step2[1] + step2[2], 8);
- step1[2] = WRAPLOW(step2[1] - step2[2], 8);
- step1[3] = WRAPLOW(step2[0] - step2[3], 8);
- step1[4] = step2[4];
- temp1 = (step2[6] - step2[5]) * cospi_16_64;
- temp2 = (step2[5] + step2[6]) * cospi_16_64;
- step1[5] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[6] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[7] = step2[7];
-
- step1[8] = WRAPLOW(step2[8] + step2[11], 8);
- step1[9] = WRAPLOW(step2[9] + step2[10], 8);
- step1[10] = WRAPLOW(step2[9] - step2[10], 8);
- step1[11] = WRAPLOW(step2[8] - step2[11], 8);
- step1[12] = WRAPLOW(-step2[12] + step2[15], 8);
- step1[13] = WRAPLOW(-step2[13] + step2[14], 8);
- step1[14] = WRAPLOW(step2[13] + step2[14], 8);
- step1[15] = WRAPLOW(step2[12] + step2[15], 8);
-
- // stage 6
- step2[0] = WRAPLOW(step1[0] + step1[7], 8);
- step2[1] = WRAPLOW(step1[1] + step1[6], 8);
- step2[2] = WRAPLOW(step1[2] + step1[5], 8);
- step2[3] = WRAPLOW(step1[3] + step1[4], 8);
- step2[4] = WRAPLOW(step1[3] - step1[4], 8);
- step2[5] = WRAPLOW(step1[2] - step1[5], 8);
- step2[6] = WRAPLOW(step1[1] - step1[6], 8);
- step2[7] = WRAPLOW(step1[0] - step1[7], 8);
- step2[8] = step1[8];
- step2[9] = step1[9];
- temp1 = (-step1[10] + step1[13]) * cospi_16_64;
- temp2 = (step1[10] + step1[13]) * cospi_16_64;
- step2[10] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[13] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = (-step1[11] + step1[12]) * cospi_16_64;
- temp2 = (step1[11] + step1[12]) * cospi_16_64;
- step2[11] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[12] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[14] = step1[14];
- step2[15] = step1[15];
-
- // stage 7
- output[0] = WRAPLOW(step2[0] + step2[15], 8);
- output[1] = WRAPLOW(step2[1] + step2[14], 8);
- output[2] = WRAPLOW(step2[2] + step2[13], 8);
- output[3] = WRAPLOW(step2[3] + step2[12], 8);
- output[4] = WRAPLOW(step2[4] + step2[11], 8);
- output[5] = WRAPLOW(step2[5] + step2[10], 8);
- output[6] = WRAPLOW(step2[6] + step2[9], 8);
- output[7] = WRAPLOW(step2[7] + step2[8], 8);
- output[8] = WRAPLOW(step2[7] - step2[8], 8);
- output[9] = WRAPLOW(step2[6] - step2[9], 8);
- output[10] = WRAPLOW(step2[5] - step2[10], 8);
- output[11] = WRAPLOW(step2[4] - step2[11], 8);
- output[12] = WRAPLOW(step2[3] - step2[12], 8);
- output[13] = WRAPLOW(step2[2] - step2[13], 8);
- output[14] = WRAPLOW(step2[1] - step2[14], 8);
- output[15] = WRAPLOW(step2[0] - step2[15], 8);
-}
-
-void vp10_idct16x16_256_add_c(const tran_low_t *input, uint8_t *dest,
- int stride) {
- tran_low_t out[16 * 16];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[16], temp_out[16];
-
- // First transform rows
- for (i = 0; i < 16; ++i) {
- vp10_idct16_c(input, outptr);
- input += 16;
- outptr += 16;
- }
-
- // Then transform columns
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- vp10_idct16_c(temp_in, temp_out);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 6));
- }
- }
-}
-
-void vp10_iadst16_c(const tran_low_t *input, tran_low_t *output) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7, s8;
- tran_high_t s9, s10, s11, s12, s13, s14, s15;
-
- tran_high_t x0 = input[15];
- tran_high_t x1 = input[0];
- tran_high_t x2 = input[13];
- tran_high_t x3 = input[2];
- tran_high_t x4 = input[11];
- tran_high_t x5 = input[4];
- tran_high_t x6 = input[9];
- tran_high_t x7 = input[6];
- tran_high_t x8 = input[7];
- tran_high_t x9 = input[8];
- tran_high_t x10 = input[5];
- tran_high_t x11 = input[10];
- tran_high_t x12 = input[3];
- tran_high_t x13 = input[12];
- tran_high_t x14 = input[1];
- tran_high_t x15 = input[14];
-
- if (!(x0 | x1 | x2 | x3 | x4 | x5 | x6 | x7 | x8
- | x9 | x10 | x11 | x12 | x13 | x14 | x15)) {
- output[0] = output[1] = output[2] = output[3] = output[4]
- = output[5] = output[6] = output[7] = output[8]
- = output[9] = output[10] = output[11] = output[12]
- = output[13] = output[14] = output[15] = 0;
- return;
- }
-
- // stage 1
- s0 = x0 * cospi_1_64 + x1 * cospi_31_64;
- s1 = x0 * cospi_31_64 - x1 * cospi_1_64;
- s2 = x2 * cospi_5_64 + x3 * cospi_27_64;
- s3 = x2 * cospi_27_64 - x3 * cospi_5_64;
- s4 = x4 * cospi_9_64 + x5 * cospi_23_64;
- s5 = x4 * cospi_23_64 - x5 * cospi_9_64;
- s6 = x6 * cospi_13_64 + x7 * cospi_19_64;
- s7 = x6 * cospi_19_64 - x7 * cospi_13_64;
- s8 = x8 * cospi_17_64 + x9 * cospi_15_64;
- s9 = x8 * cospi_15_64 - x9 * cospi_17_64;
- s10 = x10 * cospi_21_64 + x11 * cospi_11_64;
- s11 = x10 * cospi_11_64 - x11 * cospi_21_64;
- s12 = x12 * cospi_25_64 + x13 * cospi_7_64;
- s13 = x12 * cospi_7_64 - x13 * cospi_25_64;
- s14 = x14 * cospi_29_64 + x15 * cospi_3_64;
- s15 = x14 * cospi_3_64 - x15 * cospi_29_64;
-
- x0 = WRAPLOW(dct_const_round_shift(s0 + s8), 8);
- x1 = WRAPLOW(dct_const_round_shift(s1 + s9), 8);
- x2 = WRAPLOW(dct_const_round_shift(s2 + s10), 8);
- x3 = WRAPLOW(dct_const_round_shift(s3 + s11), 8);
- x4 = WRAPLOW(dct_const_round_shift(s4 + s12), 8);
- x5 = WRAPLOW(dct_const_round_shift(s5 + s13), 8);
- x6 = WRAPLOW(dct_const_round_shift(s6 + s14), 8);
- x7 = WRAPLOW(dct_const_round_shift(s7 + s15), 8);
- x8 = WRAPLOW(dct_const_round_shift(s0 - s8), 8);
- x9 = WRAPLOW(dct_const_round_shift(s1 - s9), 8);
- x10 = WRAPLOW(dct_const_round_shift(s2 - s10), 8);
- x11 = WRAPLOW(dct_const_round_shift(s3 - s11), 8);
- x12 = WRAPLOW(dct_const_round_shift(s4 - s12), 8);
- x13 = WRAPLOW(dct_const_round_shift(s5 - s13), 8);
- x14 = WRAPLOW(dct_const_round_shift(s6 - s14), 8);
- x15 = WRAPLOW(dct_const_round_shift(s7 - s15), 8);
-
- // stage 2
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = x4;
- s5 = x5;
- s6 = x6;
- s7 = x7;
- s8 = x8 * cospi_4_64 + x9 * cospi_28_64;
- s9 = x8 * cospi_28_64 - x9 * cospi_4_64;
- s10 = x10 * cospi_20_64 + x11 * cospi_12_64;
- s11 = x10 * cospi_12_64 - x11 * cospi_20_64;
- s12 = - x12 * cospi_28_64 + x13 * cospi_4_64;
- s13 = x12 * cospi_4_64 + x13 * cospi_28_64;
- s14 = - x14 * cospi_12_64 + x15 * cospi_20_64;
- s15 = x14 * cospi_20_64 + x15 * cospi_12_64;
-
- x0 = WRAPLOW(s0 + s4, 8);
- x1 = WRAPLOW(s1 + s5, 8);
- x2 = WRAPLOW(s2 + s6, 8);
- x3 = WRAPLOW(s3 + s7, 8);
- x4 = WRAPLOW(s0 - s4, 8);
- x5 = WRAPLOW(s1 - s5, 8);
- x6 = WRAPLOW(s2 - s6, 8);
- x7 = WRAPLOW(s3 - s7, 8);
- x8 = WRAPLOW(dct_const_round_shift(s8 + s12), 8);
- x9 = WRAPLOW(dct_const_round_shift(s9 + s13), 8);
- x10 = WRAPLOW(dct_const_round_shift(s10 + s14), 8);
- x11 = WRAPLOW(dct_const_round_shift(s11 + s15), 8);
- x12 = WRAPLOW(dct_const_round_shift(s8 - s12), 8);
- x13 = WRAPLOW(dct_const_round_shift(s9 - s13), 8);
- x14 = WRAPLOW(dct_const_round_shift(s10 - s14), 8);
- x15 = WRAPLOW(dct_const_round_shift(s11 - s15), 8);
-
- // stage 3
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = x4 * cospi_8_64 + x5 * cospi_24_64;
- s5 = x4 * cospi_24_64 - x5 * cospi_8_64;
- s6 = - x6 * cospi_24_64 + x7 * cospi_8_64;
- s7 = x6 * cospi_8_64 + x7 * cospi_24_64;
- s8 = x8;
- s9 = x9;
- s10 = x10;
- s11 = x11;
- s12 = x12 * cospi_8_64 + x13 * cospi_24_64;
- s13 = x12 * cospi_24_64 - x13 * cospi_8_64;
- s14 = - x14 * cospi_24_64 + x15 * cospi_8_64;
- s15 = x14 * cospi_8_64 + x15 * cospi_24_64;
-
- x0 = WRAPLOW(check_range(s0 + s2), 8);
- x1 = WRAPLOW(check_range(s1 + s3), 8);
- x2 = WRAPLOW(check_range(s0 - s2), 8);
- x3 = WRAPLOW(check_range(s1 - s3), 8);
- x4 = WRAPLOW(dct_const_round_shift(s4 + s6), 8);
- x5 = WRAPLOW(dct_const_round_shift(s5 + s7), 8);
- x6 = WRAPLOW(dct_const_round_shift(s4 - s6), 8);
- x7 = WRAPLOW(dct_const_round_shift(s5 - s7), 8);
- x8 = WRAPLOW(check_range(s8 + s10), 8);
- x9 = WRAPLOW(check_range(s9 + s11), 8);
- x10 = WRAPLOW(check_range(s8 - s10), 8);
- x11 = WRAPLOW(check_range(s9 - s11), 8);
- x12 = WRAPLOW(dct_const_round_shift(s12 + s14), 8);
- x13 = WRAPLOW(dct_const_round_shift(s13 + s15), 8);
- x14 = WRAPLOW(dct_const_round_shift(s12 - s14), 8);
- x15 = WRAPLOW(dct_const_round_shift(s13 - s15), 8);
-
- // stage 4
- s2 = (- cospi_16_64) * (x2 + x3);
- s3 = cospi_16_64 * (x2 - x3);
- s6 = cospi_16_64 * (x6 + x7);
- s7 = cospi_16_64 * (- x6 + x7);
- s10 = cospi_16_64 * (x10 + x11);
- s11 = cospi_16_64 * (- x10 + x11);
- s14 = (- cospi_16_64) * (x14 + x15);
- s15 = cospi_16_64 * (x14 - x15);
-
- x2 = WRAPLOW(dct_const_round_shift(s2), 8);
- x3 = WRAPLOW(dct_const_round_shift(s3), 8);
- x6 = WRAPLOW(dct_const_round_shift(s6), 8);
- x7 = WRAPLOW(dct_const_round_shift(s7), 8);
- x10 = WRAPLOW(dct_const_round_shift(s10), 8);
- x11 = WRAPLOW(dct_const_round_shift(s11), 8);
- x14 = WRAPLOW(dct_const_round_shift(s14), 8);
- x15 = WRAPLOW(dct_const_round_shift(s15), 8);
-
- output[0] = WRAPLOW(x0, 8);
- output[1] = WRAPLOW(-x8, 8);
- output[2] = WRAPLOW(x12, 8);
- output[3] = WRAPLOW(-x4, 8);
- output[4] = WRAPLOW(x6, 8);
- output[5] = WRAPLOW(x14, 8);
- output[6] = WRAPLOW(x10, 8);
- output[7] = WRAPLOW(x2, 8);
- output[8] = WRAPLOW(x3, 8);
- output[9] = WRAPLOW(x11, 8);
- output[10] = WRAPLOW(x15, 8);
- output[11] = WRAPLOW(x7, 8);
- output[12] = WRAPLOW(x5, 8);
- output[13] = WRAPLOW(-x13, 8);
- output[14] = WRAPLOW(x9, 8);
- output[15] = WRAPLOW(-x1, 8);
-}
-
-void vp10_idct16x16_10_add_c(const tran_low_t *input, uint8_t *dest,
- int stride) {
- tran_low_t out[16 * 16] = { 0 };
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[16], temp_out[16];
-
- // First transform rows. Since all non-zero dct coefficients are in
- // upper-left 4x4 area, we only need to calculate first 4 rows here.
- for (i = 0; i < 4; ++i) {
- vp10_idct16_c(input, outptr);
- input += 16;
- outptr += 16;
- }
-
- // Then transform columns
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j*16 + i];
- vp10_idct16_c(temp_in, temp_out);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = clip_pixel_add(
- dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 6));
- }
- }
-}
-
-void vp10_idct16x16_1_add_c(const tran_low_t *input,
- uint8_t *dest,
- int stride) {
- int i, j;
- tran_high_t a1;
- tran_low_t out = WRAPLOW(dct_const_round_shift(input[0] * cospi_16_64), 8);
- out = WRAPLOW(dct_const_round_shift(out * cospi_16_64), 8);
- a1 = ROUND_POWER_OF_TWO(out, 6);
- for (j = 0; j < 16; ++j) {
- for (i = 0; i < 16; ++i)
- dest[i] = clip_pixel_add(dest[i], a1);
- dest += stride;
- }
-}
-
-void vp10_idct32_c(const tran_low_t *input, tran_low_t *output) {
- tran_low_t step1[32], step2[32];
- tran_high_t temp1, temp2;
-
- // stage 1
- step1[0] = input[0];
- step1[1] = input[16];
- step1[2] = input[8];
- step1[3] = input[24];
- step1[4] = input[4];
- step1[5] = input[20];
- step1[6] = input[12];
- step1[7] = input[28];
- step1[8] = input[2];
- step1[9] = input[18];
- step1[10] = input[10];
- step1[11] = input[26];
- step1[12] = input[6];
- step1[13] = input[22];
- step1[14] = input[14];
- step1[15] = input[30];
-
- temp1 = input[1] * cospi_31_64 - input[31] * cospi_1_64;
- temp2 = input[1] * cospi_1_64 + input[31] * cospi_31_64;
- step1[16] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[31] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[17] * cospi_15_64 - input[15] * cospi_17_64;
- temp2 = input[17] * cospi_17_64 + input[15] * cospi_15_64;
- step1[17] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[30] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[9] * cospi_23_64 - input[23] * cospi_9_64;
- temp2 = input[9] * cospi_9_64 + input[23] * cospi_23_64;
- step1[18] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[29] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[25] * cospi_7_64 - input[7] * cospi_25_64;
- temp2 = input[25] * cospi_25_64 + input[7] * cospi_7_64;
- step1[19] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[28] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[5] * cospi_27_64 - input[27] * cospi_5_64;
- temp2 = input[5] * cospi_5_64 + input[27] * cospi_27_64;
- step1[20] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[27] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[21] * cospi_11_64 - input[11] * cospi_21_64;
- temp2 = input[21] * cospi_21_64 + input[11] * cospi_11_64;
- step1[21] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[26] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[13] * cospi_19_64 - input[19] * cospi_13_64;
- temp2 = input[13] * cospi_13_64 + input[19] * cospi_19_64;
- step1[22] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[25] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = input[29] * cospi_3_64 - input[3] * cospi_29_64;
- temp2 = input[29] * cospi_29_64 + input[3] * cospi_3_64;
- step1[23] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[24] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- // stage 2
- step2[0] = step1[0];
- step2[1] = step1[1];
- step2[2] = step1[2];
- step2[3] = step1[3];
- step2[4] = step1[4];
- step2[5] = step1[5];
- step2[6] = step1[6];
- step2[7] = step1[7];
-
- temp1 = step1[8] * cospi_30_64 - step1[15] * cospi_2_64;
- temp2 = step1[8] * cospi_2_64 + step1[15] * cospi_30_64;
- step2[8] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[15] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = step1[9] * cospi_14_64 - step1[14] * cospi_18_64;
- temp2 = step1[9] * cospi_18_64 + step1[14] * cospi_14_64;
- step2[9] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[14] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = step1[10] * cospi_22_64 - step1[13] * cospi_10_64;
- temp2 = step1[10] * cospi_10_64 + step1[13] * cospi_22_64;
- step2[10] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[13] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- temp1 = step1[11] * cospi_6_64 - step1[12] * cospi_26_64;
- temp2 = step1[11] * cospi_26_64 + step1[12] * cospi_6_64;
- step2[11] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[12] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- step2[16] = WRAPLOW(step1[16] + step1[17], 8);
- step2[17] = WRAPLOW(step1[16] - step1[17], 8);
- step2[18] = WRAPLOW(-step1[18] + step1[19], 8);
- step2[19] = WRAPLOW(step1[18] + step1[19], 8);
- step2[20] = WRAPLOW(step1[20] + step1[21], 8);
- step2[21] = WRAPLOW(step1[20] - step1[21], 8);
- step2[22] = WRAPLOW(-step1[22] + step1[23], 8);
- step2[23] = WRAPLOW(step1[22] + step1[23], 8);
- step2[24] = WRAPLOW(step1[24] + step1[25], 8);
- step2[25] = WRAPLOW(step1[24] - step1[25], 8);
- step2[26] = WRAPLOW(-step1[26] + step1[27], 8);
- step2[27] = WRAPLOW(step1[26] + step1[27], 8);
- step2[28] = WRAPLOW(step1[28] + step1[29], 8);
- step2[29] = WRAPLOW(step1[28] - step1[29], 8);
- step2[30] = WRAPLOW(-step1[30] + step1[31], 8);
- step2[31] = WRAPLOW(step1[30] + step1[31], 8);
-
- // stage 3
- step1[0] = step2[0];
- step1[1] = step2[1];
- step1[2] = step2[2];
- step1[3] = step2[3];
-
- temp1 = step2[4] * cospi_28_64 - step2[7] * cospi_4_64;
- temp2 = step2[4] * cospi_4_64 + step2[7] * cospi_28_64;
- step1[4] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[7] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = step2[5] * cospi_12_64 - step2[6] * cospi_20_64;
- temp2 = step2[5] * cospi_20_64 + step2[6] * cospi_12_64;
- step1[5] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[6] = WRAPLOW(dct_const_round_shift(temp2), 8);
-
- step1[8] = WRAPLOW(step2[8] + step2[9], 8);
- step1[9] = WRAPLOW(step2[8] - step2[9], 8);
- step1[10] = WRAPLOW(-step2[10] + step2[11], 8);
- step1[11] = WRAPLOW(step2[10] + step2[11], 8);
- step1[12] = WRAPLOW(step2[12] + step2[13], 8);
- step1[13] = WRAPLOW(step2[12] - step2[13], 8);
- step1[14] = WRAPLOW(-step2[14] + step2[15], 8);
- step1[15] = WRAPLOW(step2[14] + step2[15], 8);
-
- step1[16] = step2[16];
- step1[31] = step2[31];
- temp1 = -step2[17] * cospi_4_64 + step2[30] * cospi_28_64;
- temp2 = step2[17] * cospi_28_64 + step2[30] * cospi_4_64;
- step1[17] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[30] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step2[18] * cospi_28_64 - step2[29] * cospi_4_64;
- temp2 = -step2[18] * cospi_4_64 + step2[29] * cospi_28_64;
- step1[18] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[29] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[19] = step2[19];
- step1[20] = step2[20];
- temp1 = -step2[21] * cospi_20_64 + step2[26] * cospi_12_64;
- temp2 = step2[21] * cospi_12_64 + step2[26] * cospi_20_64;
- step1[21] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[26] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step2[22] * cospi_12_64 - step2[25] * cospi_20_64;
- temp2 = -step2[22] * cospi_20_64 + step2[25] * cospi_12_64;
- step1[22] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[25] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[23] = step2[23];
- step1[24] = step2[24];
- step1[27] = step2[27];
- step1[28] = step2[28];
-
- // stage 4
- temp1 = (step1[0] + step1[1]) * cospi_16_64;
- temp2 = (step1[0] - step1[1]) * cospi_16_64;
- step2[0] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[1] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = step1[2] * cospi_24_64 - step1[3] * cospi_8_64;
- temp2 = step1[2] * cospi_8_64 + step1[3] * cospi_24_64;
- step2[2] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[3] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[4] = WRAPLOW(step1[4] + step1[5], 8);
- step2[5] = WRAPLOW(step1[4] - step1[5], 8);
- step2[6] = WRAPLOW(-step1[6] + step1[7], 8);
- step2[7] = WRAPLOW(step1[6] + step1[7], 8);
-
- step2[8] = step1[8];
- step2[15] = step1[15];
- temp1 = -step1[9] * cospi_8_64 + step1[14] * cospi_24_64;
- temp2 = step1[9] * cospi_24_64 + step1[14] * cospi_8_64;
- step2[9] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[14] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step1[10] * cospi_24_64 - step1[13] * cospi_8_64;
- temp2 = -step1[10] * cospi_8_64 + step1[13] * cospi_24_64;
- step2[10] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[13] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[11] = step1[11];
- step2[12] = step1[12];
-
- step2[16] = WRAPLOW(step1[16] + step1[19], 8);
- step2[17] = WRAPLOW(step1[17] + step1[18], 8);
- step2[18] = WRAPLOW(step1[17] - step1[18], 8);
- step2[19] = WRAPLOW(step1[16] - step1[19], 8);
- step2[20] = WRAPLOW(-step1[20] + step1[23], 8);
- step2[21] = WRAPLOW(-step1[21] + step1[22], 8);
- step2[22] = WRAPLOW(step1[21] + step1[22], 8);
- step2[23] = WRAPLOW(step1[20] + step1[23], 8);
-
- step2[24] = WRAPLOW(step1[24] + step1[27], 8);
- step2[25] = WRAPLOW(step1[25] + step1[26], 8);
- step2[26] = WRAPLOW(step1[25] - step1[26], 8);
- step2[27] = WRAPLOW(step1[24] - step1[27], 8);
- step2[28] = WRAPLOW(-step1[28] + step1[31], 8);
- step2[29] = WRAPLOW(-step1[29] + step1[30], 8);
- step2[30] = WRAPLOW(step1[29] + step1[30], 8);
- step2[31] = WRAPLOW(step1[28] + step1[31], 8);
-
- // stage 5
- step1[0] = WRAPLOW(step2[0] + step2[3], 8);
- step1[1] = WRAPLOW(step2[1] + step2[2], 8);
- step1[2] = WRAPLOW(step2[1] - step2[2], 8);
- step1[3] = WRAPLOW(step2[0] - step2[3], 8);
- step1[4] = step2[4];
- temp1 = (step2[6] - step2[5]) * cospi_16_64;
- temp2 = (step2[5] + step2[6]) * cospi_16_64;
- step1[5] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[6] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[7] = step2[7];
-
- step1[8] = WRAPLOW(step2[8] + step2[11], 8);
- step1[9] = WRAPLOW(step2[9] + step2[10], 8);
- step1[10] = WRAPLOW(step2[9] - step2[10], 8);
- step1[11] = WRAPLOW(step2[8] - step2[11], 8);
- step1[12] = WRAPLOW(-step2[12] + step2[15], 8);
- step1[13] = WRAPLOW(-step2[13] + step2[14], 8);
- step1[14] = WRAPLOW(step2[13] + step2[14], 8);
- step1[15] = WRAPLOW(step2[12] + step2[15], 8);
-
- step1[16] = step2[16];
- step1[17] = step2[17];
- temp1 = -step2[18] * cospi_8_64 + step2[29] * cospi_24_64;
- temp2 = step2[18] * cospi_24_64 + step2[29] * cospi_8_64;
- step1[18] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[29] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step2[19] * cospi_8_64 + step2[28] * cospi_24_64;
- temp2 = step2[19] * cospi_24_64 + step2[28] * cospi_8_64;
- step1[19] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[28] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step2[20] * cospi_24_64 - step2[27] * cospi_8_64;
- temp2 = -step2[20] * cospi_8_64 + step2[27] * cospi_24_64;
- step1[20] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[27] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = -step2[21] * cospi_24_64 - step2[26] * cospi_8_64;
- temp2 = -step2[21] * cospi_8_64 + step2[26] * cospi_24_64;
- step1[21] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[26] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[22] = step2[22];
- step1[23] = step2[23];
- step1[24] = step2[24];
- step1[25] = step2[25];
- step1[30] = step2[30];
- step1[31] = step2[31];
-
- // stage 6
- step2[0] = WRAPLOW(step1[0] + step1[7], 8);
- step2[1] = WRAPLOW(step1[1] + step1[6], 8);
- step2[2] = WRAPLOW(step1[2] + step1[5], 8);
- step2[3] = WRAPLOW(step1[3] + step1[4], 8);
- step2[4] = WRAPLOW(step1[3] - step1[4], 8);
- step2[5] = WRAPLOW(step1[2] - step1[5], 8);
- step2[6] = WRAPLOW(step1[1] - step1[6], 8);
- step2[7] = WRAPLOW(step1[0] - step1[7], 8);
- step2[8] = step1[8];
- step2[9] = step1[9];
- temp1 = (-step1[10] + step1[13]) * cospi_16_64;
- temp2 = (step1[10] + step1[13]) * cospi_16_64;
- step2[10] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[13] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = (-step1[11] + step1[12]) * cospi_16_64;
- temp2 = (step1[11] + step1[12]) * cospi_16_64;
- step2[11] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step2[12] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step2[14] = step1[14];
- step2[15] = step1[15];
-
- step2[16] = WRAPLOW(step1[16] + step1[23], 8);
- step2[17] = WRAPLOW(step1[17] + step1[22], 8);
- step2[18] = WRAPLOW(step1[18] + step1[21], 8);
- step2[19] = WRAPLOW(step1[19] + step1[20], 8);
- step2[20] = WRAPLOW(step1[19] - step1[20], 8);
- step2[21] = WRAPLOW(step1[18] - step1[21], 8);
- step2[22] = WRAPLOW(step1[17] - step1[22], 8);
- step2[23] = WRAPLOW(step1[16] - step1[23], 8);
-
- step2[24] = WRAPLOW(-step1[24] + step1[31], 8);
- step2[25] = WRAPLOW(-step1[25] + step1[30], 8);
- step2[26] = WRAPLOW(-step1[26] + step1[29], 8);
- step2[27] = WRAPLOW(-step1[27] + step1[28], 8);
- step2[28] = WRAPLOW(step1[27] + step1[28], 8);
- step2[29] = WRAPLOW(step1[26] + step1[29], 8);
- step2[30] = WRAPLOW(step1[25] + step1[30], 8);
- step2[31] = WRAPLOW(step1[24] + step1[31], 8);
-
- // stage 7
- step1[0] = WRAPLOW(step2[0] + step2[15], 8);
- step1[1] = WRAPLOW(step2[1] + step2[14], 8);
- step1[2] = WRAPLOW(step2[2] + step2[13], 8);
- step1[3] = WRAPLOW(step2[3] + step2[12], 8);
- step1[4] = WRAPLOW(step2[4] + step2[11], 8);
- step1[5] = WRAPLOW(step2[5] + step2[10], 8);
- step1[6] = WRAPLOW(step2[6] + step2[9], 8);
- step1[7] = WRAPLOW(step2[7] + step2[8], 8);
- step1[8] = WRAPLOW(step2[7] - step2[8], 8);
- step1[9] = WRAPLOW(step2[6] - step2[9], 8);
- step1[10] = WRAPLOW(step2[5] - step2[10], 8);
- step1[11] = WRAPLOW(step2[4] - step2[11], 8);
- step1[12] = WRAPLOW(step2[3] - step2[12], 8);
- step1[13] = WRAPLOW(step2[2] - step2[13], 8);
- step1[14] = WRAPLOW(step2[1] - step2[14], 8);
- step1[15] = WRAPLOW(step2[0] - step2[15], 8);
-
- step1[16] = step2[16];
- step1[17] = step2[17];
- step1[18] = step2[18];
- step1[19] = step2[19];
- temp1 = (-step2[20] + step2[27]) * cospi_16_64;
- temp2 = (step2[20] + step2[27]) * cospi_16_64;
- step1[20] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[27] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = (-step2[21] + step2[26]) * cospi_16_64;
- temp2 = (step2[21] + step2[26]) * cospi_16_64;
- step1[21] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[26] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = (-step2[22] + step2[25]) * cospi_16_64;
- temp2 = (step2[22] + step2[25]) * cospi_16_64;
- step1[22] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[25] = WRAPLOW(dct_const_round_shift(temp2), 8);
- temp1 = (-step2[23] + step2[24]) * cospi_16_64;
- temp2 = (step2[23] + step2[24]) * cospi_16_64;
- step1[23] = WRAPLOW(dct_const_round_shift(temp1), 8);
- step1[24] = WRAPLOW(dct_const_round_shift(temp2), 8);
- step1[28] = step2[28];
- step1[29] = step2[29];
- step1[30] = step2[30];
- step1[31] = step2[31];
-
- // final stage
- output[0] = WRAPLOW(step1[0] + step1[31], 8);
- output[1] = WRAPLOW(step1[1] + step1[30], 8);
- output[2] = WRAPLOW(step1[2] + step1[29], 8);
- output[3] = WRAPLOW(step1[3] + step1[28], 8);
- output[4] = WRAPLOW(step1[4] + step1[27], 8);
- output[5] = WRAPLOW(step1[5] + step1[26], 8);
- output[6] = WRAPLOW(step1[6] + step1[25], 8);
- output[7] = WRAPLOW(step1[7] + step1[24], 8);
- output[8] = WRAPLOW(step1[8] + step1[23], 8);
- output[9] = WRAPLOW(step1[9] + step1[22], 8);
- output[10] = WRAPLOW(step1[10] + step1[21], 8);
- output[11] = WRAPLOW(step1[11] + step1[20], 8);
- output[12] = WRAPLOW(step1[12] + step1[19], 8);
- output[13] = WRAPLOW(step1[13] + step1[18], 8);
- output[14] = WRAPLOW(step1[14] + step1[17], 8);
- output[15] = WRAPLOW(step1[15] + step1[16], 8);
- output[16] = WRAPLOW(step1[15] - step1[16], 8);
- output[17] = WRAPLOW(step1[14] - step1[17], 8);
- output[18] = WRAPLOW(step1[13] - step1[18], 8);
- output[19] = WRAPLOW(step1[12] - step1[19], 8);
- output[20] = WRAPLOW(step1[11] - step1[20], 8);
- output[21] = WRAPLOW(step1[10] - step1[21], 8);
- output[22] = WRAPLOW(step1[9] - step1[22], 8);
- output[23] = WRAPLOW(step1[8] - step1[23], 8);
- output[24] = WRAPLOW(step1[7] - step1[24], 8);
- output[25] = WRAPLOW(step1[6] - step1[25], 8);
- output[26] = WRAPLOW(step1[5] - step1[26], 8);
- output[27] = WRAPLOW(step1[4] - step1[27], 8);
- output[28] = WRAPLOW(step1[3] - step1[28], 8);
- output[29] = WRAPLOW(step1[2] - step1[29], 8);
- output[30] = WRAPLOW(step1[1] - step1[30], 8);
- output[31] = WRAPLOW(step1[0] - step1[31], 8);
-}
-
-void vp10_idct32x32_1024_add_c(const tran_low_t *input, uint8_t *dest,
- int stride) {
- tran_low_t out[32 * 32];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[32], temp_out[32];
-
- // Rows
- for (i = 0; i < 32; ++i) {
- int16_t zero_coeff[16];
- for (j = 0; j < 16; ++j)
- zero_coeff[j] = input[2 * j] | input[2 * j + 1];
- for (j = 0; j < 8; ++j)
- zero_coeff[j] = zero_coeff[2 * j] | zero_coeff[2 * j + 1];
- for (j = 0; j < 4; ++j)
- zero_coeff[j] = zero_coeff[2 * j] | zero_coeff[2 * j + 1];
- for (j = 0; j < 2; ++j)
- zero_coeff[j] = zero_coeff[2 * j] | zero_coeff[2 * j + 1];
-
- if (zero_coeff[0] | zero_coeff[1])
- vp10_idct32_c(input, outptr);
- else
- memset(outptr, 0, sizeof(tran_low_t) * 32);
- input += 32;
- outptr += 32;
- }
-
- // Columns
- for (i = 0; i < 32; ++i) {
- for (j = 0; j < 32; ++j)
- temp_in[j] = out[j * 32 + i];
- vp10_idct32_c(temp_in, temp_out);
- for (j = 0; j < 32; ++j) {
- dest[j * stride + i] = clip_pixel_add(dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 6));
- }
- }
-}
-
-void vp10_idct32x32_34_add_c(const tran_low_t *input, uint8_t *dest,
- int stride) {
- tran_low_t out[32 * 32] = {0};
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[32], temp_out[32];
-
- // Rows
- // only upper-left 8x8 has non-zero coeff
- for (i = 0; i < 8; ++i) {
- vp10_idct32_c(input, outptr);
- input += 32;
- outptr += 32;
- }
-
- // Columns
- for (i = 0; i < 32; ++i) {
- for (j = 0; j < 32; ++j)
- temp_in[j] = out[j * 32 + i];
- vp10_idct32_c(temp_in, temp_out);
- for (j = 0; j < 32; ++j) {
- dest[j * stride + i] = clip_pixel_add(
- dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 6));
- }
- }
-}
-
-void vp10_idct32x32_1_add_c(const tran_low_t *input,
- uint8_t *dest,
- int stride) {
- int i, j;
- tran_high_t a1;
-
- tran_low_t out = WRAPLOW(dct_const_round_shift(input[0] * cospi_16_64), 8);
- out = WRAPLOW(dct_const_round_shift(out * cospi_16_64), 8);
- a1 = ROUND_POWER_OF_TWO(out, 6);
-
- for (j = 0; j < 32; ++j) {
- for (i = 0; i < 32; ++i)
- dest[i] = clip_pixel_add(dest[i], a1);
- dest += stride;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_iwht4x4_16_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- /* 4-point reversible, orthonormal inverse Walsh-Hadamard in 3.5 adds,
- 0.5 shifts per pixel. */
- int i;
- tran_low_t output[16];
- tran_high_t a1, b1, c1, d1, e1;
- const tran_low_t *ip = input;
- tran_low_t *op = output;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- for (i = 0; i < 4; i++) {
- a1 = ip[0] >> UNIT_QUANT_SHIFT;
- c1 = ip[1] >> UNIT_QUANT_SHIFT;
- d1 = ip[2] >> UNIT_QUANT_SHIFT;
- b1 = ip[3] >> UNIT_QUANT_SHIFT;
- a1 += c1;
- d1 -= b1;
- e1 = (a1 - d1) >> 1;
- b1 = e1 - b1;
- c1 = e1 - c1;
- a1 -= b1;
- d1 += c1;
- op[0] = WRAPLOW(a1, bd);
- op[1] = WRAPLOW(b1, bd);
- op[2] = WRAPLOW(c1, bd);
- op[3] = WRAPLOW(d1, bd);
- ip += 4;
- op += 4;
- }
-
- ip = output;
- for (i = 0; i < 4; i++) {
- a1 = ip[4 * 0];
- c1 = ip[4 * 1];
- d1 = ip[4 * 2];
- b1 = ip[4 * 3];
- a1 += c1;
- d1 -= b1;
- e1 = (a1 - d1) >> 1;
- b1 = e1 - b1;
- c1 = e1 - c1;
- a1 -= b1;
- d1 += c1;
- dest[stride * 0] = highbd_clip_pixel_add(dest[stride * 0], a1, bd);
- dest[stride * 1] = highbd_clip_pixel_add(dest[stride * 1], b1, bd);
- dest[stride * 2] = highbd_clip_pixel_add(dest[stride * 2], c1, bd);
- dest[stride * 3] = highbd_clip_pixel_add(dest[stride * 3], d1, bd);
-
- ip++;
- dest++;
- }
-}
-
-void vp10_highbd_iwht4x4_1_add_c(const tran_low_t *in, uint8_t *dest8,
- int dest_stride, int bd) {
- int i;
- tran_high_t a1, e1;
- tran_low_t tmp[4];
- const tran_low_t *ip = in;
- tran_low_t *op = tmp;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- (void) bd;
-
- a1 = ip[0] >> UNIT_QUANT_SHIFT;
- e1 = a1 >> 1;
- a1 -= e1;
- op[0] = WRAPLOW(a1, bd);
- op[1] = op[2] = op[3] = WRAPLOW(e1, bd);
-
- ip = tmp;
- for (i = 0; i < 4; i++) {
- e1 = ip[0] >> 1;
- a1 = ip[0] - e1;
- dest[dest_stride * 0] = highbd_clip_pixel_add(
- dest[dest_stride * 0], a1, bd);
- dest[dest_stride * 1] = highbd_clip_pixel_add(
- dest[dest_stride * 1], e1, bd);
- dest[dest_stride * 2] = highbd_clip_pixel_add(
- dest[dest_stride * 2], e1, bd);
- dest[dest_stride * 3] = highbd_clip_pixel_add(
- dest[dest_stride * 3], e1, bd);
- ip++;
- dest++;
- }
-}
-
-void vp10_highbd_idct4_c(const tran_low_t *input, tran_low_t *output, int bd) {
- tran_low_t step[4];
- tran_high_t temp1, temp2;
- (void) bd;
- // stage 1
- temp1 = (input[0] + input[2]) * cospi_16_64;
- temp2 = (input[0] - input[2]) * cospi_16_64;
- step[0] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step[1] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = input[1] * cospi_24_64 - input[3] * cospi_8_64;
- temp2 = input[1] * cospi_8_64 + input[3] * cospi_24_64;
- step[2] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step[3] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- // stage 2
- output[0] = WRAPLOW(step[0] + step[3], bd);
- output[1] = WRAPLOW(step[1] + step[2], bd);
- output[2] = WRAPLOW(step[1] - step[2], bd);
- output[3] = WRAPLOW(step[0] - step[3], bd);
-}
-
-void vp10_highbd_idct4x4_16_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[4 * 4];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[4], temp_out[4];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // Rows
- for (i = 0; i < 4; ++i) {
- vp10_highbd_idct4_c(input, outptr, bd);
- input += 4;
- outptr += 4;
- }
-
- // Columns
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j * 4 + i];
- vp10_highbd_idct4_c(temp_in, temp_out, bd);
- for (j = 0; j < 4; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 4), bd);
- }
- }
-}
-
-void vp10_highbd_idct4x4_1_add_c(const tran_low_t *input, uint8_t *dest8,
- int dest_stride, int bd) {
- int i;
- tran_high_t a1;
- tran_low_t out = WRAPLOW(
- highbd_dct_const_round_shift(input[0] * cospi_16_64, bd), bd);
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- out = WRAPLOW(highbd_dct_const_round_shift(out * cospi_16_64, bd), bd);
- a1 = ROUND_POWER_OF_TWO(out, 4);
-
- for (i = 0; i < 4; i++) {
- dest[0] = highbd_clip_pixel_add(dest[0], a1, bd);
- dest[1] = highbd_clip_pixel_add(dest[1], a1, bd);
- dest[2] = highbd_clip_pixel_add(dest[2], a1, bd);
- dest[3] = highbd_clip_pixel_add(dest[3], a1, bd);
- dest += dest_stride;
- }
-}
-
-void vp10_highbd_idct8_c(const tran_low_t *input, tran_low_t *output, int bd) {
- tran_low_t step1[8], step2[8];
- tran_high_t temp1, temp2;
- // stage 1
- step1[0] = input[0];
- step1[2] = input[4];
- step1[1] = input[2];
- step1[3] = input[6];
- temp1 = input[1] * cospi_28_64 - input[7] * cospi_4_64;
- temp2 = input[1] * cospi_4_64 + input[7] * cospi_28_64;
- step1[4] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[7] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = input[5] * cospi_12_64 - input[3] * cospi_20_64;
- temp2 = input[5] * cospi_20_64 + input[3] * cospi_12_64;
- step1[5] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[6] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- // stage 2 & stage 3 - even half
- vp10_highbd_idct4_c(step1, step1, bd);
-
- // stage 2 - odd half
- step2[4] = WRAPLOW(step1[4] + step1[5], bd);
- step2[5] = WRAPLOW(step1[4] - step1[5], bd);
- step2[6] = WRAPLOW(-step1[6] + step1[7], bd);
- step2[7] = WRAPLOW(step1[6] + step1[7], bd);
-
- // stage 3 - odd half
- step1[4] = step2[4];
- temp1 = (step2[6] - step2[5]) * cospi_16_64;
- temp2 = (step2[5] + step2[6]) * cospi_16_64;
- step1[5] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[6] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[7] = step2[7];
-
- // stage 4
- output[0] = WRAPLOW(step1[0] + step1[7], bd);
- output[1] = WRAPLOW(step1[1] + step1[6], bd);
- output[2] = WRAPLOW(step1[2] + step1[5], bd);
- output[3] = WRAPLOW(step1[3] + step1[4], bd);
- output[4] = WRAPLOW(step1[3] - step1[4], bd);
- output[5] = WRAPLOW(step1[2] - step1[5], bd);
- output[6] = WRAPLOW(step1[1] - step1[6], bd);
- output[7] = WRAPLOW(step1[0] - step1[7], bd);
-}
-
-void vp10_highbd_idct8x8_64_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[8 * 8];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[8], temp_out[8];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // First transform rows.
- for (i = 0; i < 8; ++i) {
- vp10_highbd_idct8_c(input, outptr, bd);
- input += 8;
- outptr += 8;
- }
-
- // Then transform columns.
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- vp10_highbd_idct8_c(temp_in, temp_out, bd);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 5), bd);
- }
- }
-}
-
-void vp10_highbd_idct8x8_1_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- int i, j;
- tran_high_t a1;
- tran_low_t out = WRAPLOW(
- highbd_dct_const_round_shift(input[0] * cospi_16_64, bd), bd);
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- out = WRAPLOW(highbd_dct_const_round_shift(out * cospi_16_64, bd), bd);
- a1 = ROUND_POWER_OF_TWO(out, 5);
- for (j = 0; j < 8; ++j) {
- for (i = 0; i < 8; ++i)
- dest[i] = highbd_clip_pixel_add(dest[i], a1, bd);
- dest += stride;
- }
-}
-
-void vp10_highbd_iadst4_c(const tran_low_t *input, tran_low_t *output, int bd) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7;
-
- tran_low_t x0 = input[0];
- tran_low_t x1 = input[1];
- tran_low_t x2 = input[2];
- tran_low_t x3 = input[3];
- (void) bd;
-
- if (!(x0 | x1 | x2 | x3)) {
- memset(output, 0, 4 * sizeof(*output));
- return;
- }
-
- s0 = sinpi_1_9 * x0;
- s1 = sinpi_2_9 * x0;
- s2 = sinpi_3_9 * x1;
- s3 = sinpi_4_9 * x2;
- s4 = sinpi_1_9 * x2;
- s5 = sinpi_2_9 * x3;
- s6 = sinpi_4_9 * x3;
- s7 = (tran_high_t)(x0 - x2 + x3);
-
- s0 = s0 + s3 + s5;
- s1 = s1 - s4 - s6;
- s3 = s2;
- s2 = sinpi_3_9 * s7;
-
- // 1-D transform scaling factor is sqrt(2).
- // The overall dynamic range is 14b (input) + 14b (multiplication scaling)
- // + 1b (addition) = 29b.
- // Hence the output bit depth is 15b.
- output[0] = WRAPLOW(highbd_dct_const_round_shift(s0 + s3, bd), bd);
- output[1] = WRAPLOW(highbd_dct_const_round_shift(s1 + s3, bd), bd);
- output[2] = WRAPLOW(highbd_dct_const_round_shift(s2, bd), bd);
- output[3] = WRAPLOW(highbd_dct_const_round_shift(s0 + s1 - s3, bd), bd);
-}
-
-void vp10_highbd_iadst8_c(const tran_low_t *input, tran_low_t *output, int bd) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7;
-
- tran_low_t x0 = input[7];
- tran_low_t x1 = input[0];
- tran_low_t x2 = input[5];
- tran_low_t x3 = input[2];
- tran_low_t x4 = input[3];
- tran_low_t x5 = input[4];
- tran_low_t x6 = input[1];
- tran_low_t x7 = input[6];
- (void) bd;
-
- if (!(x0 | x1 | x2 | x3 | x4 | x5 | x6 | x7)) {
- memset(output, 0, 8 * sizeof(*output));
- return;
- }
-
- // stage 1
- s0 = cospi_2_64 * x0 + cospi_30_64 * x1;
- s1 = cospi_30_64 * x0 - cospi_2_64 * x1;
- s2 = cospi_10_64 * x2 + cospi_22_64 * x3;
- s3 = cospi_22_64 * x2 - cospi_10_64 * x3;
- s4 = cospi_18_64 * x4 + cospi_14_64 * x5;
- s5 = cospi_14_64 * x4 - cospi_18_64 * x5;
- s6 = cospi_26_64 * x6 + cospi_6_64 * x7;
- s7 = cospi_6_64 * x6 - cospi_26_64 * x7;
-
- x0 = WRAPLOW(highbd_dct_const_round_shift(s0 + s4, bd), bd);
- x1 = WRAPLOW(highbd_dct_const_round_shift(s1 + s5, bd), bd);
- x2 = WRAPLOW(highbd_dct_const_round_shift(s2 + s6, bd), bd);
- x3 = WRAPLOW(highbd_dct_const_round_shift(s3 + s7, bd), bd);
- x4 = WRAPLOW(highbd_dct_const_round_shift(s0 - s4, bd), bd);
- x5 = WRAPLOW(highbd_dct_const_round_shift(s1 - s5, bd), bd);
- x6 = WRAPLOW(highbd_dct_const_round_shift(s2 - s6, bd), bd);
- x7 = WRAPLOW(highbd_dct_const_round_shift(s3 - s7, bd), bd);
-
- // stage 2
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = cospi_8_64 * x4 + cospi_24_64 * x5;
- s5 = cospi_24_64 * x4 - cospi_8_64 * x5;
- s6 = -cospi_24_64 * x6 + cospi_8_64 * x7;
- s7 = cospi_8_64 * x6 + cospi_24_64 * x7;
-
- x0 = WRAPLOW(s0 + s2, bd);
- x1 = WRAPLOW(s1 + s3, bd);
- x2 = WRAPLOW(s0 - s2, bd);
- x3 = WRAPLOW(s1 - s3, bd);
- x4 = WRAPLOW(highbd_dct_const_round_shift(s4 + s6, bd), bd);
- x5 = WRAPLOW(highbd_dct_const_round_shift(s5 + s7, bd), bd);
- x6 = WRAPLOW(highbd_dct_const_round_shift(s4 - s6, bd), bd);
- x7 = WRAPLOW(highbd_dct_const_round_shift(s5 - s7, bd), bd);
-
- // stage 3
- s2 = cospi_16_64 * (x2 + x3);
- s3 = cospi_16_64 * (x2 - x3);
- s6 = cospi_16_64 * (x6 + x7);
- s7 = cospi_16_64 * (x6 - x7);
-
- x2 = WRAPLOW(highbd_dct_const_round_shift(s2, bd), bd);
- x3 = WRAPLOW(highbd_dct_const_round_shift(s3, bd), bd);
- x6 = WRAPLOW(highbd_dct_const_round_shift(s6, bd), bd);
- x7 = WRAPLOW(highbd_dct_const_round_shift(s7, bd), bd);
-
- output[0] = WRAPLOW(x0, bd);
- output[1] = WRAPLOW(-x4, bd);
- output[2] = WRAPLOW(x6, bd);
- output[3] = WRAPLOW(-x2, bd);
- output[4] = WRAPLOW(x3, bd);
- output[5] = WRAPLOW(-x7, bd);
- output[6] = WRAPLOW(x5, bd);
- output[7] = WRAPLOW(-x1, bd);
-}
-
-void vp10_highbd_idct8x8_10_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[8 * 8] = { 0 };
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[8], temp_out[8];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // First transform rows.
- // Only first 4 row has non-zero coefs.
- for (i = 0; i < 4; ++i) {
- vp10_highbd_idct8_c(input, outptr, bd);
- input += 8;
- outptr += 8;
- }
- // Then transform columns.
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- vp10_highbd_idct8_c(temp_in, temp_out, bd);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 5), bd);
- }
- }
-}
-
-void vp10_highbd_idct16_c(const tran_low_t *input, tran_low_t *output, int bd) {
- tran_low_t step1[16], step2[16];
- tran_high_t temp1, temp2;
- (void) bd;
-
- // stage 1
- step1[0] = input[0/2];
- step1[1] = input[16/2];
- step1[2] = input[8/2];
- step1[3] = input[24/2];
- step1[4] = input[4/2];
- step1[5] = input[20/2];
- step1[6] = input[12/2];
- step1[7] = input[28/2];
- step1[8] = input[2/2];
- step1[9] = input[18/2];
- step1[10] = input[10/2];
- step1[11] = input[26/2];
- step1[12] = input[6/2];
- step1[13] = input[22/2];
- step1[14] = input[14/2];
- step1[15] = input[30/2];
-
- // stage 2
- step2[0] = step1[0];
- step2[1] = step1[1];
- step2[2] = step1[2];
- step2[3] = step1[3];
- step2[4] = step1[4];
- step2[5] = step1[5];
- step2[6] = step1[6];
- step2[7] = step1[7];
-
- temp1 = step1[8] * cospi_30_64 - step1[15] * cospi_2_64;
- temp2 = step1[8] * cospi_2_64 + step1[15] * cospi_30_64;
- step2[8] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[15] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = step1[9] * cospi_14_64 - step1[14] * cospi_18_64;
- temp2 = step1[9] * cospi_18_64 + step1[14] * cospi_14_64;
- step2[9] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[14] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = step1[10] * cospi_22_64 - step1[13] * cospi_10_64;
- temp2 = step1[10] * cospi_10_64 + step1[13] * cospi_22_64;
- step2[10] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[13] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = step1[11] * cospi_6_64 - step1[12] * cospi_26_64;
- temp2 = step1[11] * cospi_26_64 + step1[12] * cospi_6_64;
- step2[11] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[12] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- // stage 3
- step1[0] = step2[0];
- step1[1] = step2[1];
- step1[2] = step2[2];
- step1[3] = step2[3];
-
- temp1 = step2[4] * cospi_28_64 - step2[7] * cospi_4_64;
- temp2 = step2[4] * cospi_4_64 + step2[7] * cospi_28_64;
- step1[4] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[7] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = step2[5] * cospi_12_64 - step2[6] * cospi_20_64;
- temp2 = step2[5] * cospi_20_64 + step2[6] * cospi_12_64;
- step1[5] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[6] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- step1[8] = WRAPLOW(step2[8] + step2[9], bd);
- step1[9] = WRAPLOW(step2[8] - step2[9], bd);
- step1[10] = WRAPLOW(-step2[10] + step2[11], bd);
- step1[11] = WRAPLOW(step2[10] + step2[11], bd);
- step1[12] = WRAPLOW(step2[12] + step2[13], bd);
- step1[13] = WRAPLOW(step2[12] - step2[13], bd);
- step1[14] = WRAPLOW(-step2[14] + step2[15], bd);
- step1[15] = WRAPLOW(step2[14] + step2[15], bd);
-
- // stage 4
- temp1 = (step1[0] + step1[1]) * cospi_16_64;
- temp2 = (step1[0] - step1[1]) * cospi_16_64;
- step2[0] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[1] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = step1[2] * cospi_24_64 - step1[3] * cospi_8_64;
- temp2 = step1[2] * cospi_8_64 + step1[3] * cospi_24_64;
- step2[2] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[3] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step2[4] = WRAPLOW(step1[4] + step1[5], bd);
- step2[5] = WRAPLOW(step1[4] - step1[5], bd);
- step2[6] = WRAPLOW(-step1[6] + step1[7], bd);
- step2[7] = WRAPLOW(step1[6] + step1[7], bd);
-
- step2[8] = step1[8];
- step2[15] = step1[15];
- temp1 = -step1[9] * cospi_8_64 + step1[14] * cospi_24_64;
- temp2 = step1[9] * cospi_24_64 + step1[14] * cospi_8_64;
- step2[9] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[14] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step1[10] * cospi_24_64 - step1[13] * cospi_8_64;
- temp2 = -step1[10] * cospi_8_64 + step1[13] * cospi_24_64;
- step2[10] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[13] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step2[11] = step1[11];
- step2[12] = step1[12];
-
- // stage 5
- step1[0] = WRAPLOW(step2[0] + step2[3], bd);
- step1[1] = WRAPLOW(step2[1] + step2[2], bd);
- step1[2] = WRAPLOW(step2[1] - step2[2], bd);
- step1[3] = WRAPLOW(step2[0] - step2[3], bd);
- step1[4] = step2[4];
- temp1 = (step2[6] - step2[5]) * cospi_16_64;
- temp2 = (step2[5] + step2[6]) * cospi_16_64;
- step1[5] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[6] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[7] = step2[7];
-
- step1[8] = WRAPLOW(step2[8] + step2[11], bd);
- step1[9] = WRAPLOW(step2[9] + step2[10], bd);
- step1[10] = WRAPLOW(step2[9] - step2[10], bd);
- step1[11] = WRAPLOW(step2[8] - step2[11], bd);
- step1[12] = WRAPLOW(-step2[12] + step2[15], bd);
- step1[13] = WRAPLOW(-step2[13] + step2[14], bd);
- step1[14] = WRAPLOW(step2[13] + step2[14], bd);
- step1[15] = WRAPLOW(step2[12] + step2[15], bd);
-
- // stage 6
- step2[0] = WRAPLOW(step1[0] + step1[7], bd);
- step2[1] = WRAPLOW(step1[1] + step1[6], bd);
- step2[2] = WRAPLOW(step1[2] + step1[5], bd);
- step2[3] = WRAPLOW(step1[3] + step1[4], bd);
- step2[4] = WRAPLOW(step1[3] - step1[4], bd);
- step2[5] = WRAPLOW(step1[2] - step1[5], bd);
- step2[6] = WRAPLOW(step1[1] - step1[6], bd);
- step2[7] = WRAPLOW(step1[0] - step1[7], bd);
- step2[8] = step1[8];
- step2[9] = step1[9];
- temp1 = (-step1[10] + step1[13]) * cospi_16_64;
- temp2 = (step1[10] + step1[13]) * cospi_16_64;
- step2[10] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[13] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = (-step1[11] + step1[12]) * cospi_16_64;
- temp2 = (step1[11] + step1[12]) * cospi_16_64;
- step2[11] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[12] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step2[14] = step1[14];
- step2[15] = step1[15];
-
- // stage 7
- output[0] = WRAPLOW(step2[0] + step2[15], bd);
- output[1] = WRAPLOW(step2[1] + step2[14], bd);
- output[2] = WRAPLOW(step2[2] + step2[13], bd);
- output[3] = WRAPLOW(step2[3] + step2[12], bd);
- output[4] = WRAPLOW(step2[4] + step2[11], bd);
- output[5] = WRAPLOW(step2[5] + step2[10], bd);
- output[6] = WRAPLOW(step2[6] + step2[9], bd);
- output[7] = WRAPLOW(step2[7] + step2[8], bd);
- output[8] = WRAPLOW(step2[7] - step2[8], bd);
- output[9] = WRAPLOW(step2[6] - step2[9], bd);
- output[10] = WRAPLOW(step2[5] - step2[10], bd);
- output[11] = WRAPLOW(step2[4] - step2[11], bd);
- output[12] = WRAPLOW(step2[3] - step2[12], bd);
- output[13] = WRAPLOW(step2[2] - step2[13], bd);
- output[14] = WRAPLOW(step2[1] - step2[14], bd);
- output[15] = WRAPLOW(step2[0] - step2[15], bd);
-}
-
-void vp10_highbd_idct16x16_256_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[16 * 16];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[16], temp_out[16];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // First transform rows.
- for (i = 0; i < 16; ++i) {
- vp10_highbd_idct16_c(input, outptr, bd);
- input += 16;
- outptr += 16;
- }
-
- // Then transform columns.
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- vp10_highbd_idct16_c(temp_in, temp_out, bd);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i],
- ROUND_POWER_OF_TWO(temp_out[j], 6),
- bd);
- }
- }
-}
-
-void vp10_highbd_iadst16_c(const tran_low_t *input,
- tran_low_t *output,
- int bd) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7, s8;
- tran_high_t s9, s10, s11, s12, s13, s14, s15;
-
- tran_low_t x0 = input[15];
- tran_low_t x1 = input[0];
- tran_low_t x2 = input[13];
- tran_low_t x3 = input[2];
- tran_low_t x4 = input[11];
- tran_low_t x5 = input[4];
- tran_low_t x6 = input[9];
- tran_low_t x7 = input[6];
- tran_low_t x8 = input[7];
- tran_low_t x9 = input[8];
- tran_low_t x10 = input[5];
- tran_low_t x11 = input[10];
- tran_low_t x12 = input[3];
- tran_low_t x13 = input[12];
- tran_low_t x14 = input[1];
- tran_low_t x15 = input[14];
- (void) bd;
-
- if (!(x0 | x1 | x2 | x3 | x4 | x5 | x6 | x7 | x8
- | x9 | x10 | x11 | x12 | x13 | x14 | x15)) {
- memset(output, 0, 16 * sizeof(*output));
- return;
- }
-
- // stage 1
- s0 = x0 * cospi_1_64 + x1 * cospi_31_64;
- s1 = x0 * cospi_31_64 - x1 * cospi_1_64;
- s2 = x2 * cospi_5_64 + x3 * cospi_27_64;
- s3 = x2 * cospi_27_64 - x3 * cospi_5_64;
- s4 = x4 * cospi_9_64 + x5 * cospi_23_64;
- s5 = x4 * cospi_23_64 - x5 * cospi_9_64;
- s6 = x6 * cospi_13_64 + x7 * cospi_19_64;
- s7 = x6 * cospi_19_64 - x7 * cospi_13_64;
- s8 = x8 * cospi_17_64 + x9 * cospi_15_64;
- s9 = x8 * cospi_15_64 - x9 * cospi_17_64;
- s10 = x10 * cospi_21_64 + x11 * cospi_11_64;
- s11 = x10 * cospi_11_64 - x11 * cospi_21_64;
- s12 = x12 * cospi_25_64 + x13 * cospi_7_64;
- s13 = x12 * cospi_7_64 - x13 * cospi_25_64;
- s14 = x14 * cospi_29_64 + x15 * cospi_3_64;
- s15 = x14 * cospi_3_64 - x15 * cospi_29_64;
-
- x0 = WRAPLOW(highbd_dct_const_round_shift(s0 + s8, bd), bd);
- x1 = WRAPLOW(highbd_dct_const_round_shift(s1 + s9, bd), bd);
- x2 = WRAPLOW(highbd_dct_const_round_shift(s2 + s10, bd), bd);
- x3 = WRAPLOW(highbd_dct_const_round_shift(s3 + s11, bd), bd);
- x4 = WRAPLOW(highbd_dct_const_round_shift(s4 + s12, bd), bd);
- x5 = WRAPLOW(highbd_dct_const_round_shift(s5 + s13, bd), bd);
- x6 = WRAPLOW(highbd_dct_const_round_shift(s6 + s14, bd), bd);
- x7 = WRAPLOW(highbd_dct_const_round_shift(s7 + s15, bd), bd);
- x8 = WRAPLOW(highbd_dct_const_round_shift(s0 - s8, bd), bd);
- x9 = WRAPLOW(highbd_dct_const_round_shift(s1 - s9, bd), bd);
- x10 = WRAPLOW(highbd_dct_const_round_shift(s2 - s10, bd), bd);
- x11 = WRAPLOW(highbd_dct_const_round_shift(s3 - s11, bd), bd);
- x12 = WRAPLOW(highbd_dct_const_round_shift(s4 - s12, bd), bd);
- x13 = WRAPLOW(highbd_dct_const_round_shift(s5 - s13, bd), bd);
- x14 = WRAPLOW(highbd_dct_const_round_shift(s6 - s14, bd), bd);
- x15 = WRAPLOW(highbd_dct_const_round_shift(s7 - s15, bd), bd);
-
- // stage 2
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = x4;
- s5 = x5;
- s6 = x6;
- s7 = x7;
- s8 = x8 * cospi_4_64 + x9 * cospi_28_64;
- s9 = x8 * cospi_28_64 - x9 * cospi_4_64;
- s10 = x10 * cospi_20_64 + x11 * cospi_12_64;
- s11 = x10 * cospi_12_64 - x11 * cospi_20_64;
- s12 = -x12 * cospi_28_64 + x13 * cospi_4_64;
- s13 = x12 * cospi_4_64 + x13 * cospi_28_64;
- s14 = -x14 * cospi_12_64 + x15 * cospi_20_64;
- s15 = x14 * cospi_20_64 + x15 * cospi_12_64;
-
- x0 = WRAPLOW(s0 + s4, bd);
- x1 = WRAPLOW(s1 + s5, bd);
- x2 = WRAPLOW(s2 + s6, bd);
- x3 = WRAPLOW(s3 + s7, bd);
- x4 = WRAPLOW(s0 - s4, bd);
- x5 = WRAPLOW(s1 - s5, bd);
- x6 = WRAPLOW(s2 - s6, bd);
- x7 = WRAPLOW(s3 - s7, bd);
- x8 = WRAPLOW(highbd_dct_const_round_shift(s8 + s12, bd), bd);
- x9 = WRAPLOW(highbd_dct_const_round_shift(s9 + s13, bd), bd);
- x10 = WRAPLOW(highbd_dct_const_round_shift(s10 + s14, bd), bd);
- x11 = WRAPLOW(highbd_dct_const_round_shift(s11 + s15, bd), bd);
- x12 = WRAPLOW(highbd_dct_const_round_shift(s8 - s12, bd), bd);
- x13 = WRAPLOW(highbd_dct_const_round_shift(s9 - s13, bd), bd);
- x14 = WRAPLOW(highbd_dct_const_round_shift(s10 - s14, bd), bd);
- x15 = WRAPLOW(highbd_dct_const_round_shift(s11 - s15, bd), bd);
-
- // stage 3
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = x4 * cospi_8_64 + x5 * cospi_24_64;
- s5 = x4 * cospi_24_64 - x5 * cospi_8_64;
- s6 = -x6 * cospi_24_64 + x7 * cospi_8_64;
- s7 = x6 * cospi_8_64 + x7 * cospi_24_64;
- s8 = x8;
- s9 = x9;
- s10 = x10;
- s11 = x11;
- s12 = x12 * cospi_8_64 + x13 * cospi_24_64;
- s13 = x12 * cospi_24_64 - x13 * cospi_8_64;
- s14 = -x14 * cospi_24_64 + x15 * cospi_8_64;
- s15 = x14 * cospi_8_64 + x15 * cospi_24_64;
-
- x0 = WRAPLOW(s0 + s2, bd);
- x1 = WRAPLOW(s1 + s3, bd);
- x2 = WRAPLOW(s0 - s2, bd);
- x3 = WRAPLOW(s1 - s3, bd);
- x4 = WRAPLOW(highbd_dct_const_round_shift(s4 + s6, bd), bd);
- x5 = WRAPLOW(highbd_dct_const_round_shift(s5 + s7, bd), bd);
- x6 = WRAPLOW(highbd_dct_const_round_shift(s4 - s6, bd), bd);
- x7 = WRAPLOW(highbd_dct_const_round_shift(s5 - s7, bd), bd);
- x8 = WRAPLOW(s8 + s10, bd);
- x9 = WRAPLOW(s9 + s11, bd);
- x10 = WRAPLOW(s8 - s10, bd);
- x11 = WRAPLOW(s9 - s11, bd);
- x12 = WRAPLOW(highbd_dct_const_round_shift(s12 + s14, bd), bd);
- x13 = WRAPLOW(highbd_dct_const_round_shift(s13 + s15, bd), bd);
- x14 = WRAPLOW(highbd_dct_const_round_shift(s12 - s14, bd), bd);
- x15 = WRAPLOW(highbd_dct_const_round_shift(s13 - s15, bd), bd);
-
- // stage 4
- s2 = (- cospi_16_64) * (x2 + x3);
- s3 = cospi_16_64 * (x2 - x3);
- s6 = cospi_16_64 * (x6 + x7);
- s7 = cospi_16_64 * (-x6 + x7);
- s10 = cospi_16_64 * (x10 + x11);
- s11 = cospi_16_64 * (-x10 + x11);
- s14 = (- cospi_16_64) * (x14 + x15);
- s15 = cospi_16_64 * (x14 - x15);
-
- x2 = WRAPLOW(highbd_dct_const_round_shift(s2, bd), bd);
- x3 = WRAPLOW(highbd_dct_const_round_shift(s3, bd), bd);
- x6 = WRAPLOW(highbd_dct_const_round_shift(s6, bd), bd);
- x7 = WRAPLOW(highbd_dct_const_round_shift(s7, bd), bd);
- x10 = WRAPLOW(highbd_dct_const_round_shift(s10, bd), bd);
- x11 = WRAPLOW(highbd_dct_const_round_shift(s11, bd), bd);
- x14 = WRAPLOW(highbd_dct_const_round_shift(s14, bd), bd);
- x15 = WRAPLOW(highbd_dct_const_round_shift(s15, bd), bd);
-
- output[0] = WRAPLOW(x0, bd);
- output[1] = WRAPLOW(-x8, bd);
- output[2] = WRAPLOW(x12, bd);
- output[3] = WRAPLOW(-x4, bd);
- output[4] = WRAPLOW(x6, bd);
- output[5] = WRAPLOW(x14, bd);
- output[6] = WRAPLOW(x10, bd);
- output[7] = WRAPLOW(x2, bd);
- output[8] = WRAPLOW(x3, bd);
- output[9] = WRAPLOW(x11, bd);
- output[10] = WRAPLOW(x15, bd);
- output[11] = WRAPLOW(x7, bd);
- output[12] = WRAPLOW(x5, bd);
- output[13] = WRAPLOW(-x13, bd);
- output[14] = WRAPLOW(x9, bd);
- output[15] = WRAPLOW(-x1, bd);
-}
-
-void vp10_highbd_idct16x16_10_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[16 * 16] = { 0 };
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[16], temp_out[16];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // First transform rows. Since all non-zero dct coefficients are in
- // upper-left 4x4 area, we only need to calculate first 4 rows here.
- for (i = 0; i < 4; ++i) {
- vp10_highbd_idct16_c(input, outptr, bd);
- input += 16;
- outptr += 16;
- }
-
- // Then transform columns.
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j*16 + i];
- vp10_highbd_idct16_c(temp_in, temp_out, bd);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 6), bd);
- }
- }
-}
-
-void vp10_highbd_idct16x16_1_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- int i, j;
- tran_high_t a1;
- tran_low_t out = WRAPLOW(
- highbd_dct_const_round_shift(input[0] * cospi_16_64, bd), bd);
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- out = WRAPLOW(highbd_dct_const_round_shift(out * cospi_16_64, bd), bd);
- a1 = ROUND_POWER_OF_TWO(out, 6);
- for (j = 0; j < 16; ++j) {
- for (i = 0; i < 16; ++i)
- dest[i] = highbd_clip_pixel_add(dest[i], a1, bd);
- dest += stride;
- }
-}
-
-static void highbd_idct32_c(const tran_low_t *input,
- tran_low_t *output, int bd) {
- tran_low_t step1[32], step2[32];
- tran_high_t temp1, temp2;
- (void) bd;
-
- // stage 1
- step1[0] = input[0];
- step1[1] = input[16];
- step1[2] = input[8];
- step1[3] = input[24];
- step1[4] = input[4];
- step1[5] = input[20];
- step1[6] = input[12];
- step1[7] = input[28];
- step1[8] = input[2];
- step1[9] = input[18];
- step1[10] = input[10];
- step1[11] = input[26];
- step1[12] = input[6];
- step1[13] = input[22];
- step1[14] = input[14];
- step1[15] = input[30];
-
- temp1 = input[1] * cospi_31_64 - input[31] * cospi_1_64;
- temp2 = input[1] * cospi_1_64 + input[31] * cospi_31_64;
- step1[16] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[31] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[17] * cospi_15_64 - input[15] * cospi_17_64;
- temp2 = input[17] * cospi_17_64 + input[15] * cospi_15_64;
- step1[17] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[30] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[9] * cospi_23_64 - input[23] * cospi_9_64;
- temp2 = input[9] * cospi_9_64 + input[23] * cospi_23_64;
- step1[18] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[29] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[25] * cospi_7_64 - input[7] * cospi_25_64;
- temp2 = input[25] * cospi_25_64 + input[7] * cospi_7_64;
- step1[19] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[28] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[5] * cospi_27_64 - input[27] * cospi_5_64;
- temp2 = input[5] * cospi_5_64 + input[27] * cospi_27_64;
- step1[20] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[27] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[21] * cospi_11_64 - input[11] * cospi_21_64;
- temp2 = input[21] * cospi_21_64 + input[11] * cospi_11_64;
- step1[21] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[26] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[13] * cospi_19_64 - input[19] * cospi_13_64;
- temp2 = input[13] * cospi_13_64 + input[19] * cospi_19_64;
- step1[22] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[25] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = input[29] * cospi_3_64 - input[3] * cospi_29_64;
- temp2 = input[29] * cospi_29_64 + input[3] * cospi_3_64;
- step1[23] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[24] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- // stage 2
- step2[0] = step1[0];
- step2[1] = step1[1];
- step2[2] = step1[2];
- step2[3] = step1[3];
- step2[4] = step1[4];
- step2[5] = step1[5];
- step2[6] = step1[6];
- step2[7] = step1[7];
-
- temp1 = step1[8] * cospi_30_64 - step1[15] * cospi_2_64;
- temp2 = step1[8] * cospi_2_64 + step1[15] * cospi_30_64;
- step2[8] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[15] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = step1[9] * cospi_14_64 - step1[14] * cospi_18_64;
- temp2 = step1[9] * cospi_18_64 + step1[14] * cospi_14_64;
- step2[9] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[14] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = step1[10] * cospi_22_64 - step1[13] * cospi_10_64;
- temp2 = step1[10] * cospi_10_64 + step1[13] * cospi_22_64;
- step2[10] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[13] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- temp1 = step1[11] * cospi_6_64 - step1[12] * cospi_26_64;
- temp2 = step1[11] * cospi_26_64 + step1[12] * cospi_6_64;
- step2[11] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[12] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- step2[16] = WRAPLOW(step1[16] + step1[17], bd);
- step2[17] = WRAPLOW(step1[16] - step1[17], bd);
- step2[18] = WRAPLOW(-step1[18] + step1[19], bd);
- step2[19] = WRAPLOW(step1[18] + step1[19], bd);
- step2[20] = WRAPLOW(step1[20] + step1[21], bd);
- step2[21] = WRAPLOW(step1[20] - step1[21], bd);
- step2[22] = WRAPLOW(-step1[22] + step1[23], bd);
- step2[23] = WRAPLOW(step1[22] + step1[23], bd);
- step2[24] = WRAPLOW(step1[24] + step1[25], bd);
- step2[25] = WRAPLOW(step1[24] - step1[25], bd);
- step2[26] = WRAPLOW(-step1[26] + step1[27], bd);
- step2[27] = WRAPLOW(step1[26] + step1[27], bd);
- step2[28] = WRAPLOW(step1[28] + step1[29], bd);
- step2[29] = WRAPLOW(step1[28] - step1[29], bd);
- step2[30] = WRAPLOW(-step1[30] + step1[31], bd);
- step2[31] = WRAPLOW(step1[30] + step1[31], bd);
-
- // stage 3
- step1[0] = step2[0];
- step1[1] = step2[1];
- step1[2] = step2[2];
- step1[3] = step2[3];
-
- temp1 = step2[4] * cospi_28_64 - step2[7] * cospi_4_64;
- temp2 = step2[4] * cospi_4_64 + step2[7] * cospi_28_64;
- step1[4] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[7] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = step2[5] * cospi_12_64 - step2[6] * cospi_20_64;
- temp2 = step2[5] * cospi_20_64 + step2[6] * cospi_12_64;
- step1[5] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[6] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
-
- step1[8] = WRAPLOW(step2[8] + step2[9], bd);
- step1[9] = WRAPLOW(step2[8] - step2[9], bd);
- step1[10] = WRAPLOW(-step2[10] + step2[11], bd);
- step1[11] = WRAPLOW(step2[10] + step2[11], bd);
- step1[12] = WRAPLOW(step2[12] + step2[13], bd);
- step1[13] = WRAPLOW(step2[12] - step2[13], bd);
- step1[14] = WRAPLOW(-step2[14] + step2[15], bd);
- step1[15] = WRAPLOW(step2[14] + step2[15], bd);
-
- step1[16] = step2[16];
- step1[31] = step2[31];
- temp1 = -step2[17] * cospi_4_64 + step2[30] * cospi_28_64;
- temp2 = step2[17] * cospi_28_64 + step2[30] * cospi_4_64;
- step1[17] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[30] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step2[18] * cospi_28_64 - step2[29] * cospi_4_64;
- temp2 = -step2[18] * cospi_4_64 + step2[29] * cospi_28_64;
- step1[18] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[29] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[19] = step2[19];
- step1[20] = step2[20];
- temp1 = -step2[21] * cospi_20_64 + step2[26] * cospi_12_64;
- temp2 = step2[21] * cospi_12_64 + step2[26] * cospi_20_64;
- step1[21] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[26] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step2[22] * cospi_12_64 - step2[25] * cospi_20_64;
- temp2 = -step2[22] * cospi_20_64 + step2[25] * cospi_12_64;
- step1[22] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[25] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[23] = step2[23];
- step1[24] = step2[24];
- step1[27] = step2[27];
- step1[28] = step2[28];
-
- // stage 4
- temp1 = (step1[0] + step1[1]) * cospi_16_64;
- temp2 = (step1[0] - step1[1]) * cospi_16_64;
- step2[0] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[1] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = step1[2] * cospi_24_64 - step1[3] * cospi_8_64;
- temp2 = step1[2] * cospi_8_64 + step1[3] * cospi_24_64;
- step2[2] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[3] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step2[4] = WRAPLOW(step1[4] + step1[5], bd);
- step2[5] = WRAPLOW(step1[4] - step1[5], bd);
- step2[6] = WRAPLOW(-step1[6] + step1[7], bd);
- step2[7] = WRAPLOW(step1[6] + step1[7], bd);
-
- step2[8] = step1[8];
- step2[15] = step1[15];
- temp1 = -step1[9] * cospi_8_64 + step1[14] * cospi_24_64;
- temp2 = step1[9] * cospi_24_64 + step1[14] * cospi_8_64;
- step2[9] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[14] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step1[10] * cospi_24_64 - step1[13] * cospi_8_64;
- temp2 = -step1[10] * cospi_8_64 + step1[13] * cospi_24_64;
- step2[10] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[13] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step2[11] = step1[11];
- step2[12] = step1[12];
-
- step2[16] = WRAPLOW(step1[16] + step1[19], bd);
- step2[17] = WRAPLOW(step1[17] + step1[18], bd);
- step2[18] = WRAPLOW(step1[17] - step1[18], bd);
- step2[19] = WRAPLOW(step1[16] - step1[19], bd);
- step2[20] = WRAPLOW(-step1[20] + step1[23], bd);
- step2[21] = WRAPLOW(-step1[21] + step1[22], bd);
- step2[22] = WRAPLOW(step1[21] + step1[22], bd);
- step2[23] = WRAPLOW(step1[20] + step1[23], bd);
-
- step2[24] = WRAPLOW(step1[24] + step1[27], bd);
- step2[25] = WRAPLOW(step1[25] + step1[26], bd);
- step2[26] = WRAPLOW(step1[25] - step1[26], bd);
- step2[27] = WRAPLOW(step1[24] - step1[27], bd);
- step2[28] = WRAPLOW(-step1[28] + step1[31], bd);
- step2[29] = WRAPLOW(-step1[29] + step1[30], bd);
- step2[30] = WRAPLOW(step1[29] + step1[30], bd);
- step2[31] = WRAPLOW(step1[28] + step1[31], bd);
-
- // stage 5
- step1[0] = WRAPLOW(step2[0] + step2[3], bd);
- step1[1] = WRAPLOW(step2[1] + step2[2], bd);
- step1[2] = WRAPLOW(step2[1] - step2[2], bd);
- step1[3] = WRAPLOW(step2[0] - step2[3], bd);
- step1[4] = step2[4];
- temp1 = (step2[6] - step2[5]) * cospi_16_64;
- temp2 = (step2[5] + step2[6]) * cospi_16_64;
- step1[5] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[6] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[7] = step2[7];
-
- step1[8] = WRAPLOW(step2[8] + step2[11], bd);
- step1[9] = WRAPLOW(step2[9] + step2[10], bd);
- step1[10] = WRAPLOW(step2[9] - step2[10], bd);
- step1[11] = WRAPLOW(step2[8] - step2[11], bd);
- step1[12] = WRAPLOW(-step2[12] + step2[15], bd);
- step1[13] = WRAPLOW(-step2[13] + step2[14], bd);
- step1[14] = WRAPLOW(step2[13] + step2[14], bd);
- step1[15] = WRAPLOW(step2[12] + step2[15], bd);
-
- step1[16] = step2[16];
- step1[17] = step2[17];
- temp1 = -step2[18] * cospi_8_64 + step2[29] * cospi_24_64;
- temp2 = step2[18] * cospi_24_64 + step2[29] * cospi_8_64;
- step1[18] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[29] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step2[19] * cospi_8_64 + step2[28] * cospi_24_64;
- temp2 = step2[19] * cospi_24_64 + step2[28] * cospi_8_64;
- step1[19] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[28] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step2[20] * cospi_24_64 - step2[27] * cospi_8_64;
- temp2 = -step2[20] * cospi_8_64 + step2[27] * cospi_24_64;
- step1[20] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[27] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = -step2[21] * cospi_24_64 - step2[26] * cospi_8_64;
- temp2 = -step2[21] * cospi_8_64 + step2[26] * cospi_24_64;
- step1[21] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[26] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[22] = step2[22];
- step1[23] = step2[23];
- step1[24] = step2[24];
- step1[25] = step2[25];
- step1[30] = step2[30];
- step1[31] = step2[31];
-
- // stage 6
- step2[0] = WRAPLOW(step1[0] + step1[7], bd);
- step2[1] = WRAPLOW(step1[1] + step1[6], bd);
- step2[2] = WRAPLOW(step1[2] + step1[5], bd);
- step2[3] = WRAPLOW(step1[3] + step1[4], bd);
- step2[4] = WRAPLOW(step1[3] - step1[4], bd);
- step2[5] = WRAPLOW(step1[2] - step1[5], bd);
- step2[6] = WRAPLOW(step1[1] - step1[6], bd);
- step2[7] = WRAPLOW(step1[0] - step1[7], bd);
- step2[8] = step1[8];
- step2[9] = step1[9];
- temp1 = (-step1[10] + step1[13]) * cospi_16_64;
- temp2 = (step1[10] + step1[13]) * cospi_16_64;
- step2[10] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[13] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = (-step1[11] + step1[12]) * cospi_16_64;
- temp2 = (step1[11] + step1[12]) * cospi_16_64;
- step2[11] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step2[12] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step2[14] = step1[14];
- step2[15] = step1[15];
-
- step2[16] = WRAPLOW(step1[16] + step1[23], bd);
- step2[17] = WRAPLOW(step1[17] + step1[22], bd);
- step2[18] = WRAPLOW(step1[18] + step1[21], bd);
- step2[19] = WRAPLOW(step1[19] + step1[20], bd);
- step2[20] = WRAPLOW(step1[19] - step1[20], bd);
- step2[21] = WRAPLOW(step1[18] - step1[21], bd);
- step2[22] = WRAPLOW(step1[17] - step1[22], bd);
- step2[23] = WRAPLOW(step1[16] - step1[23], bd);
-
- step2[24] = WRAPLOW(-step1[24] + step1[31], bd);
- step2[25] = WRAPLOW(-step1[25] + step1[30], bd);
- step2[26] = WRAPLOW(-step1[26] + step1[29], bd);
- step2[27] = WRAPLOW(-step1[27] + step1[28], bd);
- step2[28] = WRAPLOW(step1[27] + step1[28], bd);
- step2[29] = WRAPLOW(step1[26] + step1[29], bd);
- step2[30] = WRAPLOW(step1[25] + step1[30], bd);
- step2[31] = WRAPLOW(step1[24] + step1[31], bd);
-
- // stage 7
- step1[0] = WRAPLOW(step2[0] + step2[15], bd);
- step1[1] = WRAPLOW(step2[1] + step2[14], bd);
- step1[2] = WRAPLOW(step2[2] + step2[13], bd);
- step1[3] = WRAPLOW(step2[3] + step2[12], bd);
- step1[4] = WRAPLOW(step2[4] + step2[11], bd);
- step1[5] = WRAPLOW(step2[5] + step2[10], bd);
- step1[6] = WRAPLOW(step2[6] + step2[9], bd);
- step1[7] = WRAPLOW(step2[7] + step2[8], bd);
- step1[8] = WRAPLOW(step2[7] - step2[8], bd);
- step1[9] = WRAPLOW(step2[6] - step2[9], bd);
- step1[10] = WRAPLOW(step2[5] - step2[10], bd);
- step1[11] = WRAPLOW(step2[4] - step2[11], bd);
- step1[12] = WRAPLOW(step2[3] - step2[12], bd);
- step1[13] = WRAPLOW(step2[2] - step2[13], bd);
- step1[14] = WRAPLOW(step2[1] - step2[14], bd);
- step1[15] = WRAPLOW(step2[0] - step2[15], bd);
-
- step1[16] = step2[16];
- step1[17] = step2[17];
- step1[18] = step2[18];
- step1[19] = step2[19];
- temp1 = (-step2[20] + step2[27]) * cospi_16_64;
- temp2 = (step2[20] + step2[27]) * cospi_16_64;
- step1[20] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[27] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = (-step2[21] + step2[26]) * cospi_16_64;
- temp2 = (step2[21] + step2[26]) * cospi_16_64;
- step1[21] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[26] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = (-step2[22] + step2[25]) * cospi_16_64;
- temp2 = (step2[22] + step2[25]) * cospi_16_64;
- step1[22] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[25] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- temp1 = (-step2[23] + step2[24]) * cospi_16_64;
- temp2 = (step2[23] + step2[24]) * cospi_16_64;
- step1[23] = WRAPLOW(highbd_dct_const_round_shift(temp1, bd), bd);
- step1[24] = WRAPLOW(highbd_dct_const_round_shift(temp2, bd), bd);
- step1[28] = step2[28];
- step1[29] = step2[29];
- step1[30] = step2[30];
- step1[31] = step2[31];
-
- // final stage
- output[0] = WRAPLOW(step1[0] + step1[31], bd);
- output[1] = WRAPLOW(step1[1] + step1[30], bd);
- output[2] = WRAPLOW(step1[2] + step1[29], bd);
- output[3] = WRAPLOW(step1[3] + step1[28], bd);
- output[4] = WRAPLOW(step1[4] + step1[27], bd);
- output[5] = WRAPLOW(step1[5] + step1[26], bd);
- output[6] = WRAPLOW(step1[6] + step1[25], bd);
- output[7] = WRAPLOW(step1[7] + step1[24], bd);
- output[8] = WRAPLOW(step1[8] + step1[23], bd);
- output[9] = WRAPLOW(step1[9] + step1[22], bd);
- output[10] = WRAPLOW(step1[10] + step1[21], bd);
- output[11] = WRAPLOW(step1[11] + step1[20], bd);
- output[12] = WRAPLOW(step1[12] + step1[19], bd);
- output[13] = WRAPLOW(step1[13] + step1[18], bd);
- output[14] = WRAPLOW(step1[14] + step1[17], bd);
- output[15] = WRAPLOW(step1[15] + step1[16], bd);
- output[16] = WRAPLOW(step1[15] - step1[16], bd);
- output[17] = WRAPLOW(step1[14] - step1[17], bd);
- output[18] = WRAPLOW(step1[13] - step1[18], bd);
- output[19] = WRAPLOW(step1[12] - step1[19], bd);
- output[20] = WRAPLOW(step1[11] - step1[20], bd);
- output[21] = WRAPLOW(step1[10] - step1[21], bd);
- output[22] = WRAPLOW(step1[9] - step1[22], bd);
- output[23] = WRAPLOW(step1[8] - step1[23], bd);
- output[24] = WRAPLOW(step1[7] - step1[24], bd);
- output[25] = WRAPLOW(step1[6] - step1[25], bd);
- output[26] = WRAPLOW(step1[5] - step1[26], bd);
- output[27] = WRAPLOW(step1[4] - step1[27], bd);
- output[28] = WRAPLOW(step1[3] - step1[28], bd);
- output[29] = WRAPLOW(step1[2] - step1[29], bd);
- output[30] = WRAPLOW(step1[1] - step1[30], bd);
- output[31] = WRAPLOW(step1[0] - step1[31], bd);
-}
-
-void vp10_highbd_idct32x32_1024_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[32 * 32];
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[32], temp_out[32];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // Rows
- for (i = 0; i < 32; ++i) {
- tran_low_t zero_coeff[16];
- for (j = 0; j < 16; ++j)
- zero_coeff[j] = input[2 * j] | input[2 * j + 1];
- for (j = 0; j < 8; ++j)
- zero_coeff[j] = zero_coeff[2 * j] | zero_coeff[2 * j + 1];
- for (j = 0; j < 4; ++j)
- zero_coeff[j] = zero_coeff[2 * j] | zero_coeff[2 * j + 1];
- for (j = 0; j < 2; ++j)
- zero_coeff[j] = zero_coeff[2 * j] | zero_coeff[2 * j + 1];
-
- if (zero_coeff[0] | zero_coeff[1])
- highbd_idct32_c(input, outptr, bd);
- else
- memset(outptr, 0, sizeof(tran_low_t) * 32);
- input += 32;
- outptr += 32;
- }
-
- // Columns
- for (i = 0; i < 32; ++i) {
- for (j = 0; j < 32; ++j)
- temp_in[j] = out[j * 32 + i];
- highbd_idct32_c(temp_in, temp_out, bd);
- for (j = 0; j < 32; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 6), bd);
- }
- }
-}
-
-void vp10_highbd_idct32x32_34_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[32 * 32] = {0};
- tran_low_t *outptr = out;
- int i, j;
- tran_low_t temp_in[32], temp_out[32];
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- // Rows
- // Only upper-left 8x8 has non-zero coeff.
- for (i = 0; i < 8; ++i) {
- highbd_idct32_c(input, outptr, bd);
- input += 32;
- outptr += 32;
- }
- // Columns
- for (i = 0; i < 32; ++i) {
- for (j = 0; j < 32; ++j)
- temp_in[j] = out[j * 32 + i];
- highbd_idct32_c(temp_in, temp_out, bd);
- for (j = 0; j < 32; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 6), bd);
- }
- }
-}
-
-void vp10_highbd_idct32x32_1_add_c(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- int i, j;
- int a1;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
-
- tran_low_t out = WRAPLOW(
- highbd_dct_const_round_shift(input[0] * cospi_16_64, bd), bd);
- out = WRAPLOW(highbd_dct_const_round_shift(out * cospi_16_64, bd), bd);
- a1 = ROUND_POWER_OF_TWO(out, 6);
-
- for (j = 0; j < 32; ++j) {
- for (i = 0; i < 32; ++i)
- dest[i] = highbd_clip_pixel_add(dest[i], a1, bd);
- dest += stride;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VPX_DSP_INV_TXFM_H_
-#define VPX_DSP_INV_TXFM_H_
-
-#include <assert.h>
-
-#include "./vpx_config.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_ports/mem.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-static INLINE tran_low_t check_range(tran_high_t input) {
-#if CONFIG_COEFFICIENT_RANGE_CHECKING
- // For valid VP9 input streams, intermediate stage coefficients should always
- // stay within the range of a signed 16 bit integer. Coefficients can go out
- // of this range for invalid/corrupt VP9 streams. However, strictly checking
- // this range for every intermediate coefficient can burdensome for a decoder,
- // therefore the following assertion is only enabled when configured with
- // --enable-coefficient-range-checking.
- assert(INT16_MIN <= input);
- assert(input <= INT16_MAX);
-#endif // CONFIG_COEFFICIENT_RANGE_CHECKING
- return (tran_low_t)input;
-}
-
-static INLINE tran_low_t dct_const_round_shift(tran_high_t input) {
- tran_high_t rv = ROUND_POWER_OF_TWO(input, DCT_CONST_BITS);
- return check_range(rv);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static INLINE tran_low_t highbd_check_range(tran_high_t input,
- int bd) {
-#if CONFIG_COEFFICIENT_RANGE_CHECKING
- // For valid highbitdepth VP9 streams, intermediate stage coefficients will
- // stay within the ranges:
- // - 8 bit: signed 16 bit integer
- // - 10 bit: signed 18 bit integer
- // - 12 bit: signed 20 bit integer
- const int32_t int_max = (1 << (7 + bd)) - 1;
- const int32_t int_min = -int_max - 1;
- assert(int_min <= input);
- assert(input <= int_max);
- (void) int_min;
-#endif // CONFIG_COEFFICIENT_RANGE_CHECKING
- (void) bd;
- return (tran_low_t)input;
-}
-
-static INLINE tran_low_t highbd_dct_const_round_shift(tran_high_t input,
- int bd) {
- tran_high_t rv = ROUND_POWER_OF_TWO(input, DCT_CONST_BITS);
- return highbd_check_range(rv, bd);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#if CONFIG_EMULATE_HARDWARE
-// When CONFIG_EMULATE_HARDWARE is 1 the transform performs a
-// non-normative method to handle overflows. A stream that causes
-// overflows in the inverse transform is considered invalid in VP9,
-// and a hardware implementer is free to choose any reasonable
-// method to handle overflows. However to aid in hardware
-// verification they can use a specific implementation of the
-// WRAPLOW() macro below that is identical to their intended
-// hardware implementation (and also use configure options to trigger
-// the C-implementation of the transform).
-//
-// The particular WRAPLOW implementation below performs strict
-// overflow wrapping to match common hardware implementations.
-// bd of 8 uses trans_low with 16bits, need to remove 16bits
-// bd of 10 uses trans_low with 18bits, need to remove 14bits
-// bd of 12 uses trans_low with 20bits, need to remove 12bits
-// bd of x uses trans_low with 8+x bits, need to remove 24-x bits
-#define WRAPLOW(x, bd) ((((int32_t)(x)) << (24 - bd)) >> (24 - bd))
-#else
-#define WRAPLOW(x, bd) ((int32_t)(x))
-#endif // CONFIG_EMULATE_HARDWARE
-
-void vp10_idct4_c(const tran_low_t *input, tran_low_t *output);
-void vp10_idct8_c(const tran_low_t *input, tran_low_t *output);
-void vp10_idct16_c(const tran_low_t *input, tran_low_t *output);
-void vp10_idct32_c(const tran_low_t *input, tran_low_t *output);
-void vp10_iadst4_c(const tran_low_t *input, tran_low_t *output);
-void vp10_iadst8_c(const tran_low_t *input, tran_low_t *output);
-void vp10_iadst16_c(const tran_low_t *input, tran_low_t *output);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_idct4_c(const tran_low_t *input, tran_low_t *output, int bd);
-void vp10_highbd_idct8_c(const tran_low_t *input, tran_low_t *output, int bd);
-void vp10_highbd_idct16_c(const tran_low_t *input, tran_low_t *output, int bd);
-
-void vp10_highbd_iadst4_c(const tran_low_t *input, tran_low_t *output, int bd);
-void vp10_highbd_iadst8_c(const tran_low_t *input, tran_low_t *output, int bd);
-void vp10_highbd_iadst16_c(const tran_low_t *input, tran_low_t *output, int bd);
-
-static INLINE uint16_t highbd_clip_pixel_add(uint16_t dest, tran_high_t trans,
- int bd) {
- trans = WRAPLOW(trans, bd);
- return clip_pixel_highbd(WRAPLOW(dest + trans, bd), bd);
-}
-#endif
-
-static INLINE uint8_t clip_pixel_add(uint8_t dest, tran_high_t trans) {
- trans = WRAPLOW(trans, 8);
- return clip_pixel(WRAPLOW(dest + trans, 8));
-}
-#ifdef __cplusplus
-} // extern "C"
-#endif
-#endif // VPX_DSP_INV_TXFM_H_
+++ /dev/null
-/*
- * Copyright (c) 2011 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#include "./vpx_config.h"
-#define RTCD_C
-#include "./vp10_rtcd.h"
-#include "vpx_ports/vpx_once.h"
-
-void vp10_rtcd() {
- // TODO(JBB): Remove this once, by insuring that both the encoder and
- // decoder setup functions are protected by once();
- once(setup_rtcd_internal);
-}
+++ /dev/null
-sub vp10_common_forward_decls() {
-print <<EOF
-/*
- * VP10
- */
-
-#include "vpx/vpx_integer.h"
-#include "vp10/common/common.h"
-#include "vp10/common/enums.h"
-
-struct macroblockd;
-
-/* Encoder forward decls */
-struct macroblock;
-struct vp9_variance_vtable;
-struct search_site_config;
-struct mv;
-union int_mv;
-struct yv12_buffer_config;
-EOF
-}
-forward_decls qw/vp10_common_forward_decls/;
-
-# x86inc.asm had specific constraints. break it out so it's easy to disable.
-# zero all the variables to avoid tricky else conditions.
-$mmx_x86inc = $sse_x86inc = $sse2_x86inc = $ssse3_x86inc = $avx_x86inc =
- $avx2_x86inc = '';
-$mmx_x86_64_x86inc = $sse_x86_64_x86inc = $sse2_x86_64_x86inc =
- $ssse3_x86_64_x86inc = $avx_x86_64_x86inc = $avx2_x86_64_x86inc = '';
-if (vpx_config("CONFIG_USE_X86INC") eq "yes") {
- $mmx_x86inc = 'mmx';
- $sse_x86inc = 'sse';
- $sse2_x86inc = 'sse2';
- $ssse3_x86inc = 'ssse3';
- $avx_x86inc = 'avx';
- $avx2_x86inc = 'avx2';
- if ($opts{arch} eq "x86_64") {
- $mmx_x86_64_x86inc = 'mmx';
- $sse_x86_64_x86inc = 'sse';
- $sse2_x86_64_x86inc = 'sse2';
- $ssse3_x86_64_x86inc = 'ssse3';
- $avx_x86_64_x86inc = 'avx';
- $avx2_x86_64_x86inc = 'avx2';
- }
-}
-
-# functions that are 64 bit only.
-$mmx_x86_64 = $sse2_x86_64 = $ssse3_x86_64 = $avx_x86_64 = $avx2_x86_64 = '';
-if ($opts{arch} eq "x86_64") {
- $mmx_x86_64 = 'mmx';
- $sse2_x86_64 = 'sse2';
- $ssse3_x86_64 = 'ssse3';
- $avx_x86_64 = 'avx';
- $avx2_x86_64 = 'avx2';
-}
-
-#
-# post proc
-#
-if (vpx_config("CONFIG_VP9_POSTPROC") eq "yes") {
-add_proto qw/void vp10_mbpost_proc_down/, "uint8_t *dst, int pitch, int rows, int cols, int flimit";
-specialize qw/vp10_mbpost_proc_down sse2/;
-$vp10_mbpost_proc_down_sse2=vp10_mbpost_proc_down_xmm;
-
-add_proto qw/void vp10_mbpost_proc_across_ip/, "uint8_t *src, int pitch, int rows, int cols, int flimit";
-specialize qw/vp10_mbpost_proc_across_ip sse2/;
-$vp10_mbpost_proc_across_ip_sse2=vp10_mbpost_proc_across_ip_xmm;
-
-add_proto qw/void vp10_post_proc_down_and_across/, "const uint8_t *src_ptr, uint8_t *dst_ptr, int src_pixels_per_line, int dst_pixels_per_line, int rows, int cols, int flimit";
-specialize qw/vp10_post_proc_down_and_across sse2/;
-$vp10_post_proc_down_and_across_sse2=vp10_post_proc_down_and_across_xmm;
-
-add_proto qw/void vp10_plane_add_noise/, "uint8_t *Start, char *noise, char blackclamp[16], char whiteclamp[16], char bothclamp[16], unsigned int Width, unsigned int Height, int Pitch";
-specialize qw/vp10_plane_add_noise sse2/;
-$vp10_plane_add_noise_sse2=vp10_plane_add_noise_wmt;
-
-add_proto qw/void vp10_filter_by_weight16x16/, "const uint8_t *src, int src_stride, uint8_t *dst, int dst_stride, int src_weight";
-specialize qw/vp10_filter_by_weight16x16 sse2 msa/;
-
-add_proto qw/void vp10_filter_by_weight8x8/, "const uint8_t *src, int src_stride, uint8_t *dst, int dst_stride, int src_weight";
-specialize qw/vp10_filter_by_weight8x8 sse2 msa/;
-}
-
-#
-# dct
-#
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
- # Note as optimized versions of these functions are added we need to add a check to ensure
- # that when CONFIG_EMULATE_HARDWARE is on, it defaults to the C versions only.
- if (vpx_config("CONFIG_EMULATE_HARDWARE") eq "yes") {
- add_proto qw/void vp10_iht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht4x4_16_add/;
-
- add_proto qw/void vp10_iht8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht8x8_64_add/;
-
- add_proto qw/void vp10_iht16x16_256_add/, "const tran_low_t *input, uint8_t *output, int pitch, int tx_type";
- specialize qw/vp10_iht16x16_256_add/;
-
- add_proto qw/void vp10_fdct4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4/;
-
- add_proto qw/void vp10_fdct4x4_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4_1/;
-
- add_proto qw/void vp10_fdct8x8/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8/;
-
- add_proto qw/void vp10_fdct8x8_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8_1/;
-
- add_proto qw/void vp10_fdct16x16/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16/;
-
- add_proto qw/void vp10_fdct16x16_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16_1/;
-
- add_proto qw/void vp10_fdct32x32/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32/;
-
- add_proto qw/void vp10_fdct32x32_rd/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_rd/;
-
- add_proto qw/void vp10_fdct32x32_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_1/;
-
- add_proto qw/void vp10_highbd_fdct4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct4x4/;
-
- add_proto qw/void vp10_highbd_fdct8x8/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct8x8/;
-
- add_proto qw/void vp10_highbd_fdct8x8_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct8x8_1/;
-
- add_proto qw/void vp10_highbd_fdct16x16/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct16x16/;
-
- add_proto qw/void vp10_highbd_fdct16x16_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct16x16_1/;
-
- add_proto qw/void vp10_highbd_fdct32x32/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct32x32/;
-
- add_proto qw/void vp10_highbd_fdct32x32_rd/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct32x32_rd/;
-
- add_proto qw/void vp10_highbd_fdct32x32_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct32x32_1/;
- } else {
- add_proto qw/void vp10_iht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht4x4_16_add sse2/;
-
- add_proto qw/void vp10_iht8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht8x8_64_add sse2/;
-
- add_proto qw/void vp10_iht16x16_256_add/, "const tran_low_t *input, uint8_t *output, int pitch, int tx_type";
- specialize qw/vp10_iht16x16_256_add/;
-
- add_proto qw/void vp10_fdct4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4 sse2/;
-
- add_proto qw/void vp10_fdct4x4_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4_1 sse2/;
-
- add_proto qw/void vp10_fdct8x8/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8 sse2/;
-
- add_proto qw/void vp10_fdct8x8_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8_1 sse2/;
-
- add_proto qw/void vp10_fdct16x16/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16 sse2/;
-
- add_proto qw/void vp10_fdct16x16_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16_1 sse2/;
-
- add_proto qw/void vp10_fdct32x32/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32 sse2/;
-
- add_proto qw/void vp10_fdct32x32_rd/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_rd sse2/;
-
- add_proto qw/void vp10_fdct32x32_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_1 sse2/;
-
- add_proto qw/void vp10_highbd_fdct4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct4x4 sse2/;
-
- add_proto qw/void vp10_highbd_fdct8x8/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct8x8 sse2/;
-
- add_proto qw/void vp10_highbd_fdct8x8_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct8x8_1/;
-
- add_proto qw/void vp10_highbd_fdct16x16/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct16x16 sse2/;
-
- add_proto qw/void vp10_highbd_fdct16x16_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct16x16_1/;
-
- add_proto qw/void vp10_highbd_fdct32x32/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct32x32 sse2/;
-
- add_proto qw/void vp10_highbd_fdct32x32_rd/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct32x32_rd sse2/;
-
- add_proto qw/void vp10_highbd_fdct32x32_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fdct32x32_1/;
- }
-} else {
- # Force C versions if CONFIG_EMULATE_HARDWARE is 1
- if (vpx_config("CONFIG_EMULATE_HARDWARE") eq "yes") {
- add_proto qw/void vp10_iht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht4x4_16_add/;
-
- add_proto qw/void vp10_iht8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht8x8_64_add/;
-
- add_proto qw/void vp10_iht16x16_256_add/, "const tran_low_t *input, uint8_t *output, int pitch, int tx_type";
- specialize qw/vp10_iht16x16_256_add/;
-
- add_proto qw/void vp10_fdct4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4/;
-
- add_proto qw/void vp10_fdct4x4_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4_1/;
-
- add_proto qw/void vp10_fdct8x8/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8/;
-
- add_proto qw/void vp10_fdct8x8_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8_1/;
-
- add_proto qw/void vp10_fdct16x16/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16/;
-
- add_proto qw/void vp10_fdct16x16_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16_1/;
-
- add_proto qw/void vp10_fdct32x32/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32/;
-
- add_proto qw/void vp10_fdct32x32_rd/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_rd/;
-
- add_proto qw/void vp10_fdct32x32_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_1/;
- } else {
- add_proto qw/void vp10_iht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht4x4_16_add sse2 neon dspr2 msa/;
-
- add_proto qw/void vp10_iht8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type";
- specialize qw/vp10_iht8x8_64_add sse2 neon dspr2 msa/;
-
- add_proto qw/void vp10_iht16x16_256_add/, "const tran_low_t *input, uint8_t *output, int pitch, int tx_type";
- specialize qw/vp10_iht16x16_256_add sse2 dspr2 msa/;
-
- add_proto qw/void vp10_fdct4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4 sse2/;
-
- add_proto qw/void vp10_fdct4x4_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct4x4_1 sse2/;
-
- add_proto qw/void vp10_fdct8x8/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8 sse2/;
-
- add_proto qw/void vp10_fdct8x8_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct8x8_1 sse2/;
-
- add_proto qw/void vp10_fdct16x16/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16 sse2/;
-
- add_proto qw/void vp10_fdct16x16_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct16x16_1 sse2/;
-
- add_proto qw/void vp10_fdct32x32/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32 sse2/;
-
- add_proto qw/void vp10_fdct32x32_rd/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_rd sse2/;
-
- add_proto qw/void vp10_fdct32x32_1/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fdct32x32_1 sse2/;
- }
-}
-
-# High bitdepth functions
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
- #
- # Sub Pixel Filters
- #
- add_proto qw/void vp10_highbd_convolve_copy/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve_copy/;
-
- add_proto qw/void vp10_highbd_convolve_avg/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve_avg/;
-
- add_proto qw/void vp10_highbd_convolve8/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve8/, "$sse2_x86_64";
-
- add_proto qw/void vp10_highbd_convolve8_horiz/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve8_horiz/, "$sse2_x86_64";
-
- add_proto qw/void vp10_highbd_convolve8_vert/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve8_vert/, "$sse2_x86_64";
-
- add_proto qw/void vp10_highbd_convolve8_avg/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve8_avg/, "$sse2_x86_64";
-
- add_proto qw/void vp10_highbd_convolve8_avg_horiz/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve8_avg_horiz/, "$sse2_x86_64";
-
- add_proto qw/void vp10_highbd_convolve8_avg_vert/, "const uint8_t *src, ptrdiff_t src_stride, uint8_t *dst, ptrdiff_t dst_stride, const int16_t *filter_x, int x_step_q4, const int16_t *filter_y, int y_step_q4, int w, int h, int bps";
- specialize qw/vp10_highbd_convolve8_avg_vert/, "$sse2_x86_64";
-
- #
- # post proc
- #
- if (vpx_config("CONFIG_VP9_POSTPROC") eq "yes") {
- add_proto qw/void vp10_highbd_mbpost_proc_down/, "uint16_t *dst, int pitch, int rows, int cols, int flimit";
- specialize qw/vp10_highbd_mbpost_proc_down/;
-
- add_proto qw/void vp10_highbd_mbpost_proc_across_ip/, "uint16_t *src, int pitch, int rows, int cols, int flimit";
- specialize qw/vp10_highbd_mbpost_proc_across_ip/;
-
- add_proto qw/void vp10_highbd_post_proc_down_and_across/, "const uint16_t *src_ptr, uint16_t *dst_ptr, int src_pixels_per_line, int dst_pixels_per_line, int rows, int cols, int flimit";
- specialize qw/vp10_highbd_post_proc_down_and_across/;
-
- add_proto qw/void vp10_highbd_plane_add_noise/, "uint8_t *Start, char *noise, char blackclamp[16], char whiteclamp[16], char bothclamp[16], unsigned int Width, unsigned int Height, int Pitch";
- specialize qw/vp10_highbd_plane_add_noise/;
- }
-
- #
- # dct
- #
- # Note as optimized versions of these functions are added we need to add a check to ensure
- # that when CONFIG_EMULATE_HARDWARE is on, it defaults to the C versions only.
- add_proto qw/void vp10_highbd_iht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type, int bd";
- specialize qw/vp10_highbd_iht4x4_16_add/;
-
- add_proto qw/void vp10_highbd_iht8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int tx_type, int bd";
- specialize qw/vp10_highbd_iht8x8_64_add/;
-
- add_proto qw/void vp10_highbd_iht16x16_256_add/, "const tran_low_t *input, uint8_t *output, int pitch, int tx_type, int bd";
- specialize qw/vp10_highbd_iht16x16_256_add/;
-}
-
-#
-# Encoder functions below this point.
-#
-if (vpx_config("CONFIG_VP10_ENCODER") eq "yes") {
-
-add_proto qw/unsigned int vp10_avg_8x8/, "const uint8_t *, int p";
-specialize qw/vp10_avg_8x8 sse2 neon msa/;
-
-add_proto qw/unsigned int vp10_avg_4x4/, "const uint8_t *, int p";
-specialize qw/vp10_avg_4x4 sse2 msa/;
-
-add_proto qw/void vp10_minmax_8x8/, "const uint8_t *s, int p, const uint8_t *d, int dp, int *min, int *max";
-specialize qw/vp10_minmax_8x8 sse2/;
-
-add_proto qw/void vp10_hadamard_8x8/, "int16_t const *src_diff, int src_stride, int16_t *coeff";
-specialize qw/vp10_hadamard_8x8 sse2/, "$ssse3_x86_64_x86inc";
-
-add_proto qw/void vp10_hadamard_16x16/, "int16_t const *src_diff, int src_stride, int16_t *coeff";
-specialize qw/vp10_hadamard_16x16 sse2/;
-
-add_proto qw/int16_t vp10_satd/, "const int16_t *coeff, int length";
-specialize qw/vp10_satd sse2/;
-
-add_proto qw/void vp10_int_pro_row/, "int16_t *hbuf, uint8_t const *ref, const int ref_stride, const int height";
-specialize qw/vp10_int_pro_row sse2 neon/;
-
-add_proto qw/int16_t vp10_int_pro_col/, "uint8_t const *ref, const int width";
-specialize qw/vp10_int_pro_col sse2 neon/;
-
-add_proto qw/int vp10_vector_var/, "int16_t const *ref, int16_t const *src, const int bwl";
-specialize qw/vp10_vector_var neon sse2/;
-
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
- add_proto qw/unsigned int vp10_highbd_avg_8x8/, "const uint8_t *, int p";
- specialize qw/vp10_highbd_avg_8x8/;
- add_proto qw/unsigned int vp10_highbd_avg_4x4/, "const uint8_t *, int p";
- specialize qw/vp10_highbd_avg_4x4/;
- add_proto qw/void vp10_highbd_minmax_8x8/, "const uint8_t *s, int p, const uint8_t *d, int dp, int *min, int *max";
- specialize qw/vp10_highbd_minmax_8x8/;
-}
-
-# ENCODEMB INVOKE
-
-#
-# Denoiser
-#
-if (vpx_config("CONFIG_VP9_TEMPORAL_DENOISING") eq "yes") {
- add_proto qw/int vp10_denoiser_filter/, "const uint8_t *sig, int sig_stride, const uint8_t *mc_avg, int mc_avg_stride, uint8_t *avg, int avg_stride, int increase_denoising, BLOCK_SIZE bs, int motion_magnitude";
- specialize qw/vp10_denoiser_filter sse2/;
-}
-
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
-# the transform coefficients are held in 32-bit
-# values, so the assembler code for vp10_block_error can no longer be used.
- add_proto qw/int64_t vp10_block_error/, "const tran_low_t *coeff, const tran_low_t *dqcoeff, intptr_t block_size, int64_t *ssz";
- specialize qw/vp10_block_error/;
-
- add_proto qw/void vp10_quantize_fp/, "const tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_quantize_fp/;
-
- add_proto qw/void vp10_quantize_fp_32x32/, "const tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_quantize_fp_32x32/;
-
- add_proto qw/void vp10_fdct8x8_quant/, "const int16_t *input, int stride, tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_fdct8x8_quant/;
-} else {
- add_proto qw/int64_t vp10_block_error/, "const tran_low_t *coeff, const tran_low_t *dqcoeff, intptr_t block_size, int64_t *ssz";
- specialize qw/vp10_block_error avx2 msa/, "$sse2_x86inc";
-
- add_proto qw/int64_t vp10_block_error_fp/, "const int16_t *coeff, const int16_t *dqcoeff, int block_size";
- specialize qw/vp10_block_error_fp neon/, "$sse2_x86inc";
-
- add_proto qw/void vp10_quantize_fp/, "const tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_quantize_fp neon sse2/, "$ssse3_x86_64_x86inc";
-
- add_proto qw/void vp10_quantize_fp_32x32/, "const tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_quantize_fp_32x32/, "$ssse3_x86_64_x86inc";
-
- add_proto qw/void vp10_fdct8x8_quant/, "const int16_t *input, int stride, tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_fdct8x8_quant sse2 ssse3 neon/;
-}
-
-# fdct functions
-
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
- add_proto qw/void vp10_fht4x4/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_fht4x4 sse2/;
-
- add_proto qw/void vp10_fht8x8/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_fht8x8 sse2/;
-
- add_proto qw/void vp10_fht16x16/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_fht16x16 sse2/;
-
- add_proto qw/void vp10_fwht4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fwht4x4/, "$mmx_x86inc";
-} else {
- add_proto qw/void vp10_fht4x4/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_fht4x4 sse2 msa/;
-
- add_proto qw/void vp10_fht8x8/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_fht8x8 sse2 msa/;
-
- add_proto qw/void vp10_fht16x16/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_fht16x16 sse2 msa/;
-
- add_proto qw/void vp10_fwht4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_fwht4x4 msa/, "$mmx_x86inc";
-}
-
-# Inverse transform
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
- # Note as optimized versions of these functions are added we need to add a check to ensure
- # that when CONFIG_EMULATE_HARDWARE is on, it defaults to the C versions only.
- add_proto qw/void vp10_idct4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct4x4_1_add/;
-
- add_proto qw/void vp10_idct4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct4x4_16_add/;
-
- add_proto qw/void vp10_idct8x8_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_1_add/;
-
- add_proto qw/void vp10_idct8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_64_add/;
-
- add_proto qw/void vp10_idct8x8_12_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_12_add/;
-
- add_proto qw/void vp10_idct16x16_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_1_add/;
-
- add_proto qw/void vp10_idct16x16_256_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_256_add/;
-
- add_proto qw/void vp10_idct16x16_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_10_add/;
-
- add_proto qw/void vp10_idct32x32_1024_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_1024_add/;
-
- add_proto qw/void vp10_idct32x32_34_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_34_add/;
-
- add_proto qw/void vp10_idct32x32_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_1_add/;
-
- add_proto qw/void vp10_iwht4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_iwht4x4_1_add/;
-
- add_proto qw/void vp10_iwht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_iwht4x4_16_add/;
-
- add_proto qw/void vp10_highbd_idct4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct4x4_1_add/;
-
- add_proto qw/void vp10_highbd_idct8x8_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct8x8_1_add/;
-
- add_proto qw/void vp10_highbd_idct16x16_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct16x16_1_add/;
-
- add_proto qw/void vp10_highbd_idct32x32_1024_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct32x32_1024_add/;
-
- add_proto qw/void vp10_highbd_idct32x32_34_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct32x32_34_add/;
-
- add_proto qw/void vp10_highbd_idct32x32_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct32x32_1_add/;
-
- add_proto qw/void vp10_highbd_iwht4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_iwht4x4_1_add/;
-
- add_proto qw/void vp10_highbd_iwht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_iwht4x4_16_add/;
-
- # Force C versions if CONFIG_EMULATE_HARDWARE is 1
- if (vpx_config("CONFIG_EMULATE_HARDWARE") eq "yes") {
- add_proto qw/void vp10_highbd_idct4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct4x4_16_add/;
-
- add_proto qw/void vp10_highbd_idct8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct8x8_64_add/;
-
- add_proto qw/void vp10_highbd_idct8x8_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct8x8_10_add/;
-
- add_proto qw/void vp10_highbd_idct16x16_256_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct16x16_256_add/;
-
- add_proto qw/void vp10_highbd_idct16x16_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct16x16_10_add/;
- } else {
- add_proto qw/void vp10_highbd_idct4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct4x4_16_add sse2/;
-
- add_proto qw/void vp10_highbd_idct8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct8x8_64_add sse2/;
-
- add_proto qw/void vp10_highbd_idct8x8_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct8x8_10_add sse2/;
-
- add_proto qw/void vp10_highbd_idct16x16_256_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct16x16_256_add sse2/;
-
- add_proto qw/void vp10_highbd_idct16x16_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride, int bd";
- specialize qw/vp10_highbd_idct16x16_10_add sse2/;
- } # CONFIG_EMULATE_HARDWARE
-} else {
- # Force C versions if CONFIG_EMULATE_HARDWARE is 1
- if (vpx_config("CONFIG_EMULATE_HARDWARE") eq "yes") {
- add_proto qw/void vp10_idct4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct4x4_1_add/;
-
- add_proto qw/void vp10_idct4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct4x4_16_add/;
-
- add_proto qw/void vp10_idct8x8_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_1_add/;
-
- add_proto qw/void vp10_idct8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_64_add/;
-
- add_proto qw/void vp10_idct8x8_12_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_12_add/;
-
- add_proto qw/void vp10_idct16x16_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_1_add/;
-
- add_proto qw/void vp10_idct16x16_256_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_256_add/;
-
- add_proto qw/void vp10_idct16x16_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_10_add/;
-
- add_proto qw/void vp10_idct32x32_1024_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_1024_add/;
-
- add_proto qw/void vp10_idct32x32_34_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_34_add/;
-
- add_proto qw/void vp10_idct32x32_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_1_add/;
-
- add_proto qw/void vp10_iwht4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_iwht4x4_1_add/;
-
- add_proto qw/void vp10_iwht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_iwht4x4_16_add/;
- } else {
- add_proto qw/void vp10_idct4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct4x4_1_add sse2/;
-
- add_proto qw/void vp10_idct4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct4x4_16_add sse2/;
-
- add_proto qw/void vp10_idct8x8_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_1_add sse2/;
-
- add_proto qw/void vp10_idct8x8_64_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_64_add sse2/;
-
- add_proto qw/void vp10_idct8x8_12_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct8x8_12_add sse2/;
-
- add_proto qw/void vp10_idct16x16_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_1_add sse2/;
-
- add_proto qw/void vp10_idct16x16_256_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_256_add sse2/;
-
- add_proto qw/void vp10_idct16x16_10_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct16x16_10_add sse2/;
-
- add_proto qw/void vp10_idct32x32_1024_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_1024_add sse2/;
-
- add_proto qw/void vp10_idct32x32_34_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_34_add sse2/;
-
- add_proto qw/void vp10_idct32x32_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_idct32x32_1_add sse2/;
-
- add_proto qw/void vp10_iwht4x4_1_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_iwht4x4_1_add/;
-
- add_proto qw/void vp10_iwht4x4_16_add/, "const tran_low_t *input, uint8_t *dest, int dest_stride";
- specialize qw/vp10_iwht4x4_16_add/;
- } # CONFIG_EMULATE_HARDWARE
-} # CONFIG_VP9_HIGHBITDEPTH
-
-#
-# Motion search
-#
-add_proto qw/int vp10_full_search_sad/, "const struct macroblock *x, const struct mv *ref_mv, int sad_per_bit, int distance, const struct vp9_variance_vtable *fn_ptr, const struct mv *center_mv, struct mv *best_mv";
-specialize qw/vp10_full_search_sad sse3 sse4_1/;
-$vp10_full_search_sad_sse3=vp10_full_search_sadx3;
-$vp10_full_search_sad_sse4_1=vp10_full_search_sadx8;
-
-add_proto qw/int vp10_diamond_search_sad/, "const struct macroblock *x, const struct search_site_config *cfg, struct mv *ref_mv, struct mv *best_mv, int search_param, int sad_per_bit, int *num00, const struct vp9_variance_vtable *fn_ptr, const struct mv *center_mv";
-specialize qw/vp10_diamond_search_sad/;
-
-add_proto qw/int vp10_full_range_search/, "const struct macroblock *x, const struct search_site_config *cfg, struct mv *ref_mv, struct mv *best_mv, int search_param, int sad_per_bit, int *num00, const struct vp9_variance_vtable *fn_ptr, const struct mv *center_mv";
-specialize qw/vp10_full_range_search/;
-
-add_proto qw/void vp10_temporal_filter_apply/, "uint8_t *frame1, unsigned int stride, uint8_t *frame2, unsigned int block_width, unsigned int block_height, int strength, int filter_weight, unsigned int *accumulator, uint16_t *count";
-specialize qw/vp10_temporal_filter_apply sse2 msa/;
-
-if (vpx_config("CONFIG_VP9_HIGHBITDEPTH") eq "yes") {
-
- # ENCODEMB INVOKE
-
- add_proto qw/int64_t vp10_highbd_block_error/, "const tran_low_t *coeff, const tran_low_t *dqcoeff, intptr_t block_size, int64_t *ssz, int bd";
- specialize qw/vp10_highbd_block_error sse2/;
-
- add_proto qw/void vp10_highbd_quantize_fp/, "const tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_highbd_quantize_fp/;
-
- add_proto qw/void vp10_highbd_quantize_fp_32x32/, "const tran_low_t *coeff_ptr, intptr_t n_coeffs, int skip_block, const int16_t *zbin_ptr, const int16_t *round_ptr, const int16_t *quant_ptr, const int16_t *quant_shift_ptr, tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr, const int16_t *dequant_ptr, uint16_t *eob_ptr, const int16_t *scan, const int16_t *iscan";
- specialize qw/vp10_highbd_quantize_fp_32x32/;
-
- # fdct functions
- add_proto qw/void vp10_highbd_fht4x4/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_highbd_fht4x4/;
-
- add_proto qw/void vp10_highbd_fht8x8/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_highbd_fht8x8/;
-
- add_proto qw/void vp10_highbd_fht16x16/, "const int16_t *input, tran_low_t *output, int stride, int tx_type";
- specialize qw/vp10_highbd_fht16x16/;
-
- add_proto qw/void vp10_highbd_fwht4x4/, "const int16_t *input, tran_low_t *output, int stride";
- specialize qw/vp10_highbd_fwht4x4/;
-
- add_proto qw/void vp10_highbd_temporal_filter_apply/, "uint8_t *frame1, unsigned int stride, uint8_t *frame2, unsigned int block_width, unsigned int block_height, int strength, int filter_weight, unsigned int *accumulator, uint16_t *count";
- specialize qw/vp10_highbd_temporal_filter_apply/;
-
-}
-# End vp10_high encoder functions
-
-}
-# end encoder functions
-1;
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vpx_dsp/x86/inv_txfm_sse2.h"
-#include "vpx_dsp/x86/txfm_common_sse2.h"
-#include "vpx_ports/mem.h"
-
-void vp10_iht4x4_16_add_sse2(const tran_low_t *input, uint8_t *dest, int stride,
- int tx_type) {
- __m128i in[2];
- const __m128i zero = _mm_setzero_si128();
- const __m128i eight = _mm_set1_epi16(8);
-
- in[0] = load_input_data(input);
- in[1] = load_input_data(input + 8);
-
- switch (tx_type) {
- case 0: // DCT_DCT
- idct4_sse2(in);
- idct4_sse2(in);
- break;
- case 1: // ADST_DCT
- idct4_sse2(in);
- iadst4_sse2(in);
- break;
- case 2: // DCT_ADST
- iadst4_sse2(in);
- idct4_sse2(in);
- break;
- case 3: // ADST_ADST
- iadst4_sse2(in);
- iadst4_sse2(in);
- break;
- default:
- assert(0);
- break;
- }
-
- // Final round and shift
- in[0] = _mm_add_epi16(in[0], eight);
- in[1] = _mm_add_epi16(in[1], eight);
-
- in[0] = _mm_srai_epi16(in[0], 4);
- in[1] = _mm_srai_epi16(in[1], 4);
-
- // Reconstruction and Store
- {
- __m128i d0 = _mm_cvtsi32_si128(*(const int *)(dest));
- __m128i d2 = _mm_cvtsi32_si128(*(const int *)(dest + stride * 2));
- d0 = _mm_unpacklo_epi32(d0,
- _mm_cvtsi32_si128(*(const int *)(dest + stride)));
- d2 = _mm_unpacklo_epi32(
- d2, _mm_cvtsi32_si128(*(const int *)(dest + stride * 3)));
- d0 = _mm_unpacklo_epi8(d0, zero);
- d2 = _mm_unpacklo_epi8(d2, zero);
- d0 = _mm_add_epi16(d0, in[0]);
- d2 = _mm_add_epi16(d2, in[1]);
- d0 = _mm_packus_epi16(d0, d2);
- // store result[0]
- *(int *)dest = _mm_cvtsi128_si32(d0);
- // store result[1]
- d0 = _mm_srli_si128(d0, 4);
- *(int *)(dest + stride) = _mm_cvtsi128_si32(d0);
- // store result[2]
- d0 = _mm_srli_si128(d0, 4);
- *(int *)(dest + stride * 2) = _mm_cvtsi128_si32(d0);
- // store result[3]
- d0 = _mm_srli_si128(d0, 4);
- *(int *)(dest + stride * 3) = _mm_cvtsi128_si32(d0);
- }
-}
-
-void vp10_iht8x8_64_add_sse2(const tran_low_t *input, uint8_t *dest, int stride,
- int tx_type) {
- __m128i in[8];
- const __m128i zero = _mm_setzero_si128();
- const __m128i final_rounding = _mm_set1_epi16(1 << 4);
-
- // load input data
- in[0] = load_input_data(input);
- in[1] = load_input_data(input + 8 * 1);
- in[2] = load_input_data(input + 8 * 2);
- in[3] = load_input_data(input + 8 * 3);
- in[4] = load_input_data(input + 8 * 4);
- in[5] = load_input_data(input + 8 * 5);
- in[6] = load_input_data(input + 8 * 6);
- in[7] = load_input_data(input + 8 * 7);
-
- switch (tx_type) {
- case 0: // DCT_DCT
- idct8_sse2(in);
- idct8_sse2(in);
- break;
- case 1: // ADST_DCT
- idct8_sse2(in);
- iadst8_sse2(in);
- break;
- case 2: // DCT_ADST
- iadst8_sse2(in);
- idct8_sse2(in);
- break;
- case 3: // ADST_ADST
- iadst8_sse2(in);
- iadst8_sse2(in);
- break;
- default:
- assert(0);
- break;
- }
-
- // Final rounding and shift
- in[0] = _mm_adds_epi16(in[0], final_rounding);
- in[1] = _mm_adds_epi16(in[1], final_rounding);
- in[2] = _mm_adds_epi16(in[2], final_rounding);
- in[3] = _mm_adds_epi16(in[3], final_rounding);
- in[4] = _mm_adds_epi16(in[4], final_rounding);
- in[5] = _mm_adds_epi16(in[5], final_rounding);
- in[6] = _mm_adds_epi16(in[6], final_rounding);
- in[7] = _mm_adds_epi16(in[7], final_rounding);
-
- in[0] = _mm_srai_epi16(in[0], 5);
- in[1] = _mm_srai_epi16(in[1], 5);
- in[2] = _mm_srai_epi16(in[2], 5);
- in[3] = _mm_srai_epi16(in[3], 5);
- in[4] = _mm_srai_epi16(in[4], 5);
- in[5] = _mm_srai_epi16(in[5], 5);
- in[6] = _mm_srai_epi16(in[6], 5);
- in[7] = _mm_srai_epi16(in[7], 5);
-
- RECON_AND_STORE(dest + 0 * stride, in[0]);
- RECON_AND_STORE(dest + 1 * stride, in[1]);
- RECON_AND_STORE(dest + 2 * stride, in[2]);
- RECON_AND_STORE(dest + 3 * stride, in[3]);
- RECON_AND_STORE(dest + 4 * stride, in[4]);
- RECON_AND_STORE(dest + 5 * stride, in[5]);
- RECON_AND_STORE(dest + 6 * stride, in[6]);
- RECON_AND_STORE(dest + 7 * stride, in[7]);
-}
-
-void vp10_iht16x16_256_add_sse2(const tran_low_t *input, uint8_t *dest,
- int stride, int tx_type) {
- __m128i in0[16], in1[16];
-
- load_buffer_8x16(input, in0);
- input += 8;
- load_buffer_8x16(input, in1);
-
- switch (tx_type) {
- case 0: // DCT_DCT
- idct16_sse2(in0, in1);
- idct16_sse2(in0, in1);
- break;
- case 1: // ADST_DCT
- idct16_sse2(in0, in1);
- iadst16_sse2(in0, in1);
- break;
- case 2: // DCT_ADST
- iadst16_sse2(in0, in1);
- idct16_sse2(in0, in1);
- break;
- case 3: // ADST_ADST
- iadst16_sse2(in0, in1);
- iadst16_sse2(in0, in1);
- break;
- default:
- assert(0);
- break;
- }
-
- write_buffer_8x16(dest, in0, stride);
- dest += 8;
- write_buffer_8x16(dest, in1, stride);
-}
+++ /dev/null
-;
-; Copyright (c) 2015 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-; This file is a duplicate of mfqe_sse2.asm in VP8.
-; TODO(jackychen): Find a way to fix the duplicate.
-%include "vpx_ports/x86_abi_support.asm"
-
-;void vp10_filter_by_weight16x16_sse2
-;(
-; unsigned char *src,
-; int src_stride,
-; unsigned char *dst,
-; int dst_stride,
-; int src_weight
-;)
-global sym(vp10_filter_by_weight16x16_sse2) PRIVATE
-sym(vp10_filter_by_weight16x16_sse2):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 5
- SAVE_XMM 6
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
- movd xmm0, arg(4) ; src_weight
- pshuflw xmm0, xmm0, 0x0 ; replicate to all low words
- punpcklqdq xmm0, xmm0 ; replicate to all hi words
-
- movdqa xmm1, [GLOBAL(tMFQE)]
- psubw xmm1, xmm0 ; dst_weight
-
- mov rax, arg(0) ; src
- mov rsi, arg(1) ; src_stride
- mov rdx, arg(2) ; dst
- mov rdi, arg(3) ; dst_stride
-
- mov rcx, 16 ; loop count
- pxor xmm6, xmm6
-
-.combine
- movdqa xmm2, [rax]
- movdqa xmm4, [rdx]
- add rax, rsi
-
- ; src * src_weight
- movdqa xmm3, xmm2
- punpcklbw xmm2, xmm6
- punpckhbw xmm3, xmm6
- pmullw xmm2, xmm0
- pmullw xmm3, xmm0
-
- ; dst * dst_weight
- movdqa xmm5, xmm4
- punpcklbw xmm4, xmm6
- punpckhbw xmm5, xmm6
- pmullw xmm4, xmm1
- pmullw xmm5, xmm1
-
- ; sum, round and shift
- paddw xmm2, xmm4
- paddw xmm3, xmm5
- paddw xmm2, [GLOBAL(tMFQE_round)]
- paddw xmm3, [GLOBAL(tMFQE_round)]
- psrlw xmm2, 4
- psrlw xmm3, 4
-
- packuswb xmm2, xmm3
- movdqa [rdx], xmm2
- add rdx, rdi
-
- dec rcx
- jnz .combine
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
-
- ret
-
-;void vp10_filter_by_weight8x8_sse2
-;(
-; unsigned char *src,
-; int src_stride,
-; unsigned char *dst,
-; int dst_stride,
-; int src_weight
-;)
-global sym(vp10_filter_by_weight8x8_sse2) PRIVATE
-sym(vp10_filter_by_weight8x8_sse2):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 5
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
- movd xmm0, arg(4) ; src_weight
- pshuflw xmm0, xmm0, 0x0 ; replicate to all low words
- punpcklqdq xmm0, xmm0 ; replicate to all hi words
-
- movdqa xmm1, [GLOBAL(tMFQE)]
- psubw xmm1, xmm0 ; dst_weight
-
- mov rax, arg(0) ; src
- mov rsi, arg(1) ; src_stride
- mov rdx, arg(2) ; dst
- mov rdi, arg(3) ; dst_stride
-
- mov rcx, 8 ; loop count
- pxor xmm4, xmm4
-
-.combine
- movq xmm2, [rax]
- movq xmm3, [rdx]
- add rax, rsi
-
- ; src * src_weight
- punpcklbw xmm2, xmm4
- pmullw xmm2, xmm0
-
- ; dst * dst_weight
- punpcklbw xmm3, xmm4
- pmullw xmm3, xmm1
-
- ; sum, round and shift
- paddw xmm2, xmm3
- paddw xmm2, [GLOBAL(tMFQE_round)]
- psrlw xmm2, 4
-
- packuswb xmm2, xmm4
- movq [rdx], xmm2
- add rdx, rdi
-
- dec rcx
- jnz .combine
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- UNSHADOW_ARGS
- pop rbp
-
- ret
-
-;void vp10_variance_and_sad_16x16_sse2 | arg
-;(
-; unsigned char *src1, 0
-; int stride1, 1
-; unsigned char *src2, 2
-; int stride2, 3
-; unsigned int *variance, 4
-; unsigned int *sad, 5
-;)
-global sym(vp10_variance_and_sad_16x16_sse2) PRIVATE
-sym(vp10_variance_and_sad_16x16_sse2):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 6
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
- mov rax, arg(0) ; src1
- mov rcx, arg(1) ; stride1
- mov rdx, arg(2) ; src2
- mov rdi, arg(3) ; stride2
-
- mov rsi, 16 ; block height
-
- ; Prep accumulator registers
- pxor xmm3, xmm3 ; SAD
- pxor xmm4, xmm4 ; sum of src2
- pxor xmm5, xmm5 ; sum of src2^2
-
- ; Because we're working with the actual output frames
- ; we can't depend on any kind of data alignment.
-.accumulate
- movdqa xmm0, [rax] ; src1
- movdqa xmm1, [rdx] ; src2
- add rax, rcx ; src1 + stride1
- add rdx, rdi ; src2 + stride2
-
- ; SAD(src1, src2)
- psadbw xmm0, xmm1
- paddusw xmm3, xmm0
-
- ; SUM(src2)
- pxor xmm2, xmm2
- psadbw xmm2, xmm1 ; sum src2 by misusing SAD against 0
- paddusw xmm4, xmm2
-
- ; pmaddubsw would be ideal if it took two unsigned values. instead,
- ; it expects a signed and an unsigned value. so instead we zero extend
- ; and operate on words.
- pxor xmm2, xmm2
- movdqa xmm0, xmm1
- punpcklbw xmm0, xmm2
- punpckhbw xmm1, xmm2
- pmaddwd xmm0, xmm0
- pmaddwd xmm1, xmm1
- paddd xmm5, xmm0
- paddd xmm5, xmm1
-
- sub rsi, 1
- jnz .accumulate
-
- ; phaddd only operates on adjacent double words.
- ; Finalize SAD and store
- movdqa xmm0, xmm3
- psrldq xmm0, 8
- paddusw xmm0, xmm3
- paddd xmm0, [GLOBAL(t128)]
- psrld xmm0, 8
-
- mov rax, arg(5)
- movd [rax], xmm0
-
- ; Accumulate sum of src2
- movdqa xmm0, xmm4
- psrldq xmm0, 8
- paddusw xmm0, xmm4
- ; Square src2. Ignore high value
- pmuludq xmm0, xmm0
- psrld xmm0, 8
-
- ; phaddw could be used to sum adjacent values but we want
- ; all the values summed. promote to doubles, accumulate,
- ; shift and sum
- pxor xmm2, xmm2
- movdqa xmm1, xmm5
- punpckldq xmm1, xmm2
- punpckhdq xmm5, xmm2
- paddd xmm1, xmm5
- movdqa xmm2, xmm1
- psrldq xmm1, 8
- paddd xmm1, xmm2
-
- psubd xmm1, xmm0
-
- ; (variance + 128) >> 8
- paddd xmm1, [GLOBAL(t128)]
- psrld xmm1, 8
- mov rax, arg(4)
-
- movd [rax], xmm1
-
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- UNSHADOW_ARGS
- pop rbp
- ret
-
-SECTION_RODATA
-align 16
-t128:
-%ifndef __NASM_VER__
- ddq 128
-%elif CONFIG_BIG_ENDIAN
- dq 0, 128
-%else
- dq 128, 0
-%endif
-align 16
-tMFQE: ; 1 << MFQE_PRECISION
- times 8 dw 0x10
-align 16
-tMFQE_round: ; 1 << (MFQE_PRECISION - 1)
- times 8 dw 0x08
+++ /dev/null
-;
-; Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-
-%include "vpx_ports/x86_abi_support.asm"
-
-;void vp10_post_proc_down_and_across_xmm
-;(
-; unsigned char *src_ptr,
-; unsigned char *dst_ptr,
-; int src_pixels_per_line,
-; int dst_pixels_per_line,
-; int rows,
-; int cols,
-; int flimit
-;)
-global sym(vp10_post_proc_down_and_across_xmm) PRIVATE
-sym(vp10_post_proc_down_and_across_xmm):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 7
- SAVE_XMM 7
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
-%if ABI_IS_32BIT=1 && CONFIG_PIC=1
- ALIGN_STACK 16, rax
- ; move the global rd onto the stack, since we don't have enough registers
- ; to do PIC addressing
- movdqa xmm0, [GLOBAL(rd42)]
- sub rsp, 16
- movdqa [rsp], xmm0
-%define RD42 [rsp]
-%else
-%define RD42 [GLOBAL(rd42)]
-%endif
-
-
- movd xmm2, dword ptr arg(6) ;flimit
- punpcklwd xmm2, xmm2
- punpckldq xmm2, xmm2
- punpcklqdq xmm2, xmm2
-
- mov rsi, arg(0) ;src_ptr
- mov rdi, arg(1) ;dst_ptr
-
- movsxd rcx, DWORD PTR arg(4) ;rows
- movsxd rax, DWORD PTR arg(2) ;src_pixels_per_line ; destination pitch?
- pxor xmm0, xmm0 ; mm0 = 00000000
-
-.nextrow:
-
- xor rdx, rdx ; clear out rdx for use as loop counter
-.nextcol:
- movq xmm3, QWORD PTR [rsi] ; mm4 = r0 p0..p7
- punpcklbw xmm3, xmm0 ; mm3 = p0..p3
- movdqa xmm1, xmm3 ; mm1 = p0..p3
- psllw xmm3, 2 ;
-
- movq xmm5, QWORD PTR [rsi + rax] ; mm4 = r1 p0..p7
- punpcklbw xmm5, xmm0 ; mm5 = r1 p0..p3
- paddusw xmm3, xmm5 ; mm3 += mm6
-
- ; thresholding
- movdqa xmm7, xmm1 ; mm7 = r0 p0..p3
- psubusw xmm7, xmm5 ; mm7 = r0 p0..p3 - r1 p0..p3
- psubusw xmm5, xmm1 ; mm5 = r1 p0..p3 - r0 p0..p3
- paddusw xmm7, xmm5 ; mm7 = abs(r0 p0..p3 - r1 p0..p3)
- pcmpgtw xmm7, xmm2
-
- movq xmm5, QWORD PTR [rsi + 2*rax] ; mm4 = r2 p0..p7
- punpcklbw xmm5, xmm0 ; mm5 = r2 p0..p3
- paddusw xmm3, xmm5 ; mm3 += mm5
-
- ; thresholding
- movdqa xmm6, xmm1 ; mm6 = r0 p0..p3
- psubusw xmm6, xmm5 ; mm6 = r0 p0..p3 - r2 p0..p3
- psubusw xmm5, xmm1 ; mm5 = r2 p0..p3 - r2 p0..p3
- paddusw xmm6, xmm5 ; mm6 = abs(r0 p0..p3 - r2 p0..p3)
- pcmpgtw xmm6, xmm2
- por xmm7, xmm6 ; accumulate thresholds
-
-
- neg rax
- movq xmm5, QWORD PTR [rsi+2*rax] ; mm4 = r-2 p0..p7
- punpcklbw xmm5, xmm0 ; mm5 = r-2 p0..p3
- paddusw xmm3, xmm5 ; mm3 += mm5
-
- ; thresholding
- movdqa xmm6, xmm1 ; mm6 = r0 p0..p3
- psubusw xmm6, xmm5 ; mm6 = p0..p3 - r-2 p0..p3
- psubusw xmm5, xmm1 ; mm5 = r-2 p0..p3 - p0..p3
- paddusw xmm6, xmm5 ; mm6 = abs(r0 p0..p3 - r-2 p0..p3)
- pcmpgtw xmm6, xmm2
- por xmm7, xmm6 ; accumulate thresholds
-
- movq xmm4, QWORD PTR [rsi+rax] ; mm4 = r-1 p0..p7
- punpcklbw xmm4, xmm0 ; mm4 = r-1 p0..p3
- paddusw xmm3, xmm4 ; mm3 += mm5
-
- ; thresholding
- movdqa xmm6, xmm1 ; mm6 = r0 p0..p3
- psubusw xmm6, xmm4 ; mm6 = p0..p3 - r-2 p0..p3
- psubusw xmm4, xmm1 ; mm5 = r-1 p0..p3 - p0..p3
- paddusw xmm6, xmm4 ; mm6 = abs(r0 p0..p3 - r-1 p0..p3)
- pcmpgtw xmm6, xmm2
- por xmm7, xmm6 ; accumulate thresholds
-
-
- paddusw xmm3, RD42 ; mm3 += round value
- psraw xmm3, 3 ; mm3 /= 8
-
- pand xmm1, xmm7 ; mm1 select vals > thresh from source
- pandn xmm7, xmm3 ; mm7 select vals < thresh from blurred result
- paddusw xmm1, xmm7 ; combination
-
- packuswb xmm1, xmm0 ; pack to bytes
- movq QWORD PTR [rdi], xmm1 ;
-
- neg rax ; pitch is positive
- add rsi, 8
- add rdi, 8
-
- add rdx, 8
- cmp edx, dword arg(5) ;cols
-
- jl .nextcol
-
- ; done with the all cols, start the across filtering in place
- sub rsi, rdx
- sub rdi, rdx
-
- xor rdx, rdx
- movq mm0, QWORD PTR [rdi-8];
-
-.acrossnextcol:
- movq xmm7, QWORD PTR [rdi +rdx -2]
- movd xmm4, DWORD PTR [rdi +rdx +6]
-
- pslldq xmm4, 8
- por xmm4, xmm7
-
- movdqa xmm3, xmm4
- psrldq xmm3, 2
- punpcklbw xmm3, xmm0 ; mm3 = p0..p3
- movdqa xmm1, xmm3 ; mm1 = p0..p3
- psllw xmm3, 2
-
-
- movdqa xmm5, xmm4
- psrldq xmm5, 3
- punpcklbw xmm5, xmm0 ; mm5 = p1..p4
- paddusw xmm3, xmm5 ; mm3 += mm6
-
- ; thresholding
- movdqa xmm7, xmm1 ; mm7 = p0..p3
- psubusw xmm7, xmm5 ; mm7 = p0..p3 - p1..p4
- psubusw xmm5, xmm1 ; mm5 = p1..p4 - p0..p3
- paddusw xmm7, xmm5 ; mm7 = abs(p0..p3 - p1..p4)
- pcmpgtw xmm7, xmm2
-
- movdqa xmm5, xmm4
- psrldq xmm5, 4
- punpcklbw xmm5, xmm0 ; mm5 = p2..p5
- paddusw xmm3, xmm5 ; mm3 += mm5
-
- ; thresholding
- movdqa xmm6, xmm1 ; mm6 = p0..p3
- psubusw xmm6, xmm5 ; mm6 = p0..p3 - p1..p4
- psubusw xmm5, xmm1 ; mm5 = p1..p4 - p0..p3
- paddusw xmm6, xmm5 ; mm6 = abs(p0..p3 - p1..p4)
- pcmpgtw xmm6, xmm2
- por xmm7, xmm6 ; accumulate thresholds
-
-
- movdqa xmm5, xmm4 ; mm5 = p-2..p5
- punpcklbw xmm5, xmm0 ; mm5 = p-2..p1
- paddusw xmm3, xmm5 ; mm3 += mm5
-
- ; thresholding
- movdqa xmm6, xmm1 ; mm6 = p0..p3
- psubusw xmm6, xmm5 ; mm6 = p0..p3 - p1..p4
- psubusw xmm5, xmm1 ; mm5 = p1..p4 - p0..p3
- paddusw xmm6, xmm5 ; mm6 = abs(p0..p3 - p1..p4)
- pcmpgtw xmm6, xmm2
- por xmm7, xmm6 ; accumulate thresholds
-
- psrldq xmm4, 1 ; mm4 = p-1..p5
- punpcklbw xmm4, xmm0 ; mm4 = p-1..p2
- paddusw xmm3, xmm4 ; mm3 += mm5
-
- ; thresholding
- movdqa xmm6, xmm1 ; mm6 = p0..p3
- psubusw xmm6, xmm4 ; mm6 = p0..p3 - p1..p4
- psubusw xmm4, xmm1 ; mm5 = p1..p4 - p0..p3
- paddusw xmm6, xmm4 ; mm6 = abs(p0..p3 - p1..p4)
- pcmpgtw xmm6, xmm2
- por xmm7, xmm6 ; accumulate thresholds
-
- paddusw xmm3, RD42 ; mm3 += round value
- psraw xmm3, 3 ; mm3 /= 8
-
- pand xmm1, xmm7 ; mm1 select vals > thresh from source
- pandn xmm7, xmm3 ; mm7 select vals < thresh from blurred result
- paddusw xmm1, xmm7 ; combination
-
- packuswb xmm1, xmm0 ; pack to bytes
- movq QWORD PTR [rdi+rdx-8], mm0 ; store previous four bytes
- movdq2q mm0, xmm1
-
- add rdx, 8
- cmp edx, dword arg(5) ;cols
- jl .acrossnextcol;
-
- ; last 8 pixels
- movq QWORD PTR [rdi+rdx-8], mm0
-
- ; done with this rwo
- add rsi,rax ; next line
- mov eax, dword arg(3) ;dst_pixels_per_line ; destination pitch?
- add rdi,rax ; next destination
- mov eax, dword arg(2) ;src_pixels_per_line ; destination pitch?
-
- dec rcx ; decrement count
- jnz .nextrow ; next row
-
-%if ABI_IS_32BIT=1 && CONFIG_PIC=1
- add rsp,16
- pop rsp
-%endif
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
- ret
-%undef RD42
-
-
-;void vp10_mbpost_proc_down_xmm(unsigned char *dst,
-; int pitch, int rows, int cols,int flimit)
-extern sym(vp10_rv)
-global sym(vp10_mbpost_proc_down_xmm) PRIVATE
-sym(vp10_mbpost_proc_down_xmm):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 5
- SAVE_XMM 7
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
- ALIGN_STACK 16, rax
- sub rsp, 128+16
-
- ; unsigned char d[16][8] at [rsp]
- ; create flimit2 at [rsp+128]
- mov eax, dword ptr arg(4) ;flimit
- mov [rsp+128], eax
- mov [rsp+128+4], eax
- mov [rsp+128+8], eax
- mov [rsp+128+12], eax
-%define flimit4 [rsp+128]
-
-%if ABI_IS_32BIT=0
- lea r8, [GLOBAL(sym(vp10_rv))]
-%endif
-
- ;rows +=8;
- add dword arg(2), 8
-
- ;for(c=0; c<cols; c+=8)
-.loop_col:
- mov rsi, arg(0) ; s
- pxor xmm0, xmm0 ;
-
- movsxd rax, dword ptr arg(1) ;pitch ;
- neg rax ; rax = -pitch
-
- lea rsi, [rsi + rax*8]; ; rdi = s[-pitch*8]
- neg rax
-
-
- pxor xmm5, xmm5
- pxor xmm6, xmm6 ;
-
- pxor xmm7, xmm7 ;
- mov rdi, rsi
-
- mov rcx, 15 ;
-
-.loop_initvar:
- movq xmm1, QWORD PTR [rdi];
- punpcklbw xmm1, xmm0 ;
-
- paddw xmm5, xmm1 ;
- pmullw xmm1, xmm1 ;
-
- movdqa xmm2, xmm1 ;
- punpcklwd xmm1, xmm0 ;
-
- punpckhwd xmm2, xmm0 ;
- paddd xmm6, xmm1 ;
-
- paddd xmm7, xmm2 ;
- lea rdi, [rdi+rax] ;
-
- dec rcx
- jne .loop_initvar
- ;save the var and sum
- xor rdx, rdx
-.loop_row:
- movq xmm1, QWORD PTR [rsi] ; [s-pitch*8]
- movq xmm2, QWORD PTR [rdi] ; [s+pitch*7]
-
- punpcklbw xmm1, xmm0
- punpcklbw xmm2, xmm0
-
- paddw xmm5, xmm2
- psubw xmm5, xmm1
-
- pmullw xmm2, xmm2
- movdqa xmm4, xmm2
-
- punpcklwd xmm2, xmm0
- punpckhwd xmm4, xmm0
-
- paddd xmm6, xmm2
- paddd xmm7, xmm4
-
- pmullw xmm1, xmm1
- movdqa xmm2, xmm1
-
- punpcklwd xmm1, xmm0
- psubd xmm6, xmm1
-
- punpckhwd xmm2, xmm0
- psubd xmm7, xmm2
-
-
- movdqa xmm3, xmm6
- pslld xmm3, 4
-
- psubd xmm3, xmm6
- movdqa xmm1, xmm5
-
- movdqa xmm4, xmm5
- pmullw xmm1, xmm1
-
- pmulhw xmm4, xmm4
- movdqa xmm2, xmm1
-
- punpcklwd xmm1, xmm4
- punpckhwd xmm2, xmm4
-
- movdqa xmm4, xmm7
- pslld xmm4, 4
-
- psubd xmm4, xmm7
-
- psubd xmm3, xmm1
- psubd xmm4, xmm2
-
- psubd xmm3, flimit4
- psubd xmm4, flimit4
-
- psrad xmm3, 31
- psrad xmm4, 31
-
- packssdw xmm3, xmm4
- packsswb xmm3, xmm0
-
- movq xmm1, QWORD PTR [rsi+rax*8]
-
- movq xmm2, xmm1
- punpcklbw xmm1, xmm0
-
- paddw xmm1, xmm5
- mov rcx, rdx
-
- and rcx, 127
-%if ABI_IS_32BIT=1 && CONFIG_PIC=1
- push rax
- lea rax, [GLOBAL(sym(vp10_rv))]
- movdqu xmm4, [rax + rcx*2] ;vp10_rv[rcx*2]
- pop rax
-%elif ABI_IS_32BIT=0
- movdqu xmm4, [r8 + rcx*2] ;vp10_rv[rcx*2]
-%else
- movdqu xmm4, [sym(vp10_rv) + rcx*2]
-%endif
-
- paddw xmm1, xmm4
- ;paddw xmm1, eight8s
- psraw xmm1, 4
-
- packuswb xmm1, xmm0
- pand xmm1, xmm3
-
- pandn xmm3, xmm2
- por xmm1, xmm3
-
- and rcx, 15
- movq QWORD PTR [rsp + rcx*8], xmm1 ;d[rcx*8]
-
- mov rcx, rdx
- sub rcx, 8
-
- and rcx, 15
- movq mm0, [rsp + rcx*8] ;d[rcx*8]
-
- movq [rsi], mm0
- lea rsi, [rsi+rax]
-
- lea rdi, [rdi+rax]
- add rdx, 1
-
- cmp edx, dword arg(2) ;rows
- jl .loop_row
-
- add dword arg(0), 8 ; s += 8
- sub dword arg(3), 8 ; cols -= 8
- cmp dword arg(3), 0
- jg .loop_col
-
- add rsp, 128+16
- pop rsp
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
- ret
-%undef flimit4
-
-
-;void vp10_mbpost_proc_across_ip_xmm(unsigned char *src,
-; int pitch, int rows, int cols,int flimit)
-global sym(vp10_mbpost_proc_across_ip_xmm) PRIVATE
-sym(vp10_mbpost_proc_across_ip_xmm):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 5
- SAVE_XMM 7
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
- ALIGN_STACK 16, rax
- sub rsp, 16
-
- ; create flimit4 at [rsp]
- mov eax, dword ptr arg(4) ;flimit
- mov [rsp], eax
- mov [rsp+4], eax
- mov [rsp+8], eax
- mov [rsp+12], eax
-%define flimit4 [rsp]
-
-
- ;for(r=0;r<rows;r++)
-.ip_row_loop:
-
- xor rdx, rdx ;sumsq=0;
- xor rcx, rcx ;sum=0;
- mov rsi, arg(0); s
- mov rdi, -8
-.ip_var_loop:
- ;for(i=-8;i<=6;i++)
- ;{
- ; sumsq += s[i]*s[i];
- ; sum += s[i];
- ;}
- movzx eax, byte [rsi+rdi]
- add ecx, eax
- mul al
- add edx, eax
- add rdi, 1
- cmp rdi, 6
- jle .ip_var_loop
-
-
- ;mov rax, sumsq
- ;movd xmm7, rax
- movd xmm7, edx
-
- ;mov rax, sum
- ;movd xmm6, rax
- movd xmm6, ecx
-
- mov rsi, arg(0) ;s
- xor rcx, rcx
-
- movsxd rdx, dword arg(3) ;cols
- add rdx, 8
- pxor mm0, mm0
- pxor mm1, mm1
-
- pxor xmm0, xmm0
-.nextcol4:
-
- movd xmm1, DWORD PTR [rsi+rcx-8] ; -8 -7 -6 -5
- movd xmm2, DWORD PTR [rsi+rcx+7] ; +7 +8 +9 +10
-
- punpcklbw xmm1, xmm0 ; expanding
- punpcklbw xmm2, xmm0 ; expanding
-
- punpcklwd xmm1, xmm0 ; expanding to dwords
- punpcklwd xmm2, xmm0 ; expanding to dwords
-
- psubd xmm2, xmm1 ; 7--8 8--7 9--6 10--5
- paddd xmm1, xmm1 ; -8*2 -7*2 -6*2 -5*2
-
- paddd xmm1, xmm2 ; 7+-8 8+-7 9+-6 10+-5
- pmaddwd xmm1, xmm2 ; squared of 7+-8 8+-7 9+-6 10+-5
-
- paddd xmm6, xmm2
- paddd xmm7, xmm1
-
- pshufd xmm6, xmm6, 0 ; duplicate the last ones
- pshufd xmm7, xmm7, 0 ; duplicate the last ones
-
- psrldq xmm1, 4 ; 8--7 9--6 10--5 0000
- psrldq xmm2, 4 ; 8--7 9--6 10--5 0000
-
- pshufd xmm3, xmm1, 3 ; 0000 8--7 8--7 8--7 squared
- pshufd xmm4, xmm2, 3 ; 0000 8--7 8--7 8--7 squared
-
- paddd xmm6, xmm4
- paddd xmm7, xmm3
-
- pshufd xmm3, xmm1, 01011111b ; 0000 0000 9--6 9--6 squared
- pshufd xmm4, xmm2, 01011111b ; 0000 0000 9--6 9--6 squared
-
- paddd xmm7, xmm3
- paddd xmm6, xmm4
-
- pshufd xmm3, xmm1, 10111111b ; 0000 0000 8--7 8--7 squared
- pshufd xmm4, xmm2, 10111111b ; 0000 0000 8--7 8--7 squared
-
- paddd xmm7, xmm3
- paddd xmm6, xmm4
-
- movdqa xmm3, xmm6
- pmaddwd xmm3, xmm3
-
- movdqa xmm5, xmm7
- pslld xmm5, 4
-
- psubd xmm5, xmm7
- psubd xmm5, xmm3
-
- psubd xmm5, flimit4
- psrad xmm5, 31
-
- packssdw xmm5, xmm0
- packsswb xmm5, xmm0
-
- movd xmm1, DWORD PTR [rsi+rcx]
- movq xmm2, xmm1
-
- punpcklbw xmm1, xmm0
- punpcklwd xmm1, xmm0
-
- paddd xmm1, xmm6
- paddd xmm1, [GLOBAL(four8s)]
-
- psrad xmm1, 4
- packssdw xmm1, xmm0
-
- packuswb xmm1, xmm0
- pand xmm1, xmm5
-
- pandn xmm5, xmm2
- por xmm5, xmm1
-
- movd [rsi+rcx-8], mm0
- movq mm0, mm1
-
- movdq2q mm1, xmm5
- psrldq xmm7, 12
-
- psrldq xmm6, 12
- add rcx, 4
-
- cmp rcx, rdx
- jl .nextcol4
-
- ;s+=pitch;
- movsxd rax, dword arg(1)
- add arg(0), rax
-
- sub dword arg(2), 1 ;rows-=1
- cmp dword arg(2), 0
- jg .ip_row_loop
-
- add rsp, 16
- pop rsp
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
- ret
-%undef flimit4
-
-
-;void vp10_plane_add_noise_wmt (unsigned char *start, unsigned char *noise,
-; unsigned char blackclamp[16],
-; unsigned char whiteclamp[16],
-; unsigned char bothclamp[16],
-; unsigned int width, unsigned int height, int pitch)
-global sym(vp10_plane_add_noise_wmt) PRIVATE
-sym(vp10_plane_add_noise_wmt):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 8
- GET_GOT rbx
- push rsi
- push rdi
- ; end prolog
-
-.addnoise_loop:
- call sym(LIBVPX_RAND) WRT_PLT
- mov rcx, arg(1) ;noise
- and rax, 0xff
- add rcx, rax
-
- ; we rely on the fact that the clamping vectors are stored contiguously
- ; in black/white/both order. Note that we have to reload this here because
- ; rdx could be trashed by rand()
- mov rdx, arg(2) ; blackclamp
-
-
- mov rdi, rcx
- movsxd rcx, dword arg(5) ;[Width]
- mov rsi, arg(0) ;Pos
- xor rax,rax
-
-.addnoise_nextset:
- movdqu xmm1,[rsi+rax] ; get the source
-
- psubusb xmm1, [rdx] ;blackclamp ; clamp both sides so we don't outrange adding noise
- paddusb xmm1, [rdx+32] ;bothclamp
- psubusb xmm1, [rdx+16] ;whiteclamp
-
- movdqu xmm2,[rdi+rax] ; get the noise for this line
- paddb xmm1,xmm2 ; add it in
- movdqu [rsi+rax],xmm1 ; store the result
-
- add rax,16 ; move to the next line
-
- cmp rax, rcx
- jl .addnoise_nextset
-
- movsxd rax, dword arg(7) ; Pitch
- add arg(0), rax ; Start += Pitch
- sub dword arg(6), 1 ; Height -= 1
- jg .addnoise_loop
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_GOT
- UNSHADOW_ARGS
- pop rbp
- ret
-
-
-SECTION_RODATA
-align 16
-rd42:
- times 8 dw 0x04
-four8s:
- times 4 dd 8
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h> // SSE2
-
-#include "./vp10_rtcd.h"
-#include "vp10/common/vp10_fwd_txfm.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_dsp/x86/txfm_common_sse2.h"
-
-// TODO(jingning) The high bit-depth version needs re-work for performance.
-// The current SSE2 implementation also causes cross reference to the static
-// functions in the C implementation file.
-#if DCT_HIGH_BIT_DEPTH
-#define ADD_EPI16 _mm_adds_epi16
-#define SUB_EPI16 _mm_subs_epi16
-#if FDCT32x32_HIGH_PRECISION
-void vp10_fdct32x32_rows_c(const int16_t *intermediate, tran_low_t *out) {
- int i, j;
- for (i = 0; i < 32; ++i) {
- tran_high_t temp_in[32], temp_out[32];
- for (j = 0; j < 32; ++j)
- temp_in[j] = intermediate[j * 32 + i];
- vp10_fdct32(temp_in, temp_out, 0);
- for (j = 0; j < 32; ++j)
- out[j + i * 32] =
- (tran_low_t)((temp_out[j] + 1 + (temp_out[j] < 0)) >> 2);
- }
-}
- #define HIGH_FDCT32x32_2D_C vp10_highbd_fdct32x32_c
- #define HIGH_FDCT32x32_2D_ROWS_C vp10_fdct32x32_rows_c
-#else
-void vp10_fdct32x32_rd_rows_c(const int16_t *intermediate, tran_low_t *out) {
- int i, j;
- for (i = 0; i < 32; ++i) {
- tran_high_t temp_in[32], temp_out[32];
- for (j = 0; j < 32; ++j)
- temp_in[j] = intermediate[j * 32 + i];
- vp10_fdct32(temp_in, temp_out, 1);
- for (j = 0; j < 32; ++j)
- out[j + i * 32] = (tran_low_t)temp_out[j];
- }
-}
- #define HIGH_FDCT32x32_2D_C vp10_highbd_fdct32x32_rd_c
- #define HIGH_FDCT32x32_2D_ROWS_C vp10_fdct32x32_rd_rows_c
-#endif // FDCT32x32_HIGH_PRECISION
-#else
-#define ADD_EPI16 _mm_add_epi16
-#define SUB_EPI16 _mm_sub_epi16
-#endif // DCT_HIGH_BIT_DEPTH
-
-
-void FDCT32x32_2D(const int16_t *input,
- tran_low_t *output_org, int stride) {
- // Calculate pre-multiplied strides
- const int str1 = stride;
- const int str2 = 2 * stride;
- const int str3 = 2 * stride + str1;
- // We need an intermediate buffer between passes.
- DECLARE_ALIGNED(16, int16_t, intermediate[32 * 32]);
- // Constants
- // When we use them, in one case, they are all the same. In all others
- // it's a pair of them that we need to repeat four times. This is done
- // by constructing the 32 bit constant corresponding to that pair.
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(+cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_m24_m08 = pair_set_epi16(-cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(+cospi_24_64, cospi_8_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(+cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(+cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m28_m04 = pair_set_epi16(-cospi_28_64, -cospi_4_64);
- const __m128i k__cospi_m12_m20 = pair_set_epi16(-cospi_12_64, -cospi_20_64);
- const __m128i k__cospi_p30_p02 = pair_set_epi16(+cospi_30_64, cospi_2_64);
- const __m128i k__cospi_p14_p18 = pair_set_epi16(+cospi_14_64, cospi_18_64);
- const __m128i k__cospi_p22_p10 = pair_set_epi16(+cospi_22_64, cospi_10_64);
- const __m128i k__cospi_p06_p26 = pair_set_epi16(+cospi_6_64, cospi_26_64);
- const __m128i k__cospi_m26_p06 = pair_set_epi16(-cospi_26_64, cospi_6_64);
- const __m128i k__cospi_m10_p22 = pair_set_epi16(-cospi_10_64, cospi_22_64);
- const __m128i k__cospi_m18_p14 = pair_set_epi16(-cospi_18_64, cospi_14_64);
- const __m128i k__cospi_m02_p30 = pair_set_epi16(-cospi_2_64, cospi_30_64);
- const __m128i k__cospi_p31_p01 = pair_set_epi16(+cospi_31_64, cospi_1_64);
- const __m128i k__cospi_p15_p17 = pair_set_epi16(+cospi_15_64, cospi_17_64);
- const __m128i k__cospi_p23_p09 = pair_set_epi16(+cospi_23_64, cospi_9_64);
- const __m128i k__cospi_p07_p25 = pair_set_epi16(+cospi_7_64, cospi_25_64);
- const __m128i k__cospi_m25_p07 = pair_set_epi16(-cospi_25_64, cospi_7_64);
- const __m128i k__cospi_m09_p23 = pair_set_epi16(-cospi_9_64, cospi_23_64);
- const __m128i k__cospi_m17_p15 = pair_set_epi16(-cospi_17_64, cospi_15_64);
- const __m128i k__cospi_m01_p31 = pair_set_epi16(-cospi_1_64, cospi_31_64);
- const __m128i k__cospi_p27_p05 = pair_set_epi16(+cospi_27_64, cospi_5_64);
- const __m128i k__cospi_p11_p21 = pair_set_epi16(+cospi_11_64, cospi_21_64);
- const __m128i k__cospi_p19_p13 = pair_set_epi16(+cospi_19_64, cospi_13_64);
- const __m128i k__cospi_p03_p29 = pair_set_epi16(+cospi_3_64, cospi_29_64);
- const __m128i k__cospi_m29_p03 = pair_set_epi16(-cospi_29_64, cospi_3_64);
- const __m128i k__cospi_m13_p19 = pair_set_epi16(-cospi_13_64, cospi_19_64);
- const __m128i k__cospi_m21_p11 = pair_set_epi16(-cospi_21_64, cospi_11_64);
- const __m128i k__cospi_m05_p27 = pair_set_epi16(-cospi_5_64, cospi_27_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i kZero = _mm_set1_epi16(0);
- const __m128i kOne = _mm_set1_epi16(1);
- // Do the two transform/transpose passes
- int pass;
-#if DCT_HIGH_BIT_DEPTH
- int overflow;
-#endif
- for (pass = 0; pass < 2; ++pass) {
- // We process eight columns (transposed rows in second pass) at a time.
- int column_start;
- for (column_start = 0; column_start < 32; column_start += 8) {
- __m128i step1[32];
- __m128i step2[32];
- __m128i step3[32];
- __m128i out[32];
- // Stage 1
- // Note: even though all the loads below are aligned, using the aligned
- // intrinsic make the code slightly slower.
- if (0 == pass) {
- const int16_t *in = &input[column_start];
- // step1[i] = (in[ 0 * stride] + in[(32 - 1) * stride]) << 2;
- // Note: the next four blocks could be in a loop. That would help the
- // instruction cache but is actually slower.
- {
- const int16_t *ina = in + 0 * str1;
- const int16_t *inb = in + 31 * str1;
- __m128i *step1a = &step1[ 0];
- __m128i *step1b = &step1[31];
- const __m128i ina0 = _mm_loadu_si128((const __m128i *)(ina));
- const __m128i ina1 = _mm_loadu_si128((const __m128i *)(ina + str1));
- const __m128i ina2 = _mm_loadu_si128((const __m128i *)(ina + str2));
- const __m128i ina3 = _mm_loadu_si128((const __m128i *)(ina + str3));
- const __m128i inb3 = _mm_loadu_si128((const __m128i *)(inb - str3));
- const __m128i inb2 = _mm_loadu_si128((const __m128i *)(inb - str2));
- const __m128i inb1 = _mm_loadu_si128((const __m128i *)(inb - str1));
- const __m128i inb0 = _mm_loadu_si128((const __m128i *)(inb));
- step1a[ 0] = _mm_add_epi16(ina0, inb0);
- step1a[ 1] = _mm_add_epi16(ina1, inb1);
- step1a[ 2] = _mm_add_epi16(ina2, inb2);
- step1a[ 3] = _mm_add_epi16(ina3, inb3);
- step1b[-3] = _mm_sub_epi16(ina3, inb3);
- step1b[-2] = _mm_sub_epi16(ina2, inb2);
- step1b[-1] = _mm_sub_epi16(ina1, inb1);
- step1b[-0] = _mm_sub_epi16(ina0, inb0);
- step1a[ 0] = _mm_slli_epi16(step1a[ 0], 2);
- step1a[ 1] = _mm_slli_epi16(step1a[ 1], 2);
- step1a[ 2] = _mm_slli_epi16(step1a[ 2], 2);
- step1a[ 3] = _mm_slli_epi16(step1a[ 3], 2);
- step1b[-3] = _mm_slli_epi16(step1b[-3], 2);
- step1b[-2] = _mm_slli_epi16(step1b[-2], 2);
- step1b[-1] = _mm_slli_epi16(step1b[-1], 2);
- step1b[-0] = _mm_slli_epi16(step1b[-0], 2);
- }
- {
- const int16_t *ina = in + 4 * str1;
- const int16_t *inb = in + 27 * str1;
- __m128i *step1a = &step1[ 4];
- __m128i *step1b = &step1[27];
- const __m128i ina0 = _mm_loadu_si128((const __m128i *)(ina));
- const __m128i ina1 = _mm_loadu_si128((const __m128i *)(ina + str1));
- const __m128i ina2 = _mm_loadu_si128((const __m128i *)(ina + str2));
- const __m128i ina3 = _mm_loadu_si128((const __m128i *)(ina + str3));
- const __m128i inb3 = _mm_loadu_si128((const __m128i *)(inb - str3));
- const __m128i inb2 = _mm_loadu_si128((const __m128i *)(inb - str2));
- const __m128i inb1 = _mm_loadu_si128((const __m128i *)(inb - str1));
- const __m128i inb0 = _mm_loadu_si128((const __m128i *)(inb));
- step1a[ 0] = _mm_add_epi16(ina0, inb0);
- step1a[ 1] = _mm_add_epi16(ina1, inb1);
- step1a[ 2] = _mm_add_epi16(ina2, inb2);
- step1a[ 3] = _mm_add_epi16(ina3, inb3);
- step1b[-3] = _mm_sub_epi16(ina3, inb3);
- step1b[-2] = _mm_sub_epi16(ina2, inb2);
- step1b[-1] = _mm_sub_epi16(ina1, inb1);
- step1b[-0] = _mm_sub_epi16(ina0, inb0);
- step1a[ 0] = _mm_slli_epi16(step1a[ 0], 2);
- step1a[ 1] = _mm_slli_epi16(step1a[ 1], 2);
- step1a[ 2] = _mm_slli_epi16(step1a[ 2], 2);
- step1a[ 3] = _mm_slli_epi16(step1a[ 3], 2);
- step1b[-3] = _mm_slli_epi16(step1b[-3], 2);
- step1b[-2] = _mm_slli_epi16(step1b[-2], 2);
- step1b[-1] = _mm_slli_epi16(step1b[-1], 2);
- step1b[-0] = _mm_slli_epi16(step1b[-0], 2);
- }
- {
- const int16_t *ina = in + 8 * str1;
- const int16_t *inb = in + 23 * str1;
- __m128i *step1a = &step1[ 8];
- __m128i *step1b = &step1[23];
- const __m128i ina0 = _mm_loadu_si128((const __m128i *)(ina));
- const __m128i ina1 = _mm_loadu_si128((const __m128i *)(ina + str1));
- const __m128i ina2 = _mm_loadu_si128((const __m128i *)(ina + str2));
- const __m128i ina3 = _mm_loadu_si128((const __m128i *)(ina + str3));
- const __m128i inb3 = _mm_loadu_si128((const __m128i *)(inb - str3));
- const __m128i inb2 = _mm_loadu_si128((const __m128i *)(inb - str2));
- const __m128i inb1 = _mm_loadu_si128((const __m128i *)(inb - str1));
- const __m128i inb0 = _mm_loadu_si128((const __m128i *)(inb));
- step1a[ 0] = _mm_add_epi16(ina0, inb0);
- step1a[ 1] = _mm_add_epi16(ina1, inb1);
- step1a[ 2] = _mm_add_epi16(ina2, inb2);
- step1a[ 3] = _mm_add_epi16(ina3, inb3);
- step1b[-3] = _mm_sub_epi16(ina3, inb3);
- step1b[-2] = _mm_sub_epi16(ina2, inb2);
- step1b[-1] = _mm_sub_epi16(ina1, inb1);
- step1b[-0] = _mm_sub_epi16(ina0, inb0);
- step1a[ 0] = _mm_slli_epi16(step1a[ 0], 2);
- step1a[ 1] = _mm_slli_epi16(step1a[ 1], 2);
- step1a[ 2] = _mm_slli_epi16(step1a[ 2], 2);
- step1a[ 3] = _mm_slli_epi16(step1a[ 3], 2);
- step1b[-3] = _mm_slli_epi16(step1b[-3], 2);
- step1b[-2] = _mm_slli_epi16(step1b[-2], 2);
- step1b[-1] = _mm_slli_epi16(step1b[-1], 2);
- step1b[-0] = _mm_slli_epi16(step1b[-0], 2);
- }
- {
- const int16_t *ina = in + 12 * str1;
- const int16_t *inb = in + 19 * str1;
- __m128i *step1a = &step1[12];
- __m128i *step1b = &step1[19];
- const __m128i ina0 = _mm_loadu_si128((const __m128i *)(ina));
- const __m128i ina1 = _mm_loadu_si128((const __m128i *)(ina + str1));
- const __m128i ina2 = _mm_loadu_si128((const __m128i *)(ina + str2));
- const __m128i ina3 = _mm_loadu_si128((const __m128i *)(ina + str3));
- const __m128i inb3 = _mm_loadu_si128((const __m128i *)(inb - str3));
- const __m128i inb2 = _mm_loadu_si128((const __m128i *)(inb - str2));
- const __m128i inb1 = _mm_loadu_si128((const __m128i *)(inb - str1));
- const __m128i inb0 = _mm_loadu_si128((const __m128i *)(inb));
- step1a[ 0] = _mm_add_epi16(ina0, inb0);
- step1a[ 1] = _mm_add_epi16(ina1, inb1);
- step1a[ 2] = _mm_add_epi16(ina2, inb2);
- step1a[ 3] = _mm_add_epi16(ina3, inb3);
- step1b[-3] = _mm_sub_epi16(ina3, inb3);
- step1b[-2] = _mm_sub_epi16(ina2, inb2);
- step1b[-1] = _mm_sub_epi16(ina1, inb1);
- step1b[-0] = _mm_sub_epi16(ina0, inb0);
- step1a[ 0] = _mm_slli_epi16(step1a[ 0], 2);
- step1a[ 1] = _mm_slli_epi16(step1a[ 1], 2);
- step1a[ 2] = _mm_slli_epi16(step1a[ 2], 2);
- step1a[ 3] = _mm_slli_epi16(step1a[ 3], 2);
- step1b[-3] = _mm_slli_epi16(step1b[-3], 2);
- step1b[-2] = _mm_slli_epi16(step1b[-2], 2);
- step1b[-1] = _mm_slli_epi16(step1b[-1], 2);
- step1b[-0] = _mm_slli_epi16(step1b[-0], 2);
- }
- } else {
- int16_t *in = &intermediate[column_start];
- // step1[i] = in[ 0 * 32] + in[(32 - 1) * 32];
- // Note: using the same approach as above to have common offset is
- // counter-productive as all offsets can be calculated at compile
- // time.
- // Note: the next four blocks could be in a loop. That would help the
- // instruction cache but is actually slower.
- {
- __m128i in00 = _mm_loadu_si128((const __m128i *)(in + 0 * 32));
- __m128i in01 = _mm_loadu_si128((const __m128i *)(in + 1 * 32));
- __m128i in02 = _mm_loadu_si128((const __m128i *)(in + 2 * 32));
- __m128i in03 = _mm_loadu_si128((const __m128i *)(in + 3 * 32));
- __m128i in28 = _mm_loadu_si128((const __m128i *)(in + 28 * 32));
- __m128i in29 = _mm_loadu_si128((const __m128i *)(in + 29 * 32));
- __m128i in30 = _mm_loadu_si128((const __m128i *)(in + 30 * 32));
- __m128i in31 = _mm_loadu_si128((const __m128i *)(in + 31 * 32));
- step1[0] = ADD_EPI16(in00, in31);
- step1[1] = ADD_EPI16(in01, in30);
- step1[2] = ADD_EPI16(in02, in29);
- step1[3] = ADD_EPI16(in03, in28);
- step1[28] = SUB_EPI16(in03, in28);
- step1[29] = SUB_EPI16(in02, in29);
- step1[30] = SUB_EPI16(in01, in30);
- step1[31] = SUB_EPI16(in00, in31);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1[0], &step1[1], &step1[2],
- &step1[3], &step1[28], &step1[29],
- &step1[30], &step1[31]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- __m128i in04 = _mm_loadu_si128((const __m128i *)(in + 4 * 32));
- __m128i in05 = _mm_loadu_si128((const __m128i *)(in + 5 * 32));
- __m128i in06 = _mm_loadu_si128((const __m128i *)(in + 6 * 32));
- __m128i in07 = _mm_loadu_si128((const __m128i *)(in + 7 * 32));
- __m128i in24 = _mm_loadu_si128((const __m128i *)(in + 24 * 32));
- __m128i in25 = _mm_loadu_si128((const __m128i *)(in + 25 * 32));
- __m128i in26 = _mm_loadu_si128((const __m128i *)(in + 26 * 32));
- __m128i in27 = _mm_loadu_si128((const __m128i *)(in + 27 * 32));
- step1[4] = ADD_EPI16(in04, in27);
- step1[5] = ADD_EPI16(in05, in26);
- step1[6] = ADD_EPI16(in06, in25);
- step1[7] = ADD_EPI16(in07, in24);
- step1[24] = SUB_EPI16(in07, in24);
- step1[25] = SUB_EPI16(in06, in25);
- step1[26] = SUB_EPI16(in05, in26);
- step1[27] = SUB_EPI16(in04, in27);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1[4], &step1[5], &step1[6],
- &step1[7], &step1[24], &step1[25],
- &step1[26], &step1[27]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- __m128i in08 = _mm_loadu_si128((const __m128i *)(in + 8 * 32));
- __m128i in09 = _mm_loadu_si128((const __m128i *)(in + 9 * 32));
- __m128i in10 = _mm_loadu_si128((const __m128i *)(in + 10 * 32));
- __m128i in11 = _mm_loadu_si128((const __m128i *)(in + 11 * 32));
- __m128i in20 = _mm_loadu_si128((const __m128i *)(in + 20 * 32));
- __m128i in21 = _mm_loadu_si128((const __m128i *)(in + 21 * 32));
- __m128i in22 = _mm_loadu_si128((const __m128i *)(in + 22 * 32));
- __m128i in23 = _mm_loadu_si128((const __m128i *)(in + 23 * 32));
- step1[8] = ADD_EPI16(in08, in23);
- step1[9] = ADD_EPI16(in09, in22);
- step1[10] = ADD_EPI16(in10, in21);
- step1[11] = ADD_EPI16(in11, in20);
- step1[20] = SUB_EPI16(in11, in20);
- step1[21] = SUB_EPI16(in10, in21);
- step1[22] = SUB_EPI16(in09, in22);
- step1[23] = SUB_EPI16(in08, in23);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1[8], &step1[9], &step1[10],
- &step1[11], &step1[20], &step1[21],
- &step1[22], &step1[23]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- __m128i in12 = _mm_loadu_si128((const __m128i *)(in + 12 * 32));
- __m128i in13 = _mm_loadu_si128((const __m128i *)(in + 13 * 32));
- __m128i in14 = _mm_loadu_si128((const __m128i *)(in + 14 * 32));
- __m128i in15 = _mm_loadu_si128((const __m128i *)(in + 15 * 32));
- __m128i in16 = _mm_loadu_si128((const __m128i *)(in + 16 * 32));
- __m128i in17 = _mm_loadu_si128((const __m128i *)(in + 17 * 32));
- __m128i in18 = _mm_loadu_si128((const __m128i *)(in + 18 * 32));
- __m128i in19 = _mm_loadu_si128((const __m128i *)(in + 19 * 32));
- step1[12] = ADD_EPI16(in12, in19);
- step1[13] = ADD_EPI16(in13, in18);
- step1[14] = ADD_EPI16(in14, in17);
- step1[15] = ADD_EPI16(in15, in16);
- step1[16] = SUB_EPI16(in15, in16);
- step1[17] = SUB_EPI16(in14, in17);
- step1[18] = SUB_EPI16(in13, in18);
- step1[19] = SUB_EPI16(in12, in19);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1[12], &step1[13], &step1[14],
- &step1[15], &step1[16], &step1[17],
- &step1[18], &step1[19]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
- // Stage 2
- {
- step2[0] = ADD_EPI16(step1[0], step1[15]);
- step2[1] = ADD_EPI16(step1[1], step1[14]);
- step2[2] = ADD_EPI16(step1[2], step1[13]);
- step2[3] = ADD_EPI16(step1[3], step1[12]);
- step2[4] = ADD_EPI16(step1[4], step1[11]);
- step2[5] = ADD_EPI16(step1[5], step1[10]);
- step2[6] = ADD_EPI16(step1[6], step1[ 9]);
- step2[7] = ADD_EPI16(step1[7], step1[ 8]);
- step2[8] = SUB_EPI16(step1[7], step1[ 8]);
- step2[9] = SUB_EPI16(step1[6], step1[ 9]);
- step2[10] = SUB_EPI16(step1[5], step1[10]);
- step2[11] = SUB_EPI16(step1[4], step1[11]);
- step2[12] = SUB_EPI16(step1[3], step1[12]);
- step2[13] = SUB_EPI16(step1[2], step1[13]);
- step2[14] = SUB_EPI16(step1[1], step1[14]);
- step2[15] = SUB_EPI16(step1[0], step1[15]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x16(
- &step2[0], &step2[1], &step2[2], &step2[3],
- &step2[4], &step2[5], &step2[6], &step2[7],
- &step2[8], &step2[9], &step2[10], &step2[11],
- &step2[12], &step2[13], &step2[14], &step2[15]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i s2_20_0 = _mm_unpacklo_epi16(step1[27], step1[20]);
- const __m128i s2_20_1 = _mm_unpackhi_epi16(step1[27], step1[20]);
- const __m128i s2_21_0 = _mm_unpacklo_epi16(step1[26], step1[21]);
- const __m128i s2_21_1 = _mm_unpackhi_epi16(step1[26], step1[21]);
- const __m128i s2_22_0 = _mm_unpacklo_epi16(step1[25], step1[22]);
- const __m128i s2_22_1 = _mm_unpackhi_epi16(step1[25], step1[22]);
- const __m128i s2_23_0 = _mm_unpacklo_epi16(step1[24], step1[23]);
- const __m128i s2_23_1 = _mm_unpackhi_epi16(step1[24], step1[23]);
- const __m128i s2_20_2 = _mm_madd_epi16(s2_20_0, k__cospi_p16_m16);
- const __m128i s2_20_3 = _mm_madd_epi16(s2_20_1, k__cospi_p16_m16);
- const __m128i s2_21_2 = _mm_madd_epi16(s2_21_0, k__cospi_p16_m16);
- const __m128i s2_21_3 = _mm_madd_epi16(s2_21_1, k__cospi_p16_m16);
- const __m128i s2_22_2 = _mm_madd_epi16(s2_22_0, k__cospi_p16_m16);
- const __m128i s2_22_3 = _mm_madd_epi16(s2_22_1, k__cospi_p16_m16);
- const __m128i s2_23_2 = _mm_madd_epi16(s2_23_0, k__cospi_p16_m16);
- const __m128i s2_23_3 = _mm_madd_epi16(s2_23_1, k__cospi_p16_m16);
- const __m128i s2_24_2 = _mm_madd_epi16(s2_23_0, k__cospi_p16_p16);
- const __m128i s2_24_3 = _mm_madd_epi16(s2_23_1, k__cospi_p16_p16);
- const __m128i s2_25_2 = _mm_madd_epi16(s2_22_0, k__cospi_p16_p16);
- const __m128i s2_25_3 = _mm_madd_epi16(s2_22_1, k__cospi_p16_p16);
- const __m128i s2_26_2 = _mm_madd_epi16(s2_21_0, k__cospi_p16_p16);
- const __m128i s2_26_3 = _mm_madd_epi16(s2_21_1, k__cospi_p16_p16);
- const __m128i s2_27_2 = _mm_madd_epi16(s2_20_0, k__cospi_p16_p16);
- const __m128i s2_27_3 = _mm_madd_epi16(s2_20_1, k__cospi_p16_p16);
- // dct_const_round_shift
- const __m128i s2_20_4 = _mm_add_epi32(s2_20_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_20_5 = _mm_add_epi32(s2_20_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_21_4 = _mm_add_epi32(s2_21_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_21_5 = _mm_add_epi32(s2_21_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_22_4 = _mm_add_epi32(s2_22_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_22_5 = _mm_add_epi32(s2_22_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_23_4 = _mm_add_epi32(s2_23_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_23_5 = _mm_add_epi32(s2_23_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_24_4 = _mm_add_epi32(s2_24_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_24_5 = _mm_add_epi32(s2_24_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_25_4 = _mm_add_epi32(s2_25_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_25_5 = _mm_add_epi32(s2_25_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_26_4 = _mm_add_epi32(s2_26_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_26_5 = _mm_add_epi32(s2_26_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_27_4 = _mm_add_epi32(s2_27_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_27_5 = _mm_add_epi32(s2_27_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_20_6 = _mm_srai_epi32(s2_20_4, DCT_CONST_BITS);
- const __m128i s2_20_7 = _mm_srai_epi32(s2_20_5, DCT_CONST_BITS);
- const __m128i s2_21_6 = _mm_srai_epi32(s2_21_4, DCT_CONST_BITS);
- const __m128i s2_21_7 = _mm_srai_epi32(s2_21_5, DCT_CONST_BITS);
- const __m128i s2_22_6 = _mm_srai_epi32(s2_22_4, DCT_CONST_BITS);
- const __m128i s2_22_7 = _mm_srai_epi32(s2_22_5, DCT_CONST_BITS);
- const __m128i s2_23_6 = _mm_srai_epi32(s2_23_4, DCT_CONST_BITS);
- const __m128i s2_23_7 = _mm_srai_epi32(s2_23_5, DCT_CONST_BITS);
- const __m128i s2_24_6 = _mm_srai_epi32(s2_24_4, DCT_CONST_BITS);
- const __m128i s2_24_7 = _mm_srai_epi32(s2_24_5, DCT_CONST_BITS);
- const __m128i s2_25_6 = _mm_srai_epi32(s2_25_4, DCT_CONST_BITS);
- const __m128i s2_25_7 = _mm_srai_epi32(s2_25_5, DCT_CONST_BITS);
- const __m128i s2_26_6 = _mm_srai_epi32(s2_26_4, DCT_CONST_BITS);
- const __m128i s2_26_7 = _mm_srai_epi32(s2_26_5, DCT_CONST_BITS);
- const __m128i s2_27_6 = _mm_srai_epi32(s2_27_4, DCT_CONST_BITS);
- const __m128i s2_27_7 = _mm_srai_epi32(s2_27_5, DCT_CONST_BITS);
- // Combine
- step2[20] = _mm_packs_epi32(s2_20_6, s2_20_7);
- step2[21] = _mm_packs_epi32(s2_21_6, s2_21_7);
- step2[22] = _mm_packs_epi32(s2_22_6, s2_22_7);
- step2[23] = _mm_packs_epi32(s2_23_6, s2_23_7);
- step2[24] = _mm_packs_epi32(s2_24_6, s2_24_7);
- step2[25] = _mm_packs_epi32(s2_25_6, s2_25_7);
- step2[26] = _mm_packs_epi32(s2_26_6, s2_26_7);
- step2[27] = _mm_packs_epi32(s2_27_6, s2_27_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step2[20], &step2[21], &step2[22],
- &step2[23], &step2[24], &step2[25],
- &step2[26], &step2[27]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
-
-#if !FDCT32x32_HIGH_PRECISION
- // dump the magnitude by half, hence the intermediate values are within
- // the range of 16 bits.
- if (1 == pass) {
- __m128i s3_00_0 = _mm_cmplt_epi16(step2[ 0], kZero);
- __m128i s3_01_0 = _mm_cmplt_epi16(step2[ 1], kZero);
- __m128i s3_02_0 = _mm_cmplt_epi16(step2[ 2], kZero);
- __m128i s3_03_0 = _mm_cmplt_epi16(step2[ 3], kZero);
- __m128i s3_04_0 = _mm_cmplt_epi16(step2[ 4], kZero);
- __m128i s3_05_0 = _mm_cmplt_epi16(step2[ 5], kZero);
- __m128i s3_06_0 = _mm_cmplt_epi16(step2[ 6], kZero);
- __m128i s3_07_0 = _mm_cmplt_epi16(step2[ 7], kZero);
- __m128i s2_08_0 = _mm_cmplt_epi16(step2[ 8], kZero);
- __m128i s2_09_0 = _mm_cmplt_epi16(step2[ 9], kZero);
- __m128i s3_10_0 = _mm_cmplt_epi16(step2[10], kZero);
- __m128i s3_11_0 = _mm_cmplt_epi16(step2[11], kZero);
- __m128i s3_12_0 = _mm_cmplt_epi16(step2[12], kZero);
- __m128i s3_13_0 = _mm_cmplt_epi16(step2[13], kZero);
- __m128i s2_14_0 = _mm_cmplt_epi16(step2[14], kZero);
- __m128i s2_15_0 = _mm_cmplt_epi16(step2[15], kZero);
- __m128i s3_16_0 = _mm_cmplt_epi16(step1[16], kZero);
- __m128i s3_17_0 = _mm_cmplt_epi16(step1[17], kZero);
- __m128i s3_18_0 = _mm_cmplt_epi16(step1[18], kZero);
- __m128i s3_19_0 = _mm_cmplt_epi16(step1[19], kZero);
- __m128i s3_20_0 = _mm_cmplt_epi16(step2[20], kZero);
- __m128i s3_21_0 = _mm_cmplt_epi16(step2[21], kZero);
- __m128i s3_22_0 = _mm_cmplt_epi16(step2[22], kZero);
- __m128i s3_23_0 = _mm_cmplt_epi16(step2[23], kZero);
- __m128i s3_24_0 = _mm_cmplt_epi16(step2[24], kZero);
- __m128i s3_25_0 = _mm_cmplt_epi16(step2[25], kZero);
- __m128i s3_26_0 = _mm_cmplt_epi16(step2[26], kZero);
- __m128i s3_27_0 = _mm_cmplt_epi16(step2[27], kZero);
- __m128i s3_28_0 = _mm_cmplt_epi16(step1[28], kZero);
- __m128i s3_29_0 = _mm_cmplt_epi16(step1[29], kZero);
- __m128i s3_30_0 = _mm_cmplt_epi16(step1[30], kZero);
- __m128i s3_31_0 = _mm_cmplt_epi16(step1[31], kZero);
-
- step2[0] = SUB_EPI16(step2[ 0], s3_00_0);
- step2[1] = SUB_EPI16(step2[ 1], s3_01_0);
- step2[2] = SUB_EPI16(step2[ 2], s3_02_0);
- step2[3] = SUB_EPI16(step2[ 3], s3_03_0);
- step2[4] = SUB_EPI16(step2[ 4], s3_04_0);
- step2[5] = SUB_EPI16(step2[ 5], s3_05_0);
- step2[6] = SUB_EPI16(step2[ 6], s3_06_0);
- step2[7] = SUB_EPI16(step2[ 7], s3_07_0);
- step2[8] = SUB_EPI16(step2[ 8], s2_08_0);
- step2[9] = SUB_EPI16(step2[ 9], s2_09_0);
- step2[10] = SUB_EPI16(step2[10], s3_10_0);
- step2[11] = SUB_EPI16(step2[11], s3_11_0);
- step2[12] = SUB_EPI16(step2[12], s3_12_0);
- step2[13] = SUB_EPI16(step2[13], s3_13_0);
- step2[14] = SUB_EPI16(step2[14], s2_14_0);
- step2[15] = SUB_EPI16(step2[15], s2_15_0);
- step1[16] = SUB_EPI16(step1[16], s3_16_0);
- step1[17] = SUB_EPI16(step1[17], s3_17_0);
- step1[18] = SUB_EPI16(step1[18], s3_18_0);
- step1[19] = SUB_EPI16(step1[19], s3_19_0);
- step2[20] = SUB_EPI16(step2[20], s3_20_0);
- step2[21] = SUB_EPI16(step2[21], s3_21_0);
- step2[22] = SUB_EPI16(step2[22], s3_22_0);
- step2[23] = SUB_EPI16(step2[23], s3_23_0);
- step2[24] = SUB_EPI16(step2[24], s3_24_0);
- step2[25] = SUB_EPI16(step2[25], s3_25_0);
- step2[26] = SUB_EPI16(step2[26], s3_26_0);
- step2[27] = SUB_EPI16(step2[27], s3_27_0);
- step1[28] = SUB_EPI16(step1[28], s3_28_0);
- step1[29] = SUB_EPI16(step1[29], s3_29_0);
- step1[30] = SUB_EPI16(step1[30], s3_30_0);
- step1[31] = SUB_EPI16(step1[31], s3_31_0);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x32(
- &step2[0], &step2[1], &step2[2], &step2[3],
- &step2[4], &step2[5], &step2[6], &step2[7],
- &step2[8], &step2[9], &step2[10], &step2[11],
- &step2[12], &step2[13], &step2[14], &step2[15],
- &step1[16], &step1[17], &step1[18], &step1[19],
- &step2[20], &step2[21], &step2[22], &step2[23],
- &step2[24], &step2[25], &step2[26], &step2[27],
- &step1[28], &step1[29], &step1[30], &step1[31]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- step2[0] = _mm_add_epi16(step2[ 0], kOne);
- step2[1] = _mm_add_epi16(step2[ 1], kOne);
- step2[2] = _mm_add_epi16(step2[ 2], kOne);
- step2[3] = _mm_add_epi16(step2[ 3], kOne);
- step2[4] = _mm_add_epi16(step2[ 4], kOne);
- step2[5] = _mm_add_epi16(step2[ 5], kOne);
- step2[6] = _mm_add_epi16(step2[ 6], kOne);
- step2[7] = _mm_add_epi16(step2[ 7], kOne);
- step2[8] = _mm_add_epi16(step2[ 8], kOne);
- step2[9] = _mm_add_epi16(step2[ 9], kOne);
- step2[10] = _mm_add_epi16(step2[10], kOne);
- step2[11] = _mm_add_epi16(step2[11], kOne);
- step2[12] = _mm_add_epi16(step2[12], kOne);
- step2[13] = _mm_add_epi16(step2[13], kOne);
- step2[14] = _mm_add_epi16(step2[14], kOne);
- step2[15] = _mm_add_epi16(step2[15], kOne);
- step1[16] = _mm_add_epi16(step1[16], kOne);
- step1[17] = _mm_add_epi16(step1[17], kOne);
- step1[18] = _mm_add_epi16(step1[18], kOne);
- step1[19] = _mm_add_epi16(step1[19], kOne);
- step2[20] = _mm_add_epi16(step2[20], kOne);
- step2[21] = _mm_add_epi16(step2[21], kOne);
- step2[22] = _mm_add_epi16(step2[22], kOne);
- step2[23] = _mm_add_epi16(step2[23], kOne);
- step2[24] = _mm_add_epi16(step2[24], kOne);
- step2[25] = _mm_add_epi16(step2[25], kOne);
- step2[26] = _mm_add_epi16(step2[26], kOne);
- step2[27] = _mm_add_epi16(step2[27], kOne);
- step1[28] = _mm_add_epi16(step1[28], kOne);
- step1[29] = _mm_add_epi16(step1[29], kOne);
- step1[30] = _mm_add_epi16(step1[30], kOne);
- step1[31] = _mm_add_epi16(step1[31], kOne);
-
- step2[0] = _mm_srai_epi16(step2[ 0], 2);
- step2[1] = _mm_srai_epi16(step2[ 1], 2);
- step2[2] = _mm_srai_epi16(step2[ 2], 2);
- step2[3] = _mm_srai_epi16(step2[ 3], 2);
- step2[4] = _mm_srai_epi16(step2[ 4], 2);
- step2[5] = _mm_srai_epi16(step2[ 5], 2);
- step2[6] = _mm_srai_epi16(step2[ 6], 2);
- step2[7] = _mm_srai_epi16(step2[ 7], 2);
- step2[8] = _mm_srai_epi16(step2[ 8], 2);
- step2[9] = _mm_srai_epi16(step2[ 9], 2);
- step2[10] = _mm_srai_epi16(step2[10], 2);
- step2[11] = _mm_srai_epi16(step2[11], 2);
- step2[12] = _mm_srai_epi16(step2[12], 2);
- step2[13] = _mm_srai_epi16(step2[13], 2);
- step2[14] = _mm_srai_epi16(step2[14], 2);
- step2[15] = _mm_srai_epi16(step2[15], 2);
- step1[16] = _mm_srai_epi16(step1[16], 2);
- step1[17] = _mm_srai_epi16(step1[17], 2);
- step1[18] = _mm_srai_epi16(step1[18], 2);
- step1[19] = _mm_srai_epi16(step1[19], 2);
- step2[20] = _mm_srai_epi16(step2[20], 2);
- step2[21] = _mm_srai_epi16(step2[21], 2);
- step2[22] = _mm_srai_epi16(step2[22], 2);
- step2[23] = _mm_srai_epi16(step2[23], 2);
- step2[24] = _mm_srai_epi16(step2[24], 2);
- step2[25] = _mm_srai_epi16(step2[25], 2);
- step2[26] = _mm_srai_epi16(step2[26], 2);
- step2[27] = _mm_srai_epi16(step2[27], 2);
- step1[28] = _mm_srai_epi16(step1[28], 2);
- step1[29] = _mm_srai_epi16(step1[29], 2);
- step1[30] = _mm_srai_epi16(step1[30], 2);
- step1[31] = _mm_srai_epi16(step1[31], 2);
- }
-#endif // !FDCT32x32_HIGH_PRECISION
-
-#if FDCT32x32_HIGH_PRECISION
- if (pass == 0) {
-#endif
- // Stage 3
- {
- step3[0] = ADD_EPI16(step2[(8 - 1)], step2[0]);
- step3[1] = ADD_EPI16(step2[(8 - 2)], step2[1]);
- step3[2] = ADD_EPI16(step2[(8 - 3)], step2[2]);
- step3[3] = ADD_EPI16(step2[(8 - 4)], step2[3]);
- step3[4] = SUB_EPI16(step2[(8 - 5)], step2[4]);
- step3[5] = SUB_EPI16(step2[(8 - 6)], step2[5]);
- step3[6] = SUB_EPI16(step2[(8 - 7)], step2[6]);
- step3[7] = SUB_EPI16(step2[(8 - 8)], step2[7]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step3[0], &step3[1], &step3[2],
- &step3[3], &step3[4], &step3[5],
- &step3[6], &step3[7]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i s3_10_0 = _mm_unpacklo_epi16(step2[13], step2[10]);
- const __m128i s3_10_1 = _mm_unpackhi_epi16(step2[13], step2[10]);
- const __m128i s3_11_0 = _mm_unpacklo_epi16(step2[12], step2[11]);
- const __m128i s3_11_1 = _mm_unpackhi_epi16(step2[12], step2[11]);
- const __m128i s3_10_2 = _mm_madd_epi16(s3_10_0, k__cospi_p16_m16);
- const __m128i s3_10_3 = _mm_madd_epi16(s3_10_1, k__cospi_p16_m16);
- const __m128i s3_11_2 = _mm_madd_epi16(s3_11_0, k__cospi_p16_m16);
- const __m128i s3_11_3 = _mm_madd_epi16(s3_11_1, k__cospi_p16_m16);
- const __m128i s3_12_2 = _mm_madd_epi16(s3_11_0, k__cospi_p16_p16);
- const __m128i s3_12_3 = _mm_madd_epi16(s3_11_1, k__cospi_p16_p16);
- const __m128i s3_13_2 = _mm_madd_epi16(s3_10_0, k__cospi_p16_p16);
- const __m128i s3_13_3 = _mm_madd_epi16(s3_10_1, k__cospi_p16_p16);
- // dct_const_round_shift
- const __m128i s3_10_4 = _mm_add_epi32(s3_10_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_10_5 = _mm_add_epi32(s3_10_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_11_4 = _mm_add_epi32(s3_11_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_11_5 = _mm_add_epi32(s3_11_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_12_4 = _mm_add_epi32(s3_12_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_12_5 = _mm_add_epi32(s3_12_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_13_4 = _mm_add_epi32(s3_13_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_13_5 = _mm_add_epi32(s3_13_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_10_6 = _mm_srai_epi32(s3_10_4, DCT_CONST_BITS);
- const __m128i s3_10_7 = _mm_srai_epi32(s3_10_5, DCT_CONST_BITS);
- const __m128i s3_11_6 = _mm_srai_epi32(s3_11_4, DCT_CONST_BITS);
- const __m128i s3_11_7 = _mm_srai_epi32(s3_11_5, DCT_CONST_BITS);
- const __m128i s3_12_6 = _mm_srai_epi32(s3_12_4, DCT_CONST_BITS);
- const __m128i s3_12_7 = _mm_srai_epi32(s3_12_5, DCT_CONST_BITS);
- const __m128i s3_13_6 = _mm_srai_epi32(s3_13_4, DCT_CONST_BITS);
- const __m128i s3_13_7 = _mm_srai_epi32(s3_13_5, DCT_CONST_BITS);
- // Combine
- step3[10] = _mm_packs_epi32(s3_10_6, s3_10_7);
- step3[11] = _mm_packs_epi32(s3_11_6, s3_11_7);
- step3[12] = _mm_packs_epi32(s3_12_6, s3_12_7);
- step3[13] = _mm_packs_epi32(s3_13_6, s3_13_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&step3[10], &step3[11],
- &step3[12], &step3[13]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- step3[16] = ADD_EPI16(step2[23], step1[16]);
- step3[17] = ADD_EPI16(step2[22], step1[17]);
- step3[18] = ADD_EPI16(step2[21], step1[18]);
- step3[19] = ADD_EPI16(step2[20], step1[19]);
- step3[20] = SUB_EPI16(step1[19], step2[20]);
- step3[21] = SUB_EPI16(step1[18], step2[21]);
- step3[22] = SUB_EPI16(step1[17], step2[22]);
- step3[23] = SUB_EPI16(step1[16], step2[23]);
- step3[24] = SUB_EPI16(step1[31], step2[24]);
- step3[25] = SUB_EPI16(step1[30], step2[25]);
- step3[26] = SUB_EPI16(step1[29], step2[26]);
- step3[27] = SUB_EPI16(step1[28], step2[27]);
- step3[28] = ADD_EPI16(step2[27], step1[28]);
- step3[29] = ADD_EPI16(step2[26], step1[29]);
- step3[30] = ADD_EPI16(step2[25], step1[30]);
- step3[31] = ADD_EPI16(step2[24], step1[31]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x16(
- &step3[16], &step3[17], &step3[18], &step3[19],
- &step3[20], &step3[21], &step3[22], &step3[23],
- &step3[24], &step3[25], &step3[26], &step3[27],
- &step3[28], &step3[29], &step3[30], &step3[31]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
-
- // Stage 4
- {
- step1[0] = ADD_EPI16(step3[ 3], step3[ 0]);
- step1[1] = ADD_EPI16(step3[ 2], step3[ 1]);
- step1[2] = SUB_EPI16(step3[ 1], step3[ 2]);
- step1[3] = SUB_EPI16(step3[ 0], step3[ 3]);
- step1[8] = ADD_EPI16(step3[11], step2[ 8]);
- step1[9] = ADD_EPI16(step3[10], step2[ 9]);
- step1[10] = SUB_EPI16(step2[ 9], step3[10]);
- step1[11] = SUB_EPI16(step2[ 8], step3[11]);
- step1[12] = SUB_EPI16(step2[15], step3[12]);
- step1[13] = SUB_EPI16(step2[14], step3[13]);
- step1[14] = ADD_EPI16(step3[13], step2[14]);
- step1[15] = ADD_EPI16(step3[12], step2[15]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x16(
- &step1[0], &step1[1], &step1[2], &step1[3],
- &step1[4], &step1[5], &step1[6], &step1[7],
- &step1[8], &step1[9], &step1[10], &step1[11],
- &step1[12], &step1[13], &step1[14], &step1[15]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i s1_05_0 = _mm_unpacklo_epi16(step3[6], step3[5]);
- const __m128i s1_05_1 = _mm_unpackhi_epi16(step3[6], step3[5]);
- const __m128i s1_05_2 = _mm_madd_epi16(s1_05_0, k__cospi_p16_m16);
- const __m128i s1_05_3 = _mm_madd_epi16(s1_05_1, k__cospi_p16_m16);
- const __m128i s1_06_2 = _mm_madd_epi16(s1_05_0, k__cospi_p16_p16);
- const __m128i s1_06_3 = _mm_madd_epi16(s1_05_1, k__cospi_p16_p16);
- // dct_const_round_shift
- const __m128i s1_05_4 = _mm_add_epi32(s1_05_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_05_5 = _mm_add_epi32(s1_05_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_06_4 = _mm_add_epi32(s1_06_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_06_5 = _mm_add_epi32(s1_06_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_05_6 = _mm_srai_epi32(s1_05_4, DCT_CONST_BITS);
- const __m128i s1_05_7 = _mm_srai_epi32(s1_05_5, DCT_CONST_BITS);
- const __m128i s1_06_6 = _mm_srai_epi32(s1_06_4, DCT_CONST_BITS);
- const __m128i s1_06_7 = _mm_srai_epi32(s1_06_5, DCT_CONST_BITS);
- // Combine
- step1[5] = _mm_packs_epi32(s1_05_6, s1_05_7);
- step1[6] = _mm_packs_epi32(s1_06_6, s1_06_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x2(&step1[5], &step1[6]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i s1_18_0 = _mm_unpacklo_epi16(step3[18], step3[29]);
- const __m128i s1_18_1 = _mm_unpackhi_epi16(step3[18], step3[29]);
- const __m128i s1_19_0 = _mm_unpacklo_epi16(step3[19], step3[28]);
- const __m128i s1_19_1 = _mm_unpackhi_epi16(step3[19], step3[28]);
- const __m128i s1_20_0 = _mm_unpacklo_epi16(step3[20], step3[27]);
- const __m128i s1_20_1 = _mm_unpackhi_epi16(step3[20], step3[27]);
- const __m128i s1_21_0 = _mm_unpacklo_epi16(step3[21], step3[26]);
- const __m128i s1_21_1 = _mm_unpackhi_epi16(step3[21], step3[26]);
- const __m128i s1_18_2 = _mm_madd_epi16(s1_18_0, k__cospi_m08_p24);
- const __m128i s1_18_3 = _mm_madd_epi16(s1_18_1, k__cospi_m08_p24);
- const __m128i s1_19_2 = _mm_madd_epi16(s1_19_0, k__cospi_m08_p24);
- const __m128i s1_19_3 = _mm_madd_epi16(s1_19_1, k__cospi_m08_p24);
- const __m128i s1_20_2 = _mm_madd_epi16(s1_20_0, k__cospi_m24_m08);
- const __m128i s1_20_3 = _mm_madd_epi16(s1_20_1, k__cospi_m24_m08);
- const __m128i s1_21_2 = _mm_madd_epi16(s1_21_0, k__cospi_m24_m08);
- const __m128i s1_21_3 = _mm_madd_epi16(s1_21_1, k__cospi_m24_m08);
- const __m128i s1_26_2 = _mm_madd_epi16(s1_21_0, k__cospi_m08_p24);
- const __m128i s1_26_3 = _mm_madd_epi16(s1_21_1, k__cospi_m08_p24);
- const __m128i s1_27_2 = _mm_madd_epi16(s1_20_0, k__cospi_m08_p24);
- const __m128i s1_27_3 = _mm_madd_epi16(s1_20_1, k__cospi_m08_p24);
- const __m128i s1_28_2 = _mm_madd_epi16(s1_19_0, k__cospi_p24_p08);
- const __m128i s1_28_3 = _mm_madd_epi16(s1_19_1, k__cospi_p24_p08);
- const __m128i s1_29_2 = _mm_madd_epi16(s1_18_0, k__cospi_p24_p08);
- const __m128i s1_29_3 = _mm_madd_epi16(s1_18_1, k__cospi_p24_p08);
- // dct_const_round_shift
- const __m128i s1_18_4 = _mm_add_epi32(s1_18_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_18_5 = _mm_add_epi32(s1_18_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_19_4 = _mm_add_epi32(s1_19_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_19_5 = _mm_add_epi32(s1_19_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_20_4 = _mm_add_epi32(s1_20_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_20_5 = _mm_add_epi32(s1_20_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_21_4 = _mm_add_epi32(s1_21_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_21_5 = _mm_add_epi32(s1_21_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_26_4 = _mm_add_epi32(s1_26_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_26_5 = _mm_add_epi32(s1_26_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_27_4 = _mm_add_epi32(s1_27_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_27_5 = _mm_add_epi32(s1_27_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_28_4 = _mm_add_epi32(s1_28_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_28_5 = _mm_add_epi32(s1_28_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_29_4 = _mm_add_epi32(s1_29_2, k__DCT_CONST_ROUNDING);
- const __m128i s1_29_5 = _mm_add_epi32(s1_29_3, k__DCT_CONST_ROUNDING);
- const __m128i s1_18_6 = _mm_srai_epi32(s1_18_4, DCT_CONST_BITS);
- const __m128i s1_18_7 = _mm_srai_epi32(s1_18_5, DCT_CONST_BITS);
- const __m128i s1_19_6 = _mm_srai_epi32(s1_19_4, DCT_CONST_BITS);
- const __m128i s1_19_7 = _mm_srai_epi32(s1_19_5, DCT_CONST_BITS);
- const __m128i s1_20_6 = _mm_srai_epi32(s1_20_4, DCT_CONST_BITS);
- const __m128i s1_20_7 = _mm_srai_epi32(s1_20_5, DCT_CONST_BITS);
- const __m128i s1_21_6 = _mm_srai_epi32(s1_21_4, DCT_CONST_BITS);
- const __m128i s1_21_7 = _mm_srai_epi32(s1_21_5, DCT_CONST_BITS);
- const __m128i s1_26_6 = _mm_srai_epi32(s1_26_4, DCT_CONST_BITS);
- const __m128i s1_26_7 = _mm_srai_epi32(s1_26_5, DCT_CONST_BITS);
- const __m128i s1_27_6 = _mm_srai_epi32(s1_27_4, DCT_CONST_BITS);
- const __m128i s1_27_7 = _mm_srai_epi32(s1_27_5, DCT_CONST_BITS);
- const __m128i s1_28_6 = _mm_srai_epi32(s1_28_4, DCT_CONST_BITS);
- const __m128i s1_28_7 = _mm_srai_epi32(s1_28_5, DCT_CONST_BITS);
- const __m128i s1_29_6 = _mm_srai_epi32(s1_29_4, DCT_CONST_BITS);
- const __m128i s1_29_7 = _mm_srai_epi32(s1_29_5, DCT_CONST_BITS);
- // Combine
- step1[18] = _mm_packs_epi32(s1_18_6, s1_18_7);
- step1[19] = _mm_packs_epi32(s1_19_6, s1_19_7);
- step1[20] = _mm_packs_epi32(s1_20_6, s1_20_7);
- step1[21] = _mm_packs_epi32(s1_21_6, s1_21_7);
- step1[26] = _mm_packs_epi32(s1_26_6, s1_26_7);
- step1[27] = _mm_packs_epi32(s1_27_6, s1_27_7);
- step1[28] = _mm_packs_epi32(s1_28_6, s1_28_7);
- step1[29] = _mm_packs_epi32(s1_29_6, s1_29_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1[18], &step1[19], &step1[20],
- &step1[21], &step1[26], &step1[27],
- &step1[28], &step1[29]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // Stage 5
- {
- step2[4] = ADD_EPI16(step1[5], step3[4]);
- step2[5] = SUB_EPI16(step3[4], step1[5]);
- step2[6] = SUB_EPI16(step3[7], step1[6]);
- step2[7] = ADD_EPI16(step1[6], step3[7]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&step2[4], &step2[5],
- &step2[6], &step2[7]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i out_00_0 = _mm_unpacklo_epi16(step1[0], step1[1]);
- const __m128i out_00_1 = _mm_unpackhi_epi16(step1[0], step1[1]);
- const __m128i out_08_0 = _mm_unpacklo_epi16(step1[2], step1[3]);
- const __m128i out_08_1 = _mm_unpackhi_epi16(step1[2], step1[3]);
- const __m128i out_00_2 = _mm_madd_epi16(out_00_0, k__cospi_p16_p16);
- const __m128i out_00_3 = _mm_madd_epi16(out_00_1, k__cospi_p16_p16);
- const __m128i out_16_2 = _mm_madd_epi16(out_00_0, k__cospi_p16_m16);
- const __m128i out_16_3 = _mm_madd_epi16(out_00_1, k__cospi_p16_m16);
- const __m128i out_08_2 = _mm_madd_epi16(out_08_0, k__cospi_p24_p08);
- const __m128i out_08_3 = _mm_madd_epi16(out_08_1, k__cospi_p24_p08);
- const __m128i out_24_2 = _mm_madd_epi16(out_08_0, k__cospi_m08_p24);
- const __m128i out_24_3 = _mm_madd_epi16(out_08_1, k__cospi_m08_p24);
- // dct_const_round_shift
- const __m128i out_00_4 = _mm_add_epi32(out_00_2, k__DCT_CONST_ROUNDING);
- const __m128i out_00_5 = _mm_add_epi32(out_00_3, k__DCT_CONST_ROUNDING);
- const __m128i out_16_4 = _mm_add_epi32(out_16_2, k__DCT_CONST_ROUNDING);
- const __m128i out_16_5 = _mm_add_epi32(out_16_3, k__DCT_CONST_ROUNDING);
- const __m128i out_08_4 = _mm_add_epi32(out_08_2, k__DCT_CONST_ROUNDING);
- const __m128i out_08_5 = _mm_add_epi32(out_08_3, k__DCT_CONST_ROUNDING);
- const __m128i out_24_4 = _mm_add_epi32(out_24_2, k__DCT_CONST_ROUNDING);
- const __m128i out_24_5 = _mm_add_epi32(out_24_3, k__DCT_CONST_ROUNDING);
- const __m128i out_00_6 = _mm_srai_epi32(out_00_4, DCT_CONST_BITS);
- const __m128i out_00_7 = _mm_srai_epi32(out_00_5, DCT_CONST_BITS);
- const __m128i out_16_6 = _mm_srai_epi32(out_16_4, DCT_CONST_BITS);
- const __m128i out_16_7 = _mm_srai_epi32(out_16_5, DCT_CONST_BITS);
- const __m128i out_08_6 = _mm_srai_epi32(out_08_4, DCT_CONST_BITS);
- const __m128i out_08_7 = _mm_srai_epi32(out_08_5, DCT_CONST_BITS);
- const __m128i out_24_6 = _mm_srai_epi32(out_24_4, DCT_CONST_BITS);
- const __m128i out_24_7 = _mm_srai_epi32(out_24_5, DCT_CONST_BITS);
- // Combine
- out[ 0] = _mm_packs_epi32(out_00_6, out_00_7);
- out[16] = _mm_packs_epi32(out_16_6, out_16_7);
- out[ 8] = _mm_packs_epi32(out_08_6, out_08_7);
- out[24] = _mm_packs_epi32(out_24_6, out_24_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&out[0], &out[16],
- &out[8], &out[24]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i s2_09_0 = _mm_unpacklo_epi16(step1[ 9], step1[14]);
- const __m128i s2_09_1 = _mm_unpackhi_epi16(step1[ 9], step1[14]);
- const __m128i s2_10_0 = _mm_unpacklo_epi16(step1[10], step1[13]);
- const __m128i s2_10_1 = _mm_unpackhi_epi16(step1[10], step1[13]);
- const __m128i s2_09_2 = _mm_madd_epi16(s2_09_0, k__cospi_m08_p24);
- const __m128i s2_09_3 = _mm_madd_epi16(s2_09_1, k__cospi_m08_p24);
- const __m128i s2_10_2 = _mm_madd_epi16(s2_10_0, k__cospi_m24_m08);
- const __m128i s2_10_3 = _mm_madd_epi16(s2_10_1, k__cospi_m24_m08);
- const __m128i s2_13_2 = _mm_madd_epi16(s2_10_0, k__cospi_m08_p24);
- const __m128i s2_13_3 = _mm_madd_epi16(s2_10_1, k__cospi_m08_p24);
- const __m128i s2_14_2 = _mm_madd_epi16(s2_09_0, k__cospi_p24_p08);
- const __m128i s2_14_3 = _mm_madd_epi16(s2_09_1, k__cospi_p24_p08);
- // dct_const_round_shift
- const __m128i s2_09_4 = _mm_add_epi32(s2_09_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_09_5 = _mm_add_epi32(s2_09_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_10_4 = _mm_add_epi32(s2_10_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_10_5 = _mm_add_epi32(s2_10_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_13_4 = _mm_add_epi32(s2_13_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_13_5 = _mm_add_epi32(s2_13_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_14_4 = _mm_add_epi32(s2_14_2, k__DCT_CONST_ROUNDING);
- const __m128i s2_14_5 = _mm_add_epi32(s2_14_3, k__DCT_CONST_ROUNDING);
- const __m128i s2_09_6 = _mm_srai_epi32(s2_09_4, DCT_CONST_BITS);
- const __m128i s2_09_7 = _mm_srai_epi32(s2_09_5, DCT_CONST_BITS);
- const __m128i s2_10_6 = _mm_srai_epi32(s2_10_4, DCT_CONST_BITS);
- const __m128i s2_10_7 = _mm_srai_epi32(s2_10_5, DCT_CONST_BITS);
- const __m128i s2_13_6 = _mm_srai_epi32(s2_13_4, DCT_CONST_BITS);
- const __m128i s2_13_7 = _mm_srai_epi32(s2_13_5, DCT_CONST_BITS);
- const __m128i s2_14_6 = _mm_srai_epi32(s2_14_4, DCT_CONST_BITS);
- const __m128i s2_14_7 = _mm_srai_epi32(s2_14_5, DCT_CONST_BITS);
- // Combine
- step2[ 9] = _mm_packs_epi32(s2_09_6, s2_09_7);
- step2[10] = _mm_packs_epi32(s2_10_6, s2_10_7);
- step2[13] = _mm_packs_epi32(s2_13_6, s2_13_7);
- step2[14] = _mm_packs_epi32(s2_14_6, s2_14_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&step2[9], &step2[10],
- &step2[13], &step2[14]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- step2[16] = ADD_EPI16(step1[19], step3[16]);
- step2[17] = ADD_EPI16(step1[18], step3[17]);
- step2[18] = SUB_EPI16(step3[17], step1[18]);
- step2[19] = SUB_EPI16(step3[16], step1[19]);
- step2[20] = SUB_EPI16(step3[23], step1[20]);
- step2[21] = SUB_EPI16(step3[22], step1[21]);
- step2[22] = ADD_EPI16(step1[21], step3[22]);
- step2[23] = ADD_EPI16(step1[20], step3[23]);
- step2[24] = ADD_EPI16(step1[27], step3[24]);
- step2[25] = ADD_EPI16(step1[26], step3[25]);
- step2[26] = SUB_EPI16(step3[25], step1[26]);
- step2[27] = SUB_EPI16(step3[24], step1[27]);
- step2[28] = SUB_EPI16(step3[31], step1[28]);
- step2[29] = SUB_EPI16(step3[30], step1[29]);
- step2[30] = ADD_EPI16(step1[29], step3[30]);
- step2[31] = ADD_EPI16(step1[28], step3[31]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x16(
- &step2[16], &step2[17], &step2[18], &step2[19],
- &step2[20], &step2[21], &step2[22], &step2[23],
- &step2[24], &step2[25], &step2[26], &step2[27],
- &step2[28], &step2[29], &step2[30], &step2[31]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // Stage 6
- {
- const __m128i out_04_0 = _mm_unpacklo_epi16(step2[4], step2[7]);
- const __m128i out_04_1 = _mm_unpackhi_epi16(step2[4], step2[7]);
- const __m128i out_20_0 = _mm_unpacklo_epi16(step2[5], step2[6]);
- const __m128i out_20_1 = _mm_unpackhi_epi16(step2[5], step2[6]);
- const __m128i out_12_0 = _mm_unpacklo_epi16(step2[5], step2[6]);
- const __m128i out_12_1 = _mm_unpackhi_epi16(step2[5], step2[6]);
- const __m128i out_28_0 = _mm_unpacklo_epi16(step2[4], step2[7]);
- const __m128i out_28_1 = _mm_unpackhi_epi16(step2[4], step2[7]);
- const __m128i out_04_2 = _mm_madd_epi16(out_04_0, k__cospi_p28_p04);
- const __m128i out_04_3 = _mm_madd_epi16(out_04_1, k__cospi_p28_p04);
- const __m128i out_20_2 = _mm_madd_epi16(out_20_0, k__cospi_p12_p20);
- const __m128i out_20_3 = _mm_madd_epi16(out_20_1, k__cospi_p12_p20);
- const __m128i out_12_2 = _mm_madd_epi16(out_12_0, k__cospi_m20_p12);
- const __m128i out_12_3 = _mm_madd_epi16(out_12_1, k__cospi_m20_p12);
- const __m128i out_28_2 = _mm_madd_epi16(out_28_0, k__cospi_m04_p28);
- const __m128i out_28_3 = _mm_madd_epi16(out_28_1, k__cospi_m04_p28);
- // dct_const_round_shift
- const __m128i out_04_4 = _mm_add_epi32(out_04_2, k__DCT_CONST_ROUNDING);
- const __m128i out_04_5 = _mm_add_epi32(out_04_3, k__DCT_CONST_ROUNDING);
- const __m128i out_20_4 = _mm_add_epi32(out_20_2, k__DCT_CONST_ROUNDING);
- const __m128i out_20_5 = _mm_add_epi32(out_20_3, k__DCT_CONST_ROUNDING);
- const __m128i out_12_4 = _mm_add_epi32(out_12_2, k__DCT_CONST_ROUNDING);
- const __m128i out_12_5 = _mm_add_epi32(out_12_3, k__DCT_CONST_ROUNDING);
- const __m128i out_28_4 = _mm_add_epi32(out_28_2, k__DCT_CONST_ROUNDING);
- const __m128i out_28_5 = _mm_add_epi32(out_28_3, k__DCT_CONST_ROUNDING);
- const __m128i out_04_6 = _mm_srai_epi32(out_04_4, DCT_CONST_BITS);
- const __m128i out_04_7 = _mm_srai_epi32(out_04_5, DCT_CONST_BITS);
- const __m128i out_20_6 = _mm_srai_epi32(out_20_4, DCT_CONST_BITS);
- const __m128i out_20_7 = _mm_srai_epi32(out_20_5, DCT_CONST_BITS);
- const __m128i out_12_6 = _mm_srai_epi32(out_12_4, DCT_CONST_BITS);
- const __m128i out_12_7 = _mm_srai_epi32(out_12_5, DCT_CONST_BITS);
- const __m128i out_28_6 = _mm_srai_epi32(out_28_4, DCT_CONST_BITS);
- const __m128i out_28_7 = _mm_srai_epi32(out_28_5, DCT_CONST_BITS);
- // Combine
- out[4] = _mm_packs_epi32(out_04_6, out_04_7);
- out[20] = _mm_packs_epi32(out_20_6, out_20_7);
- out[12] = _mm_packs_epi32(out_12_6, out_12_7);
- out[28] = _mm_packs_epi32(out_28_6, out_28_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&out[4], &out[20],
- &out[12], &out[28]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- step3[8] = ADD_EPI16(step2[ 9], step1[ 8]);
- step3[9] = SUB_EPI16(step1[ 8], step2[ 9]);
- step3[10] = SUB_EPI16(step1[11], step2[10]);
- step3[11] = ADD_EPI16(step2[10], step1[11]);
- step3[12] = ADD_EPI16(step2[13], step1[12]);
- step3[13] = SUB_EPI16(step1[12], step2[13]);
- step3[14] = SUB_EPI16(step1[15], step2[14]);
- step3[15] = ADD_EPI16(step2[14], step1[15]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step3[8], &step3[9], &step3[10],
- &step3[11], &step3[12], &step3[13],
- &step3[14], &step3[15]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i s3_17_0 = _mm_unpacklo_epi16(step2[17], step2[30]);
- const __m128i s3_17_1 = _mm_unpackhi_epi16(step2[17], step2[30]);
- const __m128i s3_18_0 = _mm_unpacklo_epi16(step2[18], step2[29]);
- const __m128i s3_18_1 = _mm_unpackhi_epi16(step2[18], step2[29]);
- const __m128i s3_21_0 = _mm_unpacklo_epi16(step2[21], step2[26]);
- const __m128i s3_21_1 = _mm_unpackhi_epi16(step2[21], step2[26]);
- const __m128i s3_22_0 = _mm_unpacklo_epi16(step2[22], step2[25]);
- const __m128i s3_22_1 = _mm_unpackhi_epi16(step2[22], step2[25]);
- const __m128i s3_17_2 = _mm_madd_epi16(s3_17_0, k__cospi_m04_p28);
- const __m128i s3_17_3 = _mm_madd_epi16(s3_17_1, k__cospi_m04_p28);
- const __m128i s3_18_2 = _mm_madd_epi16(s3_18_0, k__cospi_m28_m04);
- const __m128i s3_18_3 = _mm_madd_epi16(s3_18_1, k__cospi_m28_m04);
- const __m128i s3_21_2 = _mm_madd_epi16(s3_21_0, k__cospi_m20_p12);
- const __m128i s3_21_3 = _mm_madd_epi16(s3_21_1, k__cospi_m20_p12);
- const __m128i s3_22_2 = _mm_madd_epi16(s3_22_0, k__cospi_m12_m20);
- const __m128i s3_22_3 = _mm_madd_epi16(s3_22_1, k__cospi_m12_m20);
- const __m128i s3_25_2 = _mm_madd_epi16(s3_22_0, k__cospi_m20_p12);
- const __m128i s3_25_3 = _mm_madd_epi16(s3_22_1, k__cospi_m20_p12);
- const __m128i s3_26_2 = _mm_madd_epi16(s3_21_0, k__cospi_p12_p20);
- const __m128i s3_26_3 = _mm_madd_epi16(s3_21_1, k__cospi_p12_p20);
- const __m128i s3_29_2 = _mm_madd_epi16(s3_18_0, k__cospi_m04_p28);
- const __m128i s3_29_3 = _mm_madd_epi16(s3_18_1, k__cospi_m04_p28);
- const __m128i s3_30_2 = _mm_madd_epi16(s3_17_0, k__cospi_p28_p04);
- const __m128i s3_30_3 = _mm_madd_epi16(s3_17_1, k__cospi_p28_p04);
- // dct_const_round_shift
- const __m128i s3_17_4 = _mm_add_epi32(s3_17_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_17_5 = _mm_add_epi32(s3_17_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_18_4 = _mm_add_epi32(s3_18_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_18_5 = _mm_add_epi32(s3_18_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_21_4 = _mm_add_epi32(s3_21_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_21_5 = _mm_add_epi32(s3_21_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_22_4 = _mm_add_epi32(s3_22_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_22_5 = _mm_add_epi32(s3_22_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_17_6 = _mm_srai_epi32(s3_17_4, DCT_CONST_BITS);
- const __m128i s3_17_7 = _mm_srai_epi32(s3_17_5, DCT_CONST_BITS);
- const __m128i s3_18_6 = _mm_srai_epi32(s3_18_4, DCT_CONST_BITS);
- const __m128i s3_18_7 = _mm_srai_epi32(s3_18_5, DCT_CONST_BITS);
- const __m128i s3_21_6 = _mm_srai_epi32(s3_21_4, DCT_CONST_BITS);
- const __m128i s3_21_7 = _mm_srai_epi32(s3_21_5, DCT_CONST_BITS);
- const __m128i s3_22_6 = _mm_srai_epi32(s3_22_4, DCT_CONST_BITS);
- const __m128i s3_22_7 = _mm_srai_epi32(s3_22_5, DCT_CONST_BITS);
- const __m128i s3_25_4 = _mm_add_epi32(s3_25_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_25_5 = _mm_add_epi32(s3_25_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_26_4 = _mm_add_epi32(s3_26_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_26_5 = _mm_add_epi32(s3_26_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_29_4 = _mm_add_epi32(s3_29_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_29_5 = _mm_add_epi32(s3_29_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_30_4 = _mm_add_epi32(s3_30_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_30_5 = _mm_add_epi32(s3_30_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_25_6 = _mm_srai_epi32(s3_25_4, DCT_CONST_BITS);
- const __m128i s3_25_7 = _mm_srai_epi32(s3_25_5, DCT_CONST_BITS);
- const __m128i s3_26_6 = _mm_srai_epi32(s3_26_4, DCT_CONST_BITS);
- const __m128i s3_26_7 = _mm_srai_epi32(s3_26_5, DCT_CONST_BITS);
- const __m128i s3_29_6 = _mm_srai_epi32(s3_29_4, DCT_CONST_BITS);
- const __m128i s3_29_7 = _mm_srai_epi32(s3_29_5, DCT_CONST_BITS);
- const __m128i s3_30_6 = _mm_srai_epi32(s3_30_4, DCT_CONST_BITS);
- const __m128i s3_30_7 = _mm_srai_epi32(s3_30_5, DCT_CONST_BITS);
- // Combine
- step3[17] = _mm_packs_epi32(s3_17_6, s3_17_7);
- step3[18] = _mm_packs_epi32(s3_18_6, s3_18_7);
- step3[21] = _mm_packs_epi32(s3_21_6, s3_21_7);
- step3[22] = _mm_packs_epi32(s3_22_6, s3_22_7);
- // Combine
- step3[25] = _mm_packs_epi32(s3_25_6, s3_25_7);
- step3[26] = _mm_packs_epi32(s3_26_6, s3_26_7);
- step3[29] = _mm_packs_epi32(s3_29_6, s3_29_7);
- step3[30] = _mm_packs_epi32(s3_30_6, s3_30_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step3[17], &step3[18], &step3[21],
- &step3[22], &step3[25], &step3[26],
- &step3[29], &step3[30]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // Stage 7
- {
- const __m128i out_02_0 = _mm_unpacklo_epi16(step3[ 8], step3[15]);
- const __m128i out_02_1 = _mm_unpackhi_epi16(step3[ 8], step3[15]);
- const __m128i out_18_0 = _mm_unpacklo_epi16(step3[ 9], step3[14]);
- const __m128i out_18_1 = _mm_unpackhi_epi16(step3[ 9], step3[14]);
- const __m128i out_10_0 = _mm_unpacklo_epi16(step3[10], step3[13]);
- const __m128i out_10_1 = _mm_unpackhi_epi16(step3[10], step3[13]);
- const __m128i out_26_0 = _mm_unpacklo_epi16(step3[11], step3[12]);
- const __m128i out_26_1 = _mm_unpackhi_epi16(step3[11], step3[12]);
- const __m128i out_02_2 = _mm_madd_epi16(out_02_0, k__cospi_p30_p02);
- const __m128i out_02_3 = _mm_madd_epi16(out_02_1, k__cospi_p30_p02);
- const __m128i out_18_2 = _mm_madd_epi16(out_18_0, k__cospi_p14_p18);
- const __m128i out_18_3 = _mm_madd_epi16(out_18_1, k__cospi_p14_p18);
- const __m128i out_10_2 = _mm_madd_epi16(out_10_0, k__cospi_p22_p10);
- const __m128i out_10_3 = _mm_madd_epi16(out_10_1, k__cospi_p22_p10);
- const __m128i out_26_2 = _mm_madd_epi16(out_26_0, k__cospi_p06_p26);
- const __m128i out_26_3 = _mm_madd_epi16(out_26_1, k__cospi_p06_p26);
- const __m128i out_06_2 = _mm_madd_epi16(out_26_0, k__cospi_m26_p06);
- const __m128i out_06_3 = _mm_madd_epi16(out_26_1, k__cospi_m26_p06);
- const __m128i out_22_2 = _mm_madd_epi16(out_10_0, k__cospi_m10_p22);
- const __m128i out_22_3 = _mm_madd_epi16(out_10_1, k__cospi_m10_p22);
- const __m128i out_14_2 = _mm_madd_epi16(out_18_0, k__cospi_m18_p14);
- const __m128i out_14_3 = _mm_madd_epi16(out_18_1, k__cospi_m18_p14);
- const __m128i out_30_2 = _mm_madd_epi16(out_02_0, k__cospi_m02_p30);
- const __m128i out_30_3 = _mm_madd_epi16(out_02_1, k__cospi_m02_p30);
- // dct_const_round_shift
- const __m128i out_02_4 = _mm_add_epi32(out_02_2, k__DCT_CONST_ROUNDING);
- const __m128i out_02_5 = _mm_add_epi32(out_02_3, k__DCT_CONST_ROUNDING);
- const __m128i out_18_4 = _mm_add_epi32(out_18_2, k__DCT_CONST_ROUNDING);
- const __m128i out_18_5 = _mm_add_epi32(out_18_3, k__DCT_CONST_ROUNDING);
- const __m128i out_10_4 = _mm_add_epi32(out_10_2, k__DCT_CONST_ROUNDING);
- const __m128i out_10_5 = _mm_add_epi32(out_10_3, k__DCT_CONST_ROUNDING);
- const __m128i out_26_4 = _mm_add_epi32(out_26_2, k__DCT_CONST_ROUNDING);
- const __m128i out_26_5 = _mm_add_epi32(out_26_3, k__DCT_CONST_ROUNDING);
- const __m128i out_06_4 = _mm_add_epi32(out_06_2, k__DCT_CONST_ROUNDING);
- const __m128i out_06_5 = _mm_add_epi32(out_06_3, k__DCT_CONST_ROUNDING);
- const __m128i out_22_4 = _mm_add_epi32(out_22_2, k__DCT_CONST_ROUNDING);
- const __m128i out_22_5 = _mm_add_epi32(out_22_3, k__DCT_CONST_ROUNDING);
- const __m128i out_14_4 = _mm_add_epi32(out_14_2, k__DCT_CONST_ROUNDING);
- const __m128i out_14_5 = _mm_add_epi32(out_14_3, k__DCT_CONST_ROUNDING);
- const __m128i out_30_4 = _mm_add_epi32(out_30_2, k__DCT_CONST_ROUNDING);
- const __m128i out_30_5 = _mm_add_epi32(out_30_3, k__DCT_CONST_ROUNDING);
- const __m128i out_02_6 = _mm_srai_epi32(out_02_4, DCT_CONST_BITS);
- const __m128i out_02_7 = _mm_srai_epi32(out_02_5, DCT_CONST_BITS);
- const __m128i out_18_6 = _mm_srai_epi32(out_18_4, DCT_CONST_BITS);
- const __m128i out_18_7 = _mm_srai_epi32(out_18_5, DCT_CONST_BITS);
- const __m128i out_10_6 = _mm_srai_epi32(out_10_4, DCT_CONST_BITS);
- const __m128i out_10_7 = _mm_srai_epi32(out_10_5, DCT_CONST_BITS);
- const __m128i out_26_6 = _mm_srai_epi32(out_26_4, DCT_CONST_BITS);
- const __m128i out_26_7 = _mm_srai_epi32(out_26_5, DCT_CONST_BITS);
- const __m128i out_06_6 = _mm_srai_epi32(out_06_4, DCT_CONST_BITS);
- const __m128i out_06_7 = _mm_srai_epi32(out_06_5, DCT_CONST_BITS);
- const __m128i out_22_6 = _mm_srai_epi32(out_22_4, DCT_CONST_BITS);
- const __m128i out_22_7 = _mm_srai_epi32(out_22_5, DCT_CONST_BITS);
- const __m128i out_14_6 = _mm_srai_epi32(out_14_4, DCT_CONST_BITS);
- const __m128i out_14_7 = _mm_srai_epi32(out_14_5, DCT_CONST_BITS);
- const __m128i out_30_6 = _mm_srai_epi32(out_30_4, DCT_CONST_BITS);
- const __m128i out_30_7 = _mm_srai_epi32(out_30_5, DCT_CONST_BITS);
- // Combine
- out[ 2] = _mm_packs_epi32(out_02_6, out_02_7);
- out[18] = _mm_packs_epi32(out_18_6, out_18_7);
- out[10] = _mm_packs_epi32(out_10_6, out_10_7);
- out[26] = _mm_packs_epi32(out_26_6, out_26_7);
- out[ 6] = _mm_packs_epi32(out_06_6, out_06_7);
- out[22] = _mm_packs_epi32(out_22_6, out_22_7);
- out[14] = _mm_packs_epi32(out_14_6, out_14_7);
- out[30] = _mm_packs_epi32(out_30_6, out_30_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&out[2], &out[18], &out[10],
- &out[26], &out[6], &out[22],
- &out[14], &out[30]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- step1[16] = ADD_EPI16(step3[17], step2[16]);
- step1[17] = SUB_EPI16(step2[16], step3[17]);
- step1[18] = SUB_EPI16(step2[19], step3[18]);
- step1[19] = ADD_EPI16(step3[18], step2[19]);
- step1[20] = ADD_EPI16(step3[21], step2[20]);
- step1[21] = SUB_EPI16(step2[20], step3[21]);
- step1[22] = SUB_EPI16(step2[23], step3[22]);
- step1[23] = ADD_EPI16(step3[22], step2[23]);
- step1[24] = ADD_EPI16(step3[25], step2[24]);
- step1[25] = SUB_EPI16(step2[24], step3[25]);
- step1[26] = SUB_EPI16(step2[27], step3[26]);
- step1[27] = ADD_EPI16(step3[26], step2[27]);
- step1[28] = ADD_EPI16(step3[29], step2[28]);
- step1[29] = SUB_EPI16(step2[28], step3[29]);
- step1[30] = SUB_EPI16(step2[31], step3[30]);
- step1[31] = ADD_EPI16(step3[30], step2[31]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x16(
- &step1[16], &step1[17], &step1[18], &step1[19],
- &step1[20], &step1[21], &step1[22], &step1[23],
- &step1[24], &step1[25], &step1[26], &step1[27],
- &step1[28], &step1[29], &step1[30], &step1[31]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // Final stage --- outputs indices are bit-reversed.
- {
- const __m128i out_01_0 = _mm_unpacklo_epi16(step1[16], step1[31]);
- const __m128i out_01_1 = _mm_unpackhi_epi16(step1[16], step1[31]);
- const __m128i out_17_0 = _mm_unpacklo_epi16(step1[17], step1[30]);
- const __m128i out_17_1 = _mm_unpackhi_epi16(step1[17], step1[30]);
- const __m128i out_09_0 = _mm_unpacklo_epi16(step1[18], step1[29]);
- const __m128i out_09_1 = _mm_unpackhi_epi16(step1[18], step1[29]);
- const __m128i out_25_0 = _mm_unpacklo_epi16(step1[19], step1[28]);
- const __m128i out_25_1 = _mm_unpackhi_epi16(step1[19], step1[28]);
- const __m128i out_01_2 = _mm_madd_epi16(out_01_0, k__cospi_p31_p01);
- const __m128i out_01_3 = _mm_madd_epi16(out_01_1, k__cospi_p31_p01);
- const __m128i out_17_2 = _mm_madd_epi16(out_17_0, k__cospi_p15_p17);
- const __m128i out_17_3 = _mm_madd_epi16(out_17_1, k__cospi_p15_p17);
- const __m128i out_09_2 = _mm_madd_epi16(out_09_0, k__cospi_p23_p09);
- const __m128i out_09_3 = _mm_madd_epi16(out_09_1, k__cospi_p23_p09);
- const __m128i out_25_2 = _mm_madd_epi16(out_25_0, k__cospi_p07_p25);
- const __m128i out_25_3 = _mm_madd_epi16(out_25_1, k__cospi_p07_p25);
- const __m128i out_07_2 = _mm_madd_epi16(out_25_0, k__cospi_m25_p07);
- const __m128i out_07_3 = _mm_madd_epi16(out_25_1, k__cospi_m25_p07);
- const __m128i out_23_2 = _mm_madd_epi16(out_09_0, k__cospi_m09_p23);
- const __m128i out_23_3 = _mm_madd_epi16(out_09_1, k__cospi_m09_p23);
- const __m128i out_15_2 = _mm_madd_epi16(out_17_0, k__cospi_m17_p15);
- const __m128i out_15_3 = _mm_madd_epi16(out_17_1, k__cospi_m17_p15);
- const __m128i out_31_2 = _mm_madd_epi16(out_01_0, k__cospi_m01_p31);
- const __m128i out_31_3 = _mm_madd_epi16(out_01_1, k__cospi_m01_p31);
- // dct_const_round_shift
- const __m128i out_01_4 = _mm_add_epi32(out_01_2, k__DCT_CONST_ROUNDING);
- const __m128i out_01_5 = _mm_add_epi32(out_01_3, k__DCT_CONST_ROUNDING);
- const __m128i out_17_4 = _mm_add_epi32(out_17_2, k__DCT_CONST_ROUNDING);
- const __m128i out_17_5 = _mm_add_epi32(out_17_3, k__DCT_CONST_ROUNDING);
- const __m128i out_09_4 = _mm_add_epi32(out_09_2, k__DCT_CONST_ROUNDING);
- const __m128i out_09_5 = _mm_add_epi32(out_09_3, k__DCT_CONST_ROUNDING);
- const __m128i out_25_4 = _mm_add_epi32(out_25_2, k__DCT_CONST_ROUNDING);
- const __m128i out_25_5 = _mm_add_epi32(out_25_3, k__DCT_CONST_ROUNDING);
- const __m128i out_07_4 = _mm_add_epi32(out_07_2, k__DCT_CONST_ROUNDING);
- const __m128i out_07_5 = _mm_add_epi32(out_07_3, k__DCT_CONST_ROUNDING);
- const __m128i out_23_4 = _mm_add_epi32(out_23_2, k__DCT_CONST_ROUNDING);
- const __m128i out_23_5 = _mm_add_epi32(out_23_3, k__DCT_CONST_ROUNDING);
- const __m128i out_15_4 = _mm_add_epi32(out_15_2, k__DCT_CONST_ROUNDING);
- const __m128i out_15_5 = _mm_add_epi32(out_15_3, k__DCT_CONST_ROUNDING);
- const __m128i out_31_4 = _mm_add_epi32(out_31_2, k__DCT_CONST_ROUNDING);
- const __m128i out_31_5 = _mm_add_epi32(out_31_3, k__DCT_CONST_ROUNDING);
- const __m128i out_01_6 = _mm_srai_epi32(out_01_4, DCT_CONST_BITS);
- const __m128i out_01_7 = _mm_srai_epi32(out_01_5, DCT_CONST_BITS);
- const __m128i out_17_6 = _mm_srai_epi32(out_17_4, DCT_CONST_BITS);
- const __m128i out_17_7 = _mm_srai_epi32(out_17_5, DCT_CONST_BITS);
- const __m128i out_09_6 = _mm_srai_epi32(out_09_4, DCT_CONST_BITS);
- const __m128i out_09_7 = _mm_srai_epi32(out_09_5, DCT_CONST_BITS);
- const __m128i out_25_6 = _mm_srai_epi32(out_25_4, DCT_CONST_BITS);
- const __m128i out_25_7 = _mm_srai_epi32(out_25_5, DCT_CONST_BITS);
- const __m128i out_07_6 = _mm_srai_epi32(out_07_4, DCT_CONST_BITS);
- const __m128i out_07_7 = _mm_srai_epi32(out_07_5, DCT_CONST_BITS);
- const __m128i out_23_6 = _mm_srai_epi32(out_23_4, DCT_CONST_BITS);
- const __m128i out_23_7 = _mm_srai_epi32(out_23_5, DCT_CONST_BITS);
- const __m128i out_15_6 = _mm_srai_epi32(out_15_4, DCT_CONST_BITS);
- const __m128i out_15_7 = _mm_srai_epi32(out_15_5, DCT_CONST_BITS);
- const __m128i out_31_6 = _mm_srai_epi32(out_31_4, DCT_CONST_BITS);
- const __m128i out_31_7 = _mm_srai_epi32(out_31_5, DCT_CONST_BITS);
- // Combine
- out[ 1] = _mm_packs_epi32(out_01_6, out_01_7);
- out[17] = _mm_packs_epi32(out_17_6, out_17_7);
- out[ 9] = _mm_packs_epi32(out_09_6, out_09_7);
- out[25] = _mm_packs_epi32(out_25_6, out_25_7);
- out[ 7] = _mm_packs_epi32(out_07_6, out_07_7);
- out[23] = _mm_packs_epi32(out_23_6, out_23_7);
- out[15] = _mm_packs_epi32(out_15_6, out_15_7);
- out[31] = _mm_packs_epi32(out_31_6, out_31_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&out[1], &out[17], &out[9],
- &out[25], &out[7], &out[23],
- &out[15], &out[31]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i out_05_0 = _mm_unpacklo_epi16(step1[20], step1[27]);
- const __m128i out_05_1 = _mm_unpackhi_epi16(step1[20], step1[27]);
- const __m128i out_21_0 = _mm_unpacklo_epi16(step1[21], step1[26]);
- const __m128i out_21_1 = _mm_unpackhi_epi16(step1[21], step1[26]);
- const __m128i out_13_0 = _mm_unpacklo_epi16(step1[22], step1[25]);
- const __m128i out_13_1 = _mm_unpackhi_epi16(step1[22], step1[25]);
- const __m128i out_29_0 = _mm_unpacklo_epi16(step1[23], step1[24]);
- const __m128i out_29_1 = _mm_unpackhi_epi16(step1[23], step1[24]);
- const __m128i out_05_2 = _mm_madd_epi16(out_05_0, k__cospi_p27_p05);
- const __m128i out_05_3 = _mm_madd_epi16(out_05_1, k__cospi_p27_p05);
- const __m128i out_21_2 = _mm_madd_epi16(out_21_0, k__cospi_p11_p21);
- const __m128i out_21_3 = _mm_madd_epi16(out_21_1, k__cospi_p11_p21);
- const __m128i out_13_2 = _mm_madd_epi16(out_13_0, k__cospi_p19_p13);
- const __m128i out_13_3 = _mm_madd_epi16(out_13_1, k__cospi_p19_p13);
- const __m128i out_29_2 = _mm_madd_epi16(out_29_0, k__cospi_p03_p29);
- const __m128i out_29_3 = _mm_madd_epi16(out_29_1, k__cospi_p03_p29);
- const __m128i out_03_2 = _mm_madd_epi16(out_29_0, k__cospi_m29_p03);
- const __m128i out_03_3 = _mm_madd_epi16(out_29_1, k__cospi_m29_p03);
- const __m128i out_19_2 = _mm_madd_epi16(out_13_0, k__cospi_m13_p19);
- const __m128i out_19_3 = _mm_madd_epi16(out_13_1, k__cospi_m13_p19);
- const __m128i out_11_2 = _mm_madd_epi16(out_21_0, k__cospi_m21_p11);
- const __m128i out_11_3 = _mm_madd_epi16(out_21_1, k__cospi_m21_p11);
- const __m128i out_27_2 = _mm_madd_epi16(out_05_0, k__cospi_m05_p27);
- const __m128i out_27_3 = _mm_madd_epi16(out_05_1, k__cospi_m05_p27);
- // dct_const_round_shift
- const __m128i out_05_4 = _mm_add_epi32(out_05_2, k__DCT_CONST_ROUNDING);
- const __m128i out_05_5 = _mm_add_epi32(out_05_3, k__DCT_CONST_ROUNDING);
- const __m128i out_21_4 = _mm_add_epi32(out_21_2, k__DCT_CONST_ROUNDING);
- const __m128i out_21_5 = _mm_add_epi32(out_21_3, k__DCT_CONST_ROUNDING);
- const __m128i out_13_4 = _mm_add_epi32(out_13_2, k__DCT_CONST_ROUNDING);
- const __m128i out_13_5 = _mm_add_epi32(out_13_3, k__DCT_CONST_ROUNDING);
- const __m128i out_29_4 = _mm_add_epi32(out_29_2, k__DCT_CONST_ROUNDING);
- const __m128i out_29_5 = _mm_add_epi32(out_29_3, k__DCT_CONST_ROUNDING);
- const __m128i out_03_4 = _mm_add_epi32(out_03_2, k__DCT_CONST_ROUNDING);
- const __m128i out_03_5 = _mm_add_epi32(out_03_3, k__DCT_CONST_ROUNDING);
- const __m128i out_19_4 = _mm_add_epi32(out_19_2, k__DCT_CONST_ROUNDING);
- const __m128i out_19_5 = _mm_add_epi32(out_19_3, k__DCT_CONST_ROUNDING);
- const __m128i out_11_4 = _mm_add_epi32(out_11_2, k__DCT_CONST_ROUNDING);
- const __m128i out_11_5 = _mm_add_epi32(out_11_3, k__DCT_CONST_ROUNDING);
- const __m128i out_27_4 = _mm_add_epi32(out_27_2, k__DCT_CONST_ROUNDING);
- const __m128i out_27_5 = _mm_add_epi32(out_27_3, k__DCT_CONST_ROUNDING);
- const __m128i out_05_6 = _mm_srai_epi32(out_05_4, DCT_CONST_BITS);
- const __m128i out_05_7 = _mm_srai_epi32(out_05_5, DCT_CONST_BITS);
- const __m128i out_21_6 = _mm_srai_epi32(out_21_4, DCT_CONST_BITS);
- const __m128i out_21_7 = _mm_srai_epi32(out_21_5, DCT_CONST_BITS);
- const __m128i out_13_6 = _mm_srai_epi32(out_13_4, DCT_CONST_BITS);
- const __m128i out_13_7 = _mm_srai_epi32(out_13_5, DCT_CONST_BITS);
- const __m128i out_29_6 = _mm_srai_epi32(out_29_4, DCT_CONST_BITS);
- const __m128i out_29_7 = _mm_srai_epi32(out_29_5, DCT_CONST_BITS);
- const __m128i out_03_6 = _mm_srai_epi32(out_03_4, DCT_CONST_BITS);
- const __m128i out_03_7 = _mm_srai_epi32(out_03_5, DCT_CONST_BITS);
- const __m128i out_19_6 = _mm_srai_epi32(out_19_4, DCT_CONST_BITS);
- const __m128i out_19_7 = _mm_srai_epi32(out_19_5, DCT_CONST_BITS);
- const __m128i out_11_6 = _mm_srai_epi32(out_11_4, DCT_CONST_BITS);
- const __m128i out_11_7 = _mm_srai_epi32(out_11_5, DCT_CONST_BITS);
- const __m128i out_27_6 = _mm_srai_epi32(out_27_4, DCT_CONST_BITS);
- const __m128i out_27_7 = _mm_srai_epi32(out_27_5, DCT_CONST_BITS);
- // Combine
- out[ 5] = _mm_packs_epi32(out_05_6, out_05_7);
- out[21] = _mm_packs_epi32(out_21_6, out_21_7);
- out[13] = _mm_packs_epi32(out_13_6, out_13_7);
- out[29] = _mm_packs_epi32(out_29_6, out_29_7);
- out[ 3] = _mm_packs_epi32(out_03_6, out_03_7);
- out[19] = _mm_packs_epi32(out_19_6, out_19_7);
- out[11] = _mm_packs_epi32(out_11_6, out_11_7);
- out[27] = _mm_packs_epi32(out_27_6, out_27_7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&out[5], &out[21], &out[13],
- &out[29], &out[3], &out[19],
- &out[11], &out[27]);
- if (overflow) {
- if (pass == 0)
- HIGH_FDCT32x32_2D_C(input, output_org, stride);
- else
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
-#if FDCT32x32_HIGH_PRECISION
- } else {
- __m128i lstep1[64], lstep2[64], lstep3[64];
- __m128i u[32], v[32], sign[16];
- const __m128i K32One = _mm_set_epi32(1, 1, 1, 1);
- // start using 32-bit operations
- // stage 3
- {
- // expanding to 32-bit length priori to addition operations
- lstep2[ 0] = _mm_unpacklo_epi16(step2[ 0], kZero);
- lstep2[ 1] = _mm_unpackhi_epi16(step2[ 0], kZero);
- lstep2[ 2] = _mm_unpacklo_epi16(step2[ 1], kZero);
- lstep2[ 3] = _mm_unpackhi_epi16(step2[ 1], kZero);
- lstep2[ 4] = _mm_unpacklo_epi16(step2[ 2], kZero);
- lstep2[ 5] = _mm_unpackhi_epi16(step2[ 2], kZero);
- lstep2[ 6] = _mm_unpacklo_epi16(step2[ 3], kZero);
- lstep2[ 7] = _mm_unpackhi_epi16(step2[ 3], kZero);
- lstep2[ 8] = _mm_unpacklo_epi16(step2[ 4], kZero);
- lstep2[ 9] = _mm_unpackhi_epi16(step2[ 4], kZero);
- lstep2[10] = _mm_unpacklo_epi16(step2[ 5], kZero);
- lstep2[11] = _mm_unpackhi_epi16(step2[ 5], kZero);
- lstep2[12] = _mm_unpacklo_epi16(step2[ 6], kZero);
- lstep2[13] = _mm_unpackhi_epi16(step2[ 6], kZero);
- lstep2[14] = _mm_unpacklo_epi16(step2[ 7], kZero);
- lstep2[15] = _mm_unpackhi_epi16(step2[ 7], kZero);
- lstep2[ 0] = _mm_madd_epi16(lstep2[ 0], kOne);
- lstep2[ 1] = _mm_madd_epi16(lstep2[ 1], kOne);
- lstep2[ 2] = _mm_madd_epi16(lstep2[ 2], kOne);
- lstep2[ 3] = _mm_madd_epi16(lstep2[ 3], kOne);
- lstep2[ 4] = _mm_madd_epi16(lstep2[ 4], kOne);
- lstep2[ 5] = _mm_madd_epi16(lstep2[ 5], kOne);
- lstep2[ 6] = _mm_madd_epi16(lstep2[ 6], kOne);
- lstep2[ 7] = _mm_madd_epi16(lstep2[ 7], kOne);
- lstep2[ 8] = _mm_madd_epi16(lstep2[ 8], kOne);
- lstep2[ 9] = _mm_madd_epi16(lstep2[ 9], kOne);
- lstep2[10] = _mm_madd_epi16(lstep2[10], kOne);
- lstep2[11] = _mm_madd_epi16(lstep2[11], kOne);
- lstep2[12] = _mm_madd_epi16(lstep2[12], kOne);
- lstep2[13] = _mm_madd_epi16(lstep2[13], kOne);
- lstep2[14] = _mm_madd_epi16(lstep2[14], kOne);
- lstep2[15] = _mm_madd_epi16(lstep2[15], kOne);
-
- lstep3[ 0] = _mm_add_epi32(lstep2[14], lstep2[ 0]);
- lstep3[ 1] = _mm_add_epi32(lstep2[15], lstep2[ 1]);
- lstep3[ 2] = _mm_add_epi32(lstep2[12], lstep2[ 2]);
- lstep3[ 3] = _mm_add_epi32(lstep2[13], lstep2[ 3]);
- lstep3[ 4] = _mm_add_epi32(lstep2[10], lstep2[ 4]);
- lstep3[ 5] = _mm_add_epi32(lstep2[11], lstep2[ 5]);
- lstep3[ 6] = _mm_add_epi32(lstep2[ 8], lstep2[ 6]);
- lstep3[ 7] = _mm_add_epi32(lstep2[ 9], lstep2[ 7]);
- lstep3[ 8] = _mm_sub_epi32(lstep2[ 6], lstep2[ 8]);
- lstep3[ 9] = _mm_sub_epi32(lstep2[ 7], lstep2[ 9]);
- lstep3[10] = _mm_sub_epi32(lstep2[ 4], lstep2[10]);
- lstep3[11] = _mm_sub_epi32(lstep2[ 5], lstep2[11]);
- lstep3[12] = _mm_sub_epi32(lstep2[ 2], lstep2[12]);
- lstep3[13] = _mm_sub_epi32(lstep2[ 3], lstep2[13]);
- lstep3[14] = _mm_sub_epi32(lstep2[ 0], lstep2[14]);
- lstep3[15] = _mm_sub_epi32(lstep2[ 1], lstep2[15]);
- }
- {
- const __m128i s3_10_0 = _mm_unpacklo_epi16(step2[13], step2[10]);
- const __m128i s3_10_1 = _mm_unpackhi_epi16(step2[13], step2[10]);
- const __m128i s3_11_0 = _mm_unpacklo_epi16(step2[12], step2[11]);
- const __m128i s3_11_1 = _mm_unpackhi_epi16(step2[12], step2[11]);
- const __m128i s3_10_2 = _mm_madd_epi16(s3_10_0, k__cospi_p16_m16);
- const __m128i s3_10_3 = _mm_madd_epi16(s3_10_1, k__cospi_p16_m16);
- const __m128i s3_11_2 = _mm_madd_epi16(s3_11_0, k__cospi_p16_m16);
- const __m128i s3_11_3 = _mm_madd_epi16(s3_11_1, k__cospi_p16_m16);
- const __m128i s3_12_2 = _mm_madd_epi16(s3_11_0, k__cospi_p16_p16);
- const __m128i s3_12_3 = _mm_madd_epi16(s3_11_1, k__cospi_p16_p16);
- const __m128i s3_13_2 = _mm_madd_epi16(s3_10_0, k__cospi_p16_p16);
- const __m128i s3_13_3 = _mm_madd_epi16(s3_10_1, k__cospi_p16_p16);
- // dct_const_round_shift
- const __m128i s3_10_4 = _mm_add_epi32(s3_10_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_10_5 = _mm_add_epi32(s3_10_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_11_4 = _mm_add_epi32(s3_11_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_11_5 = _mm_add_epi32(s3_11_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_12_4 = _mm_add_epi32(s3_12_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_12_5 = _mm_add_epi32(s3_12_3, k__DCT_CONST_ROUNDING);
- const __m128i s3_13_4 = _mm_add_epi32(s3_13_2, k__DCT_CONST_ROUNDING);
- const __m128i s3_13_5 = _mm_add_epi32(s3_13_3, k__DCT_CONST_ROUNDING);
- lstep3[20] = _mm_srai_epi32(s3_10_4, DCT_CONST_BITS);
- lstep3[21] = _mm_srai_epi32(s3_10_5, DCT_CONST_BITS);
- lstep3[22] = _mm_srai_epi32(s3_11_4, DCT_CONST_BITS);
- lstep3[23] = _mm_srai_epi32(s3_11_5, DCT_CONST_BITS);
- lstep3[24] = _mm_srai_epi32(s3_12_4, DCT_CONST_BITS);
- lstep3[25] = _mm_srai_epi32(s3_12_5, DCT_CONST_BITS);
- lstep3[26] = _mm_srai_epi32(s3_13_4, DCT_CONST_BITS);
- lstep3[27] = _mm_srai_epi32(s3_13_5, DCT_CONST_BITS);
- }
- {
- lstep2[40] = _mm_unpacklo_epi16(step2[20], kZero);
- lstep2[41] = _mm_unpackhi_epi16(step2[20], kZero);
- lstep2[42] = _mm_unpacklo_epi16(step2[21], kZero);
- lstep2[43] = _mm_unpackhi_epi16(step2[21], kZero);
- lstep2[44] = _mm_unpacklo_epi16(step2[22], kZero);
- lstep2[45] = _mm_unpackhi_epi16(step2[22], kZero);
- lstep2[46] = _mm_unpacklo_epi16(step2[23], kZero);
- lstep2[47] = _mm_unpackhi_epi16(step2[23], kZero);
- lstep2[48] = _mm_unpacklo_epi16(step2[24], kZero);
- lstep2[49] = _mm_unpackhi_epi16(step2[24], kZero);
- lstep2[50] = _mm_unpacklo_epi16(step2[25], kZero);
- lstep2[51] = _mm_unpackhi_epi16(step2[25], kZero);
- lstep2[52] = _mm_unpacklo_epi16(step2[26], kZero);
- lstep2[53] = _mm_unpackhi_epi16(step2[26], kZero);
- lstep2[54] = _mm_unpacklo_epi16(step2[27], kZero);
- lstep2[55] = _mm_unpackhi_epi16(step2[27], kZero);
- lstep2[40] = _mm_madd_epi16(lstep2[40], kOne);
- lstep2[41] = _mm_madd_epi16(lstep2[41], kOne);
- lstep2[42] = _mm_madd_epi16(lstep2[42], kOne);
- lstep2[43] = _mm_madd_epi16(lstep2[43], kOne);
- lstep2[44] = _mm_madd_epi16(lstep2[44], kOne);
- lstep2[45] = _mm_madd_epi16(lstep2[45], kOne);
- lstep2[46] = _mm_madd_epi16(lstep2[46], kOne);
- lstep2[47] = _mm_madd_epi16(lstep2[47], kOne);
- lstep2[48] = _mm_madd_epi16(lstep2[48], kOne);
- lstep2[49] = _mm_madd_epi16(lstep2[49], kOne);
- lstep2[50] = _mm_madd_epi16(lstep2[50], kOne);
- lstep2[51] = _mm_madd_epi16(lstep2[51], kOne);
- lstep2[52] = _mm_madd_epi16(lstep2[52], kOne);
- lstep2[53] = _mm_madd_epi16(lstep2[53], kOne);
- lstep2[54] = _mm_madd_epi16(lstep2[54], kOne);
- lstep2[55] = _mm_madd_epi16(lstep2[55], kOne);
-
- lstep1[32] = _mm_unpacklo_epi16(step1[16], kZero);
- lstep1[33] = _mm_unpackhi_epi16(step1[16], kZero);
- lstep1[34] = _mm_unpacklo_epi16(step1[17], kZero);
- lstep1[35] = _mm_unpackhi_epi16(step1[17], kZero);
- lstep1[36] = _mm_unpacklo_epi16(step1[18], kZero);
- lstep1[37] = _mm_unpackhi_epi16(step1[18], kZero);
- lstep1[38] = _mm_unpacklo_epi16(step1[19], kZero);
- lstep1[39] = _mm_unpackhi_epi16(step1[19], kZero);
- lstep1[56] = _mm_unpacklo_epi16(step1[28], kZero);
- lstep1[57] = _mm_unpackhi_epi16(step1[28], kZero);
- lstep1[58] = _mm_unpacklo_epi16(step1[29], kZero);
- lstep1[59] = _mm_unpackhi_epi16(step1[29], kZero);
- lstep1[60] = _mm_unpacklo_epi16(step1[30], kZero);
- lstep1[61] = _mm_unpackhi_epi16(step1[30], kZero);
- lstep1[62] = _mm_unpacklo_epi16(step1[31], kZero);
- lstep1[63] = _mm_unpackhi_epi16(step1[31], kZero);
- lstep1[32] = _mm_madd_epi16(lstep1[32], kOne);
- lstep1[33] = _mm_madd_epi16(lstep1[33], kOne);
- lstep1[34] = _mm_madd_epi16(lstep1[34], kOne);
- lstep1[35] = _mm_madd_epi16(lstep1[35], kOne);
- lstep1[36] = _mm_madd_epi16(lstep1[36], kOne);
- lstep1[37] = _mm_madd_epi16(lstep1[37], kOne);
- lstep1[38] = _mm_madd_epi16(lstep1[38], kOne);
- lstep1[39] = _mm_madd_epi16(lstep1[39], kOne);
- lstep1[56] = _mm_madd_epi16(lstep1[56], kOne);
- lstep1[57] = _mm_madd_epi16(lstep1[57], kOne);
- lstep1[58] = _mm_madd_epi16(lstep1[58], kOne);
- lstep1[59] = _mm_madd_epi16(lstep1[59], kOne);
- lstep1[60] = _mm_madd_epi16(lstep1[60], kOne);
- lstep1[61] = _mm_madd_epi16(lstep1[61], kOne);
- lstep1[62] = _mm_madd_epi16(lstep1[62], kOne);
- lstep1[63] = _mm_madd_epi16(lstep1[63], kOne);
-
- lstep3[32] = _mm_add_epi32(lstep2[46], lstep1[32]);
- lstep3[33] = _mm_add_epi32(lstep2[47], lstep1[33]);
-
- lstep3[34] = _mm_add_epi32(lstep2[44], lstep1[34]);
- lstep3[35] = _mm_add_epi32(lstep2[45], lstep1[35]);
- lstep3[36] = _mm_add_epi32(lstep2[42], lstep1[36]);
- lstep3[37] = _mm_add_epi32(lstep2[43], lstep1[37]);
- lstep3[38] = _mm_add_epi32(lstep2[40], lstep1[38]);
- lstep3[39] = _mm_add_epi32(lstep2[41], lstep1[39]);
- lstep3[40] = _mm_sub_epi32(lstep1[38], lstep2[40]);
- lstep3[41] = _mm_sub_epi32(lstep1[39], lstep2[41]);
- lstep3[42] = _mm_sub_epi32(lstep1[36], lstep2[42]);
- lstep3[43] = _mm_sub_epi32(lstep1[37], lstep2[43]);
- lstep3[44] = _mm_sub_epi32(lstep1[34], lstep2[44]);
- lstep3[45] = _mm_sub_epi32(lstep1[35], lstep2[45]);
- lstep3[46] = _mm_sub_epi32(lstep1[32], lstep2[46]);
- lstep3[47] = _mm_sub_epi32(lstep1[33], lstep2[47]);
- lstep3[48] = _mm_sub_epi32(lstep1[62], lstep2[48]);
- lstep3[49] = _mm_sub_epi32(lstep1[63], lstep2[49]);
- lstep3[50] = _mm_sub_epi32(lstep1[60], lstep2[50]);
- lstep3[51] = _mm_sub_epi32(lstep1[61], lstep2[51]);
- lstep3[52] = _mm_sub_epi32(lstep1[58], lstep2[52]);
- lstep3[53] = _mm_sub_epi32(lstep1[59], lstep2[53]);
- lstep3[54] = _mm_sub_epi32(lstep1[56], lstep2[54]);
- lstep3[55] = _mm_sub_epi32(lstep1[57], lstep2[55]);
- lstep3[56] = _mm_add_epi32(lstep2[54], lstep1[56]);
- lstep3[57] = _mm_add_epi32(lstep2[55], lstep1[57]);
- lstep3[58] = _mm_add_epi32(lstep2[52], lstep1[58]);
- lstep3[59] = _mm_add_epi32(lstep2[53], lstep1[59]);
- lstep3[60] = _mm_add_epi32(lstep2[50], lstep1[60]);
- lstep3[61] = _mm_add_epi32(lstep2[51], lstep1[61]);
- lstep3[62] = _mm_add_epi32(lstep2[48], lstep1[62]);
- lstep3[63] = _mm_add_epi32(lstep2[49], lstep1[63]);
- }
-
- // stage 4
- {
- // expanding to 32-bit length priori to addition operations
- lstep2[16] = _mm_unpacklo_epi16(step2[ 8], kZero);
- lstep2[17] = _mm_unpackhi_epi16(step2[ 8], kZero);
- lstep2[18] = _mm_unpacklo_epi16(step2[ 9], kZero);
- lstep2[19] = _mm_unpackhi_epi16(step2[ 9], kZero);
- lstep2[28] = _mm_unpacklo_epi16(step2[14], kZero);
- lstep2[29] = _mm_unpackhi_epi16(step2[14], kZero);
- lstep2[30] = _mm_unpacklo_epi16(step2[15], kZero);
- lstep2[31] = _mm_unpackhi_epi16(step2[15], kZero);
- lstep2[16] = _mm_madd_epi16(lstep2[16], kOne);
- lstep2[17] = _mm_madd_epi16(lstep2[17], kOne);
- lstep2[18] = _mm_madd_epi16(lstep2[18], kOne);
- lstep2[19] = _mm_madd_epi16(lstep2[19], kOne);
- lstep2[28] = _mm_madd_epi16(lstep2[28], kOne);
- lstep2[29] = _mm_madd_epi16(lstep2[29], kOne);
- lstep2[30] = _mm_madd_epi16(lstep2[30], kOne);
- lstep2[31] = _mm_madd_epi16(lstep2[31], kOne);
-
- lstep1[ 0] = _mm_add_epi32(lstep3[ 6], lstep3[ 0]);
- lstep1[ 1] = _mm_add_epi32(lstep3[ 7], lstep3[ 1]);
- lstep1[ 2] = _mm_add_epi32(lstep3[ 4], lstep3[ 2]);
- lstep1[ 3] = _mm_add_epi32(lstep3[ 5], lstep3[ 3]);
- lstep1[ 4] = _mm_sub_epi32(lstep3[ 2], lstep3[ 4]);
- lstep1[ 5] = _mm_sub_epi32(lstep3[ 3], lstep3[ 5]);
- lstep1[ 6] = _mm_sub_epi32(lstep3[ 0], lstep3[ 6]);
- lstep1[ 7] = _mm_sub_epi32(lstep3[ 1], lstep3[ 7]);
- lstep1[16] = _mm_add_epi32(lstep3[22], lstep2[16]);
- lstep1[17] = _mm_add_epi32(lstep3[23], lstep2[17]);
- lstep1[18] = _mm_add_epi32(lstep3[20], lstep2[18]);
- lstep1[19] = _mm_add_epi32(lstep3[21], lstep2[19]);
- lstep1[20] = _mm_sub_epi32(lstep2[18], lstep3[20]);
- lstep1[21] = _mm_sub_epi32(lstep2[19], lstep3[21]);
- lstep1[22] = _mm_sub_epi32(lstep2[16], lstep3[22]);
- lstep1[23] = _mm_sub_epi32(lstep2[17], lstep3[23]);
- lstep1[24] = _mm_sub_epi32(lstep2[30], lstep3[24]);
- lstep1[25] = _mm_sub_epi32(lstep2[31], lstep3[25]);
- lstep1[26] = _mm_sub_epi32(lstep2[28], lstep3[26]);
- lstep1[27] = _mm_sub_epi32(lstep2[29], lstep3[27]);
- lstep1[28] = _mm_add_epi32(lstep3[26], lstep2[28]);
- lstep1[29] = _mm_add_epi32(lstep3[27], lstep2[29]);
- lstep1[30] = _mm_add_epi32(lstep3[24], lstep2[30]);
- lstep1[31] = _mm_add_epi32(lstep3[25], lstep2[31]);
- }
- {
- // to be continued...
- //
- const __m128i k32_p16_p16 = pair_set_epi32(cospi_16_64, cospi_16_64);
- const __m128i k32_p16_m16 = pair_set_epi32(cospi_16_64, -cospi_16_64);
-
- u[0] = _mm_unpacklo_epi32(lstep3[12], lstep3[10]);
- u[1] = _mm_unpackhi_epi32(lstep3[12], lstep3[10]);
- u[2] = _mm_unpacklo_epi32(lstep3[13], lstep3[11]);
- u[3] = _mm_unpackhi_epi32(lstep3[13], lstep3[11]);
-
- // TODO(jingning): manually inline k_madd_epi32_ to further hide
- // instruction latency.
- v[0] = k_madd_epi32(u[0], k32_p16_m16);
- v[1] = k_madd_epi32(u[1], k32_p16_m16);
- v[2] = k_madd_epi32(u[2], k32_p16_m16);
- v[3] = k_madd_epi32(u[3], k32_p16_m16);
- v[4] = k_madd_epi32(u[0], k32_p16_p16);
- v[5] = k_madd_epi32(u[1], k32_p16_p16);
- v[6] = k_madd_epi32(u[2], k32_p16_p16);
- v[7] = k_madd_epi32(u[3], k32_p16_p16);
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_8(&v[0], &v[1], &v[2], &v[3],
- &v[4], &v[5], &v[6], &v[7], &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[0] = k_packs_epi64(v[0], v[1]);
- u[1] = k_packs_epi64(v[2], v[3]);
- u[2] = k_packs_epi64(v[4], v[5]);
- u[3] = k_packs_epi64(v[6], v[7]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
-
- lstep1[10] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- lstep1[11] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- lstep1[12] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- lstep1[13] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- }
- {
- const __m128i k32_m08_p24 = pair_set_epi32(-cospi_8_64, cospi_24_64);
- const __m128i k32_m24_m08 = pair_set_epi32(-cospi_24_64, -cospi_8_64);
- const __m128i k32_p24_p08 = pair_set_epi32(cospi_24_64, cospi_8_64);
-
- u[ 0] = _mm_unpacklo_epi32(lstep3[36], lstep3[58]);
- u[ 1] = _mm_unpackhi_epi32(lstep3[36], lstep3[58]);
- u[ 2] = _mm_unpacklo_epi32(lstep3[37], lstep3[59]);
- u[ 3] = _mm_unpackhi_epi32(lstep3[37], lstep3[59]);
- u[ 4] = _mm_unpacklo_epi32(lstep3[38], lstep3[56]);
- u[ 5] = _mm_unpackhi_epi32(lstep3[38], lstep3[56]);
- u[ 6] = _mm_unpacklo_epi32(lstep3[39], lstep3[57]);
- u[ 7] = _mm_unpackhi_epi32(lstep3[39], lstep3[57]);
- u[ 8] = _mm_unpacklo_epi32(lstep3[40], lstep3[54]);
- u[ 9] = _mm_unpackhi_epi32(lstep3[40], lstep3[54]);
- u[10] = _mm_unpacklo_epi32(lstep3[41], lstep3[55]);
- u[11] = _mm_unpackhi_epi32(lstep3[41], lstep3[55]);
- u[12] = _mm_unpacklo_epi32(lstep3[42], lstep3[52]);
- u[13] = _mm_unpackhi_epi32(lstep3[42], lstep3[52]);
- u[14] = _mm_unpacklo_epi32(lstep3[43], lstep3[53]);
- u[15] = _mm_unpackhi_epi32(lstep3[43], lstep3[53]);
-
- v[ 0] = k_madd_epi32(u[ 0], k32_m08_p24);
- v[ 1] = k_madd_epi32(u[ 1], k32_m08_p24);
- v[ 2] = k_madd_epi32(u[ 2], k32_m08_p24);
- v[ 3] = k_madd_epi32(u[ 3], k32_m08_p24);
- v[ 4] = k_madd_epi32(u[ 4], k32_m08_p24);
- v[ 5] = k_madd_epi32(u[ 5], k32_m08_p24);
- v[ 6] = k_madd_epi32(u[ 6], k32_m08_p24);
- v[ 7] = k_madd_epi32(u[ 7], k32_m08_p24);
- v[ 8] = k_madd_epi32(u[ 8], k32_m24_m08);
- v[ 9] = k_madd_epi32(u[ 9], k32_m24_m08);
- v[10] = k_madd_epi32(u[10], k32_m24_m08);
- v[11] = k_madd_epi32(u[11], k32_m24_m08);
- v[12] = k_madd_epi32(u[12], k32_m24_m08);
- v[13] = k_madd_epi32(u[13], k32_m24_m08);
- v[14] = k_madd_epi32(u[14], k32_m24_m08);
- v[15] = k_madd_epi32(u[15], k32_m24_m08);
- v[16] = k_madd_epi32(u[12], k32_m08_p24);
- v[17] = k_madd_epi32(u[13], k32_m08_p24);
- v[18] = k_madd_epi32(u[14], k32_m08_p24);
- v[19] = k_madd_epi32(u[15], k32_m08_p24);
- v[20] = k_madd_epi32(u[ 8], k32_m08_p24);
- v[21] = k_madd_epi32(u[ 9], k32_m08_p24);
- v[22] = k_madd_epi32(u[10], k32_m08_p24);
- v[23] = k_madd_epi32(u[11], k32_m08_p24);
- v[24] = k_madd_epi32(u[ 4], k32_p24_p08);
- v[25] = k_madd_epi32(u[ 5], k32_p24_p08);
- v[26] = k_madd_epi32(u[ 6], k32_p24_p08);
- v[27] = k_madd_epi32(u[ 7], k32_p24_p08);
- v[28] = k_madd_epi32(u[ 0], k32_p24_p08);
- v[29] = k_madd_epi32(u[ 1], k32_p24_p08);
- v[30] = k_madd_epi32(u[ 2], k32_p24_p08);
- v[31] = k_madd_epi32(u[ 3], k32_p24_p08);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_32(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &v[16], &v[17], &v[18], &v[19], &v[20], &v[21], &v[22], &v[23],
- &v[24], &v[25], &v[26], &v[27], &v[28], &v[29], &v[30], &v[31],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[ 0] = k_packs_epi64(v[ 0], v[ 1]);
- u[ 1] = k_packs_epi64(v[ 2], v[ 3]);
- u[ 2] = k_packs_epi64(v[ 4], v[ 5]);
- u[ 3] = k_packs_epi64(v[ 6], v[ 7]);
- u[ 4] = k_packs_epi64(v[ 8], v[ 9]);
- u[ 5] = k_packs_epi64(v[10], v[11]);
- u[ 6] = k_packs_epi64(v[12], v[13]);
- u[ 7] = k_packs_epi64(v[14], v[15]);
- u[ 8] = k_packs_epi64(v[16], v[17]);
- u[ 9] = k_packs_epi64(v[18], v[19]);
- u[10] = k_packs_epi64(v[20], v[21]);
- u[11] = k_packs_epi64(v[22], v[23]);
- u[12] = k_packs_epi64(v[24], v[25]);
- u[13] = k_packs_epi64(v[26], v[27]);
- u[14] = k_packs_epi64(v[28], v[29]);
- u[15] = k_packs_epi64(v[30], v[31]);
-
- v[ 0] = _mm_add_epi32(u[ 0], k__DCT_CONST_ROUNDING);
- v[ 1] = _mm_add_epi32(u[ 1], k__DCT_CONST_ROUNDING);
- v[ 2] = _mm_add_epi32(u[ 2], k__DCT_CONST_ROUNDING);
- v[ 3] = _mm_add_epi32(u[ 3], k__DCT_CONST_ROUNDING);
- v[ 4] = _mm_add_epi32(u[ 4], k__DCT_CONST_ROUNDING);
- v[ 5] = _mm_add_epi32(u[ 5], k__DCT_CONST_ROUNDING);
- v[ 6] = _mm_add_epi32(u[ 6], k__DCT_CONST_ROUNDING);
- v[ 7] = _mm_add_epi32(u[ 7], k__DCT_CONST_ROUNDING);
- v[ 8] = _mm_add_epi32(u[ 8], k__DCT_CONST_ROUNDING);
- v[ 9] = _mm_add_epi32(u[ 9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- lstep1[36] = _mm_srai_epi32(v[ 0], DCT_CONST_BITS);
- lstep1[37] = _mm_srai_epi32(v[ 1], DCT_CONST_BITS);
- lstep1[38] = _mm_srai_epi32(v[ 2], DCT_CONST_BITS);
- lstep1[39] = _mm_srai_epi32(v[ 3], DCT_CONST_BITS);
- lstep1[40] = _mm_srai_epi32(v[ 4], DCT_CONST_BITS);
- lstep1[41] = _mm_srai_epi32(v[ 5], DCT_CONST_BITS);
- lstep1[42] = _mm_srai_epi32(v[ 6], DCT_CONST_BITS);
- lstep1[43] = _mm_srai_epi32(v[ 7], DCT_CONST_BITS);
- lstep1[52] = _mm_srai_epi32(v[ 8], DCT_CONST_BITS);
- lstep1[53] = _mm_srai_epi32(v[ 9], DCT_CONST_BITS);
- lstep1[54] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- lstep1[55] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- lstep1[56] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- lstep1[57] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- lstep1[58] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- lstep1[59] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
- }
- // stage 5
- {
- lstep2[ 8] = _mm_add_epi32(lstep1[10], lstep3[ 8]);
- lstep2[ 9] = _mm_add_epi32(lstep1[11], lstep3[ 9]);
- lstep2[10] = _mm_sub_epi32(lstep3[ 8], lstep1[10]);
- lstep2[11] = _mm_sub_epi32(lstep3[ 9], lstep1[11]);
- lstep2[12] = _mm_sub_epi32(lstep3[14], lstep1[12]);
- lstep2[13] = _mm_sub_epi32(lstep3[15], lstep1[13]);
- lstep2[14] = _mm_add_epi32(lstep1[12], lstep3[14]);
- lstep2[15] = _mm_add_epi32(lstep1[13], lstep3[15]);
- }
- {
- const __m128i k32_p16_p16 = pair_set_epi32(cospi_16_64, cospi_16_64);
- const __m128i k32_p16_m16 = pair_set_epi32(cospi_16_64, -cospi_16_64);
- const __m128i k32_p24_p08 = pair_set_epi32(cospi_24_64, cospi_8_64);
- const __m128i k32_m08_p24 = pair_set_epi32(-cospi_8_64, cospi_24_64);
-
- u[0] = _mm_unpacklo_epi32(lstep1[0], lstep1[2]);
- u[1] = _mm_unpackhi_epi32(lstep1[0], lstep1[2]);
- u[2] = _mm_unpacklo_epi32(lstep1[1], lstep1[3]);
- u[3] = _mm_unpackhi_epi32(lstep1[1], lstep1[3]);
- u[4] = _mm_unpacklo_epi32(lstep1[4], lstep1[6]);
- u[5] = _mm_unpackhi_epi32(lstep1[4], lstep1[6]);
- u[6] = _mm_unpacklo_epi32(lstep1[5], lstep1[7]);
- u[7] = _mm_unpackhi_epi32(lstep1[5], lstep1[7]);
-
- // TODO(jingning): manually inline k_madd_epi32_ to further hide
- // instruction latency.
- v[ 0] = k_madd_epi32(u[0], k32_p16_p16);
- v[ 1] = k_madd_epi32(u[1], k32_p16_p16);
- v[ 2] = k_madd_epi32(u[2], k32_p16_p16);
- v[ 3] = k_madd_epi32(u[3], k32_p16_p16);
- v[ 4] = k_madd_epi32(u[0], k32_p16_m16);
- v[ 5] = k_madd_epi32(u[1], k32_p16_m16);
- v[ 6] = k_madd_epi32(u[2], k32_p16_m16);
- v[ 7] = k_madd_epi32(u[3], k32_p16_m16);
- v[ 8] = k_madd_epi32(u[4], k32_p24_p08);
- v[ 9] = k_madd_epi32(u[5], k32_p24_p08);
- v[10] = k_madd_epi32(u[6], k32_p24_p08);
- v[11] = k_madd_epi32(u[7], k32_p24_p08);
- v[12] = k_madd_epi32(u[4], k32_m08_p24);
- v[13] = k_madd_epi32(u[5], k32_m08_p24);
- v[14] = k_madd_epi32(u[6], k32_m08_p24);
- v[15] = k_madd_epi32(u[7], k32_m08_p24);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_16(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[0] = k_packs_epi64(v[0], v[1]);
- u[1] = k_packs_epi64(v[2], v[3]);
- u[2] = k_packs_epi64(v[4], v[5]);
- u[3] = k_packs_epi64(v[6], v[7]);
- u[4] = k_packs_epi64(v[8], v[9]);
- u[5] = k_packs_epi64(v[10], v[11]);
- u[6] = k_packs_epi64(v[12], v[13]);
- u[7] = k_packs_epi64(v[14], v[15]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
-
- sign[0] = _mm_cmplt_epi32(u[0], kZero);
- sign[1] = _mm_cmplt_epi32(u[1], kZero);
- sign[2] = _mm_cmplt_epi32(u[2], kZero);
- sign[3] = _mm_cmplt_epi32(u[3], kZero);
- sign[4] = _mm_cmplt_epi32(u[4], kZero);
- sign[5] = _mm_cmplt_epi32(u[5], kZero);
- sign[6] = _mm_cmplt_epi32(u[6], kZero);
- sign[7] = _mm_cmplt_epi32(u[7], kZero);
-
- u[0] = _mm_sub_epi32(u[0], sign[0]);
- u[1] = _mm_sub_epi32(u[1], sign[1]);
- u[2] = _mm_sub_epi32(u[2], sign[2]);
- u[3] = _mm_sub_epi32(u[3], sign[3]);
- u[4] = _mm_sub_epi32(u[4], sign[4]);
- u[5] = _mm_sub_epi32(u[5], sign[5]);
- u[6] = _mm_sub_epi32(u[6], sign[6]);
- u[7] = _mm_sub_epi32(u[7], sign[7]);
-
- u[0] = _mm_add_epi32(u[0], K32One);
- u[1] = _mm_add_epi32(u[1], K32One);
- u[2] = _mm_add_epi32(u[2], K32One);
- u[3] = _mm_add_epi32(u[3], K32One);
- u[4] = _mm_add_epi32(u[4], K32One);
- u[5] = _mm_add_epi32(u[5], K32One);
- u[6] = _mm_add_epi32(u[6], K32One);
- u[7] = _mm_add_epi32(u[7], K32One);
-
- u[0] = _mm_srai_epi32(u[0], 2);
- u[1] = _mm_srai_epi32(u[1], 2);
- u[2] = _mm_srai_epi32(u[2], 2);
- u[3] = _mm_srai_epi32(u[3], 2);
- u[4] = _mm_srai_epi32(u[4], 2);
- u[5] = _mm_srai_epi32(u[5], 2);
- u[6] = _mm_srai_epi32(u[6], 2);
- u[7] = _mm_srai_epi32(u[7], 2);
-
- // Combine
- out[ 0] = _mm_packs_epi32(u[0], u[1]);
- out[16] = _mm_packs_epi32(u[2], u[3]);
- out[ 8] = _mm_packs_epi32(u[4], u[5]);
- out[24] = _mm_packs_epi32(u[6], u[7]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&out[0], &out[16],
- &out[8], &out[24]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i k32_m08_p24 = pair_set_epi32(-cospi_8_64, cospi_24_64);
- const __m128i k32_m24_m08 = pair_set_epi32(-cospi_24_64, -cospi_8_64);
- const __m128i k32_p24_p08 = pair_set_epi32(cospi_24_64, cospi_8_64);
-
- u[0] = _mm_unpacklo_epi32(lstep1[18], lstep1[28]);
- u[1] = _mm_unpackhi_epi32(lstep1[18], lstep1[28]);
- u[2] = _mm_unpacklo_epi32(lstep1[19], lstep1[29]);
- u[3] = _mm_unpackhi_epi32(lstep1[19], lstep1[29]);
- u[4] = _mm_unpacklo_epi32(lstep1[20], lstep1[26]);
- u[5] = _mm_unpackhi_epi32(lstep1[20], lstep1[26]);
- u[6] = _mm_unpacklo_epi32(lstep1[21], lstep1[27]);
- u[7] = _mm_unpackhi_epi32(lstep1[21], lstep1[27]);
-
- v[0] = k_madd_epi32(u[0], k32_m08_p24);
- v[1] = k_madd_epi32(u[1], k32_m08_p24);
- v[2] = k_madd_epi32(u[2], k32_m08_p24);
- v[3] = k_madd_epi32(u[3], k32_m08_p24);
- v[4] = k_madd_epi32(u[4], k32_m24_m08);
- v[5] = k_madd_epi32(u[5], k32_m24_m08);
- v[6] = k_madd_epi32(u[6], k32_m24_m08);
- v[7] = k_madd_epi32(u[7], k32_m24_m08);
- v[ 8] = k_madd_epi32(u[4], k32_m08_p24);
- v[ 9] = k_madd_epi32(u[5], k32_m08_p24);
- v[10] = k_madd_epi32(u[6], k32_m08_p24);
- v[11] = k_madd_epi32(u[7], k32_m08_p24);
- v[12] = k_madd_epi32(u[0], k32_p24_p08);
- v[13] = k_madd_epi32(u[1], k32_p24_p08);
- v[14] = k_madd_epi32(u[2], k32_p24_p08);
- v[15] = k_madd_epi32(u[3], k32_p24_p08);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_16(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[0] = k_packs_epi64(v[0], v[1]);
- u[1] = k_packs_epi64(v[2], v[3]);
- u[2] = k_packs_epi64(v[4], v[5]);
- u[3] = k_packs_epi64(v[6], v[7]);
- u[4] = k_packs_epi64(v[8], v[9]);
- u[5] = k_packs_epi64(v[10], v[11]);
- u[6] = k_packs_epi64(v[12], v[13]);
- u[7] = k_packs_epi64(v[14], v[15]);
-
- u[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
-
- lstep2[18] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- lstep2[19] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- lstep2[20] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- lstep2[21] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- lstep2[26] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- lstep2[27] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- lstep2[28] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- lstep2[29] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- }
- {
- lstep2[32] = _mm_add_epi32(lstep1[38], lstep3[32]);
- lstep2[33] = _mm_add_epi32(lstep1[39], lstep3[33]);
- lstep2[34] = _mm_add_epi32(lstep1[36], lstep3[34]);
- lstep2[35] = _mm_add_epi32(lstep1[37], lstep3[35]);
- lstep2[36] = _mm_sub_epi32(lstep3[34], lstep1[36]);
- lstep2[37] = _mm_sub_epi32(lstep3[35], lstep1[37]);
- lstep2[38] = _mm_sub_epi32(lstep3[32], lstep1[38]);
- lstep2[39] = _mm_sub_epi32(lstep3[33], lstep1[39]);
- lstep2[40] = _mm_sub_epi32(lstep3[46], lstep1[40]);
- lstep2[41] = _mm_sub_epi32(lstep3[47], lstep1[41]);
- lstep2[42] = _mm_sub_epi32(lstep3[44], lstep1[42]);
- lstep2[43] = _mm_sub_epi32(lstep3[45], lstep1[43]);
- lstep2[44] = _mm_add_epi32(lstep1[42], lstep3[44]);
- lstep2[45] = _mm_add_epi32(lstep1[43], lstep3[45]);
- lstep2[46] = _mm_add_epi32(lstep1[40], lstep3[46]);
- lstep2[47] = _mm_add_epi32(lstep1[41], lstep3[47]);
- lstep2[48] = _mm_add_epi32(lstep1[54], lstep3[48]);
- lstep2[49] = _mm_add_epi32(lstep1[55], lstep3[49]);
- lstep2[50] = _mm_add_epi32(lstep1[52], lstep3[50]);
- lstep2[51] = _mm_add_epi32(lstep1[53], lstep3[51]);
- lstep2[52] = _mm_sub_epi32(lstep3[50], lstep1[52]);
- lstep2[53] = _mm_sub_epi32(lstep3[51], lstep1[53]);
- lstep2[54] = _mm_sub_epi32(lstep3[48], lstep1[54]);
- lstep2[55] = _mm_sub_epi32(lstep3[49], lstep1[55]);
- lstep2[56] = _mm_sub_epi32(lstep3[62], lstep1[56]);
- lstep2[57] = _mm_sub_epi32(lstep3[63], lstep1[57]);
- lstep2[58] = _mm_sub_epi32(lstep3[60], lstep1[58]);
- lstep2[59] = _mm_sub_epi32(lstep3[61], lstep1[59]);
- lstep2[60] = _mm_add_epi32(lstep1[58], lstep3[60]);
- lstep2[61] = _mm_add_epi32(lstep1[59], lstep3[61]);
- lstep2[62] = _mm_add_epi32(lstep1[56], lstep3[62]);
- lstep2[63] = _mm_add_epi32(lstep1[57], lstep3[63]);
- }
- // stage 6
- {
- const __m128i k32_p28_p04 = pair_set_epi32(cospi_28_64, cospi_4_64);
- const __m128i k32_p12_p20 = pair_set_epi32(cospi_12_64, cospi_20_64);
- const __m128i k32_m20_p12 = pair_set_epi32(-cospi_20_64, cospi_12_64);
- const __m128i k32_m04_p28 = pair_set_epi32(-cospi_4_64, cospi_28_64);
-
- u[0] = _mm_unpacklo_epi32(lstep2[ 8], lstep2[14]);
- u[1] = _mm_unpackhi_epi32(lstep2[ 8], lstep2[14]);
- u[2] = _mm_unpacklo_epi32(lstep2[ 9], lstep2[15]);
- u[3] = _mm_unpackhi_epi32(lstep2[ 9], lstep2[15]);
- u[4] = _mm_unpacklo_epi32(lstep2[10], lstep2[12]);
- u[5] = _mm_unpackhi_epi32(lstep2[10], lstep2[12]);
- u[6] = _mm_unpacklo_epi32(lstep2[11], lstep2[13]);
- u[7] = _mm_unpackhi_epi32(lstep2[11], lstep2[13]);
- u[8] = _mm_unpacklo_epi32(lstep2[10], lstep2[12]);
- u[9] = _mm_unpackhi_epi32(lstep2[10], lstep2[12]);
- u[10] = _mm_unpacklo_epi32(lstep2[11], lstep2[13]);
- u[11] = _mm_unpackhi_epi32(lstep2[11], lstep2[13]);
- u[12] = _mm_unpacklo_epi32(lstep2[ 8], lstep2[14]);
- u[13] = _mm_unpackhi_epi32(lstep2[ 8], lstep2[14]);
- u[14] = _mm_unpacklo_epi32(lstep2[ 9], lstep2[15]);
- u[15] = _mm_unpackhi_epi32(lstep2[ 9], lstep2[15]);
-
- v[0] = k_madd_epi32(u[0], k32_p28_p04);
- v[1] = k_madd_epi32(u[1], k32_p28_p04);
- v[2] = k_madd_epi32(u[2], k32_p28_p04);
- v[3] = k_madd_epi32(u[3], k32_p28_p04);
- v[4] = k_madd_epi32(u[4], k32_p12_p20);
- v[5] = k_madd_epi32(u[5], k32_p12_p20);
- v[6] = k_madd_epi32(u[6], k32_p12_p20);
- v[7] = k_madd_epi32(u[7], k32_p12_p20);
- v[ 8] = k_madd_epi32(u[ 8], k32_m20_p12);
- v[ 9] = k_madd_epi32(u[ 9], k32_m20_p12);
- v[10] = k_madd_epi32(u[10], k32_m20_p12);
- v[11] = k_madd_epi32(u[11], k32_m20_p12);
- v[12] = k_madd_epi32(u[12], k32_m04_p28);
- v[13] = k_madd_epi32(u[13], k32_m04_p28);
- v[14] = k_madd_epi32(u[14], k32_m04_p28);
- v[15] = k_madd_epi32(u[15], k32_m04_p28);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_16(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[0] = k_packs_epi64(v[0], v[1]);
- u[1] = k_packs_epi64(v[2], v[3]);
- u[2] = k_packs_epi64(v[4], v[5]);
- u[3] = k_packs_epi64(v[6], v[7]);
- u[4] = k_packs_epi64(v[8], v[9]);
- u[5] = k_packs_epi64(v[10], v[11]);
- u[6] = k_packs_epi64(v[12], v[13]);
- u[7] = k_packs_epi64(v[14], v[15]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
-
- sign[0] = _mm_cmplt_epi32(u[0], kZero);
- sign[1] = _mm_cmplt_epi32(u[1], kZero);
- sign[2] = _mm_cmplt_epi32(u[2], kZero);
- sign[3] = _mm_cmplt_epi32(u[3], kZero);
- sign[4] = _mm_cmplt_epi32(u[4], kZero);
- sign[5] = _mm_cmplt_epi32(u[5], kZero);
- sign[6] = _mm_cmplt_epi32(u[6], kZero);
- sign[7] = _mm_cmplt_epi32(u[7], kZero);
-
- u[0] = _mm_sub_epi32(u[0], sign[0]);
- u[1] = _mm_sub_epi32(u[1], sign[1]);
- u[2] = _mm_sub_epi32(u[2], sign[2]);
- u[3] = _mm_sub_epi32(u[3], sign[3]);
- u[4] = _mm_sub_epi32(u[4], sign[4]);
- u[5] = _mm_sub_epi32(u[5], sign[5]);
- u[6] = _mm_sub_epi32(u[6], sign[6]);
- u[7] = _mm_sub_epi32(u[7], sign[7]);
-
- u[0] = _mm_add_epi32(u[0], K32One);
- u[1] = _mm_add_epi32(u[1], K32One);
- u[2] = _mm_add_epi32(u[2], K32One);
- u[3] = _mm_add_epi32(u[3], K32One);
- u[4] = _mm_add_epi32(u[4], K32One);
- u[5] = _mm_add_epi32(u[5], K32One);
- u[6] = _mm_add_epi32(u[6], K32One);
- u[7] = _mm_add_epi32(u[7], K32One);
-
- u[0] = _mm_srai_epi32(u[0], 2);
- u[1] = _mm_srai_epi32(u[1], 2);
- u[2] = _mm_srai_epi32(u[2], 2);
- u[3] = _mm_srai_epi32(u[3], 2);
- u[4] = _mm_srai_epi32(u[4], 2);
- u[5] = _mm_srai_epi32(u[5], 2);
- u[6] = _mm_srai_epi32(u[6], 2);
- u[7] = _mm_srai_epi32(u[7], 2);
-
- out[ 4] = _mm_packs_epi32(u[0], u[1]);
- out[20] = _mm_packs_epi32(u[2], u[3]);
- out[12] = _mm_packs_epi32(u[4], u[5]);
- out[28] = _mm_packs_epi32(u[6], u[7]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&out[4], &out[20],
- &out[12], &out[28]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- lstep3[16] = _mm_add_epi32(lstep2[18], lstep1[16]);
- lstep3[17] = _mm_add_epi32(lstep2[19], lstep1[17]);
- lstep3[18] = _mm_sub_epi32(lstep1[16], lstep2[18]);
- lstep3[19] = _mm_sub_epi32(lstep1[17], lstep2[19]);
- lstep3[20] = _mm_sub_epi32(lstep1[22], lstep2[20]);
- lstep3[21] = _mm_sub_epi32(lstep1[23], lstep2[21]);
- lstep3[22] = _mm_add_epi32(lstep2[20], lstep1[22]);
- lstep3[23] = _mm_add_epi32(lstep2[21], lstep1[23]);
- lstep3[24] = _mm_add_epi32(lstep2[26], lstep1[24]);
- lstep3[25] = _mm_add_epi32(lstep2[27], lstep1[25]);
- lstep3[26] = _mm_sub_epi32(lstep1[24], lstep2[26]);
- lstep3[27] = _mm_sub_epi32(lstep1[25], lstep2[27]);
- lstep3[28] = _mm_sub_epi32(lstep1[30], lstep2[28]);
- lstep3[29] = _mm_sub_epi32(lstep1[31], lstep2[29]);
- lstep3[30] = _mm_add_epi32(lstep2[28], lstep1[30]);
- lstep3[31] = _mm_add_epi32(lstep2[29], lstep1[31]);
- }
- {
- const __m128i k32_m04_p28 = pair_set_epi32(-cospi_4_64, cospi_28_64);
- const __m128i k32_m28_m04 = pair_set_epi32(-cospi_28_64, -cospi_4_64);
- const __m128i k32_m20_p12 = pair_set_epi32(-cospi_20_64, cospi_12_64);
- const __m128i k32_m12_m20 = pair_set_epi32(-cospi_12_64,
- -cospi_20_64);
- const __m128i k32_p12_p20 = pair_set_epi32(cospi_12_64, cospi_20_64);
- const __m128i k32_p28_p04 = pair_set_epi32(cospi_28_64, cospi_4_64);
-
- u[ 0] = _mm_unpacklo_epi32(lstep2[34], lstep2[60]);
- u[ 1] = _mm_unpackhi_epi32(lstep2[34], lstep2[60]);
- u[ 2] = _mm_unpacklo_epi32(lstep2[35], lstep2[61]);
- u[ 3] = _mm_unpackhi_epi32(lstep2[35], lstep2[61]);
- u[ 4] = _mm_unpacklo_epi32(lstep2[36], lstep2[58]);
- u[ 5] = _mm_unpackhi_epi32(lstep2[36], lstep2[58]);
- u[ 6] = _mm_unpacklo_epi32(lstep2[37], lstep2[59]);
- u[ 7] = _mm_unpackhi_epi32(lstep2[37], lstep2[59]);
- u[ 8] = _mm_unpacklo_epi32(lstep2[42], lstep2[52]);
- u[ 9] = _mm_unpackhi_epi32(lstep2[42], lstep2[52]);
- u[10] = _mm_unpacklo_epi32(lstep2[43], lstep2[53]);
- u[11] = _mm_unpackhi_epi32(lstep2[43], lstep2[53]);
- u[12] = _mm_unpacklo_epi32(lstep2[44], lstep2[50]);
- u[13] = _mm_unpackhi_epi32(lstep2[44], lstep2[50]);
- u[14] = _mm_unpacklo_epi32(lstep2[45], lstep2[51]);
- u[15] = _mm_unpackhi_epi32(lstep2[45], lstep2[51]);
-
- v[ 0] = k_madd_epi32(u[ 0], k32_m04_p28);
- v[ 1] = k_madd_epi32(u[ 1], k32_m04_p28);
- v[ 2] = k_madd_epi32(u[ 2], k32_m04_p28);
- v[ 3] = k_madd_epi32(u[ 3], k32_m04_p28);
- v[ 4] = k_madd_epi32(u[ 4], k32_m28_m04);
- v[ 5] = k_madd_epi32(u[ 5], k32_m28_m04);
- v[ 6] = k_madd_epi32(u[ 6], k32_m28_m04);
- v[ 7] = k_madd_epi32(u[ 7], k32_m28_m04);
- v[ 8] = k_madd_epi32(u[ 8], k32_m20_p12);
- v[ 9] = k_madd_epi32(u[ 9], k32_m20_p12);
- v[10] = k_madd_epi32(u[10], k32_m20_p12);
- v[11] = k_madd_epi32(u[11], k32_m20_p12);
- v[12] = k_madd_epi32(u[12], k32_m12_m20);
- v[13] = k_madd_epi32(u[13], k32_m12_m20);
- v[14] = k_madd_epi32(u[14], k32_m12_m20);
- v[15] = k_madd_epi32(u[15], k32_m12_m20);
- v[16] = k_madd_epi32(u[12], k32_m20_p12);
- v[17] = k_madd_epi32(u[13], k32_m20_p12);
- v[18] = k_madd_epi32(u[14], k32_m20_p12);
- v[19] = k_madd_epi32(u[15], k32_m20_p12);
- v[20] = k_madd_epi32(u[ 8], k32_p12_p20);
- v[21] = k_madd_epi32(u[ 9], k32_p12_p20);
- v[22] = k_madd_epi32(u[10], k32_p12_p20);
- v[23] = k_madd_epi32(u[11], k32_p12_p20);
- v[24] = k_madd_epi32(u[ 4], k32_m04_p28);
- v[25] = k_madd_epi32(u[ 5], k32_m04_p28);
- v[26] = k_madd_epi32(u[ 6], k32_m04_p28);
- v[27] = k_madd_epi32(u[ 7], k32_m04_p28);
- v[28] = k_madd_epi32(u[ 0], k32_p28_p04);
- v[29] = k_madd_epi32(u[ 1], k32_p28_p04);
- v[30] = k_madd_epi32(u[ 2], k32_p28_p04);
- v[31] = k_madd_epi32(u[ 3], k32_p28_p04);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_32(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &v[16], &v[17], &v[18], &v[19], &v[20], &v[21], &v[22], &v[23],
- &v[24], &v[25], &v[26], &v[27], &v[28], &v[29], &v[30], &v[31],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[ 0] = k_packs_epi64(v[ 0], v[ 1]);
- u[ 1] = k_packs_epi64(v[ 2], v[ 3]);
- u[ 2] = k_packs_epi64(v[ 4], v[ 5]);
- u[ 3] = k_packs_epi64(v[ 6], v[ 7]);
- u[ 4] = k_packs_epi64(v[ 8], v[ 9]);
- u[ 5] = k_packs_epi64(v[10], v[11]);
- u[ 6] = k_packs_epi64(v[12], v[13]);
- u[ 7] = k_packs_epi64(v[14], v[15]);
- u[ 8] = k_packs_epi64(v[16], v[17]);
- u[ 9] = k_packs_epi64(v[18], v[19]);
- u[10] = k_packs_epi64(v[20], v[21]);
- u[11] = k_packs_epi64(v[22], v[23]);
- u[12] = k_packs_epi64(v[24], v[25]);
- u[13] = k_packs_epi64(v[26], v[27]);
- u[14] = k_packs_epi64(v[28], v[29]);
- u[15] = k_packs_epi64(v[30], v[31]);
-
- v[ 0] = _mm_add_epi32(u[ 0], k__DCT_CONST_ROUNDING);
- v[ 1] = _mm_add_epi32(u[ 1], k__DCT_CONST_ROUNDING);
- v[ 2] = _mm_add_epi32(u[ 2], k__DCT_CONST_ROUNDING);
- v[ 3] = _mm_add_epi32(u[ 3], k__DCT_CONST_ROUNDING);
- v[ 4] = _mm_add_epi32(u[ 4], k__DCT_CONST_ROUNDING);
- v[ 5] = _mm_add_epi32(u[ 5], k__DCT_CONST_ROUNDING);
- v[ 6] = _mm_add_epi32(u[ 6], k__DCT_CONST_ROUNDING);
- v[ 7] = _mm_add_epi32(u[ 7], k__DCT_CONST_ROUNDING);
- v[ 8] = _mm_add_epi32(u[ 8], k__DCT_CONST_ROUNDING);
- v[ 9] = _mm_add_epi32(u[ 9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- lstep3[34] = _mm_srai_epi32(v[ 0], DCT_CONST_BITS);
- lstep3[35] = _mm_srai_epi32(v[ 1], DCT_CONST_BITS);
- lstep3[36] = _mm_srai_epi32(v[ 2], DCT_CONST_BITS);
- lstep3[37] = _mm_srai_epi32(v[ 3], DCT_CONST_BITS);
- lstep3[42] = _mm_srai_epi32(v[ 4], DCT_CONST_BITS);
- lstep3[43] = _mm_srai_epi32(v[ 5], DCT_CONST_BITS);
- lstep3[44] = _mm_srai_epi32(v[ 6], DCT_CONST_BITS);
- lstep3[45] = _mm_srai_epi32(v[ 7], DCT_CONST_BITS);
- lstep3[50] = _mm_srai_epi32(v[ 8], DCT_CONST_BITS);
- lstep3[51] = _mm_srai_epi32(v[ 9], DCT_CONST_BITS);
- lstep3[52] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- lstep3[53] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- lstep3[58] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- lstep3[59] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- lstep3[60] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- lstep3[61] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
- }
- // stage 7
- {
- const __m128i k32_p30_p02 = pair_set_epi32(cospi_30_64, cospi_2_64);
- const __m128i k32_p14_p18 = pair_set_epi32(cospi_14_64, cospi_18_64);
- const __m128i k32_p22_p10 = pair_set_epi32(cospi_22_64, cospi_10_64);
- const __m128i k32_p06_p26 = pair_set_epi32(cospi_6_64, cospi_26_64);
- const __m128i k32_m26_p06 = pair_set_epi32(-cospi_26_64, cospi_6_64);
- const __m128i k32_m10_p22 = pair_set_epi32(-cospi_10_64, cospi_22_64);
- const __m128i k32_m18_p14 = pair_set_epi32(-cospi_18_64, cospi_14_64);
- const __m128i k32_m02_p30 = pair_set_epi32(-cospi_2_64, cospi_30_64);
-
- u[ 0] = _mm_unpacklo_epi32(lstep3[16], lstep3[30]);
- u[ 1] = _mm_unpackhi_epi32(lstep3[16], lstep3[30]);
- u[ 2] = _mm_unpacklo_epi32(lstep3[17], lstep3[31]);
- u[ 3] = _mm_unpackhi_epi32(lstep3[17], lstep3[31]);
- u[ 4] = _mm_unpacklo_epi32(lstep3[18], lstep3[28]);
- u[ 5] = _mm_unpackhi_epi32(lstep3[18], lstep3[28]);
- u[ 6] = _mm_unpacklo_epi32(lstep3[19], lstep3[29]);
- u[ 7] = _mm_unpackhi_epi32(lstep3[19], lstep3[29]);
- u[ 8] = _mm_unpacklo_epi32(lstep3[20], lstep3[26]);
- u[ 9] = _mm_unpackhi_epi32(lstep3[20], lstep3[26]);
- u[10] = _mm_unpacklo_epi32(lstep3[21], lstep3[27]);
- u[11] = _mm_unpackhi_epi32(lstep3[21], lstep3[27]);
- u[12] = _mm_unpacklo_epi32(lstep3[22], lstep3[24]);
- u[13] = _mm_unpackhi_epi32(lstep3[22], lstep3[24]);
- u[14] = _mm_unpacklo_epi32(lstep3[23], lstep3[25]);
- u[15] = _mm_unpackhi_epi32(lstep3[23], lstep3[25]);
-
- v[ 0] = k_madd_epi32(u[ 0], k32_p30_p02);
- v[ 1] = k_madd_epi32(u[ 1], k32_p30_p02);
- v[ 2] = k_madd_epi32(u[ 2], k32_p30_p02);
- v[ 3] = k_madd_epi32(u[ 3], k32_p30_p02);
- v[ 4] = k_madd_epi32(u[ 4], k32_p14_p18);
- v[ 5] = k_madd_epi32(u[ 5], k32_p14_p18);
- v[ 6] = k_madd_epi32(u[ 6], k32_p14_p18);
- v[ 7] = k_madd_epi32(u[ 7], k32_p14_p18);
- v[ 8] = k_madd_epi32(u[ 8], k32_p22_p10);
- v[ 9] = k_madd_epi32(u[ 9], k32_p22_p10);
- v[10] = k_madd_epi32(u[10], k32_p22_p10);
- v[11] = k_madd_epi32(u[11], k32_p22_p10);
- v[12] = k_madd_epi32(u[12], k32_p06_p26);
- v[13] = k_madd_epi32(u[13], k32_p06_p26);
- v[14] = k_madd_epi32(u[14], k32_p06_p26);
- v[15] = k_madd_epi32(u[15], k32_p06_p26);
- v[16] = k_madd_epi32(u[12], k32_m26_p06);
- v[17] = k_madd_epi32(u[13], k32_m26_p06);
- v[18] = k_madd_epi32(u[14], k32_m26_p06);
- v[19] = k_madd_epi32(u[15], k32_m26_p06);
- v[20] = k_madd_epi32(u[ 8], k32_m10_p22);
- v[21] = k_madd_epi32(u[ 9], k32_m10_p22);
- v[22] = k_madd_epi32(u[10], k32_m10_p22);
- v[23] = k_madd_epi32(u[11], k32_m10_p22);
- v[24] = k_madd_epi32(u[ 4], k32_m18_p14);
- v[25] = k_madd_epi32(u[ 5], k32_m18_p14);
- v[26] = k_madd_epi32(u[ 6], k32_m18_p14);
- v[27] = k_madd_epi32(u[ 7], k32_m18_p14);
- v[28] = k_madd_epi32(u[ 0], k32_m02_p30);
- v[29] = k_madd_epi32(u[ 1], k32_m02_p30);
- v[30] = k_madd_epi32(u[ 2], k32_m02_p30);
- v[31] = k_madd_epi32(u[ 3], k32_m02_p30);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_32(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &v[16], &v[17], &v[18], &v[19], &v[20], &v[21], &v[22], &v[23],
- &v[24], &v[25], &v[26], &v[27], &v[28], &v[29], &v[30], &v[31],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[ 0] = k_packs_epi64(v[ 0], v[ 1]);
- u[ 1] = k_packs_epi64(v[ 2], v[ 3]);
- u[ 2] = k_packs_epi64(v[ 4], v[ 5]);
- u[ 3] = k_packs_epi64(v[ 6], v[ 7]);
- u[ 4] = k_packs_epi64(v[ 8], v[ 9]);
- u[ 5] = k_packs_epi64(v[10], v[11]);
- u[ 6] = k_packs_epi64(v[12], v[13]);
- u[ 7] = k_packs_epi64(v[14], v[15]);
- u[ 8] = k_packs_epi64(v[16], v[17]);
- u[ 9] = k_packs_epi64(v[18], v[19]);
- u[10] = k_packs_epi64(v[20], v[21]);
- u[11] = k_packs_epi64(v[22], v[23]);
- u[12] = k_packs_epi64(v[24], v[25]);
- u[13] = k_packs_epi64(v[26], v[27]);
- u[14] = k_packs_epi64(v[28], v[29]);
- u[15] = k_packs_epi64(v[30], v[31]);
-
- v[ 0] = _mm_add_epi32(u[ 0], k__DCT_CONST_ROUNDING);
- v[ 1] = _mm_add_epi32(u[ 1], k__DCT_CONST_ROUNDING);
- v[ 2] = _mm_add_epi32(u[ 2], k__DCT_CONST_ROUNDING);
- v[ 3] = _mm_add_epi32(u[ 3], k__DCT_CONST_ROUNDING);
- v[ 4] = _mm_add_epi32(u[ 4], k__DCT_CONST_ROUNDING);
- v[ 5] = _mm_add_epi32(u[ 5], k__DCT_CONST_ROUNDING);
- v[ 6] = _mm_add_epi32(u[ 6], k__DCT_CONST_ROUNDING);
- v[ 7] = _mm_add_epi32(u[ 7], k__DCT_CONST_ROUNDING);
- v[ 8] = _mm_add_epi32(u[ 8], k__DCT_CONST_ROUNDING);
- v[ 9] = _mm_add_epi32(u[ 9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- u[ 0] = _mm_srai_epi32(v[ 0], DCT_CONST_BITS);
- u[ 1] = _mm_srai_epi32(v[ 1], DCT_CONST_BITS);
- u[ 2] = _mm_srai_epi32(v[ 2], DCT_CONST_BITS);
- u[ 3] = _mm_srai_epi32(v[ 3], DCT_CONST_BITS);
- u[ 4] = _mm_srai_epi32(v[ 4], DCT_CONST_BITS);
- u[ 5] = _mm_srai_epi32(v[ 5], DCT_CONST_BITS);
- u[ 6] = _mm_srai_epi32(v[ 6], DCT_CONST_BITS);
- u[ 7] = _mm_srai_epi32(v[ 7], DCT_CONST_BITS);
- u[ 8] = _mm_srai_epi32(v[ 8], DCT_CONST_BITS);
- u[ 9] = _mm_srai_epi32(v[ 9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
-
- v[ 0] = _mm_cmplt_epi32(u[ 0], kZero);
- v[ 1] = _mm_cmplt_epi32(u[ 1], kZero);
- v[ 2] = _mm_cmplt_epi32(u[ 2], kZero);
- v[ 3] = _mm_cmplt_epi32(u[ 3], kZero);
- v[ 4] = _mm_cmplt_epi32(u[ 4], kZero);
- v[ 5] = _mm_cmplt_epi32(u[ 5], kZero);
- v[ 6] = _mm_cmplt_epi32(u[ 6], kZero);
- v[ 7] = _mm_cmplt_epi32(u[ 7], kZero);
- v[ 8] = _mm_cmplt_epi32(u[ 8], kZero);
- v[ 9] = _mm_cmplt_epi32(u[ 9], kZero);
- v[10] = _mm_cmplt_epi32(u[10], kZero);
- v[11] = _mm_cmplt_epi32(u[11], kZero);
- v[12] = _mm_cmplt_epi32(u[12], kZero);
- v[13] = _mm_cmplt_epi32(u[13], kZero);
- v[14] = _mm_cmplt_epi32(u[14], kZero);
- v[15] = _mm_cmplt_epi32(u[15], kZero);
-
- u[ 0] = _mm_sub_epi32(u[ 0], v[ 0]);
- u[ 1] = _mm_sub_epi32(u[ 1], v[ 1]);
- u[ 2] = _mm_sub_epi32(u[ 2], v[ 2]);
- u[ 3] = _mm_sub_epi32(u[ 3], v[ 3]);
- u[ 4] = _mm_sub_epi32(u[ 4], v[ 4]);
- u[ 5] = _mm_sub_epi32(u[ 5], v[ 5]);
- u[ 6] = _mm_sub_epi32(u[ 6], v[ 6]);
- u[ 7] = _mm_sub_epi32(u[ 7], v[ 7]);
- u[ 8] = _mm_sub_epi32(u[ 8], v[ 8]);
- u[ 9] = _mm_sub_epi32(u[ 9], v[ 9]);
- u[10] = _mm_sub_epi32(u[10], v[10]);
- u[11] = _mm_sub_epi32(u[11], v[11]);
- u[12] = _mm_sub_epi32(u[12], v[12]);
- u[13] = _mm_sub_epi32(u[13], v[13]);
- u[14] = _mm_sub_epi32(u[14], v[14]);
- u[15] = _mm_sub_epi32(u[15], v[15]);
-
- v[ 0] = _mm_add_epi32(u[ 0], K32One);
- v[ 1] = _mm_add_epi32(u[ 1], K32One);
- v[ 2] = _mm_add_epi32(u[ 2], K32One);
- v[ 3] = _mm_add_epi32(u[ 3], K32One);
- v[ 4] = _mm_add_epi32(u[ 4], K32One);
- v[ 5] = _mm_add_epi32(u[ 5], K32One);
- v[ 6] = _mm_add_epi32(u[ 6], K32One);
- v[ 7] = _mm_add_epi32(u[ 7], K32One);
- v[ 8] = _mm_add_epi32(u[ 8], K32One);
- v[ 9] = _mm_add_epi32(u[ 9], K32One);
- v[10] = _mm_add_epi32(u[10], K32One);
- v[11] = _mm_add_epi32(u[11], K32One);
- v[12] = _mm_add_epi32(u[12], K32One);
- v[13] = _mm_add_epi32(u[13], K32One);
- v[14] = _mm_add_epi32(u[14], K32One);
- v[15] = _mm_add_epi32(u[15], K32One);
-
- u[ 0] = _mm_srai_epi32(v[ 0], 2);
- u[ 1] = _mm_srai_epi32(v[ 1], 2);
- u[ 2] = _mm_srai_epi32(v[ 2], 2);
- u[ 3] = _mm_srai_epi32(v[ 3], 2);
- u[ 4] = _mm_srai_epi32(v[ 4], 2);
- u[ 5] = _mm_srai_epi32(v[ 5], 2);
- u[ 6] = _mm_srai_epi32(v[ 6], 2);
- u[ 7] = _mm_srai_epi32(v[ 7], 2);
- u[ 8] = _mm_srai_epi32(v[ 8], 2);
- u[ 9] = _mm_srai_epi32(v[ 9], 2);
- u[10] = _mm_srai_epi32(v[10], 2);
- u[11] = _mm_srai_epi32(v[11], 2);
- u[12] = _mm_srai_epi32(v[12], 2);
- u[13] = _mm_srai_epi32(v[13], 2);
- u[14] = _mm_srai_epi32(v[14], 2);
- u[15] = _mm_srai_epi32(v[15], 2);
-
- out[ 2] = _mm_packs_epi32(u[0], u[1]);
- out[18] = _mm_packs_epi32(u[2], u[3]);
- out[10] = _mm_packs_epi32(u[4], u[5]);
- out[26] = _mm_packs_epi32(u[6], u[7]);
- out[ 6] = _mm_packs_epi32(u[8], u[9]);
- out[22] = _mm_packs_epi32(u[10], u[11]);
- out[14] = _mm_packs_epi32(u[12], u[13]);
- out[30] = _mm_packs_epi32(u[14], u[15]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&out[2], &out[18], &out[10],
- &out[26], &out[6], &out[22],
- &out[14], &out[30]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- lstep1[32] = _mm_add_epi32(lstep3[34], lstep2[32]);
- lstep1[33] = _mm_add_epi32(lstep3[35], lstep2[33]);
- lstep1[34] = _mm_sub_epi32(lstep2[32], lstep3[34]);
- lstep1[35] = _mm_sub_epi32(lstep2[33], lstep3[35]);
- lstep1[36] = _mm_sub_epi32(lstep2[38], lstep3[36]);
- lstep1[37] = _mm_sub_epi32(lstep2[39], lstep3[37]);
- lstep1[38] = _mm_add_epi32(lstep3[36], lstep2[38]);
- lstep1[39] = _mm_add_epi32(lstep3[37], lstep2[39]);
- lstep1[40] = _mm_add_epi32(lstep3[42], lstep2[40]);
- lstep1[41] = _mm_add_epi32(lstep3[43], lstep2[41]);
- lstep1[42] = _mm_sub_epi32(lstep2[40], lstep3[42]);
- lstep1[43] = _mm_sub_epi32(lstep2[41], lstep3[43]);
- lstep1[44] = _mm_sub_epi32(lstep2[46], lstep3[44]);
- lstep1[45] = _mm_sub_epi32(lstep2[47], lstep3[45]);
- lstep1[46] = _mm_add_epi32(lstep3[44], lstep2[46]);
- lstep1[47] = _mm_add_epi32(lstep3[45], lstep2[47]);
- lstep1[48] = _mm_add_epi32(lstep3[50], lstep2[48]);
- lstep1[49] = _mm_add_epi32(lstep3[51], lstep2[49]);
- lstep1[50] = _mm_sub_epi32(lstep2[48], lstep3[50]);
- lstep1[51] = _mm_sub_epi32(lstep2[49], lstep3[51]);
- lstep1[52] = _mm_sub_epi32(lstep2[54], lstep3[52]);
- lstep1[53] = _mm_sub_epi32(lstep2[55], lstep3[53]);
- lstep1[54] = _mm_add_epi32(lstep3[52], lstep2[54]);
- lstep1[55] = _mm_add_epi32(lstep3[53], lstep2[55]);
- lstep1[56] = _mm_add_epi32(lstep3[58], lstep2[56]);
- lstep1[57] = _mm_add_epi32(lstep3[59], lstep2[57]);
- lstep1[58] = _mm_sub_epi32(lstep2[56], lstep3[58]);
- lstep1[59] = _mm_sub_epi32(lstep2[57], lstep3[59]);
- lstep1[60] = _mm_sub_epi32(lstep2[62], lstep3[60]);
- lstep1[61] = _mm_sub_epi32(lstep2[63], lstep3[61]);
- lstep1[62] = _mm_add_epi32(lstep3[60], lstep2[62]);
- lstep1[63] = _mm_add_epi32(lstep3[61], lstep2[63]);
- }
- // stage 8
- {
- const __m128i k32_p31_p01 = pair_set_epi32(cospi_31_64, cospi_1_64);
- const __m128i k32_p15_p17 = pair_set_epi32(cospi_15_64, cospi_17_64);
- const __m128i k32_p23_p09 = pair_set_epi32(cospi_23_64, cospi_9_64);
- const __m128i k32_p07_p25 = pair_set_epi32(cospi_7_64, cospi_25_64);
- const __m128i k32_m25_p07 = pair_set_epi32(-cospi_25_64, cospi_7_64);
- const __m128i k32_m09_p23 = pair_set_epi32(-cospi_9_64, cospi_23_64);
- const __m128i k32_m17_p15 = pair_set_epi32(-cospi_17_64, cospi_15_64);
- const __m128i k32_m01_p31 = pair_set_epi32(-cospi_1_64, cospi_31_64);
-
- u[ 0] = _mm_unpacklo_epi32(lstep1[32], lstep1[62]);
- u[ 1] = _mm_unpackhi_epi32(lstep1[32], lstep1[62]);
- u[ 2] = _mm_unpacklo_epi32(lstep1[33], lstep1[63]);
- u[ 3] = _mm_unpackhi_epi32(lstep1[33], lstep1[63]);
- u[ 4] = _mm_unpacklo_epi32(lstep1[34], lstep1[60]);
- u[ 5] = _mm_unpackhi_epi32(lstep1[34], lstep1[60]);
- u[ 6] = _mm_unpacklo_epi32(lstep1[35], lstep1[61]);
- u[ 7] = _mm_unpackhi_epi32(lstep1[35], lstep1[61]);
- u[ 8] = _mm_unpacklo_epi32(lstep1[36], lstep1[58]);
- u[ 9] = _mm_unpackhi_epi32(lstep1[36], lstep1[58]);
- u[10] = _mm_unpacklo_epi32(lstep1[37], lstep1[59]);
- u[11] = _mm_unpackhi_epi32(lstep1[37], lstep1[59]);
- u[12] = _mm_unpacklo_epi32(lstep1[38], lstep1[56]);
- u[13] = _mm_unpackhi_epi32(lstep1[38], lstep1[56]);
- u[14] = _mm_unpacklo_epi32(lstep1[39], lstep1[57]);
- u[15] = _mm_unpackhi_epi32(lstep1[39], lstep1[57]);
-
- v[ 0] = k_madd_epi32(u[ 0], k32_p31_p01);
- v[ 1] = k_madd_epi32(u[ 1], k32_p31_p01);
- v[ 2] = k_madd_epi32(u[ 2], k32_p31_p01);
- v[ 3] = k_madd_epi32(u[ 3], k32_p31_p01);
- v[ 4] = k_madd_epi32(u[ 4], k32_p15_p17);
- v[ 5] = k_madd_epi32(u[ 5], k32_p15_p17);
- v[ 6] = k_madd_epi32(u[ 6], k32_p15_p17);
- v[ 7] = k_madd_epi32(u[ 7], k32_p15_p17);
- v[ 8] = k_madd_epi32(u[ 8], k32_p23_p09);
- v[ 9] = k_madd_epi32(u[ 9], k32_p23_p09);
- v[10] = k_madd_epi32(u[10], k32_p23_p09);
- v[11] = k_madd_epi32(u[11], k32_p23_p09);
- v[12] = k_madd_epi32(u[12], k32_p07_p25);
- v[13] = k_madd_epi32(u[13], k32_p07_p25);
- v[14] = k_madd_epi32(u[14], k32_p07_p25);
- v[15] = k_madd_epi32(u[15], k32_p07_p25);
- v[16] = k_madd_epi32(u[12], k32_m25_p07);
- v[17] = k_madd_epi32(u[13], k32_m25_p07);
- v[18] = k_madd_epi32(u[14], k32_m25_p07);
- v[19] = k_madd_epi32(u[15], k32_m25_p07);
- v[20] = k_madd_epi32(u[ 8], k32_m09_p23);
- v[21] = k_madd_epi32(u[ 9], k32_m09_p23);
- v[22] = k_madd_epi32(u[10], k32_m09_p23);
- v[23] = k_madd_epi32(u[11], k32_m09_p23);
- v[24] = k_madd_epi32(u[ 4], k32_m17_p15);
- v[25] = k_madd_epi32(u[ 5], k32_m17_p15);
- v[26] = k_madd_epi32(u[ 6], k32_m17_p15);
- v[27] = k_madd_epi32(u[ 7], k32_m17_p15);
- v[28] = k_madd_epi32(u[ 0], k32_m01_p31);
- v[29] = k_madd_epi32(u[ 1], k32_m01_p31);
- v[30] = k_madd_epi32(u[ 2], k32_m01_p31);
- v[31] = k_madd_epi32(u[ 3], k32_m01_p31);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_32(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &v[16], &v[17], &v[18], &v[19], &v[20], &v[21], &v[22], &v[23],
- &v[24], &v[25], &v[26], &v[27], &v[28], &v[29], &v[30], &v[31],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[ 0] = k_packs_epi64(v[ 0], v[ 1]);
- u[ 1] = k_packs_epi64(v[ 2], v[ 3]);
- u[ 2] = k_packs_epi64(v[ 4], v[ 5]);
- u[ 3] = k_packs_epi64(v[ 6], v[ 7]);
- u[ 4] = k_packs_epi64(v[ 8], v[ 9]);
- u[ 5] = k_packs_epi64(v[10], v[11]);
- u[ 6] = k_packs_epi64(v[12], v[13]);
- u[ 7] = k_packs_epi64(v[14], v[15]);
- u[ 8] = k_packs_epi64(v[16], v[17]);
- u[ 9] = k_packs_epi64(v[18], v[19]);
- u[10] = k_packs_epi64(v[20], v[21]);
- u[11] = k_packs_epi64(v[22], v[23]);
- u[12] = k_packs_epi64(v[24], v[25]);
- u[13] = k_packs_epi64(v[26], v[27]);
- u[14] = k_packs_epi64(v[28], v[29]);
- u[15] = k_packs_epi64(v[30], v[31]);
-
- v[ 0] = _mm_add_epi32(u[ 0], k__DCT_CONST_ROUNDING);
- v[ 1] = _mm_add_epi32(u[ 1], k__DCT_CONST_ROUNDING);
- v[ 2] = _mm_add_epi32(u[ 2], k__DCT_CONST_ROUNDING);
- v[ 3] = _mm_add_epi32(u[ 3], k__DCT_CONST_ROUNDING);
- v[ 4] = _mm_add_epi32(u[ 4], k__DCT_CONST_ROUNDING);
- v[ 5] = _mm_add_epi32(u[ 5], k__DCT_CONST_ROUNDING);
- v[ 6] = _mm_add_epi32(u[ 6], k__DCT_CONST_ROUNDING);
- v[ 7] = _mm_add_epi32(u[ 7], k__DCT_CONST_ROUNDING);
- v[ 8] = _mm_add_epi32(u[ 8], k__DCT_CONST_ROUNDING);
- v[ 9] = _mm_add_epi32(u[ 9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- u[ 0] = _mm_srai_epi32(v[ 0], DCT_CONST_BITS);
- u[ 1] = _mm_srai_epi32(v[ 1], DCT_CONST_BITS);
- u[ 2] = _mm_srai_epi32(v[ 2], DCT_CONST_BITS);
- u[ 3] = _mm_srai_epi32(v[ 3], DCT_CONST_BITS);
- u[ 4] = _mm_srai_epi32(v[ 4], DCT_CONST_BITS);
- u[ 5] = _mm_srai_epi32(v[ 5], DCT_CONST_BITS);
- u[ 6] = _mm_srai_epi32(v[ 6], DCT_CONST_BITS);
- u[ 7] = _mm_srai_epi32(v[ 7], DCT_CONST_BITS);
- u[ 8] = _mm_srai_epi32(v[ 8], DCT_CONST_BITS);
- u[ 9] = _mm_srai_epi32(v[ 9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
-
- v[ 0] = _mm_cmplt_epi32(u[ 0], kZero);
- v[ 1] = _mm_cmplt_epi32(u[ 1], kZero);
- v[ 2] = _mm_cmplt_epi32(u[ 2], kZero);
- v[ 3] = _mm_cmplt_epi32(u[ 3], kZero);
- v[ 4] = _mm_cmplt_epi32(u[ 4], kZero);
- v[ 5] = _mm_cmplt_epi32(u[ 5], kZero);
- v[ 6] = _mm_cmplt_epi32(u[ 6], kZero);
- v[ 7] = _mm_cmplt_epi32(u[ 7], kZero);
- v[ 8] = _mm_cmplt_epi32(u[ 8], kZero);
- v[ 9] = _mm_cmplt_epi32(u[ 9], kZero);
- v[10] = _mm_cmplt_epi32(u[10], kZero);
- v[11] = _mm_cmplt_epi32(u[11], kZero);
- v[12] = _mm_cmplt_epi32(u[12], kZero);
- v[13] = _mm_cmplt_epi32(u[13], kZero);
- v[14] = _mm_cmplt_epi32(u[14], kZero);
- v[15] = _mm_cmplt_epi32(u[15], kZero);
-
- u[ 0] = _mm_sub_epi32(u[ 0], v[ 0]);
- u[ 1] = _mm_sub_epi32(u[ 1], v[ 1]);
- u[ 2] = _mm_sub_epi32(u[ 2], v[ 2]);
- u[ 3] = _mm_sub_epi32(u[ 3], v[ 3]);
- u[ 4] = _mm_sub_epi32(u[ 4], v[ 4]);
- u[ 5] = _mm_sub_epi32(u[ 5], v[ 5]);
- u[ 6] = _mm_sub_epi32(u[ 6], v[ 6]);
- u[ 7] = _mm_sub_epi32(u[ 7], v[ 7]);
- u[ 8] = _mm_sub_epi32(u[ 8], v[ 8]);
- u[ 9] = _mm_sub_epi32(u[ 9], v[ 9]);
- u[10] = _mm_sub_epi32(u[10], v[10]);
- u[11] = _mm_sub_epi32(u[11], v[11]);
- u[12] = _mm_sub_epi32(u[12], v[12]);
- u[13] = _mm_sub_epi32(u[13], v[13]);
- u[14] = _mm_sub_epi32(u[14], v[14]);
- u[15] = _mm_sub_epi32(u[15], v[15]);
-
- v[0] = _mm_add_epi32(u[0], K32One);
- v[1] = _mm_add_epi32(u[1], K32One);
- v[2] = _mm_add_epi32(u[2], K32One);
- v[3] = _mm_add_epi32(u[3], K32One);
- v[4] = _mm_add_epi32(u[4], K32One);
- v[5] = _mm_add_epi32(u[5], K32One);
- v[6] = _mm_add_epi32(u[6], K32One);
- v[7] = _mm_add_epi32(u[7], K32One);
- v[8] = _mm_add_epi32(u[8], K32One);
- v[9] = _mm_add_epi32(u[9], K32One);
- v[10] = _mm_add_epi32(u[10], K32One);
- v[11] = _mm_add_epi32(u[11], K32One);
- v[12] = _mm_add_epi32(u[12], K32One);
- v[13] = _mm_add_epi32(u[13], K32One);
- v[14] = _mm_add_epi32(u[14], K32One);
- v[15] = _mm_add_epi32(u[15], K32One);
-
- u[0] = _mm_srai_epi32(v[0], 2);
- u[1] = _mm_srai_epi32(v[1], 2);
- u[2] = _mm_srai_epi32(v[2], 2);
- u[3] = _mm_srai_epi32(v[3], 2);
- u[4] = _mm_srai_epi32(v[4], 2);
- u[5] = _mm_srai_epi32(v[5], 2);
- u[6] = _mm_srai_epi32(v[6], 2);
- u[7] = _mm_srai_epi32(v[7], 2);
- u[8] = _mm_srai_epi32(v[8], 2);
- u[9] = _mm_srai_epi32(v[9], 2);
- u[10] = _mm_srai_epi32(v[10], 2);
- u[11] = _mm_srai_epi32(v[11], 2);
- u[12] = _mm_srai_epi32(v[12], 2);
- u[13] = _mm_srai_epi32(v[13], 2);
- u[14] = _mm_srai_epi32(v[14], 2);
- u[15] = _mm_srai_epi32(v[15], 2);
-
- out[ 1] = _mm_packs_epi32(u[0], u[1]);
- out[17] = _mm_packs_epi32(u[2], u[3]);
- out[ 9] = _mm_packs_epi32(u[4], u[5]);
- out[25] = _mm_packs_epi32(u[6], u[7]);
- out[ 7] = _mm_packs_epi32(u[8], u[9]);
- out[23] = _mm_packs_epi32(u[10], u[11]);
- out[15] = _mm_packs_epi32(u[12], u[13]);
- out[31] = _mm_packs_epi32(u[14], u[15]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&out[1], &out[17], &out[9],
- &out[25], &out[7], &out[23],
- &out[15], &out[31]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i k32_p27_p05 = pair_set_epi32(cospi_27_64, cospi_5_64);
- const __m128i k32_p11_p21 = pair_set_epi32(cospi_11_64, cospi_21_64);
- const __m128i k32_p19_p13 = pair_set_epi32(cospi_19_64, cospi_13_64);
- const __m128i k32_p03_p29 = pair_set_epi32(cospi_3_64, cospi_29_64);
- const __m128i k32_m29_p03 = pair_set_epi32(-cospi_29_64, cospi_3_64);
- const __m128i k32_m13_p19 = pair_set_epi32(-cospi_13_64, cospi_19_64);
- const __m128i k32_m21_p11 = pair_set_epi32(-cospi_21_64, cospi_11_64);
- const __m128i k32_m05_p27 = pair_set_epi32(-cospi_5_64, cospi_27_64);
-
- u[ 0] = _mm_unpacklo_epi32(lstep1[40], lstep1[54]);
- u[ 1] = _mm_unpackhi_epi32(lstep1[40], lstep1[54]);
- u[ 2] = _mm_unpacklo_epi32(lstep1[41], lstep1[55]);
- u[ 3] = _mm_unpackhi_epi32(lstep1[41], lstep1[55]);
- u[ 4] = _mm_unpacklo_epi32(lstep1[42], lstep1[52]);
- u[ 5] = _mm_unpackhi_epi32(lstep1[42], lstep1[52]);
- u[ 6] = _mm_unpacklo_epi32(lstep1[43], lstep1[53]);
- u[ 7] = _mm_unpackhi_epi32(lstep1[43], lstep1[53]);
- u[ 8] = _mm_unpacklo_epi32(lstep1[44], lstep1[50]);
- u[ 9] = _mm_unpackhi_epi32(lstep1[44], lstep1[50]);
- u[10] = _mm_unpacklo_epi32(lstep1[45], lstep1[51]);
- u[11] = _mm_unpackhi_epi32(lstep1[45], lstep1[51]);
- u[12] = _mm_unpacklo_epi32(lstep1[46], lstep1[48]);
- u[13] = _mm_unpackhi_epi32(lstep1[46], lstep1[48]);
- u[14] = _mm_unpacklo_epi32(lstep1[47], lstep1[49]);
- u[15] = _mm_unpackhi_epi32(lstep1[47], lstep1[49]);
-
- v[ 0] = k_madd_epi32(u[ 0], k32_p27_p05);
- v[ 1] = k_madd_epi32(u[ 1], k32_p27_p05);
- v[ 2] = k_madd_epi32(u[ 2], k32_p27_p05);
- v[ 3] = k_madd_epi32(u[ 3], k32_p27_p05);
- v[ 4] = k_madd_epi32(u[ 4], k32_p11_p21);
- v[ 5] = k_madd_epi32(u[ 5], k32_p11_p21);
- v[ 6] = k_madd_epi32(u[ 6], k32_p11_p21);
- v[ 7] = k_madd_epi32(u[ 7], k32_p11_p21);
- v[ 8] = k_madd_epi32(u[ 8], k32_p19_p13);
- v[ 9] = k_madd_epi32(u[ 9], k32_p19_p13);
- v[10] = k_madd_epi32(u[10], k32_p19_p13);
- v[11] = k_madd_epi32(u[11], k32_p19_p13);
- v[12] = k_madd_epi32(u[12], k32_p03_p29);
- v[13] = k_madd_epi32(u[13], k32_p03_p29);
- v[14] = k_madd_epi32(u[14], k32_p03_p29);
- v[15] = k_madd_epi32(u[15], k32_p03_p29);
- v[16] = k_madd_epi32(u[12], k32_m29_p03);
- v[17] = k_madd_epi32(u[13], k32_m29_p03);
- v[18] = k_madd_epi32(u[14], k32_m29_p03);
- v[19] = k_madd_epi32(u[15], k32_m29_p03);
- v[20] = k_madd_epi32(u[ 8], k32_m13_p19);
- v[21] = k_madd_epi32(u[ 9], k32_m13_p19);
- v[22] = k_madd_epi32(u[10], k32_m13_p19);
- v[23] = k_madd_epi32(u[11], k32_m13_p19);
- v[24] = k_madd_epi32(u[ 4], k32_m21_p11);
- v[25] = k_madd_epi32(u[ 5], k32_m21_p11);
- v[26] = k_madd_epi32(u[ 6], k32_m21_p11);
- v[27] = k_madd_epi32(u[ 7], k32_m21_p11);
- v[28] = k_madd_epi32(u[ 0], k32_m05_p27);
- v[29] = k_madd_epi32(u[ 1], k32_m05_p27);
- v[30] = k_madd_epi32(u[ 2], k32_m05_p27);
- v[31] = k_madd_epi32(u[ 3], k32_m05_p27);
-
-#if DCT_HIGH_BIT_DEPTH
- overflow = k_check_epi32_overflow_32(
- &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &v[6], &v[7],
- &v[8], &v[9], &v[10], &v[11], &v[12], &v[13], &v[14], &v[15],
- &v[16], &v[17], &v[18], &v[19], &v[20], &v[21], &v[22], &v[23],
- &v[24], &v[25], &v[26], &v[27], &v[28], &v[29], &v[30], &v[31],
- &kZero);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- u[ 0] = k_packs_epi64(v[ 0], v[ 1]);
- u[ 1] = k_packs_epi64(v[ 2], v[ 3]);
- u[ 2] = k_packs_epi64(v[ 4], v[ 5]);
- u[ 3] = k_packs_epi64(v[ 6], v[ 7]);
- u[ 4] = k_packs_epi64(v[ 8], v[ 9]);
- u[ 5] = k_packs_epi64(v[10], v[11]);
- u[ 6] = k_packs_epi64(v[12], v[13]);
- u[ 7] = k_packs_epi64(v[14], v[15]);
- u[ 8] = k_packs_epi64(v[16], v[17]);
- u[ 9] = k_packs_epi64(v[18], v[19]);
- u[10] = k_packs_epi64(v[20], v[21]);
- u[11] = k_packs_epi64(v[22], v[23]);
- u[12] = k_packs_epi64(v[24], v[25]);
- u[13] = k_packs_epi64(v[26], v[27]);
- u[14] = k_packs_epi64(v[28], v[29]);
- u[15] = k_packs_epi64(v[30], v[31]);
-
- v[ 0] = _mm_add_epi32(u[ 0], k__DCT_CONST_ROUNDING);
- v[ 1] = _mm_add_epi32(u[ 1], k__DCT_CONST_ROUNDING);
- v[ 2] = _mm_add_epi32(u[ 2], k__DCT_CONST_ROUNDING);
- v[ 3] = _mm_add_epi32(u[ 3], k__DCT_CONST_ROUNDING);
- v[ 4] = _mm_add_epi32(u[ 4], k__DCT_CONST_ROUNDING);
- v[ 5] = _mm_add_epi32(u[ 5], k__DCT_CONST_ROUNDING);
- v[ 6] = _mm_add_epi32(u[ 6], k__DCT_CONST_ROUNDING);
- v[ 7] = _mm_add_epi32(u[ 7], k__DCT_CONST_ROUNDING);
- v[ 8] = _mm_add_epi32(u[ 8], k__DCT_CONST_ROUNDING);
- v[ 9] = _mm_add_epi32(u[ 9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- u[ 0] = _mm_srai_epi32(v[ 0], DCT_CONST_BITS);
- u[ 1] = _mm_srai_epi32(v[ 1], DCT_CONST_BITS);
- u[ 2] = _mm_srai_epi32(v[ 2], DCT_CONST_BITS);
- u[ 3] = _mm_srai_epi32(v[ 3], DCT_CONST_BITS);
- u[ 4] = _mm_srai_epi32(v[ 4], DCT_CONST_BITS);
- u[ 5] = _mm_srai_epi32(v[ 5], DCT_CONST_BITS);
- u[ 6] = _mm_srai_epi32(v[ 6], DCT_CONST_BITS);
- u[ 7] = _mm_srai_epi32(v[ 7], DCT_CONST_BITS);
- u[ 8] = _mm_srai_epi32(v[ 8], DCT_CONST_BITS);
- u[ 9] = _mm_srai_epi32(v[ 9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
-
- v[ 0] = _mm_cmplt_epi32(u[ 0], kZero);
- v[ 1] = _mm_cmplt_epi32(u[ 1], kZero);
- v[ 2] = _mm_cmplt_epi32(u[ 2], kZero);
- v[ 3] = _mm_cmplt_epi32(u[ 3], kZero);
- v[ 4] = _mm_cmplt_epi32(u[ 4], kZero);
- v[ 5] = _mm_cmplt_epi32(u[ 5], kZero);
- v[ 6] = _mm_cmplt_epi32(u[ 6], kZero);
- v[ 7] = _mm_cmplt_epi32(u[ 7], kZero);
- v[ 8] = _mm_cmplt_epi32(u[ 8], kZero);
- v[ 9] = _mm_cmplt_epi32(u[ 9], kZero);
- v[10] = _mm_cmplt_epi32(u[10], kZero);
- v[11] = _mm_cmplt_epi32(u[11], kZero);
- v[12] = _mm_cmplt_epi32(u[12], kZero);
- v[13] = _mm_cmplt_epi32(u[13], kZero);
- v[14] = _mm_cmplt_epi32(u[14], kZero);
- v[15] = _mm_cmplt_epi32(u[15], kZero);
-
- u[ 0] = _mm_sub_epi32(u[ 0], v[ 0]);
- u[ 1] = _mm_sub_epi32(u[ 1], v[ 1]);
- u[ 2] = _mm_sub_epi32(u[ 2], v[ 2]);
- u[ 3] = _mm_sub_epi32(u[ 3], v[ 3]);
- u[ 4] = _mm_sub_epi32(u[ 4], v[ 4]);
- u[ 5] = _mm_sub_epi32(u[ 5], v[ 5]);
- u[ 6] = _mm_sub_epi32(u[ 6], v[ 6]);
- u[ 7] = _mm_sub_epi32(u[ 7], v[ 7]);
- u[ 8] = _mm_sub_epi32(u[ 8], v[ 8]);
- u[ 9] = _mm_sub_epi32(u[ 9], v[ 9]);
- u[10] = _mm_sub_epi32(u[10], v[10]);
- u[11] = _mm_sub_epi32(u[11], v[11]);
- u[12] = _mm_sub_epi32(u[12], v[12]);
- u[13] = _mm_sub_epi32(u[13], v[13]);
- u[14] = _mm_sub_epi32(u[14], v[14]);
- u[15] = _mm_sub_epi32(u[15], v[15]);
-
- v[0] = _mm_add_epi32(u[0], K32One);
- v[1] = _mm_add_epi32(u[1], K32One);
- v[2] = _mm_add_epi32(u[2], K32One);
- v[3] = _mm_add_epi32(u[3], K32One);
- v[4] = _mm_add_epi32(u[4], K32One);
- v[5] = _mm_add_epi32(u[5], K32One);
- v[6] = _mm_add_epi32(u[6], K32One);
- v[7] = _mm_add_epi32(u[7], K32One);
- v[8] = _mm_add_epi32(u[8], K32One);
- v[9] = _mm_add_epi32(u[9], K32One);
- v[10] = _mm_add_epi32(u[10], K32One);
- v[11] = _mm_add_epi32(u[11], K32One);
- v[12] = _mm_add_epi32(u[12], K32One);
- v[13] = _mm_add_epi32(u[13], K32One);
- v[14] = _mm_add_epi32(u[14], K32One);
- v[15] = _mm_add_epi32(u[15], K32One);
-
- u[0] = _mm_srai_epi32(v[0], 2);
- u[1] = _mm_srai_epi32(v[1], 2);
- u[2] = _mm_srai_epi32(v[2], 2);
- u[3] = _mm_srai_epi32(v[3], 2);
- u[4] = _mm_srai_epi32(v[4], 2);
- u[5] = _mm_srai_epi32(v[5], 2);
- u[6] = _mm_srai_epi32(v[6], 2);
- u[7] = _mm_srai_epi32(v[7], 2);
- u[8] = _mm_srai_epi32(v[8], 2);
- u[9] = _mm_srai_epi32(v[9], 2);
- u[10] = _mm_srai_epi32(v[10], 2);
- u[11] = _mm_srai_epi32(v[11], 2);
- u[12] = _mm_srai_epi32(v[12], 2);
- u[13] = _mm_srai_epi32(v[13], 2);
- u[14] = _mm_srai_epi32(v[14], 2);
- u[15] = _mm_srai_epi32(v[15], 2);
-
- out[ 5] = _mm_packs_epi32(u[0], u[1]);
- out[21] = _mm_packs_epi32(u[2], u[3]);
- out[13] = _mm_packs_epi32(u[4], u[5]);
- out[29] = _mm_packs_epi32(u[6], u[7]);
- out[ 3] = _mm_packs_epi32(u[8], u[9]);
- out[19] = _mm_packs_epi32(u[10], u[11]);
- out[11] = _mm_packs_epi32(u[12], u[13]);
- out[27] = _mm_packs_epi32(u[14], u[15]);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&out[5], &out[21], &out[13],
- &out[29], &out[3], &out[19],
- &out[11], &out[27]);
- if (overflow) {
- HIGH_FDCT32x32_2D_ROWS_C(intermediate, output_org);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
-#endif // FDCT32x32_HIGH_PRECISION
- // Transpose the results, do it as four 8x8 transposes.
- {
- int transpose_block;
- int16_t *output0 = &intermediate[column_start * 32];
- tran_low_t *output1 = &output_org[column_start * 32];
- for (transpose_block = 0; transpose_block < 4; ++transpose_block) {
- __m128i *this_out = &out[8 * transpose_block];
- // 00 01 02 03 04 05 06 07
- // 10 11 12 13 14 15 16 17
- // 20 21 22 23 24 25 26 27
- // 30 31 32 33 34 35 36 37
- // 40 41 42 43 44 45 46 47
- // 50 51 52 53 54 55 56 57
- // 60 61 62 63 64 65 66 67
- // 70 71 72 73 74 75 76 77
- const __m128i tr0_0 = _mm_unpacklo_epi16(this_out[0], this_out[1]);
- const __m128i tr0_1 = _mm_unpacklo_epi16(this_out[2], this_out[3]);
- const __m128i tr0_2 = _mm_unpackhi_epi16(this_out[0], this_out[1]);
- const __m128i tr0_3 = _mm_unpackhi_epi16(this_out[2], this_out[3]);
- const __m128i tr0_4 = _mm_unpacklo_epi16(this_out[4], this_out[5]);
- const __m128i tr0_5 = _mm_unpacklo_epi16(this_out[6], this_out[7]);
- const __m128i tr0_6 = _mm_unpackhi_epi16(this_out[4], this_out[5]);
- const __m128i tr0_7 = _mm_unpackhi_epi16(this_out[6], this_out[7]);
- // 00 10 01 11 02 12 03 13
- // 20 30 21 31 22 32 23 33
- // 04 14 05 15 06 16 07 17
- // 24 34 25 35 26 36 27 37
- // 40 50 41 51 42 52 43 53
- // 60 70 61 71 62 72 63 73
- // 54 54 55 55 56 56 57 57
- // 64 74 65 75 66 76 67 77
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_2, tr0_3);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_2, tr0_3);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5);
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7);
- // 00 10 20 30 01 11 21 31
- // 40 50 60 70 41 51 61 71
- // 02 12 22 32 03 13 23 33
- // 42 52 62 72 43 53 63 73
- // 04 14 24 34 05 15 21 36
- // 44 54 64 74 45 55 61 76
- // 06 16 26 36 07 17 27 37
- // 46 56 66 76 47 57 67 77
- __m128i tr2_0 = _mm_unpacklo_epi64(tr1_0, tr1_4);
- __m128i tr2_1 = _mm_unpackhi_epi64(tr1_0, tr1_4);
- __m128i tr2_2 = _mm_unpacklo_epi64(tr1_2, tr1_6);
- __m128i tr2_3 = _mm_unpackhi_epi64(tr1_2, tr1_6);
- __m128i tr2_4 = _mm_unpacklo_epi64(tr1_1, tr1_5);
- __m128i tr2_5 = _mm_unpackhi_epi64(tr1_1, tr1_5);
- __m128i tr2_6 = _mm_unpacklo_epi64(tr1_3, tr1_7);
- __m128i tr2_7 = _mm_unpackhi_epi64(tr1_3, tr1_7);
- // 00 10 20 30 40 50 60 70
- // 01 11 21 31 41 51 61 71
- // 02 12 22 32 42 52 62 72
- // 03 13 23 33 43 53 63 73
- // 04 14 24 34 44 54 64 74
- // 05 15 25 35 45 55 65 75
- // 06 16 26 36 46 56 66 76
- // 07 17 27 37 47 57 67 77
- if (0 == pass) {
- // output[j] = (output[j] + 1 + (output[j] > 0)) >> 2;
- // TODO(cd): see quality impact of only doing
- // output[j] = (output[j] + 1) >> 2;
- // which would remove the code between here ...
- __m128i tr2_0_0 = _mm_cmpgt_epi16(tr2_0, kZero);
- __m128i tr2_1_0 = _mm_cmpgt_epi16(tr2_1, kZero);
- __m128i tr2_2_0 = _mm_cmpgt_epi16(tr2_2, kZero);
- __m128i tr2_3_0 = _mm_cmpgt_epi16(tr2_3, kZero);
- __m128i tr2_4_0 = _mm_cmpgt_epi16(tr2_4, kZero);
- __m128i tr2_5_0 = _mm_cmpgt_epi16(tr2_5, kZero);
- __m128i tr2_6_0 = _mm_cmpgt_epi16(tr2_6, kZero);
- __m128i tr2_7_0 = _mm_cmpgt_epi16(tr2_7, kZero);
- tr2_0 = _mm_sub_epi16(tr2_0, tr2_0_0);
- tr2_1 = _mm_sub_epi16(tr2_1, tr2_1_0);
- tr2_2 = _mm_sub_epi16(tr2_2, tr2_2_0);
- tr2_3 = _mm_sub_epi16(tr2_3, tr2_3_0);
- tr2_4 = _mm_sub_epi16(tr2_4, tr2_4_0);
- tr2_5 = _mm_sub_epi16(tr2_5, tr2_5_0);
- tr2_6 = _mm_sub_epi16(tr2_6, tr2_6_0);
- tr2_7 = _mm_sub_epi16(tr2_7, tr2_7_0);
- // ... and here.
- // PS: also change code in vp9/encoder/vp9_dct.c
- tr2_0 = _mm_add_epi16(tr2_0, kOne);
- tr2_1 = _mm_add_epi16(tr2_1, kOne);
- tr2_2 = _mm_add_epi16(tr2_2, kOne);
- tr2_3 = _mm_add_epi16(tr2_3, kOne);
- tr2_4 = _mm_add_epi16(tr2_4, kOne);
- tr2_5 = _mm_add_epi16(tr2_5, kOne);
- tr2_6 = _mm_add_epi16(tr2_6, kOne);
- tr2_7 = _mm_add_epi16(tr2_7, kOne);
- tr2_0 = _mm_srai_epi16(tr2_0, 2);
- tr2_1 = _mm_srai_epi16(tr2_1, 2);
- tr2_2 = _mm_srai_epi16(tr2_2, 2);
- tr2_3 = _mm_srai_epi16(tr2_3, 2);
- tr2_4 = _mm_srai_epi16(tr2_4, 2);
- tr2_5 = _mm_srai_epi16(tr2_5, 2);
- tr2_6 = _mm_srai_epi16(tr2_6, 2);
- tr2_7 = _mm_srai_epi16(tr2_7, 2);
- }
- // Note: even though all these stores are aligned, using the aligned
- // intrinsic make the code slightly slower.
- if (pass == 0) {
- _mm_storeu_si128((__m128i *)(output0 + 0 * 32), tr2_0);
- _mm_storeu_si128((__m128i *)(output0 + 1 * 32), tr2_1);
- _mm_storeu_si128((__m128i *)(output0 + 2 * 32), tr2_2);
- _mm_storeu_si128((__m128i *)(output0 + 3 * 32), tr2_3);
- _mm_storeu_si128((__m128i *)(output0 + 4 * 32), tr2_4);
- _mm_storeu_si128((__m128i *)(output0 + 5 * 32), tr2_5);
- _mm_storeu_si128((__m128i *)(output0 + 6 * 32), tr2_6);
- _mm_storeu_si128((__m128i *)(output0 + 7 * 32), tr2_7);
- // Process next 8x8
- output0 += 8;
- } else {
- storeu_output(&tr2_0, (output1 + 0 * 32));
- storeu_output(&tr2_1, (output1 + 1 * 32));
- storeu_output(&tr2_2, (output1 + 2 * 32));
- storeu_output(&tr2_3, (output1 + 3 * 32));
- storeu_output(&tr2_4, (output1 + 4 * 32));
- storeu_output(&tr2_5, (output1 + 5 * 32));
- storeu_output(&tr2_6, (output1 + 6 * 32));
- storeu_output(&tr2_7, (output1 + 7 * 32));
- // Process next 8x8
- output1 += 8;
- }
- }
- }
- }
- }
-} // NOLINT
-
-#undef ADD_EPI16
-#undef SUB_EPI16
-#undef HIGH_FDCT32x32_2D_C
-#undef HIGH_FDCT32x32_2D_ROWS_C
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h> // SSE2
-
-#include "./vpx_dsp_rtcd.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_dsp/x86/fwd_txfm_sse2.h"
-#include "vpx_dsp/x86/txfm_common_sse2.h"
-#include "vpx_ports/mem.h"
-
-// TODO(jingning) The high bit-depth functions need rework for performance.
-// After we properly fix the high bit-depth function implementations, this
-// file's dependency should be substantially simplified.
-#if DCT_HIGH_BIT_DEPTH
-#define ADD_EPI16 _mm_adds_epi16
-#define SUB_EPI16 _mm_subs_epi16
-
-#else
-#define ADD_EPI16 _mm_add_epi16
-#define SUB_EPI16 _mm_sub_epi16
-#endif
-
-void FDCT4x4_2D(const int16_t *input, tran_low_t *output, int stride) {
- // This 2D transform implements 4 vertical 1D transforms followed
- // by 4 horizontal 1D transforms. The multiplies and adds are as given
- // by Chen, Smith and Fralick ('77). The commands for moving the data
- // around have been minimized by hand.
- // For the purposes of the comments, the 16 inputs are referred to at i0
- // through iF (in raster order), intermediate variables are a0, b0, c0
- // through f, and correspond to the in-place computations mapped to input
- // locations. The outputs, o0 through oF are labeled according to the
- // output locations.
-
- // Constants
- // These are the coefficients used for the multiplies.
- // In the comments, pN means cos(N pi /64) and mN is -cos(N pi /64),
- // where cospi_N_64 = cos(N pi /64)
- const __m128i k__cospi_A = octa_set_epi16(cospi_16_64, cospi_16_64,
- cospi_16_64, cospi_16_64,
- cospi_16_64, -cospi_16_64,
- cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_B = octa_set_epi16(cospi_16_64, -cospi_16_64,
- cospi_16_64, -cospi_16_64,
- cospi_16_64, cospi_16_64,
- cospi_16_64, cospi_16_64);
- const __m128i k__cospi_C = octa_set_epi16(cospi_8_64, cospi_24_64,
- cospi_8_64, cospi_24_64,
- cospi_24_64, -cospi_8_64,
- cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_D = octa_set_epi16(cospi_24_64, -cospi_8_64,
- cospi_24_64, -cospi_8_64,
- cospi_8_64, cospi_24_64,
- cospi_8_64, cospi_24_64);
- const __m128i k__cospi_E = octa_set_epi16(cospi_16_64, cospi_16_64,
- cospi_16_64, cospi_16_64,
- cospi_16_64, cospi_16_64,
- cospi_16_64, cospi_16_64);
- const __m128i k__cospi_F = octa_set_epi16(cospi_16_64, -cospi_16_64,
- cospi_16_64, -cospi_16_64,
- cospi_16_64, -cospi_16_64,
- cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_G = octa_set_epi16(cospi_8_64, cospi_24_64,
- cospi_8_64, cospi_24_64,
- -cospi_8_64, -cospi_24_64,
- -cospi_8_64, -cospi_24_64);
- const __m128i k__cospi_H = octa_set_epi16(cospi_24_64, -cospi_8_64,
- cospi_24_64, -cospi_8_64,
- -cospi_24_64, cospi_8_64,
- -cospi_24_64, cospi_8_64);
-
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- // This second rounding constant saves doing some extra adds at the end
- const __m128i k__DCT_CONST_ROUNDING2 = _mm_set1_epi32(DCT_CONST_ROUNDING
- +(DCT_CONST_ROUNDING << 1));
- const int DCT_CONST_BITS2 = DCT_CONST_BITS + 2;
- const __m128i k__nonzero_bias_a = _mm_setr_epi16(0, 1, 1, 1, 1, 1, 1, 1);
- const __m128i k__nonzero_bias_b = _mm_setr_epi16(1, 0, 0, 0, 0, 0, 0, 0);
- __m128i in0, in1;
-#if DCT_HIGH_BIT_DEPTH
- __m128i cmp0, cmp1;
- int test, overflow;
-#endif
-
- // Load inputs.
- in0 = _mm_loadl_epi64((const __m128i *)(input + 0 * stride));
- in1 = _mm_loadl_epi64((const __m128i *)(input + 1 * stride));
- in1 = _mm_unpacklo_epi64(in1, _mm_loadl_epi64((const __m128i *)
- (input + 2 * stride)));
- in0 = _mm_unpacklo_epi64(in0, _mm_loadl_epi64((const __m128i *)
- (input + 3 * stride)));
- // in0 = [i0 i1 i2 i3 iC iD iE iF]
- // in1 = [i4 i5 i6 i7 i8 i9 iA iB]
-#if DCT_HIGH_BIT_DEPTH
- // Check inputs small enough to use optimised code
- cmp0 = _mm_xor_si128(_mm_cmpgt_epi16(in0, _mm_set1_epi16(0x3ff)),
- _mm_cmplt_epi16(in0, _mm_set1_epi16(0xfc00)));
- cmp1 = _mm_xor_si128(_mm_cmpgt_epi16(in1, _mm_set1_epi16(0x3ff)),
- _mm_cmplt_epi16(in1, _mm_set1_epi16(0xfc00)));
- test = _mm_movemask_epi8(_mm_or_si128(cmp0, cmp1));
- if (test) {
- vpx_highbd_fdct4x4_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
-
- // multiply by 16 to give some extra precision
- in0 = _mm_slli_epi16(in0, 4);
- in1 = _mm_slli_epi16(in1, 4);
- // if (i == 0 && input[0]) input[0] += 1;
- // add 1 to the upper left pixel if it is non-zero, which helps reduce
- // the round-trip error
- {
- // The mask will only contain whether the first value is zero, all
- // other comparison will fail as something shifted by 4 (above << 4)
- // can never be equal to one. To increment in the non-zero case, we
- // add the mask and one for the first element:
- // - if zero, mask = -1, v = v - 1 + 1 = v
- // - if non-zero, mask = 0, v = v + 0 + 1 = v + 1
- __m128i mask = _mm_cmpeq_epi16(in0, k__nonzero_bias_a);
- in0 = _mm_add_epi16(in0, mask);
- in0 = _mm_add_epi16(in0, k__nonzero_bias_b);
- }
- // There are 4 total stages, alternating between an add/subtract stage
- // followed by an multiply-and-add stage.
- {
- // Stage 1: Add/subtract
-
- // in0 = [i0 i1 i2 i3 iC iD iE iF]
- // in1 = [i4 i5 i6 i7 i8 i9 iA iB]
- const __m128i r0 = _mm_unpacklo_epi16(in0, in1);
- const __m128i r1 = _mm_unpackhi_epi16(in0, in1);
- // r0 = [i0 i4 i1 i5 i2 i6 i3 i7]
- // r1 = [iC i8 iD i9 iE iA iF iB]
- const __m128i r2 = _mm_shuffle_epi32(r0, 0xB4);
- const __m128i r3 = _mm_shuffle_epi32(r1, 0xB4);
- // r2 = [i0 i4 i1 i5 i3 i7 i2 i6]
- // r3 = [iC i8 iD i9 iF iB iE iA]
-
- const __m128i t0 = _mm_add_epi16(r2, r3);
- const __m128i t1 = _mm_sub_epi16(r2, r3);
- // t0 = [a0 a4 a1 a5 a3 a7 a2 a6]
- // t1 = [aC a8 aD a9 aF aB aE aA]
-
- // Stage 2: multiply by constants (which gets us into 32 bits).
- // The constants needed here are:
- // k__cospi_A = [p16 p16 p16 p16 p16 m16 p16 m16]
- // k__cospi_B = [p16 m16 p16 m16 p16 p16 p16 p16]
- // k__cospi_C = [p08 p24 p08 p24 p24 m08 p24 m08]
- // k__cospi_D = [p24 m08 p24 m08 p08 p24 p08 p24]
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_A);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_B);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_C);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_D);
- // Then add and right-shift to get back to 16-bit range
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- // w0 = [b0 b1 b7 b6]
- // w1 = [b8 b9 bF bE]
- // w2 = [b4 b5 b3 b2]
- // w3 = [bC bD bB bA]
- const __m128i x0 = _mm_packs_epi32(w0, w1);
- const __m128i x1 = _mm_packs_epi32(w2, w3);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x2(&x0, &x1);
- if (overflow) {
- vpx_highbd_fdct4x4_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // x0 = [b0 b1 b7 b6 b8 b9 bF bE]
- // x1 = [b4 b5 b3 b2 bC bD bB bA]
- in0 = _mm_shuffle_epi32(x0, 0xD8);
- in1 = _mm_shuffle_epi32(x1, 0x8D);
- // in0 = [b0 b1 b8 b9 b7 b6 bF bE]
- // in1 = [b3 b2 bB bA b4 b5 bC bD]
- }
- {
- // vertical DCTs finished. Now we do the horizontal DCTs.
- // Stage 3: Add/subtract
-
- const __m128i t0 = ADD_EPI16(in0, in1);
- const __m128i t1 = SUB_EPI16(in0, in1);
- // t0 = [c0 c1 c8 c9 c4 c5 cC cD]
- // t1 = [c3 c2 cB cA -c7 -c6 -cF -cE]
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x2(&t0, &t1);
- if (overflow) {
- vpx_highbd_fdct4x4_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
-
- // Stage 4: multiply by constants (which gets us into 32 bits).
- {
- // The constants needed here are:
- // k__cospi_E = [p16 p16 p16 p16 p16 p16 p16 p16]
- // k__cospi_F = [p16 m16 p16 m16 p16 m16 p16 m16]
- // k__cospi_G = [p08 p24 p08 p24 m08 m24 m08 m24]
- // k__cospi_H = [p24 m08 p24 m08 m24 p08 m24 p08]
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_E);
- const __m128i u1 = _mm_madd_epi16(t0, k__cospi_F);
- const __m128i u2 = _mm_madd_epi16(t1, k__cospi_G);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_H);
- // Then add and right-shift to get back to 16-bit range
- // but this combines the final right-shift as well to save operations
- // This unusual rounding operations is to maintain bit-accurate
- // compatibility with the c version of this function which has two
- // rounding steps in a row.
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING2);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING2);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING2);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING2);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS2);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS2);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS2);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS2);
- // w0 = [o0 o4 o8 oC]
- // w1 = [o2 o6 oA oE]
- // w2 = [o1 o5 o9 oD]
- // w3 = [o3 o7 oB oF]
- // remember the o's are numbered according to the correct output location
- const __m128i x0 = _mm_packs_epi32(w0, w1);
- const __m128i x1 = _mm_packs_epi32(w2, w3);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x2(&x0, &x1);
- if (overflow) {
- vpx_highbd_fdct4x4_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- {
- // x0 = [o0 o4 o8 oC o2 o6 oA oE]
- // x1 = [o1 o5 o9 oD o3 o7 oB oF]
- const __m128i y0 = _mm_unpacklo_epi16(x0, x1);
- const __m128i y1 = _mm_unpackhi_epi16(x0, x1);
- // y0 = [o0 o1 o4 o5 o8 o9 oC oD]
- // y1 = [o2 o3 o6 o7 oA oB oE oF]
- in0 = _mm_unpacklo_epi32(y0, y1);
- // in0 = [o0 o1 o2 o3 o4 o5 o6 o7]
- in1 = _mm_unpackhi_epi32(y0, y1);
- // in1 = [o8 o9 oA oB oC oD oE oF]
- }
- }
- }
- // Post-condition (v + 1) >> 2 is now incorporated into previous
- // add and right-shift commands. Only 2 store instructions needed
- // because we are using the fact that 1/3 are stored just after 0/2.
- storeu_output(&in0, output + 0 * 4);
- storeu_output(&in1, output + 2 * 4);
-}
-
-
-void FDCT8x8_2D(const int16_t *input, tran_low_t *output, int stride) {
- int pass;
- // Constants
- // When we use them, in one case, they are all the same. In all others
- // it's a pair of them that we need to repeat four times. This is done
- // by constructing the 32 bit constant corresponding to that pair.
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
-#if DCT_HIGH_BIT_DEPTH
- int overflow;
-#endif
- // Load input
- __m128i in0 = _mm_load_si128((const __m128i *)(input + 0 * stride));
- __m128i in1 = _mm_load_si128((const __m128i *)(input + 1 * stride));
- __m128i in2 = _mm_load_si128((const __m128i *)(input + 2 * stride));
- __m128i in3 = _mm_load_si128((const __m128i *)(input + 3 * stride));
- __m128i in4 = _mm_load_si128((const __m128i *)(input + 4 * stride));
- __m128i in5 = _mm_load_si128((const __m128i *)(input + 5 * stride));
- __m128i in6 = _mm_load_si128((const __m128i *)(input + 6 * stride));
- __m128i in7 = _mm_load_si128((const __m128i *)(input + 7 * stride));
- // Pre-condition input (shift by two)
- in0 = _mm_slli_epi16(in0, 2);
- in1 = _mm_slli_epi16(in1, 2);
- in2 = _mm_slli_epi16(in2, 2);
- in3 = _mm_slli_epi16(in3, 2);
- in4 = _mm_slli_epi16(in4, 2);
- in5 = _mm_slli_epi16(in5, 2);
- in6 = _mm_slli_epi16(in6, 2);
- in7 = _mm_slli_epi16(in7, 2);
-
- // We do two passes, first the columns, then the rows. The results of the
- // first pass are transposed so that the same column code can be reused. The
- // results of the second pass are also transposed so that the rows (processed
- // as columns) are put back in row positions.
- for (pass = 0; pass < 2; pass++) {
- // To store results of each pass before the transpose.
- __m128i res0, res1, res2, res3, res4, res5, res6, res7;
- // Add/subtract
- const __m128i q0 = ADD_EPI16(in0, in7);
- const __m128i q1 = ADD_EPI16(in1, in6);
- const __m128i q2 = ADD_EPI16(in2, in5);
- const __m128i q3 = ADD_EPI16(in3, in4);
- const __m128i q4 = SUB_EPI16(in3, in4);
- const __m128i q5 = SUB_EPI16(in2, in5);
- const __m128i q6 = SUB_EPI16(in1, in6);
- const __m128i q7 = SUB_EPI16(in0, in7);
-#if DCT_HIGH_BIT_DEPTH
- if (pass == 1) {
- overflow = check_epi16_overflow_x8(&q0, &q1, &q2, &q3,
- &q4, &q5, &q6, &q7);
- if (overflow) {
- vpx_highbd_fdct8x8_c(input, output, stride);
- return;
- }
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // Work on first four results
- {
- // Add/subtract
- const __m128i r0 = ADD_EPI16(q0, q3);
- const __m128i r1 = ADD_EPI16(q1, q2);
- const __m128i r2 = SUB_EPI16(q1, q2);
- const __m128i r3 = SUB_EPI16(q0, q3);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&r0, &r1, &r2, &r3);
- if (overflow) {
- vpx_highbd_fdct8x8_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // Interleave to do the multiply by constants which gets us into 32bits
- {
- const __m128i t0 = _mm_unpacklo_epi16(r0, r1);
- const __m128i t1 = _mm_unpackhi_epi16(r0, r1);
- const __m128i t2 = _mm_unpacklo_epi16(r2, r3);
- const __m128i t3 = _mm_unpackhi_epi16(r2, r3);
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_p16_p16);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_p16_p16);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_p16_m16);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_p16_m16);
- const __m128i u4 = _mm_madd_epi16(t2, k__cospi_p24_p08);
- const __m128i u5 = _mm_madd_epi16(t3, k__cospi_p24_p08);
- const __m128i u6 = _mm_madd_epi16(t2, k__cospi_m08_p24);
- const __m128i u7 = _mm_madd_epi16(t3, k__cospi_m08_p24);
- // dct_const_round_shift
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- const __m128i v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- const __m128i v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- const __m128i v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- const __m128i v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- const __m128i w4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- const __m128i w5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- const __m128i w6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- const __m128i w7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- // Combine
- res0 = _mm_packs_epi32(w0, w1);
- res4 = _mm_packs_epi32(w2, w3);
- res2 = _mm_packs_epi32(w4, w5);
- res6 = _mm_packs_epi32(w6, w7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&res0, &res4, &res2, &res6);
- if (overflow) {
- vpx_highbd_fdct8x8_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
- // Work on next four results
- {
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i d0 = _mm_unpacklo_epi16(q6, q5);
- const __m128i d1 = _mm_unpackhi_epi16(q6, q5);
- const __m128i e0 = _mm_madd_epi16(d0, k__cospi_p16_m16);
- const __m128i e1 = _mm_madd_epi16(d1, k__cospi_p16_m16);
- const __m128i e2 = _mm_madd_epi16(d0, k__cospi_p16_p16);
- const __m128i e3 = _mm_madd_epi16(d1, k__cospi_p16_p16);
- // dct_const_round_shift
- const __m128i f0 = _mm_add_epi32(e0, k__DCT_CONST_ROUNDING);
- const __m128i f1 = _mm_add_epi32(e1, k__DCT_CONST_ROUNDING);
- const __m128i f2 = _mm_add_epi32(e2, k__DCT_CONST_ROUNDING);
- const __m128i f3 = _mm_add_epi32(e3, k__DCT_CONST_ROUNDING);
- const __m128i s0 = _mm_srai_epi32(f0, DCT_CONST_BITS);
- const __m128i s1 = _mm_srai_epi32(f1, DCT_CONST_BITS);
- const __m128i s2 = _mm_srai_epi32(f2, DCT_CONST_BITS);
- const __m128i s3 = _mm_srai_epi32(f3, DCT_CONST_BITS);
- // Combine
- const __m128i r0 = _mm_packs_epi32(s0, s1);
- const __m128i r1 = _mm_packs_epi32(s2, s3);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x2(&r0, &r1);
- if (overflow) {
- vpx_highbd_fdct8x8_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- {
- // Add/subtract
- const __m128i x0 = ADD_EPI16(q4, r0);
- const __m128i x1 = SUB_EPI16(q4, r0);
- const __m128i x2 = SUB_EPI16(q7, r1);
- const __m128i x3 = ADD_EPI16(q7, r1);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&x0, &x1, &x2, &x3);
- if (overflow) {
- vpx_highbd_fdct8x8_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // Interleave to do the multiply by constants which gets us into 32bits
- {
- const __m128i t0 = _mm_unpacklo_epi16(x0, x3);
- const __m128i t1 = _mm_unpackhi_epi16(x0, x3);
- const __m128i t2 = _mm_unpacklo_epi16(x1, x2);
- const __m128i t3 = _mm_unpackhi_epi16(x1, x2);
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_p28_p04);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_p28_p04);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_m04_p28);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_m04_p28);
- const __m128i u4 = _mm_madd_epi16(t2, k__cospi_p12_p20);
- const __m128i u5 = _mm_madd_epi16(t3, k__cospi_p12_p20);
- const __m128i u6 = _mm_madd_epi16(t2, k__cospi_m20_p12);
- const __m128i u7 = _mm_madd_epi16(t3, k__cospi_m20_p12);
- // dct_const_round_shift
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- const __m128i v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- const __m128i v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- const __m128i v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- const __m128i v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- const __m128i w4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- const __m128i w5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- const __m128i w6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- const __m128i w7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- // Combine
- res1 = _mm_packs_epi32(w0, w1);
- res7 = _mm_packs_epi32(w2, w3);
- res5 = _mm_packs_epi32(w4, w5);
- res3 = _mm_packs_epi32(w6, w7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&res1, &res7, &res5, &res3);
- if (overflow) {
- vpx_highbd_fdct8x8_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
- }
- // Transpose the 8x8.
- {
- // 00 01 02 03 04 05 06 07
- // 10 11 12 13 14 15 16 17
- // 20 21 22 23 24 25 26 27
- // 30 31 32 33 34 35 36 37
- // 40 41 42 43 44 45 46 47
- // 50 51 52 53 54 55 56 57
- // 60 61 62 63 64 65 66 67
- // 70 71 72 73 74 75 76 77
- const __m128i tr0_0 = _mm_unpacklo_epi16(res0, res1);
- const __m128i tr0_1 = _mm_unpacklo_epi16(res2, res3);
- const __m128i tr0_2 = _mm_unpackhi_epi16(res0, res1);
- const __m128i tr0_3 = _mm_unpackhi_epi16(res2, res3);
- const __m128i tr0_4 = _mm_unpacklo_epi16(res4, res5);
- const __m128i tr0_5 = _mm_unpacklo_epi16(res6, res7);
- const __m128i tr0_6 = _mm_unpackhi_epi16(res4, res5);
- const __m128i tr0_7 = _mm_unpackhi_epi16(res6, res7);
- // 00 10 01 11 02 12 03 13
- // 20 30 21 31 22 32 23 33
- // 04 14 05 15 06 16 07 17
- // 24 34 25 35 26 36 27 37
- // 40 50 41 51 42 52 43 53
- // 60 70 61 71 62 72 63 73
- // 54 54 55 55 56 56 57 57
- // 64 74 65 75 66 76 67 77
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_2, tr0_3);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_2, tr0_3);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5);
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7);
- // 00 10 20 30 01 11 21 31
- // 40 50 60 70 41 51 61 71
- // 02 12 22 32 03 13 23 33
- // 42 52 62 72 43 53 63 73
- // 04 14 24 34 05 15 21 36
- // 44 54 64 74 45 55 61 76
- // 06 16 26 36 07 17 27 37
- // 46 56 66 76 47 57 67 77
- in0 = _mm_unpacklo_epi64(tr1_0, tr1_4);
- in1 = _mm_unpackhi_epi64(tr1_0, tr1_4);
- in2 = _mm_unpacklo_epi64(tr1_2, tr1_6);
- in3 = _mm_unpackhi_epi64(tr1_2, tr1_6);
- in4 = _mm_unpacklo_epi64(tr1_1, tr1_5);
- in5 = _mm_unpackhi_epi64(tr1_1, tr1_5);
- in6 = _mm_unpacklo_epi64(tr1_3, tr1_7);
- in7 = _mm_unpackhi_epi64(tr1_3, tr1_7);
- // 00 10 20 30 40 50 60 70
- // 01 11 21 31 41 51 61 71
- // 02 12 22 32 42 52 62 72
- // 03 13 23 33 43 53 63 73
- // 04 14 24 34 44 54 64 74
- // 05 15 25 35 45 55 65 75
- // 06 16 26 36 46 56 66 76
- // 07 17 27 37 47 57 67 77
- }
- }
- // Post-condition output and store it
- {
- // Post-condition (division by two)
- // division of two 16 bits signed numbers using shifts
- // n / 2 = (n - (n >> 15)) >> 1
- const __m128i sign_in0 = _mm_srai_epi16(in0, 15);
- const __m128i sign_in1 = _mm_srai_epi16(in1, 15);
- const __m128i sign_in2 = _mm_srai_epi16(in2, 15);
- const __m128i sign_in3 = _mm_srai_epi16(in3, 15);
- const __m128i sign_in4 = _mm_srai_epi16(in4, 15);
- const __m128i sign_in5 = _mm_srai_epi16(in5, 15);
- const __m128i sign_in6 = _mm_srai_epi16(in6, 15);
- const __m128i sign_in7 = _mm_srai_epi16(in7, 15);
- in0 = _mm_sub_epi16(in0, sign_in0);
- in1 = _mm_sub_epi16(in1, sign_in1);
- in2 = _mm_sub_epi16(in2, sign_in2);
- in3 = _mm_sub_epi16(in3, sign_in3);
- in4 = _mm_sub_epi16(in4, sign_in4);
- in5 = _mm_sub_epi16(in5, sign_in5);
- in6 = _mm_sub_epi16(in6, sign_in6);
- in7 = _mm_sub_epi16(in7, sign_in7);
- in0 = _mm_srai_epi16(in0, 1);
- in1 = _mm_srai_epi16(in1, 1);
- in2 = _mm_srai_epi16(in2, 1);
- in3 = _mm_srai_epi16(in3, 1);
- in4 = _mm_srai_epi16(in4, 1);
- in5 = _mm_srai_epi16(in5, 1);
- in6 = _mm_srai_epi16(in6, 1);
- in7 = _mm_srai_epi16(in7, 1);
- // store results
- store_output(&in0, (output + 0 * 8));
- store_output(&in1, (output + 1 * 8));
- store_output(&in2, (output + 2 * 8));
- store_output(&in3, (output + 3 * 8));
- store_output(&in4, (output + 4 * 8));
- store_output(&in5, (output + 5 * 8));
- store_output(&in6, (output + 6 * 8));
- store_output(&in7, (output + 7 * 8));
- }
-}
-
-void FDCT16x16_2D(const int16_t *input, tran_low_t *output, int stride) {
- // The 2D transform is done with two passes which are actually pretty
- // similar. In the first one, we transform the columns and transpose
- // the results. In the second one, we transform the rows. To achieve that,
- // as the first pass results are transposed, we transpose the columns (that
- // is the transposed rows) and transpose the results (so that it goes back
- // in normal/row positions).
- int pass;
- // We need an intermediate buffer between passes.
- DECLARE_ALIGNED(16, int16_t, intermediate[256]);
- const int16_t *in = input;
- int16_t *out0 = intermediate;
- tran_low_t *out1 = output;
- // Constants
- // When we use them, in one case, they are all the same. In all others
- // it's a pair of them that we need to repeat four times. This is done
- // by constructing the 32 bit constant corresponding to that pair.
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_p08_m24 = pair_set_epi16(cospi_8_64, -cospi_24_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__cospi_p30_p02 = pair_set_epi16(cospi_30_64, cospi_2_64);
- const __m128i k__cospi_p14_p18 = pair_set_epi16(cospi_14_64, cospi_18_64);
- const __m128i k__cospi_m02_p30 = pair_set_epi16(-cospi_2_64, cospi_30_64);
- const __m128i k__cospi_m18_p14 = pair_set_epi16(-cospi_18_64, cospi_14_64);
- const __m128i k__cospi_p22_p10 = pair_set_epi16(cospi_22_64, cospi_10_64);
- const __m128i k__cospi_p06_p26 = pair_set_epi16(cospi_6_64, cospi_26_64);
- const __m128i k__cospi_m10_p22 = pair_set_epi16(-cospi_10_64, cospi_22_64);
- const __m128i k__cospi_m26_p06 = pair_set_epi16(-cospi_26_64, cospi_6_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i kOne = _mm_set1_epi16(1);
- // Do the two transform/transpose passes
- for (pass = 0; pass < 2; ++pass) {
- // We process eight columns (transposed rows in second pass) at a time.
- int column_start;
-#if DCT_HIGH_BIT_DEPTH
- int overflow;
-#endif
- for (column_start = 0; column_start < 16; column_start += 8) {
- __m128i in00, in01, in02, in03, in04, in05, in06, in07;
- __m128i in08, in09, in10, in11, in12, in13, in14, in15;
- __m128i input0, input1, input2, input3, input4, input5, input6, input7;
- __m128i step1_0, step1_1, step1_2, step1_3;
- __m128i step1_4, step1_5, step1_6, step1_7;
- __m128i step2_1, step2_2, step2_3, step2_4, step2_5, step2_6;
- __m128i step3_0, step3_1, step3_2, step3_3;
- __m128i step3_4, step3_5, step3_6, step3_7;
- __m128i res00, res01, res02, res03, res04, res05, res06, res07;
- __m128i res08, res09, res10, res11, res12, res13, res14, res15;
- // Load and pre-condition input.
- if (0 == pass) {
- in00 = _mm_load_si128((const __m128i *)(in + 0 * stride));
- in01 = _mm_load_si128((const __m128i *)(in + 1 * stride));
- in02 = _mm_load_si128((const __m128i *)(in + 2 * stride));
- in03 = _mm_load_si128((const __m128i *)(in + 3 * stride));
- in04 = _mm_load_si128((const __m128i *)(in + 4 * stride));
- in05 = _mm_load_si128((const __m128i *)(in + 5 * stride));
- in06 = _mm_load_si128((const __m128i *)(in + 6 * stride));
- in07 = _mm_load_si128((const __m128i *)(in + 7 * stride));
- in08 = _mm_load_si128((const __m128i *)(in + 8 * stride));
- in09 = _mm_load_si128((const __m128i *)(in + 9 * stride));
- in10 = _mm_load_si128((const __m128i *)(in + 10 * stride));
- in11 = _mm_load_si128((const __m128i *)(in + 11 * stride));
- in12 = _mm_load_si128((const __m128i *)(in + 12 * stride));
- in13 = _mm_load_si128((const __m128i *)(in + 13 * stride));
- in14 = _mm_load_si128((const __m128i *)(in + 14 * stride));
- in15 = _mm_load_si128((const __m128i *)(in + 15 * stride));
- // x = x << 2
- in00 = _mm_slli_epi16(in00, 2);
- in01 = _mm_slli_epi16(in01, 2);
- in02 = _mm_slli_epi16(in02, 2);
- in03 = _mm_slli_epi16(in03, 2);
- in04 = _mm_slli_epi16(in04, 2);
- in05 = _mm_slli_epi16(in05, 2);
- in06 = _mm_slli_epi16(in06, 2);
- in07 = _mm_slli_epi16(in07, 2);
- in08 = _mm_slli_epi16(in08, 2);
- in09 = _mm_slli_epi16(in09, 2);
- in10 = _mm_slli_epi16(in10, 2);
- in11 = _mm_slli_epi16(in11, 2);
- in12 = _mm_slli_epi16(in12, 2);
- in13 = _mm_slli_epi16(in13, 2);
- in14 = _mm_slli_epi16(in14, 2);
- in15 = _mm_slli_epi16(in15, 2);
- } else {
- in00 = _mm_load_si128((const __m128i *)(in + 0 * 16));
- in01 = _mm_load_si128((const __m128i *)(in + 1 * 16));
- in02 = _mm_load_si128((const __m128i *)(in + 2 * 16));
- in03 = _mm_load_si128((const __m128i *)(in + 3 * 16));
- in04 = _mm_load_si128((const __m128i *)(in + 4 * 16));
- in05 = _mm_load_si128((const __m128i *)(in + 5 * 16));
- in06 = _mm_load_si128((const __m128i *)(in + 6 * 16));
- in07 = _mm_load_si128((const __m128i *)(in + 7 * 16));
- in08 = _mm_load_si128((const __m128i *)(in + 8 * 16));
- in09 = _mm_load_si128((const __m128i *)(in + 9 * 16));
- in10 = _mm_load_si128((const __m128i *)(in + 10 * 16));
- in11 = _mm_load_si128((const __m128i *)(in + 11 * 16));
- in12 = _mm_load_si128((const __m128i *)(in + 12 * 16));
- in13 = _mm_load_si128((const __m128i *)(in + 13 * 16));
- in14 = _mm_load_si128((const __m128i *)(in + 14 * 16));
- in15 = _mm_load_si128((const __m128i *)(in + 15 * 16));
- // x = (x + 1) >> 2
- in00 = _mm_add_epi16(in00, kOne);
- in01 = _mm_add_epi16(in01, kOne);
- in02 = _mm_add_epi16(in02, kOne);
- in03 = _mm_add_epi16(in03, kOne);
- in04 = _mm_add_epi16(in04, kOne);
- in05 = _mm_add_epi16(in05, kOne);
- in06 = _mm_add_epi16(in06, kOne);
- in07 = _mm_add_epi16(in07, kOne);
- in08 = _mm_add_epi16(in08, kOne);
- in09 = _mm_add_epi16(in09, kOne);
- in10 = _mm_add_epi16(in10, kOne);
- in11 = _mm_add_epi16(in11, kOne);
- in12 = _mm_add_epi16(in12, kOne);
- in13 = _mm_add_epi16(in13, kOne);
- in14 = _mm_add_epi16(in14, kOne);
- in15 = _mm_add_epi16(in15, kOne);
- in00 = _mm_srai_epi16(in00, 2);
- in01 = _mm_srai_epi16(in01, 2);
- in02 = _mm_srai_epi16(in02, 2);
- in03 = _mm_srai_epi16(in03, 2);
- in04 = _mm_srai_epi16(in04, 2);
- in05 = _mm_srai_epi16(in05, 2);
- in06 = _mm_srai_epi16(in06, 2);
- in07 = _mm_srai_epi16(in07, 2);
- in08 = _mm_srai_epi16(in08, 2);
- in09 = _mm_srai_epi16(in09, 2);
- in10 = _mm_srai_epi16(in10, 2);
- in11 = _mm_srai_epi16(in11, 2);
- in12 = _mm_srai_epi16(in12, 2);
- in13 = _mm_srai_epi16(in13, 2);
- in14 = _mm_srai_epi16(in14, 2);
- in15 = _mm_srai_epi16(in15, 2);
- }
- in += 8;
- // Calculate input for the first 8 results.
- {
- input0 = ADD_EPI16(in00, in15);
- input1 = ADD_EPI16(in01, in14);
- input2 = ADD_EPI16(in02, in13);
- input3 = ADD_EPI16(in03, in12);
- input4 = ADD_EPI16(in04, in11);
- input5 = ADD_EPI16(in05, in10);
- input6 = ADD_EPI16(in06, in09);
- input7 = ADD_EPI16(in07, in08);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&input0, &input1, &input2, &input3,
- &input4, &input5, &input6, &input7);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // Calculate input for the next 8 results.
- {
- step1_0 = SUB_EPI16(in07, in08);
- step1_1 = SUB_EPI16(in06, in09);
- step1_2 = SUB_EPI16(in05, in10);
- step1_3 = SUB_EPI16(in04, in11);
- step1_4 = SUB_EPI16(in03, in12);
- step1_5 = SUB_EPI16(in02, in13);
- step1_6 = SUB_EPI16(in01, in14);
- step1_7 = SUB_EPI16(in00, in15);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1_0, &step1_1,
- &step1_2, &step1_3,
- &step1_4, &step1_5,
- &step1_6, &step1_7);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // Work on the first eight values; fdct8(input, even_results);
- {
- // Add/subtract
- const __m128i q0 = ADD_EPI16(input0, input7);
- const __m128i q1 = ADD_EPI16(input1, input6);
- const __m128i q2 = ADD_EPI16(input2, input5);
- const __m128i q3 = ADD_EPI16(input3, input4);
- const __m128i q4 = SUB_EPI16(input3, input4);
- const __m128i q5 = SUB_EPI16(input2, input5);
- const __m128i q6 = SUB_EPI16(input1, input6);
- const __m128i q7 = SUB_EPI16(input0, input7);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&q0, &q1, &q2, &q3,
- &q4, &q5, &q6, &q7);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // Work on first four results
- {
- // Add/subtract
- const __m128i r0 = ADD_EPI16(q0, q3);
- const __m128i r1 = ADD_EPI16(q1, q2);
- const __m128i r2 = SUB_EPI16(q1, q2);
- const __m128i r3 = SUB_EPI16(q0, q3);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&r0, &r1, &r2, &r3);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // Interleave to do the multiply by constants which gets us
- // into 32 bits.
- {
- const __m128i t0 = _mm_unpacklo_epi16(r0, r1);
- const __m128i t1 = _mm_unpackhi_epi16(r0, r1);
- const __m128i t2 = _mm_unpacklo_epi16(r2, r3);
- const __m128i t3 = _mm_unpackhi_epi16(r2, r3);
- res00 = mult_round_shift(&t0, &t1, &k__cospi_p16_p16,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res08 = mult_round_shift(&t0, &t1, &k__cospi_p16_m16,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res04 = mult_round_shift(&t2, &t3, &k__cospi_p24_p08,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res12 = mult_round_shift(&t2, &t3, &k__cospi_m08_p24,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&res00, &res08, &res04, &res12);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
- // Work on next four results
- {
- // Interleave to do the multiply by constants which gets us
- // into 32 bits.
- const __m128i d0 = _mm_unpacklo_epi16(q6, q5);
- const __m128i d1 = _mm_unpackhi_epi16(q6, q5);
- const __m128i r0 = mult_round_shift(&d0, &d1, &k__cospi_p16_m16,
- &k__DCT_CONST_ROUNDING,
- DCT_CONST_BITS);
- const __m128i r1 = mult_round_shift(&d0, &d1, &k__cospi_p16_p16,
- &k__DCT_CONST_ROUNDING,
- DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x2(&r0, &r1);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- {
- // Add/subtract
- const __m128i x0 = ADD_EPI16(q4, r0);
- const __m128i x1 = SUB_EPI16(q4, r0);
- const __m128i x2 = SUB_EPI16(q7, r1);
- const __m128i x3 = ADD_EPI16(q7, r1);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&x0, &x1, &x2, &x3);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- // Interleave to do the multiply by constants which gets us
- // into 32 bits.
- {
- const __m128i t0 = _mm_unpacklo_epi16(x0, x3);
- const __m128i t1 = _mm_unpackhi_epi16(x0, x3);
- const __m128i t2 = _mm_unpacklo_epi16(x1, x2);
- const __m128i t3 = _mm_unpackhi_epi16(x1, x2);
- res02 = mult_round_shift(&t0, &t1, &k__cospi_p28_p04,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res14 = mult_round_shift(&t0, &t1, &k__cospi_m04_p28,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res10 = mult_round_shift(&t2, &t3, &k__cospi_p12_p20,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res06 = mult_round_shift(&t2, &t3, &k__cospi_m20_p12,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&res02, &res14,
- &res10, &res06);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
- }
- }
- // Work on the next eight values; step1 -> odd_results
- {
- // step 2
- {
- const __m128i t0 = _mm_unpacklo_epi16(step1_5, step1_2);
- const __m128i t1 = _mm_unpackhi_epi16(step1_5, step1_2);
- const __m128i t2 = _mm_unpacklo_epi16(step1_4, step1_3);
- const __m128i t3 = _mm_unpackhi_epi16(step1_4, step1_3);
- step2_2 = mult_round_shift(&t0, &t1, &k__cospi_p16_m16,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- step2_3 = mult_round_shift(&t2, &t3, &k__cospi_p16_m16,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- step2_5 = mult_round_shift(&t0, &t1, &k__cospi_p16_p16,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- step2_4 = mult_round_shift(&t2, &t3, &k__cospi_p16_p16,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&step2_2, &step2_3, &step2_5,
- &step2_4);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // step 3
- {
- step3_0 = ADD_EPI16(step1_0, step2_3);
- step3_1 = ADD_EPI16(step1_1, step2_2);
- step3_2 = SUB_EPI16(step1_1, step2_2);
- step3_3 = SUB_EPI16(step1_0, step2_3);
- step3_4 = SUB_EPI16(step1_7, step2_4);
- step3_5 = SUB_EPI16(step1_6, step2_5);
- step3_6 = ADD_EPI16(step1_6, step2_5);
- step3_7 = ADD_EPI16(step1_7, step2_4);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step3_0, &step3_1,
- &step3_2, &step3_3,
- &step3_4, &step3_5,
- &step3_6, &step3_7);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // step 4
- {
- const __m128i t0 = _mm_unpacklo_epi16(step3_1, step3_6);
- const __m128i t1 = _mm_unpackhi_epi16(step3_1, step3_6);
- const __m128i t2 = _mm_unpacklo_epi16(step3_2, step3_5);
- const __m128i t3 = _mm_unpackhi_epi16(step3_2, step3_5);
- step2_1 = mult_round_shift(&t0, &t1, &k__cospi_m08_p24,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- step2_2 = mult_round_shift(&t2, &t3, &k__cospi_p24_p08,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- step2_6 = mult_round_shift(&t0, &t1, &k__cospi_p24_p08,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- step2_5 = mult_round_shift(&t2, &t3, &k__cospi_p08_m24,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&step2_1, &step2_2, &step2_6,
- &step2_5);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // step 5
- {
- step1_0 = ADD_EPI16(step3_0, step2_1);
- step1_1 = SUB_EPI16(step3_0, step2_1);
- step1_2 = ADD_EPI16(step3_3, step2_2);
- step1_3 = SUB_EPI16(step3_3, step2_2);
- step1_4 = SUB_EPI16(step3_4, step2_5);
- step1_5 = ADD_EPI16(step3_4, step2_5);
- step1_6 = SUB_EPI16(step3_7, step2_6);
- step1_7 = ADD_EPI16(step3_7, step2_6);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x8(&step1_0, &step1_1,
- &step1_2, &step1_3,
- &step1_4, &step1_5,
- &step1_6, &step1_7);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- // step 6
- {
- const __m128i t0 = _mm_unpacklo_epi16(step1_0, step1_7);
- const __m128i t1 = _mm_unpackhi_epi16(step1_0, step1_7);
- const __m128i t2 = _mm_unpacklo_epi16(step1_1, step1_6);
- const __m128i t3 = _mm_unpackhi_epi16(step1_1, step1_6);
- res01 = mult_round_shift(&t0, &t1, &k__cospi_p30_p02,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res09 = mult_round_shift(&t2, &t3, &k__cospi_p14_p18,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res15 = mult_round_shift(&t0, &t1, &k__cospi_m02_p30,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res07 = mult_round_shift(&t2, &t3, &k__cospi_m18_p14,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&res01, &res09, &res15, &res07);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- {
- const __m128i t0 = _mm_unpacklo_epi16(step1_2, step1_5);
- const __m128i t1 = _mm_unpackhi_epi16(step1_2, step1_5);
- const __m128i t2 = _mm_unpacklo_epi16(step1_3, step1_4);
- const __m128i t3 = _mm_unpackhi_epi16(step1_3, step1_4);
- res05 = mult_round_shift(&t0, &t1, &k__cospi_p22_p10,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res13 = mult_round_shift(&t2, &t3, &k__cospi_p06_p26,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res11 = mult_round_shift(&t0, &t1, &k__cospi_m10_p22,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
- res03 = mult_round_shift(&t2, &t3, &k__cospi_m26_p06,
- &k__DCT_CONST_ROUNDING, DCT_CONST_BITS);
-#if DCT_HIGH_BIT_DEPTH
- overflow = check_epi16_overflow_x4(&res05, &res13, &res11, &res03);
- if (overflow) {
- vpx_highbd_fdct16x16_c(input, output, stride);
- return;
- }
-#endif // DCT_HIGH_BIT_DEPTH
- }
- }
- // Transpose the results, do it as two 8x8 transposes.
- transpose_and_output8x8(&res00, &res01, &res02, &res03,
- &res04, &res05, &res06, &res07,
- pass, out0, out1);
- transpose_and_output8x8(&res08, &res09, &res10, &res11,
- &res12, &res13, &res14, &res15,
- pass, out0 + 8, out1 + 8);
- if (pass == 0) {
- out0 += 8*16;
- } else {
- out1 += 8*16;
- }
- }
- // Setup in/out for next pass.
- in = intermediate;
- }
-}
-
-#undef ADD_EPI16
-#undef SUB_EPI16
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h> // SSE2
-
-#include "./vpx_config.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_dsp/x86/fwd_txfm_sse2.h"
-
-void vp10_fdct4x4_1_sse2(const int16_t *input, tran_low_t *output, int stride) {
- __m128i in0, in1;
- __m128i tmp;
- const __m128i zero = _mm_setzero_si128();
- in0 = _mm_loadl_epi64((const __m128i *)(input + 0 * stride));
- in1 = _mm_loadl_epi64((const __m128i *)(input + 1 * stride));
- in1 = _mm_unpacklo_epi64(in1, _mm_loadl_epi64((const __m128i *)
- (input + 2 * stride)));
- in0 = _mm_unpacklo_epi64(in0, _mm_loadl_epi64((const __m128i *)
- (input + 3 * stride)));
-
- tmp = _mm_add_epi16(in0, in1);
- in0 = _mm_unpacklo_epi16(zero, tmp);
- in1 = _mm_unpackhi_epi16(zero, tmp);
- in0 = _mm_srai_epi32(in0, 16);
- in1 = _mm_srai_epi32(in1, 16);
-
- tmp = _mm_add_epi32(in0, in1);
- in0 = _mm_unpacklo_epi32(tmp, zero);
- in1 = _mm_unpackhi_epi32(tmp, zero);
-
- tmp = _mm_add_epi32(in0, in1);
- in0 = _mm_srli_si128(tmp, 8);
-
- in1 = _mm_add_epi32(tmp, in0);
- in0 = _mm_slli_epi32(in1, 1);
- store_output(&in0, output);
-}
-
-void vp10_fdct8x8_1_sse2(const int16_t *input, tran_low_t *output, int stride) {
- __m128i in0 = _mm_load_si128((const __m128i *)(input + 0 * stride));
- __m128i in1 = _mm_load_si128((const __m128i *)(input + 1 * stride));
- __m128i in2 = _mm_load_si128((const __m128i *)(input + 2 * stride));
- __m128i in3 = _mm_load_si128((const __m128i *)(input + 3 * stride));
- __m128i u0, u1, sum;
-
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
-
- in0 = _mm_load_si128((const __m128i *)(input + 4 * stride));
- in1 = _mm_load_si128((const __m128i *)(input + 5 * stride));
- in2 = _mm_load_si128((const __m128i *)(input + 6 * stride));
- in3 = _mm_load_si128((const __m128i *)(input + 7 * stride));
-
- sum = _mm_add_epi16(u0, u1);
-
- in0 = _mm_add_epi16(in0, in1);
- in2 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, in0);
-
- u0 = _mm_setzero_si128();
- sum = _mm_add_epi16(sum, in2);
-
- in0 = _mm_unpacklo_epi16(u0, sum);
- in1 = _mm_unpackhi_epi16(u0, sum);
- in0 = _mm_srai_epi32(in0, 16);
- in1 = _mm_srai_epi32(in1, 16);
-
- sum = _mm_add_epi32(in0, in1);
- in0 = _mm_unpacklo_epi32(sum, u0);
- in1 = _mm_unpackhi_epi32(sum, u0);
-
- sum = _mm_add_epi32(in0, in1);
- in0 = _mm_srli_si128(sum, 8);
-
- in1 = _mm_add_epi32(sum, in0);
- store_output(&in1, output);
-}
-
-void vp10_fdct16x16_1_sse2(const int16_t *input, tran_low_t *output,
- int stride) {
- __m128i in0, in1, in2, in3;
- __m128i u0, u1;
- __m128i sum = _mm_setzero_si128();
- int i;
-
- for (i = 0; i < 2; ++i) {
- input += 8 * i;
- in0 = _mm_load_si128((const __m128i *)(input + 0 * stride));
- in1 = _mm_load_si128((const __m128i *)(input + 1 * stride));
- in2 = _mm_load_si128((const __m128i *)(input + 2 * stride));
- in3 = _mm_load_si128((const __m128i *)(input + 3 * stride));
-
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- in0 = _mm_load_si128((const __m128i *)(input + 4 * stride));
- in1 = _mm_load_si128((const __m128i *)(input + 5 * stride));
- in2 = _mm_load_si128((const __m128i *)(input + 6 * stride));
- in3 = _mm_load_si128((const __m128i *)(input + 7 * stride));
-
- sum = _mm_add_epi16(sum, u1);
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- in0 = _mm_load_si128((const __m128i *)(input + 8 * stride));
- in1 = _mm_load_si128((const __m128i *)(input + 9 * stride));
- in2 = _mm_load_si128((const __m128i *)(input + 10 * stride));
- in3 = _mm_load_si128((const __m128i *)(input + 11 * stride));
-
- sum = _mm_add_epi16(sum, u1);
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- in0 = _mm_load_si128((const __m128i *)(input + 12 * stride));
- in1 = _mm_load_si128((const __m128i *)(input + 13 * stride));
- in2 = _mm_load_si128((const __m128i *)(input + 14 * stride));
- in3 = _mm_load_si128((const __m128i *)(input + 15 * stride));
-
- sum = _mm_add_epi16(sum, u1);
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- sum = _mm_add_epi16(sum, u1);
- }
-
- u0 = _mm_setzero_si128();
- in0 = _mm_unpacklo_epi16(u0, sum);
- in1 = _mm_unpackhi_epi16(u0, sum);
- in0 = _mm_srai_epi32(in0, 16);
- in1 = _mm_srai_epi32(in1, 16);
-
- sum = _mm_add_epi32(in0, in1);
- in0 = _mm_unpacklo_epi32(sum, u0);
- in1 = _mm_unpackhi_epi32(sum, u0);
-
- sum = _mm_add_epi32(in0, in1);
- in0 = _mm_srli_si128(sum, 8);
-
- in1 = _mm_add_epi32(sum, in0);
- in1 = _mm_srai_epi32(in1, 1);
- store_output(&in1, output);
-}
-
-void vp10_fdct32x32_1_sse2(const int16_t *input, tran_low_t *output,
- int stride) {
- __m128i in0, in1, in2, in3;
- __m128i u0, u1;
- __m128i sum = _mm_setzero_si128();
- int i;
-
- for (i = 0; i < 8; ++i) {
- in0 = _mm_load_si128((const __m128i *)(input + 0));
- in1 = _mm_load_si128((const __m128i *)(input + 8));
- in2 = _mm_load_si128((const __m128i *)(input + 16));
- in3 = _mm_load_si128((const __m128i *)(input + 24));
-
- input += stride;
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- in0 = _mm_load_si128((const __m128i *)(input + 0));
- in1 = _mm_load_si128((const __m128i *)(input + 8));
- in2 = _mm_load_si128((const __m128i *)(input + 16));
- in3 = _mm_load_si128((const __m128i *)(input + 24));
-
- input += stride;
- sum = _mm_add_epi16(sum, u1);
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- in0 = _mm_load_si128((const __m128i *)(input + 0));
- in1 = _mm_load_si128((const __m128i *)(input + 8));
- in2 = _mm_load_si128((const __m128i *)(input + 16));
- in3 = _mm_load_si128((const __m128i *)(input + 24));
-
- input += stride;
- sum = _mm_add_epi16(sum, u1);
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- in0 = _mm_load_si128((const __m128i *)(input + 0));
- in1 = _mm_load_si128((const __m128i *)(input + 8));
- in2 = _mm_load_si128((const __m128i *)(input + 16));
- in3 = _mm_load_si128((const __m128i *)(input + 24));
-
- input += stride;
- sum = _mm_add_epi16(sum, u1);
- u0 = _mm_add_epi16(in0, in1);
- u1 = _mm_add_epi16(in2, in3);
- sum = _mm_add_epi16(sum, u0);
-
- sum = _mm_add_epi16(sum, u1);
- }
-
- u0 = _mm_setzero_si128();
- in0 = _mm_unpacklo_epi16(u0, sum);
- in1 = _mm_unpackhi_epi16(u0, sum);
- in0 = _mm_srai_epi32(in0, 16);
- in1 = _mm_srai_epi32(in1, 16);
-
- sum = _mm_add_epi32(in0, in1);
- in0 = _mm_unpacklo_epi32(sum, u0);
- in1 = _mm_unpackhi_epi32(sum, u0);
-
- sum = _mm_add_epi32(in0, in1);
- in0 = _mm_srli_si128(sum, 8);
-
- in1 = _mm_add_epi32(sum, in0);
- in1 = _mm_srai_epi32(in1, 3);
- store_output(&in1, output);
-}
-
-#define DCT_HIGH_BIT_DEPTH 0
-#define FDCT4x4_2D vp10_fdct4x4_sse2
-#define FDCT8x8_2D vp10_fdct8x8_sse2
-#define FDCT16x16_2D vp10_fdct16x16_sse2
-#include "vp10/common/x86/vp10_fwd_txfm_impl_sse2.h"
-#undef FDCT4x4_2D
-#undef FDCT8x8_2D
-#undef FDCT16x16_2D
-
-#define FDCT32x32_2D vp10_fdct32x32_rd_sse2
-#define FDCT32x32_HIGH_PRECISION 0
-#include "vp10/common/x86/vp10_fwd_dct32x32_impl_sse2.h"
-#undef FDCT32x32_2D
-#undef FDCT32x32_HIGH_PRECISION
-
-#define FDCT32x32_2D vp10_fdct32x32_sse2
-#define FDCT32x32_HIGH_PRECISION 1
-#include "vp10/common/x86/vp10_fwd_dct32x32_impl_sse2.h" // NOLINT
-#undef FDCT32x32_2D
-#undef FDCT32x32_HIGH_PRECISION
-#undef DCT_HIGH_BIT_DEPTH
-
-#if CONFIG_VP9_HIGHBITDEPTH
-#define DCT_HIGH_BIT_DEPTH 1
-#define FDCT4x4_2D vp10_highbd_fdct4x4_sse2
-#define FDCT8x8_2D vp10_highbd_fdct8x8_sse2
-#define FDCT16x16_2D vp10_highbd_fdct16x16_sse2
-#include "vp10/common/x86/vp10_fwd_txfm_impl_sse2.h" // NOLINT
-#undef FDCT4x4_2D
-#undef FDCT8x8_2D
-#undef FDCT16x16_2D
-
-#define FDCT32x32_2D vp10_highbd_fdct32x32_rd_sse2
-#define FDCT32x32_HIGH_PRECISION 0
-#include "vp10/common/x86/vp10_fwd_dct32x32_impl_sse2.h" // NOLINT
-#undef FDCT32x32_2D
-#undef FDCT32x32_HIGH_PRECISION
-
-#define FDCT32x32_2D vp10_highbd_fdct32x32_sse2
-#define FDCT32x32_HIGH_PRECISION 1
-#include "vp10/common/x86/vp10_fwd_dct32x32_impl_sse2.h" // NOLINT
-#undef FDCT32x32_2D
-#undef FDCT32x32_HIGH_PRECISION
-#undef DCT_HIGH_BIT_DEPTH
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vp10_rtcd.h"
-#include "vp10/common/x86/vp10_inv_txfm_sse2.h"
-#include "vpx_dsp/x86/txfm_common_sse2.h"
-
-#define RECON_AND_STORE4X4(dest, in_x) \
-{ \
- __m128i d0 = _mm_cvtsi32_si128(*(const int *)(dest)); \
- d0 = _mm_unpacklo_epi8(d0, zero); \
- d0 = _mm_add_epi16(in_x, d0); \
- d0 = _mm_packus_epi16(d0, d0); \
- *(int *)(dest) = _mm_cvtsi128_si32(d0); \
-}
-
-void vp10_idct4x4_16_add_sse2(const int16_t *input, uint8_t *dest, int stride) {
- const __m128i zero = _mm_setzero_si128();
- const __m128i eight = _mm_set1_epi16(8);
- const __m128i cst = _mm_setr_epi16(
- (int16_t)cospi_16_64, (int16_t)cospi_16_64, (int16_t)cospi_16_64,
- (int16_t)-cospi_16_64, (int16_t)cospi_24_64, (int16_t)-cospi_8_64,
- (int16_t)cospi_8_64, (int16_t)cospi_24_64);
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- __m128i input0, input1, input2, input3;
-
- // Rows
- input0 = _mm_load_si128((const __m128i *)input);
- input2 = _mm_load_si128((const __m128i *)(input + 8));
-
- // Construct i3, i1, i3, i1, i2, i0, i2, i0
- input0 = _mm_shufflelo_epi16(input0, 0xd8);
- input0 = _mm_shufflehi_epi16(input0, 0xd8);
- input2 = _mm_shufflelo_epi16(input2, 0xd8);
- input2 = _mm_shufflehi_epi16(input2, 0xd8);
-
- input1 = _mm_unpackhi_epi32(input0, input0);
- input0 = _mm_unpacklo_epi32(input0, input0);
- input3 = _mm_unpackhi_epi32(input2, input2);
- input2 = _mm_unpacklo_epi32(input2, input2);
-
- // Stage 1
- input0 = _mm_madd_epi16(input0, cst);
- input1 = _mm_madd_epi16(input1, cst);
- input2 = _mm_madd_epi16(input2, cst);
- input3 = _mm_madd_epi16(input3, cst);
-
- input0 = _mm_add_epi32(input0, rounding);
- input1 = _mm_add_epi32(input1, rounding);
- input2 = _mm_add_epi32(input2, rounding);
- input3 = _mm_add_epi32(input3, rounding);
-
- input0 = _mm_srai_epi32(input0, DCT_CONST_BITS);
- input1 = _mm_srai_epi32(input1, DCT_CONST_BITS);
- input2 = _mm_srai_epi32(input2, DCT_CONST_BITS);
- input3 = _mm_srai_epi32(input3, DCT_CONST_BITS);
-
- // Stage 2
- input0 = _mm_packs_epi32(input0, input1);
- input1 = _mm_packs_epi32(input2, input3);
-
- // Transpose
- input2 = _mm_unpacklo_epi16(input0, input1);
- input3 = _mm_unpackhi_epi16(input0, input1);
- input0 = _mm_unpacklo_epi32(input2, input3);
- input1 = _mm_unpackhi_epi32(input2, input3);
-
- // Switch column2, column 3, and then, we got:
- // input2: column1, column 0; input3: column2, column 3.
- input1 = _mm_shuffle_epi32(input1, 0x4e);
- input2 = _mm_add_epi16(input0, input1);
- input3 = _mm_sub_epi16(input0, input1);
-
- // Columns
- // Construct i3, i1, i3, i1, i2, i0, i2, i0
- input0 = _mm_unpacklo_epi32(input2, input2);
- input1 = _mm_unpackhi_epi32(input2, input2);
- input2 = _mm_unpackhi_epi32(input3, input3);
- input3 = _mm_unpacklo_epi32(input3, input3);
-
- // Stage 1
- input0 = _mm_madd_epi16(input0, cst);
- input1 = _mm_madd_epi16(input1, cst);
- input2 = _mm_madd_epi16(input2, cst);
- input3 = _mm_madd_epi16(input3, cst);
-
- input0 = _mm_add_epi32(input0, rounding);
- input1 = _mm_add_epi32(input1, rounding);
- input2 = _mm_add_epi32(input2, rounding);
- input3 = _mm_add_epi32(input3, rounding);
-
- input0 = _mm_srai_epi32(input0, DCT_CONST_BITS);
- input1 = _mm_srai_epi32(input1, DCT_CONST_BITS);
- input2 = _mm_srai_epi32(input2, DCT_CONST_BITS);
- input3 = _mm_srai_epi32(input3, DCT_CONST_BITS);
-
- // Stage 2
- input0 = _mm_packs_epi32(input0, input2);
- input1 = _mm_packs_epi32(input1, input3);
-
- // Transpose
- input2 = _mm_unpacklo_epi16(input0, input1);
- input3 = _mm_unpackhi_epi16(input0, input1);
- input0 = _mm_unpacklo_epi32(input2, input3);
- input1 = _mm_unpackhi_epi32(input2, input3);
-
- // Switch column2, column 3, and then, we got:
- // input2: column1, column 0; input3: column2, column 3.
- input1 = _mm_shuffle_epi32(input1, 0x4e);
- input2 = _mm_add_epi16(input0, input1);
- input3 = _mm_sub_epi16(input0, input1);
-
- // Final round and shift
- input2 = _mm_add_epi16(input2, eight);
- input3 = _mm_add_epi16(input3, eight);
-
- input2 = _mm_srai_epi16(input2, 4);
- input3 = _mm_srai_epi16(input3, 4);
-
- // Reconstruction and Store
- {
- __m128i d0 = _mm_cvtsi32_si128(*(const int *)(dest));
- __m128i d2 = _mm_cvtsi32_si128(*(const int *)(dest + stride * 2));
- d0 = _mm_unpacklo_epi32(d0,
- _mm_cvtsi32_si128(*(const int *)(dest + stride)));
- d2 = _mm_unpacklo_epi32(
- _mm_cvtsi32_si128(*(const int *)(dest + stride * 3)), d2);
- d0 = _mm_unpacklo_epi8(d0, zero);
- d2 = _mm_unpacklo_epi8(d2, zero);
- d0 = _mm_add_epi16(d0, input2);
- d2 = _mm_add_epi16(d2, input3);
- d0 = _mm_packus_epi16(d0, d2);
- // store input0
- *(int *)dest = _mm_cvtsi128_si32(d0);
- // store input1
- d0 = _mm_srli_si128(d0, 4);
- *(int *)(dest + stride) = _mm_cvtsi128_si32(d0);
- // store input2
- d0 = _mm_srli_si128(d0, 4);
- *(int *)(dest + stride * 3) = _mm_cvtsi128_si32(d0);
- // store input3
- d0 = _mm_srli_si128(d0, 4);
- *(int *)(dest + stride * 2) = _mm_cvtsi128_si32(d0);
- }
-}
-
-void vp10_idct4x4_1_add_sse2(const int16_t *input, uint8_t *dest, int stride) {
- __m128i dc_value;
- const __m128i zero = _mm_setzero_si128();
- int a;
-
- a = dct_const_round_shift(input[0] * cospi_16_64);
- a = dct_const_round_shift(a * cospi_16_64);
- a = ROUND_POWER_OF_TWO(a, 4);
-
- dc_value = _mm_set1_epi16(a);
-
- RECON_AND_STORE4X4(dest + 0 * stride, dc_value);
- RECON_AND_STORE4X4(dest + 1 * stride, dc_value);
- RECON_AND_STORE4X4(dest + 2 * stride, dc_value);
- RECON_AND_STORE4X4(dest + 3 * stride, dc_value);
-}
-
-static INLINE void transpose_4x4(__m128i *res) {
- const __m128i tr0_0 = _mm_unpacklo_epi16(res[0], res[1]);
- const __m128i tr0_1 = _mm_unpackhi_epi16(res[0], res[1]);
-
- res[0] = _mm_unpacklo_epi16(tr0_0, tr0_1);
- res[1] = _mm_unpackhi_epi16(tr0_0, tr0_1);
-}
-
-void vp10_idct4_sse2(__m128i *in) {
- const __m128i k__cospi_p16_p16 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- __m128i u[8], v[8];
-
- transpose_4x4(in);
- // stage 1
- u[0] = _mm_unpacklo_epi16(in[0], in[1]);
- u[1] = _mm_unpackhi_epi16(in[0], in[1]);
- v[0] = _mm_madd_epi16(u[0], k__cospi_p16_p16);
- v[1] = _mm_madd_epi16(u[0], k__cospi_p16_m16);
- v[2] = _mm_madd_epi16(u[1], k__cospi_p24_m08);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p08_p24);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
-
- u[0] = _mm_packs_epi32(v[0], v[1]);
- u[1] = _mm_packs_epi32(v[3], v[2]);
-
- // stage 2
- in[0] = _mm_add_epi16(u[0], u[1]);
- in[1] = _mm_sub_epi16(u[0], u[1]);
- in[1] = _mm_shuffle_epi32(in[1], 0x4E);
-}
-
-void vp10_iadst4_sse2(__m128i *in) {
- const __m128i k__sinpi_p01_p04 = pair_set_epi16(sinpi_1_9, sinpi_4_9);
- const __m128i k__sinpi_p03_p02 = pair_set_epi16(sinpi_3_9, sinpi_2_9);
- const __m128i k__sinpi_p02_m01 = pair_set_epi16(sinpi_2_9, -sinpi_1_9);
- const __m128i k__sinpi_p03_m04 = pair_set_epi16(sinpi_3_9, -sinpi_4_9);
- const __m128i k__sinpi_p03_p03 = _mm_set1_epi16((int16_t)sinpi_3_9);
- const __m128i kZero = _mm_set1_epi16(0);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- __m128i u[8], v[8], in7;
-
- transpose_4x4(in);
- in7 = _mm_srli_si128(in[1], 8);
- in7 = _mm_add_epi16(in7, in[0]);
- in7 = _mm_sub_epi16(in7, in[1]);
-
- u[0] = _mm_unpacklo_epi16(in[0], in[1]);
- u[1] = _mm_unpackhi_epi16(in[0], in[1]);
- u[2] = _mm_unpacklo_epi16(in7, kZero);
- u[3] = _mm_unpackhi_epi16(in[0], kZero);
-
- v[0] = _mm_madd_epi16(u[0], k__sinpi_p01_p04); // s0 + s3
- v[1] = _mm_madd_epi16(u[1], k__sinpi_p03_p02); // s2 + s5
- v[2] = _mm_madd_epi16(u[2], k__sinpi_p03_p03); // x2
- v[3] = _mm_madd_epi16(u[0], k__sinpi_p02_m01); // s1 - s4
- v[4] = _mm_madd_epi16(u[1], k__sinpi_p03_m04); // s2 - s6
- v[5] = _mm_madd_epi16(u[3], k__sinpi_p03_p03); // s2
-
- u[0] = _mm_add_epi32(v[0], v[1]);
- u[1] = _mm_add_epi32(v[3], v[4]);
- u[2] = v[2];
- u[3] = _mm_add_epi32(u[0], u[1]);
- u[4] = _mm_slli_epi32(v[5], 2);
- u[5] = _mm_add_epi32(u[3], v[5]);
- u[6] = _mm_sub_epi32(u[5], u[4]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
-
- in[0] = _mm_packs_epi32(u[0], u[1]);
- in[1] = _mm_packs_epi32(u[2], u[3]);
-}
-
-#define TRANSPOSE_8X8(in0, in1, in2, in3, in4, in5, in6, in7, \
- out0, out1, out2, out3, out4, out5, out6, out7) \
- { \
- const __m128i tr0_0 = _mm_unpacklo_epi16(in0, in1); \
- const __m128i tr0_1 = _mm_unpacklo_epi16(in2, in3); \
- const __m128i tr0_2 = _mm_unpackhi_epi16(in0, in1); \
- const __m128i tr0_3 = _mm_unpackhi_epi16(in2, in3); \
- const __m128i tr0_4 = _mm_unpacklo_epi16(in4, in5); \
- const __m128i tr0_5 = _mm_unpacklo_epi16(in6, in7); \
- const __m128i tr0_6 = _mm_unpackhi_epi16(in4, in5); \
- const __m128i tr0_7 = _mm_unpackhi_epi16(in6, in7); \
- \
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1); \
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_2, tr0_3); \
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1); \
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_2, tr0_3); \
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5); \
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7); \
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5); \
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7); \
- \
- out0 = _mm_unpacklo_epi64(tr1_0, tr1_4); \
- out1 = _mm_unpackhi_epi64(tr1_0, tr1_4); \
- out2 = _mm_unpacklo_epi64(tr1_2, tr1_6); \
- out3 = _mm_unpackhi_epi64(tr1_2, tr1_6); \
- out4 = _mm_unpacklo_epi64(tr1_1, tr1_5); \
- out5 = _mm_unpackhi_epi64(tr1_1, tr1_5); \
- out6 = _mm_unpacklo_epi64(tr1_3, tr1_7); \
- out7 = _mm_unpackhi_epi64(tr1_3, tr1_7); \
- }
-
-#define TRANSPOSE_4X8_10(tmp0, tmp1, tmp2, tmp3, \
- out0, out1, out2, out3) \
- { \
- const __m128i tr0_0 = _mm_unpackhi_epi16(tmp0, tmp1); \
- const __m128i tr0_1 = _mm_unpacklo_epi16(tmp1, tmp0); \
- const __m128i tr0_4 = _mm_unpacklo_epi16(tmp2, tmp3); \
- const __m128i tr0_5 = _mm_unpackhi_epi16(tmp3, tmp2); \
- \
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1); \
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1); \
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5); \
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5); \
- \
- out0 = _mm_unpacklo_epi64(tr1_0, tr1_4); \
- out1 = _mm_unpackhi_epi64(tr1_0, tr1_4); \
- out2 = _mm_unpacklo_epi64(tr1_2, tr1_6); \
- out3 = _mm_unpackhi_epi64(tr1_2, tr1_6); \
- }
-
-#define TRANSPOSE_8X8_10(in0, in1, in2, in3, out0, out1) \
- { \
- const __m128i tr0_0 = _mm_unpacklo_epi16(in0, in1); \
- const __m128i tr0_1 = _mm_unpacklo_epi16(in2, in3); \
- out0 = _mm_unpacklo_epi32(tr0_0, tr0_1); \
- out1 = _mm_unpackhi_epi32(tr0_0, tr0_1); \
- }
-
-// Define Macro for multiplying elements by constants and adding them together.
-#define MULTIPLICATION_AND_ADD(lo_0, hi_0, lo_1, hi_1, \
- cst0, cst1, cst2, cst3, res0, res1, res2, res3) \
- { \
- tmp0 = _mm_madd_epi16(lo_0, cst0); \
- tmp1 = _mm_madd_epi16(hi_0, cst0); \
- tmp2 = _mm_madd_epi16(lo_0, cst1); \
- tmp3 = _mm_madd_epi16(hi_0, cst1); \
- tmp4 = _mm_madd_epi16(lo_1, cst2); \
- tmp5 = _mm_madd_epi16(hi_1, cst2); \
- tmp6 = _mm_madd_epi16(lo_1, cst3); \
- tmp7 = _mm_madd_epi16(hi_1, cst3); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- tmp4 = _mm_add_epi32(tmp4, rounding); \
- tmp5 = _mm_add_epi32(tmp5, rounding); \
- tmp6 = _mm_add_epi32(tmp6, rounding); \
- tmp7 = _mm_add_epi32(tmp7, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- tmp4 = _mm_srai_epi32(tmp4, DCT_CONST_BITS); \
- tmp5 = _mm_srai_epi32(tmp5, DCT_CONST_BITS); \
- tmp6 = _mm_srai_epi32(tmp6, DCT_CONST_BITS); \
- tmp7 = _mm_srai_epi32(tmp7, DCT_CONST_BITS); \
- \
- res0 = _mm_packs_epi32(tmp0, tmp1); \
- res1 = _mm_packs_epi32(tmp2, tmp3); \
- res2 = _mm_packs_epi32(tmp4, tmp5); \
- res3 = _mm_packs_epi32(tmp6, tmp7); \
- }
-
-#define MULTIPLICATION_AND_ADD_2(lo_0, hi_0, cst0, cst1, res0, res1) \
- { \
- tmp0 = _mm_madd_epi16(lo_0, cst0); \
- tmp1 = _mm_madd_epi16(hi_0, cst0); \
- tmp2 = _mm_madd_epi16(lo_0, cst1); \
- tmp3 = _mm_madd_epi16(hi_0, cst1); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- \
- res0 = _mm_packs_epi32(tmp0, tmp1); \
- res1 = _mm_packs_epi32(tmp2, tmp3); \
- }
-
-#define IDCT8(in0, in1, in2, in3, in4, in5, in6, in7, \
- out0, out1, out2, out3, out4, out5, out6, out7) \
- { \
- /* Stage1 */ \
- { \
- const __m128i lo_17 = _mm_unpacklo_epi16(in1, in7); \
- const __m128i hi_17 = _mm_unpackhi_epi16(in1, in7); \
- const __m128i lo_35 = _mm_unpacklo_epi16(in3, in5); \
- const __m128i hi_35 = _mm_unpackhi_epi16(in3, in5); \
- \
- MULTIPLICATION_AND_ADD(lo_17, hi_17, lo_35, hi_35, stg1_0, \
- stg1_1, stg1_2, stg1_3, stp1_4, \
- stp1_7, stp1_5, stp1_6) \
- } \
- \
- /* Stage2 */ \
- { \
- const __m128i lo_04 = _mm_unpacklo_epi16(in0, in4); \
- const __m128i hi_04 = _mm_unpackhi_epi16(in0, in4); \
- const __m128i lo_26 = _mm_unpacklo_epi16(in2, in6); \
- const __m128i hi_26 = _mm_unpackhi_epi16(in2, in6); \
- \
- MULTIPLICATION_AND_ADD(lo_04, hi_04, lo_26, hi_26, stg2_0, \
- stg2_1, stg2_2, stg2_3, stp2_0, \
- stp2_1, stp2_2, stp2_3) \
- \
- stp2_4 = _mm_adds_epi16(stp1_4, stp1_5); \
- stp2_5 = _mm_subs_epi16(stp1_4, stp1_5); \
- stp2_6 = _mm_subs_epi16(stp1_7, stp1_6); \
- stp2_7 = _mm_adds_epi16(stp1_7, stp1_6); \
- } \
- \
- /* Stage3 */ \
- { \
- const __m128i lo_56 = _mm_unpacklo_epi16(stp2_6, stp2_5); \
- const __m128i hi_56 = _mm_unpackhi_epi16(stp2_6, stp2_5); \
- \
- stp1_0 = _mm_adds_epi16(stp2_0, stp2_3); \
- stp1_1 = _mm_adds_epi16(stp2_1, stp2_2); \
- stp1_2 = _mm_subs_epi16(stp2_1, stp2_2); \
- stp1_3 = _mm_subs_epi16(stp2_0, stp2_3); \
- \
- tmp0 = _mm_madd_epi16(lo_56, stg2_1); \
- tmp1 = _mm_madd_epi16(hi_56, stg2_1); \
- tmp2 = _mm_madd_epi16(lo_56, stg2_0); \
- tmp3 = _mm_madd_epi16(hi_56, stg2_0); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- \
- stp1_5 = _mm_packs_epi32(tmp0, tmp1); \
- stp1_6 = _mm_packs_epi32(tmp2, tmp3); \
- } \
- \
- /* Stage4 */ \
- out0 = _mm_adds_epi16(stp1_0, stp2_7); \
- out1 = _mm_adds_epi16(stp1_1, stp1_6); \
- out2 = _mm_adds_epi16(stp1_2, stp1_5); \
- out3 = _mm_adds_epi16(stp1_3, stp2_4); \
- out4 = _mm_subs_epi16(stp1_3, stp2_4); \
- out5 = _mm_subs_epi16(stp1_2, stp1_5); \
- out6 = _mm_subs_epi16(stp1_1, stp1_6); \
- out7 = _mm_subs_epi16(stp1_0, stp2_7); \
- }
-
-void vp10_idct8x8_64_add_sse2(const int16_t *input, uint8_t *dest, int stride) {
- const __m128i zero = _mm_setzero_si128();
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i final_rounding = _mm_set1_epi16(1 << 4);
- const __m128i stg1_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg1_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i stg1_2 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i stg1_3 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i stg2_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg2_2 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i stg2_3 = pair_set_epi16(cospi_8_64, cospi_24_64);
-
- __m128i in0, in1, in2, in3, in4, in5, in6, in7;
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6, stp1_7;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
- int i;
-
- // Load input data.
- in0 = _mm_load_si128((const __m128i *)input);
- in1 = _mm_load_si128((const __m128i *)(input + 8 * 1));
- in2 = _mm_load_si128((const __m128i *)(input + 8 * 2));
- in3 = _mm_load_si128((const __m128i *)(input + 8 * 3));
- in4 = _mm_load_si128((const __m128i *)(input + 8 * 4));
- in5 = _mm_load_si128((const __m128i *)(input + 8 * 5));
- in6 = _mm_load_si128((const __m128i *)(input + 8 * 6));
- in7 = _mm_load_si128((const __m128i *)(input + 8 * 7));
-
- // 2-D
- for (i = 0; i < 2; i++) {
- // 8x8 Transpose is copied from vp10_fdct8x8_sse2()
- TRANSPOSE_8X8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
-
- // 4-stage 1D vp10_idct8x8
- IDCT8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- }
-
- // Final rounding and shift
- in0 = _mm_adds_epi16(in0, final_rounding);
- in1 = _mm_adds_epi16(in1, final_rounding);
- in2 = _mm_adds_epi16(in2, final_rounding);
- in3 = _mm_adds_epi16(in3, final_rounding);
- in4 = _mm_adds_epi16(in4, final_rounding);
- in5 = _mm_adds_epi16(in5, final_rounding);
- in6 = _mm_adds_epi16(in6, final_rounding);
- in7 = _mm_adds_epi16(in7, final_rounding);
-
- in0 = _mm_srai_epi16(in0, 5);
- in1 = _mm_srai_epi16(in1, 5);
- in2 = _mm_srai_epi16(in2, 5);
- in3 = _mm_srai_epi16(in3, 5);
- in4 = _mm_srai_epi16(in4, 5);
- in5 = _mm_srai_epi16(in5, 5);
- in6 = _mm_srai_epi16(in6, 5);
- in7 = _mm_srai_epi16(in7, 5);
-
- RECON_AND_STORE(dest + 0 * stride, in0);
- RECON_AND_STORE(dest + 1 * stride, in1);
- RECON_AND_STORE(dest + 2 * stride, in2);
- RECON_AND_STORE(dest + 3 * stride, in3);
- RECON_AND_STORE(dest + 4 * stride, in4);
- RECON_AND_STORE(dest + 5 * stride, in5);
- RECON_AND_STORE(dest + 6 * stride, in6);
- RECON_AND_STORE(dest + 7 * stride, in7);
-}
-
-void vp10_idct8x8_1_add_sse2(const int16_t *input, uint8_t *dest, int stride) {
- __m128i dc_value;
- const __m128i zero = _mm_setzero_si128();
- int a;
-
- a = dct_const_round_shift(input[0] * cospi_16_64);
- a = dct_const_round_shift(a * cospi_16_64);
- a = ROUND_POWER_OF_TWO(a, 5);
-
- dc_value = _mm_set1_epi16(a);
-
- RECON_AND_STORE(dest + 0 * stride, dc_value);
- RECON_AND_STORE(dest + 1 * stride, dc_value);
- RECON_AND_STORE(dest + 2 * stride, dc_value);
- RECON_AND_STORE(dest + 3 * stride, dc_value);
- RECON_AND_STORE(dest + 4 * stride, dc_value);
- RECON_AND_STORE(dest + 5 * stride, dc_value);
- RECON_AND_STORE(dest + 6 * stride, dc_value);
- RECON_AND_STORE(dest + 7 * stride, dc_value);
-}
-
-void vp10_idct8_sse2(__m128i *in) {
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i stg1_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg1_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i stg1_2 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i stg1_3 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i stg2_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg2_2 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i stg2_3 = pair_set_epi16(cospi_8_64, cospi_24_64);
-
- __m128i in0, in1, in2, in3, in4, in5, in6, in7;
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6, stp1_7;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
-
- // 8x8 Transpose is copied from vp10_fdct8x8_sse2()
- TRANSPOSE_8X8(in[0], in[1], in[2], in[3], in[4], in[5], in[6], in[7],
- in0, in1, in2, in3, in4, in5, in6, in7);
-
- // 4-stage 1D vp10_idct8x8
- IDCT8(in0, in1, in2, in3, in4, in5, in6, in7,
- in[0], in[1], in[2], in[3], in[4], in[5], in[6], in[7]);
-}
-
-void vp10_iadst8_sse2(__m128i *in) {
- const __m128i k__cospi_p02_p30 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i k__cospi_p30_m02 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i k__cospi_p10_p22 = pair_set_epi16(cospi_10_64, cospi_22_64);
- const __m128i k__cospi_p22_m10 = pair_set_epi16(cospi_22_64, -cospi_10_64);
- const __m128i k__cospi_p18_p14 = pair_set_epi16(cospi_18_64, cospi_14_64);
- const __m128i k__cospi_p14_m18 = pair_set_epi16(cospi_14_64, -cospi_18_64);
- const __m128i k__cospi_p26_p06 = pair_set_epi16(cospi_26_64, cospi_6_64);
- const __m128i k__cospi_p06_m26 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_m24_p08 = pair_set_epi16(-cospi_24_64, cospi_8_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__const_0 = _mm_set1_epi16(0);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
-
- __m128i u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, u10, u11, u12, u13, u14, u15;
- __m128i v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15;
- __m128i w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15;
- __m128i s0, s1, s2, s3, s4, s5, s6, s7;
- __m128i in0, in1, in2, in3, in4, in5, in6, in7;
-
- // transpose
- array_transpose_8x8(in, in);
-
- // properly aligned for butterfly input
- in0 = in[7];
- in1 = in[0];
- in2 = in[5];
- in3 = in[2];
- in4 = in[3];
- in5 = in[4];
- in6 = in[1];
- in7 = in[6];
-
- // column transformation
- // stage 1
- // interleave and multiply/add into 32-bit integer
- s0 = _mm_unpacklo_epi16(in0, in1);
- s1 = _mm_unpackhi_epi16(in0, in1);
- s2 = _mm_unpacklo_epi16(in2, in3);
- s3 = _mm_unpackhi_epi16(in2, in3);
- s4 = _mm_unpacklo_epi16(in4, in5);
- s5 = _mm_unpackhi_epi16(in4, in5);
- s6 = _mm_unpacklo_epi16(in6, in7);
- s7 = _mm_unpackhi_epi16(in6, in7);
-
- u0 = _mm_madd_epi16(s0, k__cospi_p02_p30);
- u1 = _mm_madd_epi16(s1, k__cospi_p02_p30);
- u2 = _mm_madd_epi16(s0, k__cospi_p30_m02);
- u3 = _mm_madd_epi16(s1, k__cospi_p30_m02);
- u4 = _mm_madd_epi16(s2, k__cospi_p10_p22);
- u5 = _mm_madd_epi16(s3, k__cospi_p10_p22);
- u6 = _mm_madd_epi16(s2, k__cospi_p22_m10);
- u7 = _mm_madd_epi16(s3, k__cospi_p22_m10);
- u8 = _mm_madd_epi16(s4, k__cospi_p18_p14);
- u9 = _mm_madd_epi16(s5, k__cospi_p18_p14);
- u10 = _mm_madd_epi16(s4, k__cospi_p14_m18);
- u11 = _mm_madd_epi16(s5, k__cospi_p14_m18);
- u12 = _mm_madd_epi16(s6, k__cospi_p26_p06);
- u13 = _mm_madd_epi16(s7, k__cospi_p26_p06);
- u14 = _mm_madd_epi16(s6, k__cospi_p06_m26);
- u15 = _mm_madd_epi16(s7, k__cospi_p06_m26);
-
- // addition
- w0 = _mm_add_epi32(u0, u8);
- w1 = _mm_add_epi32(u1, u9);
- w2 = _mm_add_epi32(u2, u10);
- w3 = _mm_add_epi32(u3, u11);
- w4 = _mm_add_epi32(u4, u12);
- w5 = _mm_add_epi32(u5, u13);
- w6 = _mm_add_epi32(u6, u14);
- w7 = _mm_add_epi32(u7, u15);
- w8 = _mm_sub_epi32(u0, u8);
- w9 = _mm_sub_epi32(u1, u9);
- w10 = _mm_sub_epi32(u2, u10);
- w11 = _mm_sub_epi32(u3, u11);
- w12 = _mm_sub_epi32(u4, u12);
- w13 = _mm_sub_epi32(u5, u13);
- w14 = _mm_sub_epi32(u6, u14);
- w15 = _mm_sub_epi32(u7, u15);
-
- // shift and rounding
- v0 = _mm_add_epi32(w0, k__DCT_CONST_ROUNDING);
- v1 = _mm_add_epi32(w1, k__DCT_CONST_ROUNDING);
- v2 = _mm_add_epi32(w2, k__DCT_CONST_ROUNDING);
- v3 = _mm_add_epi32(w3, k__DCT_CONST_ROUNDING);
- v4 = _mm_add_epi32(w4, k__DCT_CONST_ROUNDING);
- v5 = _mm_add_epi32(w5, k__DCT_CONST_ROUNDING);
- v6 = _mm_add_epi32(w6, k__DCT_CONST_ROUNDING);
- v7 = _mm_add_epi32(w7, k__DCT_CONST_ROUNDING);
- v8 = _mm_add_epi32(w8, k__DCT_CONST_ROUNDING);
- v9 = _mm_add_epi32(w9, k__DCT_CONST_ROUNDING);
- v10 = _mm_add_epi32(w10, k__DCT_CONST_ROUNDING);
- v11 = _mm_add_epi32(w11, k__DCT_CONST_ROUNDING);
- v12 = _mm_add_epi32(w12, k__DCT_CONST_ROUNDING);
- v13 = _mm_add_epi32(w13, k__DCT_CONST_ROUNDING);
- v14 = _mm_add_epi32(w14, k__DCT_CONST_ROUNDING);
- v15 = _mm_add_epi32(w15, k__DCT_CONST_ROUNDING);
-
- u0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- u1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- u2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- u3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- u4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- u5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- u6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- u7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- u8 = _mm_srai_epi32(v8, DCT_CONST_BITS);
- u9 = _mm_srai_epi32(v9, DCT_CONST_BITS);
- u10 = _mm_srai_epi32(v10, DCT_CONST_BITS);
- u11 = _mm_srai_epi32(v11, DCT_CONST_BITS);
- u12 = _mm_srai_epi32(v12, DCT_CONST_BITS);
- u13 = _mm_srai_epi32(v13, DCT_CONST_BITS);
- u14 = _mm_srai_epi32(v14, DCT_CONST_BITS);
- u15 = _mm_srai_epi32(v15, DCT_CONST_BITS);
-
- // back to 16-bit and pack 8 integers into __m128i
- in[0] = _mm_packs_epi32(u0, u1);
- in[1] = _mm_packs_epi32(u2, u3);
- in[2] = _mm_packs_epi32(u4, u5);
- in[3] = _mm_packs_epi32(u6, u7);
- in[4] = _mm_packs_epi32(u8, u9);
- in[5] = _mm_packs_epi32(u10, u11);
- in[6] = _mm_packs_epi32(u12, u13);
- in[7] = _mm_packs_epi32(u14, u15);
-
- // stage 2
- s0 = _mm_add_epi16(in[0], in[2]);
- s1 = _mm_add_epi16(in[1], in[3]);
- s2 = _mm_sub_epi16(in[0], in[2]);
- s3 = _mm_sub_epi16(in[1], in[3]);
- u0 = _mm_unpacklo_epi16(in[4], in[5]);
- u1 = _mm_unpackhi_epi16(in[4], in[5]);
- u2 = _mm_unpacklo_epi16(in[6], in[7]);
- u3 = _mm_unpackhi_epi16(in[6], in[7]);
-
- v0 = _mm_madd_epi16(u0, k__cospi_p08_p24);
- v1 = _mm_madd_epi16(u1, k__cospi_p08_p24);
- v2 = _mm_madd_epi16(u0, k__cospi_p24_m08);
- v3 = _mm_madd_epi16(u1, k__cospi_p24_m08);
- v4 = _mm_madd_epi16(u2, k__cospi_m24_p08);
- v5 = _mm_madd_epi16(u3, k__cospi_m24_p08);
- v6 = _mm_madd_epi16(u2, k__cospi_p08_p24);
- v7 = _mm_madd_epi16(u3, k__cospi_p08_p24);
-
- w0 = _mm_add_epi32(v0, v4);
- w1 = _mm_add_epi32(v1, v5);
- w2 = _mm_add_epi32(v2, v6);
- w3 = _mm_add_epi32(v3, v7);
- w4 = _mm_sub_epi32(v0, v4);
- w5 = _mm_sub_epi32(v1, v5);
- w6 = _mm_sub_epi32(v2, v6);
- w7 = _mm_sub_epi32(v3, v7);
-
- v0 = _mm_add_epi32(w0, k__DCT_CONST_ROUNDING);
- v1 = _mm_add_epi32(w1, k__DCT_CONST_ROUNDING);
- v2 = _mm_add_epi32(w2, k__DCT_CONST_ROUNDING);
- v3 = _mm_add_epi32(w3, k__DCT_CONST_ROUNDING);
- v4 = _mm_add_epi32(w4, k__DCT_CONST_ROUNDING);
- v5 = _mm_add_epi32(w5, k__DCT_CONST_ROUNDING);
- v6 = _mm_add_epi32(w6, k__DCT_CONST_ROUNDING);
- v7 = _mm_add_epi32(w7, k__DCT_CONST_ROUNDING);
-
- u0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- u1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- u2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- u3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- u4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- u5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- u6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- u7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
-
- // back to 16-bit intergers
- s4 = _mm_packs_epi32(u0, u1);
- s5 = _mm_packs_epi32(u2, u3);
- s6 = _mm_packs_epi32(u4, u5);
- s7 = _mm_packs_epi32(u6, u7);
-
- // stage 3
- u0 = _mm_unpacklo_epi16(s2, s3);
- u1 = _mm_unpackhi_epi16(s2, s3);
- u2 = _mm_unpacklo_epi16(s6, s7);
- u3 = _mm_unpackhi_epi16(s6, s7);
-
- v0 = _mm_madd_epi16(u0, k__cospi_p16_p16);
- v1 = _mm_madd_epi16(u1, k__cospi_p16_p16);
- v2 = _mm_madd_epi16(u0, k__cospi_p16_m16);
- v3 = _mm_madd_epi16(u1, k__cospi_p16_m16);
- v4 = _mm_madd_epi16(u2, k__cospi_p16_p16);
- v5 = _mm_madd_epi16(u3, k__cospi_p16_p16);
- v6 = _mm_madd_epi16(u2, k__cospi_p16_m16);
- v7 = _mm_madd_epi16(u3, k__cospi_p16_m16);
-
- u0 = _mm_add_epi32(v0, k__DCT_CONST_ROUNDING);
- u1 = _mm_add_epi32(v1, k__DCT_CONST_ROUNDING);
- u2 = _mm_add_epi32(v2, k__DCT_CONST_ROUNDING);
- u3 = _mm_add_epi32(v3, k__DCT_CONST_ROUNDING);
- u4 = _mm_add_epi32(v4, k__DCT_CONST_ROUNDING);
- u5 = _mm_add_epi32(v5, k__DCT_CONST_ROUNDING);
- u6 = _mm_add_epi32(v6, k__DCT_CONST_ROUNDING);
- u7 = _mm_add_epi32(v7, k__DCT_CONST_ROUNDING);
-
- v0 = _mm_srai_epi32(u0, DCT_CONST_BITS);
- v1 = _mm_srai_epi32(u1, DCT_CONST_BITS);
- v2 = _mm_srai_epi32(u2, DCT_CONST_BITS);
- v3 = _mm_srai_epi32(u3, DCT_CONST_BITS);
- v4 = _mm_srai_epi32(u4, DCT_CONST_BITS);
- v5 = _mm_srai_epi32(u5, DCT_CONST_BITS);
- v6 = _mm_srai_epi32(u6, DCT_CONST_BITS);
- v7 = _mm_srai_epi32(u7, DCT_CONST_BITS);
-
- s2 = _mm_packs_epi32(v0, v1);
- s3 = _mm_packs_epi32(v2, v3);
- s6 = _mm_packs_epi32(v4, v5);
- s7 = _mm_packs_epi32(v6, v7);
-
- in[0] = s0;
- in[1] = _mm_sub_epi16(k__const_0, s4);
- in[2] = s6;
- in[3] = _mm_sub_epi16(k__const_0, s2);
- in[4] = s3;
- in[5] = _mm_sub_epi16(k__const_0, s7);
- in[6] = s5;
- in[7] = _mm_sub_epi16(k__const_0, s1);
-}
-
-void vp10_idct8x8_12_add_sse2(const int16_t *input, uint8_t *dest, int stride) {
- const __m128i zero = _mm_setzero_si128();
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i final_rounding = _mm_set1_epi16(1 << 4);
- const __m128i stg1_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg1_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i stg1_2 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i stg1_3 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i stg2_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg2_2 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i stg2_3 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i stg3_0 = pair_set_epi16(-cospi_16_64, cospi_16_64);
-
- __m128i in0, in1, in2, in3, in4, in5, in6, in7;
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6, stp1_7;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
-
- // Rows. Load 4-row input data.
- in0 = _mm_load_si128((const __m128i *)input);
- in1 = _mm_load_si128((const __m128i *)(input + 8 * 1));
- in2 = _mm_load_si128((const __m128i *)(input + 8 * 2));
- in3 = _mm_load_si128((const __m128i *)(input + 8 * 3));
-
- // 8x4 Transpose
- TRANSPOSE_8X8_10(in0, in1, in2, in3, in0, in1);
- // Stage1
- {
- const __m128i lo_17 = _mm_unpackhi_epi16(in0, zero);
- const __m128i lo_35 = _mm_unpackhi_epi16(in1, zero);
-
- tmp0 = _mm_madd_epi16(lo_17, stg1_0);
- tmp2 = _mm_madd_epi16(lo_17, stg1_1);
- tmp4 = _mm_madd_epi16(lo_35, stg1_2);
- tmp6 = _mm_madd_epi16(lo_35, stg1_3);
-
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp4 = _mm_add_epi32(tmp4, rounding);
- tmp6 = _mm_add_epi32(tmp6, rounding);
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
- tmp4 = _mm_srai_epi32(tmp4, DCT_CONST_BITS);
- tmp6 = _mm_srai_epi32(tmp6, DCT_CONST_BITS);
-
- stp1_4 = _mm_packs_epi32(tmp0, tmp2);
- stp1_5 = _mm_packs_epi32(tmp4, tmp6);
- }
-
- // Stage2
- {
- const __m128i lo_04 = _mm_unpacklo_epi16(in0, zero);
- const __m128i lo_26 = _mm_unpacklo_epi16(in1, zero);
-
- tmp0 = _mm_madd_epi16(lo_04, stg2_0);
- tmp2 = _mm_madd_epi16(lo_04, stg2_1);
- tmp4 = _mm_madd_epi16(lo_26, stg2_2);
- tmp6 = _mm_madd_epi16(lo_26, stg2_3);
-
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp4 = _mm_add_epi32(tmp4, rounding);
- tmp6 = _mm_add_epi32(tmp6, rounding);
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
- tmp4 = _mm_srai_epi32(tmp4, DCT_CONST_BITS);
- tmp6 = _mm_srai_epi32(tmp6, DCT_CONST_BITS);
-
- stp2_0 = _mm_packs_epi32(tmp0, tmp2);
- stp2_2 = _mm_packs_epi32(tmp6, tmp4);
-
- tmp0 = _mm_adds_epi16(stp1_4, stp1_5);
- tmp1 = _mm_subs_epi16(stp1_4, stp1_5);
-
- stp2_4 = tmp0;
- stp2_5 = _mm_unpacklo_epi64(tmp1, zero);
- stp2_6 = _mm_unpackhi_epi64(tmp1, zero);
- }
-
- // Stage3
- {
- const __m128i lo_56 = _mm_unpacklo_epi16(stp2_5, stp2_6);
-
- tmp4 = _mm_adds_epi16(stp2_0, stp2_2);
- tmp6 = _mm_subs_epi16(stp2_0, stp2_2);
-
- stp1_2 = _mm_unpackhi_epi64(tmp6, tmp4);
- stp1_3 = _mm_unpacklo_epi64(tmp6, tmp4);
-
- tmp0 = _mm_madd_epi16(lo_56, stg3_0);
- tmp2 = _mm_madd_epi16(lo_56, stg2_0); // stg3_1 = stg2_0
-
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
-
- stp1_5 = _mm_packs_epi32(tmp0, tmp2);
- }
-
- // Stage4
- tmp0 = _mm_adds_epi16(stp1_3, stp2_4);
- tmp1 = _mm_adds_epi16(stp1_2, stp1_5);
- tmp2 = _mm_subs_epi16(stp1_3, stp2_4);
- tmp3 = _mm_subs_epi16(stp1_2, stp1_5);
-
- TRANSPOSE_4X8_10(tmp0, tmp1, tmp2, tmp3, in0, in1, in2, in3)
-
- IDCT8(in0, in1, in2, in3, zero, zero, zero, zero,
- in0, in1, in2, in3, in4, in5, in6, in7);
- // Final rounding and shift
- in0 = _mm_adds_epi16(in0, final_rounding);
- in1 = _mm_adds_epi16(in1, final_rounding);
- in2 = _mm_adds_epi16(in2, final_rounding);
- in3 = _mm_adds_epi16(in3, final_rounding);
- in4 = _mm_adds_epi16(in4, final_rounding);
- in5 = _mm_adds_epi16(in5, final_rounding);
- in6 = _mm_adds_epi16(in6, final_rounding);
- in7 = _mm_adds_epi16(in7, final_rounding);
-
- in0 = _mm_srai_epi16(in0, 5);
- in1 = _mm_srai_epi16(in1, 5);
- in2 = _mm_srai_epi16(in2, 5);
- in3 = _mm_srai_epi16(in3, 5);
- in4 = _mm_srai_epi16(in4, 5);
- in5 = _mm_srai_epi16(in5, 5);
- in6 = _mm_srai_epi16(in6, 5);
- in7 = _mm_srai_epi16(in7, 5);
-
- RECON_AND_STORE(dest + 0 * stride, in0);
- RECON_AND_STORE(dest + 1 * stride, in1);
- RECON_AND_STORE(dest + 2 * stride, in2);
- RECON_AND_STORE(dest + 3 * stride, in3);
- RECON_AND_STORE(dest + 4 * stride, in4);
- RECON_AND_STORE(dest + 5 * stride, in5);
- RECON_AND_STORE(dest + 6 * stride, in6);
- RECON_AND_STORE(dest + 7 * stride, in7);
-}
-
-#define IDCT16 \
- /* Stage2 */ \
- { \
- const __m128i lo_1_15 = _mm_unpacklo_epi16(in[1], in[15]); \
- const __m128i hi_1_15 = _mm_unpackhi_epi16(in[1], in[15]); \
- const __m128i lo_9_7 = _mm_unpacklo_epi16(in[9], in[7]); \
- const __m128i hi_9_7 = _mm_unpackhi_epi16(in[9], in[7]); \
- const __m128i lo_5_11 = _mm_unpacklo_epi16(in[5], in[11]); \
- const __m128i hi_5_11 = _mm_unpackhi_epi16(in[5], in[11]); \
- const __m128i lo_13_3 = _mm_unpacklo_epi16(in[13], in[3]); \
- const __m128i hi_13_3 = _mm_unpackhi_epi16(in[13], in[3]); \
- \
- MULTIPLICATION_AND_ADD(lo_1_15, hi_1_15, lo_9_7, hi_9_7, \
- stg2_0, stg2_1, stg2_2, stg2_3, \
- stp2_8, stp2_15, stp2_9, stp2_14) \
- \
- MULTIPLICATION_AND_ADD(lo_5_11, hi_5_11, lo_13_3, hi_13_3, \
- stg2_4, stg2_5, stg2_6, stg2_7, \
- stp2_10, stp2_13, stp2_11, stp2_12) \
- } \
- \
- /* Stage3 */ \
- { \
- const __m128i lo_2_14 = _mm_unpacklo_epi16(in[2], in[14]); \
- const __m128i hi_2_14 = _mm_unpackhi_epi16(in[2], in[14]); \
- const __m128i lo_10_6 = _mm_unpacklo_epi16(in[10], in[6]); \
- const __m128i hi_10_6 = _mm_unpackhi_epi16(in[10], in[6]); \
- \
- MULTIPLICATION_AND_ADD(lo_2_14, hi_2_14, lo_10_6, hi_10_6, \
- stg3_0, stg3_1, stg3_2, stg3_3, \
- stp1_4, stp1_7, stp1_5, stp1_6) \
- \
- stp1_8_0 = _mm_add_epi16(stp2_8, stp2_9); \
- stp1_9 = _mm_sub_epi16(stp2_8, stp2_9); \
- stp1_10 = _mm_sub_epi16(stp2_11, stp2_10); \
- stp1_11 = _mm_add_epi16(stp2_11, stp2_10); \
- \
- stp1_12_0 = _mm_add_epi16(stp2_12, stp2_13); \
- stp1_13 = _mm_sub_epi16(stp2_12, stp2_13); \
- stp1_14 = _mm_sub_epi16(stp2_15, stp2_14); \
- stp1_15 = _mm_add_epi16(stp2_15, stp2_14); \
- } \
- \
- /* Stage4 */ \
- { \
- const __m128i lo_0_8 = _mm_unpacklo_epi16(in[0], in[8]); \
- const __m128i hi_0_8 = _mm_unpackhi_epi16(in[0], in[8]); \
- const __m128i lo_4_12 = _mm_unpacklo_epi16(in[4], in[12]); \
- const __m128i hi_4_12 = _mm_unpackhi_epi16(in[4], in[12]); \
- \
- const __m128i lo_9_14 = _mm_unpacklo_epi16(stp1_9, stp1_14); \
- const __m128i hi_9_14 = _mm_unpackhi_epi16(stp1_9, stp1_14); \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- \
- MULTIPLICATION_AND_ADD(lo_0_8, hi_0_8, lo_4_12, hi_4_12, \
- stg4_0, stg4_1, stg4_2, stg4_3, \
- stp2_0, stp2_1, stp2_2, stp2_3) \
- \
- stp2_4 = _mm_add_epi16(stp1_4, stp1_5); \
- stp2_5 = _mm_sub_epi16(stp1_4, stp1_5); \
- stp2_6 = _mm_sub_epi16(stp1_7, stp1_6); \
- stp2_7 = _mm_add_epi16(stp1_7, stp1_6); \
- \
- MULTIPLICATION_AND_ADD(lo_9_14, hi_9_14, lo_10_13, hi_10_13, \
- stg4_4, stg4_5, stg4_6, stg4_7, \
- stp2_9, stp2_14, stp2_10, stp2_13) \
- } \
- \
- /* Stage5 */ \
- { \
- const __m128i lo_6_5 = _mm_unpacklo_epi16(stp2_6, stp2_5); \
- const __m128i hi_6_5 = _mm_unpackhi_epi16(stp2_6, stp2_5); \
- \
- stp1_0 = _mm_add_epi16(stp2_0, stp2_3); \
- stp1_1 = _mm_add_epi16(stp2_1, stp2_2); \
- stp1_2 = _mm_sub_epi16(stp2_1, stp2_2); \
- stp1_3 = _mm_sub_epi16(stp2_0, stp2_3); \
- \
- tmp0 = _mm_madd_epi16(lo_6_5, stg4_1); \
- tmp1 = _mm_madd_epi16(hi_6_5, stg4_1); \
- tmp2 = _mm_madd_epi16(lo_6_5, stg4_0); \
- tmp3 = _mm_madd_epi16(hi_6_5, stg4_0); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- \
- stp1_5 = _mm_packs_epi32(tmp0, tmp1); \
- stp1_6 = _mm_packs_epi32(tmp2, tmp3); \
- \
- stp1_8 = _mm_add_epi16(stp1_8_0, stp1_11); \
- stp1_9 = _mm_add_epi16(stp2_9, stp2_10); \
- stp1_10 = _mm_sub_epi16(stp2_9, stp2_10); \
- stp1_11 = _mm_sub_epi16(stp1_8_0, stp1_11); \
- \
- stp1_12 = _mm_sub_epi16(stp1_15, stp1_12_0); \
- stp1_13 = _mm_sub_epi16(stp2_14, stp2_13); \
- stp1_14 = _mm_add_epi16(stp2_14, stp2_13); \
- stp1_15 = _mm_add_epi16(stp1_15, stp1_12_0); \
- } \
- \
- /* Stage6 */ \
- { \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- const __m128i lo_11_12 = _mm_unpacklo_epi16(stp1_11, stp1_12); \
- const __m128i hi_11_12 = _mm_unpackhi_epi16(stp1_11, stp1_12); \
- \
- stp2_0 = _mm_add_epi16(stp1_0, stp2_7); \
- stp2_1 = _mm_add_epi16(stp1_1, stp1_6); \
- stp2_2 = _mm_add_epi16(stp1_2, stp1_5); \
- stp2_3 = _mm_add_epi16(stp1_3, stp2_4); \
- stp2_4 = _mm_sub_epi16(stp1_3, stp2_4); \
- stp2_5 = _mm_sub_epi16(stp1_2, stp1_5); \
- stp2_6 = _mm_sub_epi16(stp1_1, stp1_6); \
- stp2_7 = _mm_sub_epi16(stp1_0, stp2_7); \
- \
- MULTIPLICATION_AND_ADD(lo_10_13, hi_10_13, lo_11_12, hi_11_12, \
- stg6_0, stg4_0, stg6_0, stg4_0, \
- stp2_10, stp2_13, stp2_11, stp2_12) \
- }
-
-#define IDCT16_10 \
- /* Stage2 */ \
- { \
- const __m128i lo_1_15 = _mm_unpacklo_epi16(in[1], zero); \
- const __m128i hi_1_15 = _mm_unpackhi_epi16(in[1], zero); \
- const __m128i lo_13_3 = _mm_unpacklo_epi16(zero, in[3]); \
- const __m128i hi_13_3 = _mm_unpackhi_epi16(zero, in[3]); \
- \
- MULTIPLICATION_AND_ADD(lo_1_15, hi_1_15, lo_13_3, hi_13_3, \
- stg2_0, stg2_1, stg2_6, stg2_7, \
- stp1_8_0, stp1_15, stp1_11, stp1_12_0) \
- } \
- \
- /* Stage3 */ \
- { \
- const __m128i lo_2_14 = _mm_unpacklo_epi16(in[2], zero); \
- const __m128i hi_2_14 = _mm_unpackhi_epi16(in[2], zero); \
- \
- MULTIPLICATION_AND_ADD_2(lo_2_14, hi_2_14, \
- stg3_0, stg3_1, \
- stp2_4, stp2_7) \
- \
- stp1_9 = stp1_8_0; \
- stp1_10 = stp1_11; \
- \
- stp1_13 = stp1_12_0; \
- stp1_14 = stp1_15; \
- } \
- \
- /* Stage4 */ \
- { \
- const __m128i lo_0_8 = _mm_unpacklo_epi16(in[0], zero); \
- const __m128i hi_0_8 = _mm_unpackhi_epi16(in[0], zero); \
- \
- const __m128i lo_9_14 = _mm_unpacklo_epi16(stp1_9, stp1_14); \
- const __m128i hi_9_14 = _mm_unpackhi_epi16(stp1_9, stp1_14); \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- \
- MULTIPLICATION_AND_ADD_2(lo_0_8, hi_0_8, \
- stg4_0, stg4_1, \
- stp1_0, stp1_1) \
- stp2_5 = stp2_4; \
- stp2_6 = stp2_7; \
- \
- MULTIPLICATION_AND_ADD(lo_9_14, hi_9_14, lo_10_13, hi_10_13, \
- stg4_4, stg4_5, stg4_6, stg4_7, \
- stp2_9, stp2_14, stp2_10, stp2_13) \
- } \
- \
- /* Stage5 */ \
- { \
- const __m128i lo_6_5 = _mm_unpacklo_epi16(stp2_6, stp2_5); \
- const __m128i hi_6_5 = _mm_unpackhi_epi16(stp2_6, stp2_5); \
- \
- stp1_2 = stp1_1; \
- stp1_3 = stp1_0; \
- \
- tmp0 = _mm_madd_epi16(lo_6_5, stg4_1); \
- tmp1 = _mm_madd_epi16(hi_6_5, stg4_1); \
- tmp2 = _mm_madd_epi16(lo_6_5, stg4_0); \
- tmp3 = _mm_madd_epi16(hi_6_5, stg4_0); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- \
- stp1_5 = _mm_packs_epi32(tmp0, tmp1); \
- stp1_6 = _mm_packs_epi32(tmp2, tmp3); \
- \
- stp1_8 = _mm_add_epi16(stp1_8_0, stp1_11); \
- stp1_9 = _mm_add_epi16(stp2_9, stp2_10); \
- stp1_10 = _mm_sub_epi16(stp2_9, stp2_10); \
- stp1_11 = _mm_sub_epi16(stp1_8_0, stp1_11); \
- \
- stp1_12 = _mm_sub_epi16(stp1_15, stp1_12_0); \
- stp1_13 = _mm_sub_epi16(stp2_14, stp2_13); \
- stp1_14 = _mm_add_epi16(stp2_14, stp2_13); \
- stp1_15 = _mm_add_epi16(stp1_15, stp1_12_0); \
- } \
- \
- /* Stage6 */ \
- { \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- const __m128i lo_11_12 = _mm_unpacklo_epi16(stp1_11, stp1_12); \
- const __m128i hi_11_12 = _mm_unpackhi_epi16(stp1_11, stp1_12); \
- \
- stp2_0 = _mm_add_epi16(stp1_0, stp2_7); \
- stp2_1 = _mm_add_epi16(stp1_1, stp1_6); \
- stp2_2 = _mm_add_epi16(stp1_2, stp1_5); \
- stp2_3 = _mm_add_epi16(stp1_3, stp2_4); \
- stp2_4 = _mm_sub_epi16(stp1_3, stp2_4); \
- stp2_5 = _mm_sub_epi16(stp1_2, stp1_5); \
- stp2_6 = _mm_sub_epi16(stp1_1, stp1_6); \
- stp2_7 = _mm_sub_epi16(stp1_0, stp2_7); \
- \
- MULTIPLICATION_AND_ADD(lo_10_13, hi_10_13, lo_11_12, hi_11_12, \
- stg6_0, stg4_0, stg6_0, stg4_0, \
- stp2_10, stp2_13, stp2_11, stp2_12) \
- }
-
-void vp10_idct16x16_256_add_sse2(const int16_t *input, uint8_t *dest,
- int stride) {
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i final_rounding = _mm_set1_epi16(1 << 5);
- const __m128i zero = _mm_setzero_si128();
-
- const __m128i stg2_0 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i stg2_2 = pair_set_epi16(cospi_14_64, -cospi_18_64);
- const __m128i stg2_3 = pair_set_epi16(cospi_18_64, cospi_14_64);
- const __m128i stg2_4 = pair_set_epi16(cospi_22_64, -cospi_10_64);
- const __m128i stg2_5 = pair_set_epi16(cospi_10_64, cospi_22_64);
- const __m128i stg2_6 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i stg2_7 = pair_set_epi16(cospi_26_64, cospi_6_64);
-
- const __m128i stg3_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg3_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i stg3_2 = pair_set_epi16(cospi_12_64, -cospi_20_64);
- const __m128i stg3_3 = pair_set_epi16(cospi_20_64, cospi_12_64);
-
- const __m128i stg4_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg4_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg4_2 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i stg4_3 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i stg4_4 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i stg4_5 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i stg4_6 = pair_set_epi16(-cospi_24_64, -cospi_8_64);
- const __m128i stg4_7 = pair_set_epi16(-cospi_8_64, cospi_24_64);
-
- const __m128i stg6_0 = pair_set_epi16(-cospi_16_64, cospi_16_64);
-
- __m128i in[16], l[16], r[16], *curr1;
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6, stp1_7,
- stp1_8, stp1_9, stp1_10, stp1_11, stp1_12, stp1_13, stp1_14, stp1_15,
- stp1_8_0, stp1_12_0;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7,
- stp2_8, stp2_9, stp2_10, stp2_11, stp2_12, stp2_13, stp2_14, stp2_15;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
- int i;
-
- curr1 = l;
- for (i = 0; i < 2; i++) {
- // 1-D vp10_idct
-
- // Load input data.
- in[0] = _mm_load_si128((const __m128i *)input);
- in[8] = _mm_load_si128((const __m128i *)(input + 8 * 1));
- in[1] = _mm_load_si128((const __m128i *)(input + 8 * 2));
- in[9] = _mm_load_si128((const __m128i *)(input + 8 * 3));
- in[2] = _mm_load_si128((const __m128i *)(input + 8 * 4));
- in[10] = _mm_load_si128((const __m128i *)(input + 8 * 5));
- in[3] = _mm_load_si128((const __m128i *)(input + 8 * 6));
- in[11] = _mm_load_si128((const __m128i *)(input + 8 * 7));
- in[4] = _mm_load_si128((const __m128i *)(input + 8 * 8));
- in[12] = _mm_load_si128((const __m128i *)(input + 8 * 9));
- in[5] = _mm_load_si128((const __m128i *)(input + 8 * 10));
- in[13] = _mm_load_si128((const __m128i *)(input + 8 * 11));
- in[6] = _mm_load_si128((const __m128i *)(input + 8 * 12));
- in[14] = _mm_load_si128((const __m128i *)(input + 8 * 13));
- in[7] = _mm_load_si128((const __m128i *)(input + 8 * 14));
- in[15] = _mm_load_si128((const __m128i *)(input + 8 * 15));
-
- array_transpose_8x8(in, in);
- array_transpose_8x8(in + 8, in + 8);
-
- IDCT16
-
- // Stage7
- curr1[0] = _mm_add_epi16(stp2_0, stp1_15);
- curr1[1] = _mm_add_epi16(stp2_1, stp1_14);
- curr1[2] = _mm_add_epi16(stp2_2, stp2_13);
- curr1[3] = _mm_add_epi16(stp2_3, stp2_12);
- curr1[4] = _mm_add_epi16(stp2_4, stp2_11);
- curr1[5] = _mm_add_epi16(stp2_5, stp2_10);
- curr1[6] = _mm_add_epi16(stp2_6, stp1_9);
- curr1[7] = _mm_add_epi16(stp2_7, stp1_8);
- curr1[8] = _mm_sub_epi16(stp2_7, stp1_8);
- curr1[9] = _mm_sub_epi16(stp2_6, stp1_9);
- curr1[10] = _mm_sub_epi16(stp2_5, stp2_10);
- curr1[11] = _mm_sub_epi16(stp2_4, stp2_11);
- curr1[12] = _mm_sub_epi16(stp2_3, stp2_12);
- curr1[13] = _mm_sub_epi16(stp2_2, stp2_13);
- curr1[14] = _mm_sub_epi16(stp2_1, stp1_14);
- curr1[15] = _mm_sub_epi16(stp2_0, stp1_15);
-
- curr1 = r;
- input += 128;
- }
- for (i = 0; i < 2; i++) {
- int j;
- // 1-D vp10_idct
- array_transpose_8x8(l + i * 8, in);
- array_transpose_8x8(r + i * 8, in + 8);
-
- IDCT16
-
- // 2-D
- in[0] = _mm_add_epi16(stp2_0, stp1_15);
- in[1] = _mm_add_epi16(stp2_1, stp1_14);
- in[2] = _mm_add_epi16(stp2_2, stp2_13);
- in[3] = _mm_add_epi16(stp2_3, stp2_12);
- in[4] = _mm_add_epi16(stp2_4, stp2_11);
- in[5] = _mm_add_epi16(stp2_5, stp2_10);
- in[6] = _mm_add_epi16(stp2_6, stp1_9);
- in[7] = _mm_add_epi16(stp2_7, stp1_8);
- in[8] = _mm_sub_epi16(stp2_7, stp1_8);
- in[9] = _mm_sub_epi16(stp2_6, stp1_9);
- in[10] = _mm_sub_epi16(stp2_5, stp2_10);
- in[11] = _mm_sub_epi16(stp2_4, stp2_11);
- in[12] = _mm_sub_epi16(stp2_3, stp2_12);
- in[13] = _mm_sub_epi16(stp2_2, stp2_13);
- in[14] = _mm_sub_epi16(stp2_1, stp1_14);
- in[15] = _mm_sub_epi16(stp2_0, stp1_15);
-
- for (j = 0; j < 16; ++j) {
- // Final rounding and shift
- in[j] = _mm_adds_epi16(in[j], final_rounding);
- in[j] = _mm_srai_epi16(in[j], 6);
- RECON_AND_STORE(dest + j * stride, in[j]);
- }
-
- dest += 8;
- }
-}
-
-void vp10_idct16x16_1_add_sse2(const int16_t *input,
- uint8_t *dest,
- int stride) {
- __m128i dc_value;
- const __m128i zero = _mm_setzero_si128();
- int a, i;
-
- a = dct_const_round_shift(input[0] * cospi_16_64);
- a = dct_const_round_shift(a * cospi_16_64);
- a = ROUND_POWER_OF_TWO(a, 6);
-
- dc_value = _mm_set1_epi16(a);
-
- for (i = 0; i < 2; ++i) {
- RECON_AND_STORE(dest + 0 * stride, dc_value);
- RECON_AND_STORE(dest + 1 * stride, dc_value);
- RECON_AND_STORE(dest + 2 * stride, dc_value);
- RECON_AND_STORE(dest + 3 * stride, dc_value);
- RECON_AND_STORE(dest + 4 * stride, dc_value);
- RECON_AND_STORE(dest + 5 * stride, dc_value);
- RECON_AND_STORE(dest + 6 * stride, dc_value);
- RECON_AND_STORE(dest + 7 * stride, dc_value);
- RECON_AND_STORE(dest + 8 * stride, dc_value);
- RECON_AND_STORE(dest + 9 * stride, dc_value);
- RECON_AND_STORE(dest + 10 * stride, dc_value);
- RECON_AND_STORE(dest + 11 * stride, dc_value);
- RECON_AND_STORE(dest + 12 * stride, dc_value);
- RECON_AND_STORE(dest + 13 * stride, dc_value);
- RECON_AND_STORE(dest + 14 * stride, dc_value);
- RECON_AND_STORE(dest + 15 * stride, dc_value);
- dest += 8;
- }
-}
-
-static void vp10_iadst16_8col(__m128i *in) {
- // perform 16x16 1-D ADST for 8 columns
- __m128i s[16], x[16], u[32], v[32];
- const __m128i k__cospi_p01_p31 = pair_set_epi16(cospi_1_64, cospi_31_64);
- const __m128i k__cospi_p31_m01 = pair_set_epi16(cospi_31_64, -cospi_1_64);
- const __m128i k__cospi_p05_p27 = pair_set_epi16(cospi_5_64, cospi_27_64);
- const __m128i k__cospi_p27_m05 = pair_set_epi16(cospi_27_64, -cospi_5_64);
- const __m128i k__cospi_p09_p23 = pair_set_epi16(cospi_9_64, cospi_23_64);
- const __m128i k__cospi_p23_m09 = pair_set_epi16(cospi_23_64, -cospi_9_64);
- const __m128i k__cospi_p13_p19 = pair_set_epi16(cospi_13_64, cospi_19_64);
- const __m128i k__cospi_p19_m13 = pair_set_epi16(cospi_19_64, -cospi_13_64);
- const __m128i k__cospi_p17_p15 = pair_set_epi16(cospi_17_64, cospi_15_64);
- const __m128i k__cospi_p15_m17 = pair_set_epi16(cospi_15_64, -cospi_17_64);
- const __m128i k__cospi_p21_p11 = pair_set_epi16(cospi_21_64, cospi_11_64);
- const __m128i k__cospi_p11_m21 = pair_set_epi16(cospi_11_64, -cospi_21_64);
- const __m128i k__cospi_p25_p07 = pair_set_epi16(cospi_25_64, cospi_7_64);
- const __m128i k__cospi_p07_m25 = pair_set_epi16(cospi_7_64, -cospi_25_64);
- const __m128i k__cospi_p29_p03 = pair_set_epi16(cospi_29_64, cospi_3_64);
- const __m128i k__cospi_p03_m29 = pair_set_epi16(cospi_3_64, -cospi_29_64);
- const __m128i k__cospi_p04_p28 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p28_m04 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i k__cospi_p20_p12 = pair_set_epi16(cospi_20_64, cospi_12_64);
- const __m128i k__cospi_p12_m20 = pair_set_epi16(cospi_12_64, -cospi_20_64);
- const __m128i k__cospi_m28_p04 = pair_set_epi16(-cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m12_p20 = pair_set_epi16(-cospi_12_64, cospi_20_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_m24_p08 = pair_set_epi16(-cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m16_m16 = _mm_set1_epi16((int16_t)-cospi_16_64);
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_m16_p16 = pair_set_epi16(-cospi_16_64, cospi_16_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i kZero = _mm_set1_epi16(0);
-
- u[0] = _mm_unpacklo_epi16(in[15], in[0]);
- u[1] = _mm_unpackhi_epi16(in[15], in[0]);
- u[2] = _mm_unpacklo_epi16(in[13], in[2]);
- u[3] = _mm_unpackhi_epi16(in[13], in[2]);
- u[4] = _mm_unpacklo_epi16(in[11], in[4]);
- u[5] = _mm_unpackhi_epi16(in[11], in[4]);
- u[6] = _mm_unpacklo_epi16(in[9], in[6]);
- u[7] = _mm_unpackhi_epi16(in[9], in[6]);
- u[8] = _mm_unpacklo_epi16(in[7], in[8]);
- u[9] = _mm_unpackhi_epi16(in[7], in[8]);
- u[10] = _mm_unpacklo_epi16(in[5], in[10]);
- u[11] = _mm_unpackhi_epi16(in[5], in[10]);
- u[12] = _mm_unpacklo_epi16(in[3], in[12]);
- u[13] = _mm_unpackhi_epi16(in[3], in[12]);
- u[14] = _mm_unpacklo_epi16(in[1], in[14]);
- u[15] = _mm_unpackhi_epi16(in[1], in[14]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p01_p31);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p01_p31);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p31_m01);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p31_m01);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p05_p27);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p05_p27);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p27_m05);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p27_m05);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p09_p23);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p09_p23);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p23_m09);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p23_m09);
- v[12] = _mm_madd_epi16(u[6], k__cospi_p13_p19);
- v[13] = _mm_madd_epi16(u[7], k__cospi_p13_p19);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p19_m13);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p19_m13);
- v[16] = _mm_madd_epi16(u[8], k__cospi_p17_p15);
- v[17] = _mm_madd_epi16(u[9], k__cospi_p17_p15);
- v[18] = _mm_madd_epi16(u[8], k__cospi_p15_m17);
- v[19] = _mm_madd_epi16(u[9], k__cospi_p15_m17);
- v[20] = _mm_madd_epi16(u[10], k__cospi_p21_p11);
- v[21] = _mm_madd_epi16(u[11], k__cospi_p21_p11);
- v[22] = _mm_madd_epi16(u[10], k__cospi_p11_m21);
- v[23] = _mm_madd_epi16(u[11], k__cospi_p11_m21);
- v[24] = _mm_madd_epi16(u[12], k__cospi_p25_p07);
- v[25] = _mm_madd_epi16(u[13], k__cospi_p25_p07);
- v[26] = _mm_madd_epi16(u[12], k__cospi_p07_m25);
- v[27] = _mm_madd_epi16(u[13], k__cospi_p07_m25);
- v[28] = _mm_madd_epi16(u[14], k__cospi_p29_p03);
- v[29] = _mm_madd_epi16(u[15], k__cospi_p29_p03);
- v[30] = _mm_madd_epi16(u[14], k__cospi_p03_m29);
- v[31] = _mm_madd_epi16(u[15], k__cospi_p03_m29);
-
- u[0] = _mm_add_epi32(v[0], v[16]);
- u[1] = _mm_add_epi32(v[1], v[17]);
- u[2] = _mm_add_epi32(v[2], v[18]);
- u[3] = _mm_add_epi32(v[3], v[19]);
- u[4] = _mm_add_epi32(v[4], v[20]);
- u[5] = _mm_add_epi32(v[5], v[21]);
- u[6] = _mm_add_epi32(v[6], v[22]);
- u[7] = _mm_add_epi32(v[7], v[23]);
- u[8] = _mm_add_epi32(v[8], v[24]);
- u[9] = _mm_add_epi32(v[9], v[25]);
- u[10] = _mm_add_epi32(v[10], v[26]);
- u[11] = _mm_add_epi32(v[11], v[27]);
- u[12] = _mm_add_epi32(v[12], v[28]);
- u[13] = _mm_add_epi32(v[13], v[29]);
- u[14] = _mm_add_epi32(v[14], v[30]);
- u[15] = _mm_add_epi32(v[15], v[31]);
- u[16] = _mm_sub_epi32(v[0], v[16]);
- u[17] = _mm_sub_epi32(v[1], v[17]);
- u[18] = _mm_sub_epi32(v[2], v[18]);
- u[19] = _mm_sub_epi32(v[3], v[19]);
- u[20] = _mm_sub_epi32(v[4], v[20]);
- u[21] = _mm_sub_epi32(v[5], v[21]);
- u[22] = _mm_sub_epi32(v[6], v[22]);
- u[23] = _mm_sub_epi32(v[7], v[23]);
- u[24] = _mm_sub_epi32(v[8], v[24]);
- u[25] = _mm_sub_epi32(v[9], v[25]);
- u[26] = _mm_sub_epi32(v[10], v[26]);
- u[27] = _mm_sub_epi32(v[11], v[27]);
- u[28] = _mm_sub_epi32(v[12], v[28]);
- u[29] = _mm_sub_epi32(v[13], v[29]);
- u[30] = _mm_sub_epi32(v[14], v[30]);
- u[31] = _mm_sub_epi32(v[15], v[31]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
- v[8] = _mm_add_epi32(u[8], k__DCT_CONST_ROUNDING);
- v[9] = _mm_add_epi32(u[9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
- v[16] = _mm_add_epi32(u[16], k__DCT_CONST_ROUNDING);
- v[17] = _mm_add_epi32(u[17], k__DCT_CONST_ROUNDING);
- v[18] = _mm_add_epi32(u[18], k__DCT_CONST_ROUNDING);
- v[19] = _mm_add_epi32(u[19], k__DCT_CONST_ROUNDING);
- v[20] = _mm_add_epi32(u[20], k__DCT_CONST_ROUNDING);
- v[21] = _mm_add_epi32(u[21], k__DCT_CONST_ROUNDING);
- v[22] = _mm_add_epi32(u[22], k__DCT_CONST_ROUNDING);
- v[23] = _mm_add_epi32(u[23], k__DCT_CONST_ROUNDING);
- v[24] = _mm_add_epi32(u[24], k__DCT_CONST_ROUNDING);
- v[25] = _mm_add_epi32(u[25], k__DCT_CONST_ROUNDING);
- v[26] = _mm_add_epi32(u[26], k__DCT_CONST_ROUNDING);
- v[27] = _mm_add_epi32(u[27], k__DCT_CONST_ROUNDING);
- v[28] = _mm_add_epi32(u[28], k__DCT_CONST_ROUNDING);
- v[29] = _mm_add_epi32(u[29], k__DCT_CONST_ROUNDING);
- v[30] = _mm_add_epi32(u[30], k__DCT_CONST_ROUNDING);
- v[31] = _mm_add_epi32(u[31], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
- u[8] = _mm_srai_epi32(v[8], DCT_CONST_BITS);
- u[9] = _mm_srai_epi32(v[9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
- u[16] = _mm_srai_epi32(v[16], DCT_CONST_BITS);
- u[17] = _mm_srai_epi32(v[17], DCT_CONST_BITS);
- u[18] = _mm_srai_epi32(v[18], DCT_CONST_BITS);
- u[19] = _mm_srai_epi32(v[19], DCT_CONST_BITS);
- u[20] = _mm_srai_epi32(v[20], DCT_CONST_BITS);
- u[21] = _mm_srai_epi32(v[21], DCT_CONST_BITS);
- u[22] = _mm_srai_epi32(v[22], DCT_CONST_BITS);
- u[23] = _mm_srai_epi32(v[23], DCT_CONST_BITS);
- u[24] = _mm_srai_epi32(v[24], DCT_CONST_BITS);
- u[25] = _mm_srai_epi32(v[25], DCT_CONST_BITS);
- u[26] = _mm_srai_epi32(v[26], DCT_CONST_BITS);
- u[27] = _mm_srai_epi32(v[27], DCT_CONST_BITS);
- u[28] = _mm_srai_epi32(v[28], DCT_CONST_BITS);
- u[29] = _mm_srai_epi32(v[29], DCT_CONST_BITS);
- u[30] = _mm_srai_epi32(v[30], DCT_CONST_BITS);
- u[31] = _mm_srai_epi32(v[31], DCT_CONST_BITS);
-
- s[0] = _mm_packs_epi32(u[0], u[1]);
- s[1] = _mm_packs_epi32(u[2], u[3]);
- s[2] = _mm_packs_epi32(u[4], u[5]);
- s[3] = _mm_packs_epi32(u[6], u[7]);
- s[4] = _mm_packs_epi32(u[8], u[9]);
- s[5] = _mm_packs_epi32(u[10], u[11]);
- s[6] = _mm_packs_epi32(u[12], u[13]);
- s[7] = _mm_packs_epi32(u[14], u[15]);
- s[8] = _mm_packs_epi32(u[16], u[17]);
- s[9] = _mm_packs_epi32(u[18], u[19]);
- s[10] = _mm_packs_epi32(u[20], u[21]);
- s[11] = _mm_packs_epi32(u[22], u[23]);
- s[12] = _mm_packs_epi32(u[24], u[25]);
- s[13] = _mm_packs_epi32(u[26], u[27]);
- s[14] = _mm_packs_epi32(u[28], u[29]);
- s[15] = _mm_packs_epi32(u[30], u[31]);
-
- // stage 2
- u[0] = _mm_unpacklo_epi16(s[8], s[9]);
- u[1] = _mm_unpackhi_epi16(s[8], s[9]);
- u[2] = _mm_unpacklo_epi16(s[10], s[11]);
- u[3] = _mm_unpackhi_epi16(s[10], s[11]);
- u[4] = _mm_unpacklo_epi16(s[12], s[13]);
- u[5] = _mm_unpackhi_epi16(s[12], s[13]);
- u[6] = _mm_unpacklo_epi16(s[14], s[15]);
- u[7] = _mm_unpackhi_epi16(s[14], s[15]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p04_p28);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p04_p28);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p28_m04);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p28_m04);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p20_p12);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p20_p12);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p12_m20);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p12_m20);
- v[8] = _mm_madd_epi16(u[4], k__cospi_m28_p04);
- v[9] = _mm_madd_epi16(u[5], k__cospi_m28_p04);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p04_p28);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p04_p28);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m12_p20);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m12_p20);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p20_p12);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p20_p12);
-
- u[0] = _mm_add_epi32(v[0], v[8]);
- u[1] = _mm_add_epi32(v[1], v[9]);
- u[2] = _mm_add_epi32(v[2], v[10]);
- u[3] = _mm_add_epi32(v[3], v[11]);
- u[4] = _mm_add_epi32(v[4], v[12]);
- u[5] = _mm_add_epi32(v[5], v[13]);
- u[6] = _mm_add_epi32(v[6], v[14]);
- u[7] = _mm_add_epi32(v[7], v[15]);
- u[8] = _mm_sub_epi32(v[0], v[8]);
- u[9] = _mm_sub_epi32(v[1], v[9]);
- u[10] = _mm_sub_epi32(v[2], v[10]);
- u[11] = _mm_sub_epi32(v[3], v[11]);
- u[12] = _mm_sub_epi32(v[4], v[12]);
- u[13] = _mm_sub_epi32(v[5], v[13]);
- u[14] = _mm_sub_epi32(v[6], v[14]);
- u[15] = _mm_sub_epi32(v[7], v[15]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
- v[8] = _mm_add_epi32(u[8], k__DCT_CONST_ROUNDING);
- v[9] = _mm_add_epi32(u[9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
- u[8] = _mm_srai_epi32(v[8], DCT_CONST_BITS);
- u[9] = _mm_srai_epi32(v[9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
-
- x[0] = _mm_add_epi16(s[0], s[4]);
- x[1] = _mm_add_epi16(s[1], s[5]);
- x[2] = _mm_add_epi16(s[2], s[6]);
- x[3] = _mm_add_epi16(s[3], s[7]);
- x[4] = _mm_sub_epi16(s[0], s[4]);
- x[5] = _mm_sub_epi16(s[1], s[5]);
- x[6] = _mm_sub_epi16(s[2], s[6]);
- x[7] = _mm_sub_epi16(s[3], s[7]);
- x[8] = _mm_packs_epi32(u[0], u[1]);
- x[9] = _mm_packs_epi32(u[2], u[3]);
- x[10] = _mm_packs_epi32(u[4], u[5]);
- x[11] = _mm_packs_epi32(u[6], u[7]);
- x[12] = _mm_packs_epi32(u[8], u[9]);
- x[13] = _mm_packs_epi32(u[10], u[11]);
- x[14] = _mm_packs_epi32(u[12], u[13]);
- x[15] = _mm_packs_epi32(u[14], u[15]);
-
- // stage 3
- u[0] = _mm_unpacklo_epi16(x[4], x[5]);
- u[1] = _mm_unpackhi_epi16(x[4], x[5]);
- u[2] = _mm_unpacklo_epi16(x[6], x[7]);
- u[3] = _mm_unpackhi_epi16(x[6], x[7]);
- u[4] = _mm_unpacklo_epi16(x[12], x[13]);
- u[5] = _mm_unpackhi_epi16(x[12], x[13]);
- u[6] = _mm_unpacklo_epi16(x[14], x[15]);
- u[7] = _mm_unpackhi_epi16(x[14], x[15]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p08_p24);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p08_p24);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p24_m08);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p24_m08);
- v[4] = _mm_madd_epi16(u[2], k__cospi_m24_p08);
- v[5] = _mm_madd_epi16(u[3], k__cospi_m24_p08);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p08_p24);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p08_p24);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p08_p24);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p08_p24);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p24_m08);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p24_m08);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m24_p08);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m24_p08);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p08_p24);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p08_p24);
-
- u[0] = _mm_add_epi32(v[0], v[4]);
- u[1] = _mm_add_epi32(v[1], v[5]);
- u[2] = _mm_add_epi32(v[2], v[6]);
- u[3] = _mm_add_epi32(v[3], v[7]);
- u[4] = _mm_sub_epi32(v[0], v[4]);
- u[5] = _mm_sub_epi32(v[1], v[5]);
- u[6] = _mm_sub_epi32(v[2], v[6]);
- u[7] = _mm_sub_epi32(v[3], v[7]);
- u[8] = _mm_add_epi32(v[8], v[12]);
- u[9] = _mm_add_epi32(v[9], v[13]);
- u[10] = _mm_add_epi32(v[10], v[14]);
- u[11] = _mm_add_epi32(v[11], v[15]);
- u[12] = _mm_sub_epi32(v[8], v[12]);
- u[13] = _mm_sub_epi32(v[9], v[13]);
- u[14] = _mm_sub_epi32(v[10], v[14]);
- u[15] = _mm_sub_epi32(v[11], v[15]);
-
- u[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(u[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(u[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- v[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- v[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- v[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- v[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- v[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- v[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- v[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- v[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- s[0] = _mm_add_epi16(x[0], x[2]);
- s[1] = _mm_add_epi16(x[1], x[3]);
- s[2] = _mm_sub_epi16(x[0], x[2]);
- s[3] = _mm_sub_epi16(x[1], x[3]);
- s[4] = _mm_packs_epi32(v[0], v[1]);
- s[5] = _mm_packs_epi32(v[2], v[3]);
- s[6] = _mm_packs_epi32(v[4], v[5]);
- s[7] = _mm_packs_epi32(v[6], v[7]);
- s[8] = _mm_add_epi16(x[8], x[10]);
- s[9] = _mm_add_epi16(x[9], x[11]);
- s[10] = _mm_sub_epi16(x[8], x[10]);
- s[11] = _mm_sub_epi16(x[9], x[11]);
- s[12] = _mm_packs_epi32(v[8], v[9]);
- s[13] = _mm_packs_epi32(v[10], v[11]);
- s[14] = _mm_packs_epi32(v[12], v[13]);
- s[15] = _mm_packs_epi32(v[14], v[15]);
-
- // stage 4
- u[0] = _mm_unpacklo_epi16(s[2], s[3]);
- u[1] = _mm_unpackhi_epi16(s[2], s[3]);
- u[2] = _mm_unpacklo_epi16(s[6], s[7]);
- u[3] = _mm_unpackhi_epi16(s[6], s[7]);
- u[4] = _mm_unpacklo_epi16(s[10], s[11]);
- u[5] = _mm_unpackhi_epi16(s[10], s[11]);
- u[6] = _mm_unpacklo_epi16(s[14], s[15]);
- u[7] = _mm_unpackhi_epi16(s[14], s[15]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_m16_m16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m16_m16);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p16_m16);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p16_m16);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p16_p16);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p16_p16);
- v[6] = _mm_madd_epi16(u[2], k__cospi_m16_p16);
- v[7] = _mm_madd_epi16(u[3], k__cospi_m16_p16);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p16_p16);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p16_p16);
- v[10] = _mm_madd_epi16(u[4], k__cospi_m16_p16);
- v[11] = _mm_madd_epi16(u[5], k__cospi_m16_p16);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m16_m16);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m16_m16);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p16_m16);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p16_m16);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(v[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(v[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(v[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(v[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(v[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(v[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(v[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(v[15], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- v[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- v[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- v[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- v[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- v[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- v[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- v[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- v[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- in[0] = s[0];
- in[1] = _mm_sub_epi16(kZero, s[8]);
- in[2] = s[12];
- in[3] = _mm_sub_epi16(kZero, s[4]);
- in[4] = _mm_packs_epi32(v[4], v[5]);
- in[5] = _mm_packs_epi32(v[12], v[13]);
- in[6] = _mm_packs_epi32(v[8], v[9]);
- in[7] = _mm_packs_epi32(v[0], v[1]);
- in[8] = _mm_packs_epi32(v[2], v[3]);
- in[9] = _mm_packs_epi32(v[10], v[11]);
- in[10] = _mm_packs_epi32(v[14], v[15]);
- in[11] = _mm_packs_epi32(v[6], v[7]);
- in[12] = s[5];
- in[13] = _mm_sub_epi16(kZero, s[13]);
- in[14] = s[9];
- in[15] = _mm_sub_epi16(kZero, s[1]);
-}
-
-static void vp10_idct16_8col(__m128i *in) {
- const __m128i k__cospi_p30_m02 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i k__cospi_p02_p30 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i k__cospi_p14_m18 = pair_set_epi16(cospi_14_64, -cospi_18_64);
- const __m128i k__cospi_p18_p14 = pair_set_epi16(cospi_18_64, cospi_14_64);
- const __m128i k__cospi_p22_m10 = pair_set_epi16(cospi_22_64, -cospi_10_64);
- const __m128i k__cospi_p10_p22 = pair_set_epi16(cospi_10_64, cospi_22_64);
- const __m128i k__cospi_p06_m26 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i k__cospi_p26_p06 = pair_set_epi16(cospi_26_64, cospi_6_64);
- const __m128i k__cospi_p28_m04 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i k__cospi_p04_p28 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_m20 = pair_set_epi16(cospi_12_64, -cospi_20_64);
- const __m128i k__cospi_p20_p12 = pair_set_epi16(cospi_20_64, cospi_12_64);
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m24_m08 = pair_set_epi16(-cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_m16_p16 = pair_set_epi16(-cospi_16_64, cospi_16_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- __m128i v[16], u[16], s[16], t[16];
-
- // stage 1
- s[0] = in[0];
- s[1] = in[8];
- s[2] = in[4];
- s[3] = in[12];
- s[4] = in[2];
- s[5] = in[10];
- s[6] = in[6];
- s[7] = in[14];
- s[8] = in[1];
- s[9] = in[9];
- s[10] = in[5];
- s[11] = in[13];
- s[12] = in[3];
- s[13] = in[11];
- s[14] = in[7];
- s[15] = in[15];
-
- // stage 2
- u[0] = _mm_unpacklo_epi16(s[8], s[15]);
- u[1] = _mm_unpackhi_epi16(s[8], s[15]);
- u[2] = _mm_unpacklo_epi16(s[9], s[14]);
- u[3] = _mm_unpackhi_epi16(s[9], s[14]);
- u[4] = _mm_unpacklo_epi16(s[10], s[13]);
- u[5] = _mm_unpackhi_epi16(s[10], s[13]);
- u[6] = _mm_unpacklo_epi16(s[11], s[12]);
- u[7] = _mm_unpackhi_epi16(s[11], s[12]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p30_m02);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p30_m02);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p02_p30);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p02_p30);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p14_m18);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p14_m18);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p18_p14);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p18_p14);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p22_m10);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p22_m10);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p10_p22);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p10_p22);
- v[12] = _mm_madd_epi16(u[6], k__cospi_p06_m26);
- v[13] = _mm_madd_epi16(u[7], k__cospi_p06_m26);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p26_p06);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p26_p06);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(v[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(v[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(v[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(v[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(v[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(v[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(v[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(v[15], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- u[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- u[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- s[8] = _mm_packs_epi32(u[0], u[1]);
- s[15] = _mm_packs_epi32(u[2], u[3]);
- s[9] = _mm_packs_epi32(u[4], u[5]);
- s[14] = _mm_packs_epi32(u[6], u[7]);
- s[10] = _mm_packs_epi32(u[8], u[9]);
- s[13] = _mm_packs_epi32(u[10], u[11]);
- s[11] = _mm_packs_epi32(u[12], u[13]);
- s[12] = _mm_packs_epi32(u[14], u[15]);
-
- // stage 3
- t[0] = s[0];
- t[1] = s[1];
- t[2] = s[2];
- t[3] = s[3];
- u[0] = _mm_unpacklo_epi16(s[4], s[7]);
- u[1] = _mm_unpackhi_epi16(s[4], s[7]);
- u[2] = _mm_unpacklo_epi16(s[5], s[6]);
- u[3] = _mm_unpackhi_epi16(s[5], s[6]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p28_m04);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p28_m04);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p04_p28);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p04_p28);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p12_m20);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p12_m20);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p20_p12);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p20_p12);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
-
- t[4] = _mm_packs_epi32(u[0], u[1]);
- t[7] = _mm_packs_epi32(u[2], u[3]);
- t[5] = _mm_packs_epi32(u[4], u[5]);
- t[6] = _mm_packs_epi32(u[6], u[7]);
- t[8] = _mm_add_epi16(s[8], s[9]);
- t[9] = _mm_sub_epi16(s[8], s[9]);
- t[10] = _mm_sub_epi16(s[11], s[10]);
- t[11] = _mm_add_epi16(s[10], s[11]);
- t[12] = _mm_add_epi16(s[12], s[13]);
- t[13] = _mm_sub_epi16(s[12], s[13]);
- t[14] = _mm_sub_epi16(s[15], s[14]);
- t[15] = _mm_add_epi16(s[14], s[15]);
-
- // stage 4
- u[0] = _mm_unpacklo_epi16(t[0], t[1]);
- u[1] = _mm_unpackhi_epi16(t[0], t[1]);
- u[2] = _mm_unpacklo_epi16(t[2], t[3]);
- u[3] = _mm_unpackhi_epi16(t[2], t[3]);
- u[4] = _mm_unpacklo_epi16(t[9], t[14]);
- u[5] = _mm_unpackhi_epi16(t[9], t[14]);
- u[6] = _mm_unpacklo_epi16(t[10], t[13]);
- u[7] = _mm_unpackhi_epi16(t[10], t[13]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p16_p16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p16_p16);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p16_m16);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p16_m16);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p24_m08);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p24_m08);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p08_p24);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p08_p24);
- v[8] = _mm_madd_epi16(u[4], k__cospi_m08_p24);
- v[9] = _mm_madd_epi16(u[5], k__cospi_m08_p24);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p24_p08);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p24_p08);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m24_m08);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m24_m08);
- v[14] = _mm_madd_epi16(u[6], k__cospi_m08_p24);
- v[15] = _mm_madd_epi16(u[7], k__cospi_m08_p24);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(v[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(v[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(v[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(v[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(v[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(v[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(v[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(v[15], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- u[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- u[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- s[0] = _mm_packs_epi32(u[0], u[1]);
- s[1] = _mm_packs_epi32(u[2], u[3]);
- s[2] = _mm_packs_epi32(u[4], u[5]);
- s[3] = _mm_packs_epi32(u[6], u[7]);
- s[4] = _mm_add_epi16(t[4], t[5]);
- s[5] = _mm_sub_epi16(t[4], t[5]);
- s[6] = _mm_sub_epi16(t[7], t[6]);
- s[7] = _mm_add_epi16(t[6], t[7]);
- s[8] = t[8];
- s[15] = t[15];
- s[9] = _mm_packs_epi32(u[8], u[9]);
- s[14] = _mm_packs_epi32(u[10], u[11]);
- s[10] = _mm_packs_epi32(u[12], u[13]);
- s[13] = _mm_packs_epi32(u[14], u[15]);
- s[11] = t[11];
- s[12] = t[12];
-
- // stage 5
- t[0] = _mm_add_epi16(s[0], s[3]);
- t[1] = _mm_add_epi16(s[1], s[2]);
- t[2] = _mm_sub_epi16(s[1], s[2]);
- t[3] = _mm_sub_epi16(s[0], s[3]);
- t[4] = s[4];
- t[7] = s[7];
-
- u[0] = _mm_unpacklo_epi16(s[5], s[6]);
- u[1] = _mm_unpackhi_epi16(s[5], s[6]);
- v[0] = _mm_madd_epi16(u[0], k__cospi_m16_p16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m16_p16);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p16_p16);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p16_p16);
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- t[5] = _mm_packs_epi32(u[0], u[1]);
- t[6] = _mm_packs_epi32(u[2], u[3]);
-
- t[8] = _mm_add_epi16(s[8], s[11]);
- t[9] = _mm_add_epi16(s[9], s[10]);
- t[10] = _mm_sub_epi16(s[9], s[10]);
- t[11] = _mm_sub_epi16(s[8], s[11]);
- t[12] = _mm_sub_epi16(s[15], s[12]);
- t[13] = _mm_sub_epi16(s[14], s[13]);
- t[14] = _mm_add_epi16(s[13], s[14]);
- t[15] = _mm_add_epi16(s[12], s[15]);
-
- // stage 6
- s[0] = _mm_add_epi16(t[0], t[7]);
- s[1] = _mm_add_epi16(t[1], t[6]);
- s[2] = _mm_add_epi16(t[2], t[5]);
- s[3] = _mm_add_epi16(t[3], t[4]);
- s[4] = _mm_sub_epi16(t[3], t[4]);
- s[5] = _mm_sub_epi16(t[2], t[5]);
- s[6] = _mm_sub_epi16(t[1], t[6]);
- s[7] = _mm_sub_epi16(t[0], t[7]);
- s[8] = t[8];
- s[9] = t[9];
-
- u[0] = _mm_unpacklo_epi16(t[10], t[13]);
- u[1] = _mm_unpackhi_epi16(t[10], t[13]);
- u[2] = _mm_unpacklo_epi16(t[11], t[12]);
- u[3] = _mm_unpackhi_epi16(t[11], t[12]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_m16_p16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m16_p16);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p16_p16);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p16_p16);
- v[4] = _mm_madd_epi16(u[2], k__cospi_m16_p16);
- v[5] = _mm_madd_epi16(u[3], k__cospi_m16_p16);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p16_p16);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p16_p16);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
-
- s[10] = _mm_packs_epi32(u[0], u[1]);
- s[13] = _mm_packs_epi32(u[2], u[3]);
- s[11] = _mm_packs_epi32(u[4], u[5]);
- s[12] = _mm_packs_epi32(u[6], u[7]);
- s[14] = t[14];
- s[15] = t[15];
-
- // stage 7
- in[0] = _mm_add_epi16(s[0], s[15]);
- in[1] = _mm_add_epi16(s[1], s[14]);
- in[2] = _mm_add_epi16(s[2], s[13]);
- in[3] = _mm_add_epi16(s[3], s[12]);
- in[4] = _mm_add_epi16(s[4], s[11]);
- in[5] = _mm_add_epi16(s[5], s[10]);
- in[6] = _mm_add_epi16(s[6], s[9]);
- in[7] = _mm_add_epi16(s[7], s[8]);
- in[8] = _mm_sub_epi16(s[7], s[8]);
- in[9] = _mm_sub_epi16(s[6], s[9]);
- in[10] = _mm_sub_epi16(s[5], s[10]);
- in[11] = _mm_sub_epi16(s[4], s[11]);
- in[12] = _mm_sub_epi16(s[3], s[12]);
- in[13] = _mm_sub_epi16(s[2], s[13]);
- in[14] = _mm_sub_epi16(s[1], s[14]);
- in[15] = _mm_sub_epi16(s[0], s[15]);
-}
-
-void vp10_idct16_sse2(__m128i *in0, __m128i *in1) {
- array_transpose_16x16(in0, in1);
- vp10_idct16_8col(in0);
- vp10_idct16_8col(in1);
-}
-
-void vp10_iadst16_sse2(__m128i *in0, __m128i *in1) {
- array_transpose_16x16(in0, in1);
- vp10_iadst16_8col(in0);
- vp10_iadst16_8col(in1);
-}
-
-void vp10_idct16x16_10_add_sse2(const int16_t *input, uint8_t *dest,
- int stride) {
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i final_rounding = _mm_set1_epi16(1 << 5);
- const __m128i zero = _mm_setzero_si128();
-
- const __m128i stg2_0 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i stg2_6 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i stg2_7 = pair_set_epi16(cospi_26_64, cospi_6_64);
-
- const __m128i stg3_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg3_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
-
- const __m128i stg4_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg4_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg4_4 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i stg4_5 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i stg4_6 = pair_set_epi16(-cospi_24_64, -cospi_8_64);
- const __m128i stg4_7 = pair_set_epi16(-cospi_8_64, cospi_24_64);
-
- const __m128i stg6_0 = pair_set_epi16(-cospi_16_64, cospi_16_64);
- __m128i in[16], l[16];
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6,
- stp1_8, stp1_9, stp1_10, stp1_11, stp1_12, stp1_13, stp1_14, stp1_15,
- stp1_8_0, stp1_12_0;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7,
- stp2_8, stp2_9, stp2_10, stp2_11, stp2_12, stp2_13, stp2_14;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
- int i;
- // First 1-D inverse DCT
- // Load input data.
- in[0] = _mm_load_si128((const __m128i *)input);
- in[1] = _mm_load_si128((const __m128i *)(input + 8 * 2));
- in[2] = _mm_load_si128((const __m128i *)(input + 8 * 4));
- in[3] = _mm_load_si128((const __m128i *)(input + 8 * 6));
-
- TRANSPOSE_8X4(in[0], in[1], in[2], in[3], in[0], in[1]);
-
- // Stage2
- {
- const __m128i lo_1_15 = _mm_unpackhi_epi16(in[0], zero);
- const __m128i lo_13_3 = _mm_unpackhi_epi16(zero, in[1]);
-
- tmp0 = _mm_madd_epi16(lo_1_15, stg2_0);
- tmp2 = _mm_madd_epi16(lo_1_15, stg2_1);
- tmp5 = _mm_madd_epi16(lo_13_3, stg2_6);
- tmp7 = _mm_madd_epi16(lo_13_3, stg2_7);
-
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp5 = _mm_add_epi32(tmp5, rounding);
- tmp7 = _mm_add_epi32(tmp7, rounding);
-
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
- tmp5 = _mm_srai_epi32(tmp5, DCT_CONST_BITS);
- tmp7 = _mm_srai_epi32(tmp7, DCT_CONST_BITS);
-
- stp2_8 = _mm_packs_epi32(tmp0, tmp2);
- stp2_11 = _mm_packs_epi32(tmp5, tmp7);
- }
-
- // Stage3
- {
- const __m128i lo_2_14 = _mm_unpacklo_epi16(in[1], zero);
-
- tmp0 = _mm_madd_epi16(lo_2_14, stg3_0);
- tmp2 = _mm_madd_epi16(lo_2_14, stg3_1);
-
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
-
- stp1_13 = _mm_unpackhi_epi64(stp2_11, zero);
- stp1_14 = _mm_unpackhi_epi64(stp2_8, zero);
-
- stp1_4 = _mm_packs_epi32(tmp0, tmp2);
- }
-
- // Stage4
- {
- const __m128i lo_0_8 = _mm_unpacklo_epi16(in[0], zero);
- const __m128i lo_9_14 = _mm_unpacklo_epi16(stp2_8, stp1_14);
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp2_11, stp1_13);
-
- tmp0 = _mm_madd_epi16(lo_0_8, stg4_0);
- tmp2 = _mm_madd_epi16(lo_0_8, stg4_1);
- tmp1 = _mm_madd_epi16(lo_9_14, stg4_4);
- tmp3 = _mm_madd_epi16(lo_9_14, stg4_5);
- tmp5 = _mm_madd_epi16(lo_10_13, stg4_6);
- tmp7 = _mm_madd_epi16(lo_10_13, stg4_7);
-
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp1 = _mm_add_epi32(tmp1, rounding);
- tmp3 = _mm_add_epi32(tmp3, rounding);
- tmp5 = _mm_add_epi32(tmp5, rounding);
- tmp7 = _mm_add_epi32(tmp7, rounding);
-
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS);
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS);
- tmp5 = _mm_srai_epi32(tmp5, DCT_CONST_BITS);
- tmp7 = _mm_srai_epi32(tmp7, DCT_CONST_BITS);
-
- stp1_0 = _mm_packs_epi32(tmp0, tmp0);
- stp1_1 = _mm_packs_epi32(tmp2, tmp2);
- stp2_9 = _mm_packs_epi32(tmp1, tmp3);
- stp2_10 = _mm_packs_epi32(tmp5, tmp7);
-
- stp2_6 = _mm_unpackhi_epi64(stp1_4, zero);
- }
-
- // Stage5 and Stage6
- {
- tmp0 = _mm_add_epi16(stp2_8, stp2_11);
- tmp1 = _mm_sub_epi16(stp2_8, stp2_11);
- tmp2 = _mm_add_epi16(stp2_9, stp2_10);
- tmp3 = _mm_sub_epi16(stp2_9, stp2_10);
-
- stp1_9 = _mm_unpacklo_epi64(tmp2, zero);
- stp1_10 = _mm_unpacklo_epi64(tmp3, zero);
- stp1_8 = _mm_unpacklo_epi64(tmp0, zero);
- stp1_11 = _mm_unpacklo_epi64(tmp1, zero);
-
- stp1_13 = _mm_unpackhi_epi64(tmp3, zero);
- stp1_14 = _mm_unpackhi_epi64(tmp2, zero);
- stp1_12 = _mm_unpackhi_epi64(tmp1, zero);
- stp1_15 = _mm_unpackhi_epi64(tmp0, zero);
- }
-
- // Stage6
- {
- const __m128i lo_6_5 = _mm_unpacklo_epi16(stp2_6, stp1_4);
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13);
- const __m128i lo_11_12 = _mm_unpacklo_epi16(stp1_11, stp1_12);
-
- tmp1 = _mm_madd_epi16(lo_6_5, stg4_1);
- tmp3 = _mm_madd_epi16(lo_6_5, stg4_0);
- tmp0 = _mm_madd_epi16(lo_10_13, stg6_0);
- tmp2 = _mm_madd_epi16(lo_10_13, stg4_0);
- tmp4 = _mm_madd_epi16(lo_11_12, stg6_0);
- tmp6 = _mm_madd_epi16(lo_11_12, stg4_0);
-
- tmp1 = _mm_add_epi32(tmp1, rounding);
- tmp3 = _mm_add_epi32(tmp3, rounding);
- tmp0 = _mm_add_epi32(tmp0, rounding);
- tmp2 = _mm_add_epi32(tmp2, rounding);
- tmp4 = _mm_add_epi32(tmp4, rounding);
- tmp6 = _mm_add_epi32(tmp6, rounding);
-
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS);
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS);
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS);
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS);
- tmp4 = _mm_srai_epi32(tmp4, DCT_CONST_BITS);
- tmp6 = _mm_srai_epi32(tmp6, DCT_CONST_BITS);
-
- stp1_6 = _mm_packs_epi32(tmp3, tmp1);
-
- stp2_10 = _mm_packs_epi32(tmp0, zero);
- stp2_13 = _mm_packs_epi32(tmp2, zero);
- stp2_11 = _mm_packs_epi32(tmp4, zero);
- stp2_12 = _mm_packs_epi32(tmp6, zero);
-
- tmp0 = _mm_add_epi16(stp1_0, stp1_4);
- tmp1 = _mm_sub_epi16(stp1_0, stp1_4);
- tmp2 = _mm_add_epi16(stp1_1, stp1_6);
- tmp3 = _mm_sub_epi16(stp1_1, stp1_6);
-
- stp2_0 = _mm_unpackhi_epi64(tmp0, zero);
- stp2_1 = _mm_unpacklo_epi64(tmp2, zero);
- stp2_2 = _mm_unpackhi_epi64(tmp2, zero);
- stp2_3 = _mm_unpacklo_epi64(tmp0, zero);
- stp2_4 = _mm_unpacklo_epi64(tmp1, zero);
- stp2_5 = _mm_unpackhi_epi64(tmp3, zero);
- stp2_6 = _mm_unpacklo_epi64(tmp3, zero);
- stp2_7 = _mm_unpackhi_epi64(tmp1, zero);
- }
-
- // Stage7. Left 8x16 only.
- l[0] = _mm_add_epi16(stp2_0, stp1_15);
- l[1] = _mm_add_epi16(stp2_1, stp1_14);
- l[2] = _mm_add_epi16(stp2_2, stp2_13);
- l[3] = _mm_add_epi16(stp2_3, stp2_12);
- l[4] = _mm_add_epi16(stp2_4, stp2_11);
- l[5] = _mm_add_epi16(stp2_5, stp2_10);
- l[6] = _mm_add_epi16(stp2_6, stp1_9);
- l[7] = _mm_add_epi16(stp2_7, stp1_8);
- l[8] = _mm_sub_epi16(stp2_7, stp1_8);
- l[9] = _mm_sub_epi16(stp2_6, stp1_9);
- l[10] = _mm_sub_epi16(stp2_5, stp2_10);
- l[11] = _mm_sub_epi16(stp2_4, stp2_11);
- l[12] = _mm_sub_epi16(stp2_3, stp2_12);
- l[13] = _mm_sub_epi16(stp2_2, stp2_13);
- l[14] = _mm_sub_epi16(stp2_1, stp1_14);
- l[15] = _mm_sub_epi16(stp2_0, stp1_15);
-
- // Second 1-D inverse transform, performed per 8x16 block
- for (i = 0; i < 2; i++) {
- int j;
- array_transpose_4X8(l + 8 * i, in);
-
- IDCT16_10
-
- // Stage7
- in[0] = _mm_add_epi16(stp2_0, stp1_15);
- in[1] = _mm_add_epi16(stp2_1, stp1_14);
- in[2] = _mm_add_epi16(stp2_2, stp2_13);
- in[3] = _mm_add_epi16(stp2_3, stp2_12);
- in[4] = _mm_add_epi16(stp2_4, stp2_11);
- in[5] = _mm_add_epi16(stp2_5, stp2_10);
- in[6] = _mm_add_epi16(stp2_6, stp1_9);
- in[7] = _mm_add_epi16(stp2_7, stp1_8);
- in[8] = _mm_sub_epi16(stp2_7, stp1_8);
- in[9] = _mm_sub_epi16(stp2_6, stp1_9);
- in[10] = _mm_sub_epi16(stp2_5, stp2_10);
- in[11] = _mm_sub_epi16(stp2_4, stp2_11);
- in[12] = _mm_sub_epi16(stp2_3, stp2_12);
- in[13] = _mm_sub_epi16(stp2_2, stp2_13);
- in[14] = _mm_sub_epi16(stp2_1, stp1_14);
- in[15] = _mm_sub_epi16(stp2_0, stp1_15);
-
- for (j = 0; j < 16; ++j) {
- // Final rounding and shift
- in[j] = _mm_adds_epi16(in[j], final_rounding);
- in[j] = _mm_srai_epi16(in[j], 6);
- RECON_AND_STORE(dest + j * stride, in[j]);
- }
-
- dest += 8;
- }
-}
-
-#define LOAD_DQCOEFF(reg, input) \
- { \
- reg = _mm_load_si128((const __m128i *) input); \
- input += 8; \
- } \
-
-#define IDCT32_34 \
-/* Stage1 */ \
-{ \
- const __m128i zero = _mm_setzero_si128();\
- const __m128i lo_1_31 = _mm_unpacklo_epi16(in[1], zero); \
- const __m128i hi_1_31 = _mm_unpackhi_epi16(in[1], zero); \
- \
- const __m128i lo_25_7= _mm_unpacklo_epi16(zero, in[7]); \
- const __m128i hi_25_7 = _mm_unpackhi_epi16(zero, in[7]); \
- \
- const __m128i lo_5_27 = _mm_unpacklo_epi16(in[5], zero); \
- const __m128i hi_5_27 = _mm_unpackhi_epi16(in[5], zero); \
- \
- const __m128i lo_29_3 = _mm_unpacklo_epi16(zero, in[3]); \
- const __m128i hi_29_3 = _mm_unpackhi_epi16(zero, in[3]); \
- \
- MULTIPLICATION_AND_ADD_2(lo_1_31, hi_1_31, stg1_0, \
- stg1_1, stp1_16, stp1_31); \
- MULTIPLICATION_AND_ADD_2(lo_25_7, hi_25_7, stg1_6, \
- stg1_7, stp1_19, stp1_28); \
- MULTIPLICATION_AND_ADD_2(lo_5_27, hi_5_27, stg1_8, \
- stg1_9, stp1_20, stp1_27); \
- MULTIPLICATION_AND_ADD_2(lo_29_3, hi_29_3, stg1_14, \
- stg1_15, stp1_23, stp1_24); \
-} \
-\
-/* Stage2 */ \
-{ \
- const __m128i zero = _mm_setzero_si128();\
- const __m128i lo_2_30 = _mm_unpacklo_epi16(in[2], zero); \
- const __m128i hi_2_30 = _mm_unpackhi_epi16(in[2], zero); \
- \
- const __m128i lo_26_6 = _mm_unpacklo_epi16(zero, in[6]); \
- const __m128i hi_26_6 = _mm_unpackhi_epi16(zero, in[6]); \
- \
- MULTIPLICATION_AND_ADD_2(lo_2_30, hi_2_30, stg2_0, \
- stg2_1, stp2_8, stp2_15); \
- MULTIPLICATION_AND_ADD_2(lo_26_6, hi_26_6, stg2_6, \
- stg2_7, stp2_11, stp2_12); \
- \
- stp2_16 = stp1_16; \
- stp2_19 = stp1_19; \
- \
- stp2_20 = stp1_20; \
- stp2_23 = stp1_23; \
- \
- stp2_24 = stp1_24; \
- stp2_27 = stp1_27; \
- \
- stp2_28 = stp1_28; \
- stp2_31 = stp1_31; \
-} \
-\
-/* Stage3 */ \
-{ \
- const __m128i zero = _mm_setzero_si128();\
- const __m128i lo_4_28 = _mm_unpacklo_epi16(in[4], zero); \
- const __m128i hi_4_28 = _mm_unpackhi_epi16(in[4], zero); \
- \
- const __m128i lo_17_30 = _mm_unpacklo_epi16(stp1_16, stp1_31); \
- const __m128i hi_17_30 = _mm_unpackhi_epi16(stp1_16, stp1_31); \
- const __m128i lo_18_29 = _mm_unpacklo_epi16(stp1_19, stp1_28); \
- const __m128i hi_18_29 = _mm_unpackhi_epi16(stp1_19, stp1_28); \
- \
- const __m128i lo_21_26 = _mm_unpacklo_epi16(stp1_20, stp1_27); \
- const __m128i hi_21_26 = _mm_unpackhi_epi16(stp1_20, stp1_27); \
- const __m128i lo_22_25 = _mm_unpacklo_epi16(stp1_23, stp1_24); \
- const __m128i hi_22_25 = _mm_unpackhi_epi16(stp1_23, stp2_24); \
- \
- MULTIPLICATION_AND_ADD_2(lo_4_28, hi_4_28, stg3_0, \
- stg3_1, stp1_4, stp1_7); \
- \
- stp1_8 = stp2_8; \
- stp1_11 = stp2_11; \
- stp1_12 = stp2_12; \
- stp1_15 = stp2_15; \
- \
- MULTIPLICATION_AND_ADD(lo_17_30, hi_17_30, lo_18_29, hi_18_29, stg3_4, \
- stg3_5, stg3_6, stg3_4, stp1_17, stp1_30, \
- stp1_18, stp1_29) \
- MULTIPLICATION_AND_ADD(lo_21_26, hi_21_26, lo_22_25, hi_22_25, stg3_8, \
- stg3_9, stg3_10, stg3_8, stp1_21, stp1_26, \
- stp1_22, stp1_25) \
- \
- stp1_16 = stp2_16; \
- stp1_31 = stp2_31; \
- stp1_19 = stp2_19; \
- stp1_20 = stp2_20; \
- stp1_23 = stp2_23; \
- stp1_24 = stp2_24; \
- stp1_27 = stp2_27; \
- stp1_28 = stp2_28; \
-} \
-\
-/* Stage4 */ \
-{ \
- const __m128i zero = _mm_setzero_si128();\
- const __m128i lo_0_16 = _mm_unpacklo_epi16(in[0], zero); \
- const __m128i hi_0_16 = _mm_unpackhi_epi16(in[0], zero); \
- \
- const __m128i lo_9_14 = _mm_unpacklo_epi16(stp2_8, stp2_15); \
- const __m128i hi_9_14 = _mm_unpackhi_epi16(stp2_8, stp2_15); \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp2_11, stp2_12); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp2_11, stp2_12); \
- \
- MULTIPLICATION_AND_ADD_2(lo_0_16, hi_0_16, stg4_0, \
- stg4_1, stp2_0, stp2_1); \
- \
- stp2_4 = stp1_4; \
- stp2_5 = stp1_4; \
- stp2_6 = stp1_7; \
- stp2_7 = stp1_7; \
- \
- MULTIPLICATION_AND_ADD(lo_9_14, hi_9_14, lo_10_13, hi_10_13, stg4_4, \
- stg4_5, stg4_6, stg4_4, stp2_9, stp2_14, \
- stp2_10, stp2_13) \
- \
- stp2_8 = stp1_8; \
- stp2_15 = stp1_15; \
- stp2_11 = stp1_11; \
- stp2_12 = stp1_12; \
- \
- stp2_16 = _mm_add_epi16(stp1_16, stp1_19); \
- stp2_17 = _mm_add_epi16(stp1_17, stp1_18); \
- stp2_18 = _mm_sub_epi16(stp1_17, stp1_18); \
- stp2_19 = _mm_sub_epi16(stp1_16, stp1_19); \
- stp2_20 = _mm_sub_epi16(stp1_23, stp1_20); \
- stp2_21 = _mm_sub_epi16(stp1_22, stp1_21); \
- stp2_22 = _mm_add_epi16(stp1_22, stp1_21); \
- stp2_23 = _mm_add_epi16(stp1_23, stp1_20); \
- \
- stp2_24 = _mm_add_epi16(stp1_24, stp1_27); \
- stp2_25 = _mm_add_epi16(stp1_25, stp1_26); \
- stp2_26 = _mm_sub_epi16(stp1_25, stp1_26); \
- stp2_27 = _mm_sub_epi16(stp1_24, stp1_27); \
- stp2_28 = _mm_sub_epi16(stp1_31, stp1_28); \
- stp2_29 = _mm_sub_epi16(stp1_30, stp1_29); \
- stp2_30 = _mm_add_epi16(stp1_29, stp1_30); \
- stp2_31 = _mm_add_epi16(stp1_28, stp1_31); \
-} \
-\
-/* Stage5 */ \
-{ \
- const __m128i lo_6_5 = _mm_unpacklo_epi16(stp2_6, stp2_5); \
- const __m128i hi_6_5 = _mm_unpackhi_epi16(stp2_6, stp2_5); \
- const __m128i lo_18_29 = _mm_unpacklo_epi16(stp2_18, stp2_29); \
- const __m128i hi_18_29 = _mm_unpackhi_epi16(stp2_18, stp2_29); \
- \
- const __m128i lo_19_28 = _mm_unpacklo_epi16(stp2_19, stp2_28); \
- const __m128i hi_19_28 = _mm_unpackhi_epi16(stp2_19, stp2_28); \
- const __m128i lo_20_27 = _mm_unpacklo_epi16(stp2_20, stp2_27); \
- const __m128i hi_20_27 = _mm_unpackhi_epi16(stp2_20, stp2_27); \
- \
- const __m128i lo_21_26 = _mm_unpacklo_epi16(stp2_21, stp2_26); \
- const __m128i hi_21_26 = _mm_unpackhi_epi16(stp2_21, stp2_26); \
- \
- stp1_0 = stp2_0; \
- stp1_1 = stp2_1; \
- stp1_2 = stp2_1; \
- stp1_3 = stp2_0; \
- \
- tmp0 = _mm_madd_epi16(lo_6_5, stg4_1); \
- tmp1 = _mm_madd_epi16(hi_6_5, stg4_1); \
- tmp2 = _mm_madd_epi16(lo_6_5, stg4_0); \
- tmp3 = _mm_madd_epi16(hi_6_5, stg4_0); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- \
- stp1_5 = _mm_packs_epi32(tmp0, tmp1); \
- stp1_6 = _mm_packs_epi32(tmp2, tmp3); \
- \
- stp1_4 = stp2_4; \
- stp1_7 = stp2_7; \
- \
- stp1_8 = _mm_add_epi16(stp2_8, stp2_11); \
- stp1_9 = _mm_add_epi16(stp2_9, stp2_10); \
- stp1_10 = _mm_sub_epi16(stp2_9, stp2_10); \
- stp1_11 = _mm_sub_epi16(stp2_8, stp2_11); \
- stp1_12 = _mm_sub_epi16(stp2_15, stp2_12); \
- stp1_13 = _mm_sub_epi16(stp2_14, stp2_13); \
- stp1_14 = _mm_add_epi16(stp2_14, stp2_13); \
- stp1_15 = _mm_add_epi16(stp2_15, stp2_12); \
- \
- stp1_16 = stp2_16; \
- stp1_17 = stp2_17; \
- \
- MULTIPLICATION_AND_ADD(lo_18_29, hi_18_29, lo_19_28, hi_19_28, stg4_4, \
- stg4_5, stg4_4, stg4_5, stp1_18, stp1_29, \
- stp1_19, stp1_28) \
- MULTIPLICATION_AND_ADD(lo_20_27, hi_20_27, lo_21_26, hi_21_26, stg4_6, \
- stg4_4, stg4_6, stg4_4, stp1_20, stp1_27, \
- stp1_21, stp1_26) \
- \
- stp1_22 = stp2_22; \
- stp1_23 = stp2_23; \
- stp1_24 = stp2_24; \
- stp1_25 = stp2_25; \
- stp1_30 = stp2_30; \
- stp1_31 = stp2_31; \
-} \
-\
-/* Stage6 */ \
-{ \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- const __m128i lo_11_12 = _mm_unpacklo_epi16(stp1_11, stp1_12); \
- const __m128i hi_11_12 = _mm_unpackhi_epi16(stp1_11, stp1_12); \
- \
- stp2_0 = _mm_add_epi16(stp1_0, stp1_7); \
- stp2_1 = _mm_add_epi16(stp1_1, stp1_6); \
- stp2_2 = _mm_add_epi16(stp1_2, stp1_5); \
- stp2_3 = _mm_add_epi16(stp1_3, stp1_4); \
- stp2_4 = _mm_sub_epi16(stp1_3, stp1_4); \
- stp2_5 = _mm_sub_epi16(stp1_2, stp1_5); \
- stp2_6 = _mm_sub_epi16(stp1_1, stp1_6); \
- stp2_7 = _mm_sub_epi16(stp1_0, stp1_7); \
- \
- stp2_8 = stp1_8; \
- stp2_9 = stp1_9; \
- stp2_14 = stp1_14; \
- stp2_15 = stp1_15; \
- \
- MULTIPLICATION_AND_ADD(lo_10_13, hi_10_13, lo_11_12, hi_11_12, \
- stg6_0, stg4_0, stg6_0, stg4_0, stp2_10, \
- stp2_13, stp2_11, stp2_12) \
- \
- stp2_16 = _mm_add_epi16(stp1_16, stp1_23); \
- stp2_17 = _mm_add_epi16(stp1_17, stp1_22); \
- stp2_18 = _mm_add_epi16(stp1_18, stp1_21); \
- stp2_19 = _mm_add_epi16(stp1_19, stp1_20); \
- stp2_20 = _mm_sub_epi16(stp1_19, stp1_20); \
- stp2_21 = _mm_sub_epi16(stp1_18, stp1_21); \
- stp2_22 = _mm_sub_epi16(stp1_17, stp1_22); \
- stp2_23 = _mm_sub_epi16(stp1_16, stp1_23); \
- \
- stp2_24 = _mm_sub_epi16(stp1_31, stp1_24); \
- stp2_25 = _mm_sub_epi16(stp1_30, stp1_25); \
- stp2_26 = _mm_sub_epi16(stp1_29, stp1_26); \
- stp2_27 = _mm_sub_epi16(stp1_28, stp1_27); \
- stp2_28 = _mm_add_epi16(stp1_27, stp1_28); \
- stp2_29 = _mm_add_epi16(stp1_26, stp1_29); \
- stp2_30 = _mm_add_epi16(stp1_25, stp1_30); \
- stp2_31 = _mm_add_epi16(stp1_24, stp1_31); \
-} \
-\
-/* Stage7 */ \
-{ \
- const __m128i lo_20_27 = _mm_unpacklo_epi16(stp2_20, stp2_27); \
- const __m128i hi_20_27 = _mm_unpackhi_epi16(stp2_20, stp2_27); \
- const __m128i lo_21_26 = _mm_unpacklo_epi16(stp2_21, stp2_26); \
- const __m128i hi_21_26 = _mm_unpackhi_epi16(stp2_21, stp2_26); \
- \
- const __m128i lo_22_25 = _mm_unpacklo_epi16(stp2_22, stp2_25); \
- const __m128i hi_22_25 = _mm_unpackhi_epi16(stp2_22, stp2_25); \
- const __m128i lo_23_24 = _mm_unpacklo_epi16(stp2_23, stp2_24); \
- const __m128i hi_23_24 = _mm_unpackhi_epi16(stp2_23, stp2_24); \
- \
- stp1_0 = _mm_add_epi16(stp2_0, stp2_15); \
- stp1_1 = _mm_add_epi16(stp2_1, stp2_14); \
- stp1_2 = _mm_add_epi16(stp2_2, stp2_13); \
- stp1_3 = _mm_add_epi16(stp2_3, stp2_12); \
- stp1_4 = _mm_add_epi16(stp2_4, stp2_11); \
- stp1_5 = _mm_add_epi16(stp2_5, stp2_10); \
- stp1_6 = _mm_add_epi16(stp2_6, stp2_9); \
- stp1_7 = _mm_add_epi16(stp2_7, stp2_8); \
- stp1_8 = _mm_sub_epi16(stp2_7, stp2_8); \
- stp1_9 = _mm_sub_epi16(stp2_6, stp2_9); \
- stp1_10 = _mm_sub_epi16(stp2_5, stp2_10); \
- stp1_11 = _mm_sub_epi16(stp2_4, stp2_11); \
- stp1_12 = _mm_sub_epi16(stp2_3, stp2_12); \
- stp1_13 = _mm_sub_epi16(stp2_2, stp2_13); \
- stp1_14 = _mm_sub_epi16(stp2_1, stp2_14); \
- stp1_15 = _mm_sub_epi16(stp2_0, stp2_15); \
- \
- stp1_16 = stp2_16; \
- stp1_17 = stp2_17; \
- stp1_18 = stp2_18; \
- stp1_19 = stp2_19; \
- \
- MULTIPLICATION_AND_ADD(lo_20_27, hi_20_27, lo_21_26, hi_21_26, stg6_0, \
- stg4_0, stg6_0, stg4_0, stp1_20, stp1_27, \
- stp1_21, stp1_26) \
- MULTIPLICATION_AND_ADD(lo_22_25, hi_22_25, lo_23_24, hi_23_24, stg6_0, \
- stg4_0, stg6_0, stg4_0, stp1_22, stp1_25, \
- stp1_23, stp1_24) \
- \
- stp1_28 = stp2_28; \
- stp1_29 = stp2_29; \
- stp1_30 = stp2_30; \
- stp1_31 = stp2_31; \
-}
-
-
-#define IDCT32 \
-/* Stage1 */ \
-{ \
- const __m128i lo_1_31 = _mm_unpacklo_epi16(in[1], in[31]); \
- const __m128i hi_1_31 = _mm_unpackhi_epi16(in[1], in[31]); \
- const __m128i lo_17_15 = _mm_unpacklo_epi16(in[17], in[15]); \
- const __m128i hi_17_15 = _mm_unpackhi_epi16(in[17], in[15]); \
- \
- const __m128i lo_9_23 = _mm_unpacklo_epi16(in[9], in[23]); \
- const __m128i hi_9_23 = _mm_unpackhi_epi16(in[9], in[23]); \
- const __m128i lo_25_7= _mm_unpacklo_epi16(in[25], in[7]); \
- const __m128i hi_25_7 = _mm_unpackhi_epi16(in[25], in[7]); \
- \
- const __m128i lo_5_27 = _mm_unpacklo_epi16(in[5], in[27]); \
- const __m128i hi_5_27 = _mm_unpackhi_epi16(in[5], in[27]); \
- const __m128i lo_21_11 = _mm_unpacklo_epi16(in[21], in[11]); \
- const __m128i hi_21_11 = _mm_unpackhi_epi16(in[21], in[11]); \
- \
- const __m128i lo_13_19 = _mm_unpacklo_epi16(in[13], in[19]); \
- const __m128i hi_13_19 = _mm_unpackhi_epi16(in[13], in[19]); \
- const __m128i lo_29_3 = _mm_unpacklo_epi16(in[29], in[3]); \
- const __m128i hi_29_3 = _mm_unpackhi_epi16(in[29], in[3]); \
- \
- MULTIPLICATION_AND_ADD(lo_1_31, hi_1_31, lo_17_15, hi_17_15, stg1_0, \
- stg1_1, stg1_2, stg1_3, stp1_16, stp1_31, \
- stp1_17, stp1_30) \
- MULTIPLICATION_AND_ADD(lo_9_23, hi_9_23, lo_25_7, hi_25_7, stg1_4, \
- stg1_5, stg1_6, stg1_7, stp1_18, stp1_29, \
- stp1_19, stp1_28) \
- MULTIPLICATION_AND_ADD(lo_5_27, hi_5_27, lo_21_11, hi_21_11, stg1_8, \
- stg1_9, stg1_10, stg1_11, stp1_20, stp1_27, \
- stp1_21, stp1_26) \
- MULTIPLICATION_AND_ADD(lo_13_19, hi_13_19, lo_29_3, hi_29_3, stg1_12, \
- stg1_13, stg1_14, stg1_15, stp1_22, stp1_25, \
- stp1_23, stp1_24) \
-} \
-\
-/* Stage2 */ \
-{ \
- const __m128i lo_2_30 = _mm_unpacklo_epi16(in[2], in[30]); \
- const __m128i hi_2_30 = _mm_unpackhi_epi16(in[2], in[30]); \
- const __m128i lo_18_14 = _mm_unpacklo_epi16(in[18], in[14]); \
- const __m128i hi_18_14 = _mm_unpackhi_epi16(in[18], in[14]); \
- \
- const __m128i lo_10_22 = _mm_unpacklo_epi16(in[10], in[22]); \
- const __m128i hi_10_22 = _mm_unpackhi_epi16(in[10], in[22]); \
- const __m128i lo_26_6 = _mm_unpacklo_epi16(in[26], in[6]); \
- const __m128i hi_26_6 = _mm_unpackhi_epi16(in[26], in[6]); \
- \
- MULTIPLICATION_AND_ADD(lo_2_30, hi_2_30, lo_18_14, hi_18_14, stg2_0, \
- stg2_1, stg2_2, stg2_3, stp2_8, stp2_15, stp2_9, \
- stp2_14) \
- MULTIPLICATION_AND_ADD(lo_10_22, hi_10_22, lo_26_6, hi_26_6, stg2_4, \
- stg2_5, stg2_6, stg2_7, stp2_10, stp2_13, \
- stp2_11, stp2_12) \
- \
- stp2_16 = _mm_add_epi16(stp1_16, stp1_17); \
- stp2_17 = _mm_sub_epi16(stp1_16, stp1_17); \
- stp2_18 = _mm_sub_epi16(stp1_19, stp1_18); \
- stp2_19 = _mm_add_epi16(stp1_19, stp1_18); \
- \
- stp2_20 = _mm_add_epi16(stp1_20, stp1_21); \
- stp2_21 = _mm_sub_epi16(stp1_20, stp1_21); \
- stp2_22 = _mm_sub_epi16(stp1_23, stp1_22); \
- stp2_23 = _mm_add_epi16(stp1_23, stp1_22); \
- \
- stp2_24 = _mm_add_epi16(stp1_24, stp1_25); \
- stp2_25 = _mm_sub_epi16(stp1_24, stp1_25); \
- stp2_26 = _mm_sub_epi16(stp1_27, stp1_26); \
- stp2_27 = _mm_add_epi16(stp1_27, stp1_26); \
- \
- stp2_28 = _mm_add_epi16(stp1_28, stp1_29); \
- stp2_29 = _mm_sub_epi16(stp1_28, stp1_29); \
- stp2_30 = _mm_sub_epi16(stp1_31, stp1_30); \
- stp2_31 = _mm_add_epi16(stp1_31, stp1_30); \
-} \
-\
-/* Stage3 */ \
-{ \
- const __m128i lo_4_28 = _mm_unpacklo_epi16(in[4], in[28]); \
- const __m128i hi_4_28 = _mm_unpackhi_epi16(in[4], in[28]); \
- const __m128i lo_20_12 = _mm_unpacklo_epi16(in[20], in[12]); \
- const __m128i hi_20_12 = _mm_unpackhi_epi16(in[20], in[12]); \
- \
- const __m128i lo_17_30 = _mm_unpacklo_epi16(stp2_17, stp2_30); \
- const __m128i hi_17_30 = _mm_unpackhi_epi16(stp2_17, stp2_30); \
- const __m128i lo_18_29 = _mm_unpacklo_epi16(stp2_18, stp2_29); \
- const __m128i hi_18_29 = _mm_unpackhi_epi16(stp2_18, stp2_29); \
- \
- const __m128i lo_21_26 = _mm_unpacklo_epi16(stp2_21, stp2_26); \
- const __m128i hi_21_26 = _mm_unpackhi_epi16(stp2_21, stp2_26); \
- const __m128i lo_22_25 = _mm_unpacklo_epi16(stp2_22, stp2_25); \
- const __m128i hi_22_25 = _mm_unpackhi_epi16(stp2_22, stp2_25); \
- \
- MULTIPLICATION_AND_ADD(lo_4_28, hi_4_28, lo_20_12, hi_20_12, stg3_0, \
- stg3_1, stg3_2, stg3_3, stp1_4, stp1_7, stp1_5, \
- stp1_6) \
- \
- stp1_8 = _mm_add_epi16(stp2_8, stp2_9); \
- stp1_9 = _mm_sub_epi16(stp2_8, stp2_9); \
- stp1_10 = _mm_sub_epi16(stp2_11, stp2_10); \
- stp1_11 = _mm_add_epi16(stp2_11, stp2_10); \
- stp1_12 = _mm_add_epi16(stp2_12, stp2_13); \
- stp1_13 = _mm_sub_epi16(stp2_12, stp2_13); \
- stp1_14 = _mm_sub_epi16(stp2_15, stp2_14); \
- stp1_15 = _mm_add_epi16(stp2_15, stp2_14); \
- \
- MULTIPLICATION_AND_ADD(lo_17_30, hi_17_30, lo_18_29, hi_18_29, stg3_4, \
- stg3_5, stg3_6, stg3_4, stp1_17, stp1_30, \
- stp1_18, stp1_29) \
- MULTIPLICATION_AND_ADD(lo_21_26, hi_21_26, lo_22_25, hi_22_25, stg3_8, \
- stg3_9, stg3_10, stg3_8, stp1_21, stp1_26, \
- stp1_22, stp1_25) \
- \
- stp1_16 = stp2_16; \
- stp1_31 = stp2_31; \
- stp1_19 = stp2_19; \
- stp1_20 = stp2_20; \
- stp1_23 = stp2_23; \
- stp1_24 = stp2_24; \
- stp1_27 = stp2_27; \
- stp1_28 = stp2_28; \
-} \
-\
-/* Stage4 */ \
-{ \
- const __m128i lo_0_16 = _mm_unpacklo_epi16(in[0], in[16]); \
- const __m128i hi_0_16 = _mm_unpackhi_epi16(in[0], in[16]); \
- const __m128i lo_8_24 = _mm_unpacklo_epi16(in[8], in[24]); \
- const __m128i hi_8_24 = _mm_unpackhi_epi16(in[8], in[24]); \
- \
- const __m128i lo_9_14 = _mm_unpacklo_epi16(stp1_9, stp1_14); \
- const __m128i hi_9_14 = _mm_unpackhi_epi16(stp1_9, stp1_14); \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- \
- MULTIPLICATION_AND_ADD(lo_0_16, hi_0_16, lo_8_24, hi_8_24, stg4_0, \
- stg4_1, stg4_2, stg4_3, stp2_0, stp2_1, \
- stp2_2, stp2_3) \
- \
- stp2_4 = _mm_add_epi16(stp1_4, stp1_5); \
- stp2_5 = _mm_sub_epi16(stp1_4, stp1_5); \
- stp2_6 = _mm_sub_epi16(stp1_7, stp1_6); \
- stp2_7 = _mm_add_epi16(stp1_7, stp1_6); \
- \
- MULTIPLICATION_AND_ADD(lo_9_14, hi_9_14, lo_10_13, hi_10_13, stg4_4, \
- stg4_5, stg4_6, stg4_4, stp2_9, stp2_14, \
- stp2_10, stp2_13) \
- \
- stp2_8 = stp1_8; \
- stp2_15 = stp1_15; \
- stp2_11 = stp1_11; \
- stp2_12 = stp1_12; \
- \
- stp2_16 = _mm_add_epi16(stp1_16, stp1_19); \
- stp2_17 = _mm_add_epi16(stp1_17, stp1_18); \
- stp2_18 = _mm_sub_epi16(stp1_17, stp1_18); \
- stp2_19 = _mm_sub_epi16(stp1_16, stp1_19); \
- stp2_20 = _mm_sub_epi16(stp1_23, stp1_20); \
- stp2_21 = _mm_sub_epi16(stp1_22, stp1_21); \
- stp2_22 = _mm_add_epi16(stp1_22, stp1_21); \
- stp2_23 = _mm_add_epi16(stp1_23, stp1_20); \
- \
- stp2_24 = _mm_add_epi16(stp1_24, stp1_27); \
- stp2_25 = _mm_add_epi16(stp1_25, stp1_26); \
- stp2_26 = _mm_sub_epi16(stp1_25, stp1_26); \
- stp2_27 = _mm_sub_epi16(stp1_24, stp1_27); \
- stp2_28 = _mm_sub_epi16(stp1_31, stp1_28); \
- stp2_29 = _mm_sub_epi16(stp1_30, stp1_29); \
- stp2_30 = _mm_add_epi16(stp1_29, stp1_30); \
- stp2_31 = _mm_add_epi16(stp1_28, stp1_31); \
-} \
-\
-/* Stage5 */ \
-{ \
- const __m128i lo_6_5 = _mm_unpacklo_epi16(stp2_6, stp2_5); \
- const __m128i hi_6_5 = _mm_unpackhi_epi16(stp2_6, stp2_5); \
- const __m128i lo_18_29 = _mm_unpacklo_epi16(stp2_18, stp2_29); \
- const __m128i hi_18_29 = _mm_unpackhi_epi16(stp2_18, stp2_29); \
- \
- const __m128i lo_19_28 = _mm_unpacklo_epi16(stp2_19, stp2_28); \
- const __m128i hi_19_28 = _mm_unpackhi_epi16(stp2_19, stp2_28); \
- const __m128i lo_20_27 = _mm_unpacklo_epi16(stp2_20, stp2_27); \
- const __m128i hi_20_27 = _mm_unpackhi_epi16(stp2_20, stp2_27); \
- \
- const __m128i lo_21_26 = _mm_unpacklo_epi16(stp2_21, stp2_26); \
- const __m128i hi_21_26 = _mm_unpackhi_epi16(stp2_21, stp2_26); \
- \
- stp1_0 = _mm_add_epi16(stp2_0, stp2_3); \
- stp1_1 = _mm_add_epi16(stp2_1, stp2_2); \
- stp1_2 = _mm_sub_epi16(stp2_1, stp2_2); \
- stp1_3 = _mm_sub_epi16(stp2_0, stp2_3); \
- \
- tmp0 = _mm_madd_epi16(lo_6_5, stg4_1); \
- tmp1 = _mm_madd_epi16(hi_6_5, stg4_1); \
- tmp2 = _mm_madd_epi16(lo_6_5, stg4_0); \
- tmp3 = _mm_madd_epi16(hi_6_5, stg4_0); \
- \
- tmp0 = _mm_add_epi32(tmp0, rounding); \
- tmp1 = _mm_add_epi32(tmp1, rounding); \
- tmp2 = _mm_add_epi32(tmp2, rounding); \
- tmp3 = _mm_add_epi32(tmp3, rounding); \
- \
- tmp0 = _mm_srai_epi32(tmp0, DCT_CONST_BITS); \
- tmp1 = _mm_srai_epi32(tmp1, DCT_CONST_BITS); \
- tmp2 = _mm_srai_epi32(tmp2, DCT_CONST_BITS); \
- tmp3 = _mm_srai_epi32(tmp3, DCT_CONST_BITS); \
- \
- stp1_5 = _mm_packs_epi32(tmp0, tmp1); \
- stp1_6 = _mm_packs_epi32(tmp2, tmp3); \
- \
- stp1_4 = stp2_4; \
- stp1_7 = stp2_7; \
- \
- stp1_8 = _mm_add_epi16(stp2_8, stp2_11); \
- stp1_9 = _mm_add_epi16(stp2_9, stp2_10); \
- stp1_10 = _mm_sub_epi16(stp2_9, stp2_10); \
- stp1_11 = _mm_sub_epi16(stp2_8, stp2_11); \
- stp1_12 = _mm_sub_epi16(stp2_15, stp2_12); \
- stp1_13 = _mm_sub_epi16(stp2_14, stp2_13); \
- stp1_14 = _mm_add_epi16(stp2_14, stp2_13); \
- stp1_15 = _mm_add_epi16(stp2_15, stp2_12); \
- \
- stp1_16 = stp2_16; \
- stp1_17 = stp2_17; \
- \
- MULTIPLICATION_AND_ADD(lo_18_29, hi_18_29, lo_19_28, hi_19_28, stg4_4, \
- stg4_5, stg4_4, stg4_5, stp1_18, stp1_29, \
- stp1_19, stp1_28) \
- MULTIPLICATION_AND_ADD(lo_20_27, hi_20_27, lo_21_26, hi_21_26, stg4_6, \
- stg4_4, stg4_6, stg4_4, stp1_20, stp1_27, \
- stp1_21, stp1_26) \
- \
- stp1_22 = stp2_22; \
- stp1_23 = stp2_23; \
- stp1_24 = stp2_24; \
- stp1_25 = stp2_25; \
- stp1_30 = stp2_30; \
- stp1_31 = stp2_31; \
-} \
-\
-/* Stage6 */ \
-{ \
- const __m128i lo_10_13 = _mm_unpacklo_epi16(stp1_10, stp1_13); \
- const __m128i hi_10_13 = _mm_unpackhi_epi16(stp1_10, stp1_13); \
- const __m128i lo_11_12 = _mm_unpacklo_epi16(stp1_11, stp1_12); \
- const __m128i hi_11_12 = _mm_unpackhi_epi16(stp1_11, stp1_12); \
- \
- stp2_0 = _mm_add_epi16(stp1_0, stp1_7); \
- stp2_1 = _mm_add_epi16(stp1_1, stp1_6); \
- stp2_2 = _mm_add_epi16(stp1_2, stp1_5); \
- stp2_3 = _mm_add_epi16(stp1_3, stp1_4); \
- stp2_4 = _mm_sub_epi16(stp1_3, stp1_4); \
- stp2_5 = _mm_sub_epi16(stp1_2, stp1_5); \
- stp2_6 = _mm_sub_epi16(stp1_1, stp1_6); \
- stp2_7 = _mm_sub_epi16(stp1_0, stp1_7); \
- \
- stp2_8 = stp1_8; \
- stp2_9 = stp1_9; \
- stp2_14 = stp1_14; \
- stp2_15 = stp1_15; \
- \
- MULTIPLICATION_AND_ADD(lo_10_13, hi_10_13, lo_11_12, hi_11_12, \
- stg6_0, stg4_0, stg6_0, stg4_0, stp2_10, \
- stp2_13, stp2_11, stp2_12) \
- \
- stp2_16 = _mm_add_epi16(stp1_16, stp1_23); \
- stp2_17 = _mm_add_epi16(stp1_17, stp1_22); \
- stp2_18 = _mm_add_epi16(stp1_18, stp1_21); \
- stp2_19 = _mm_add_epi16(stp1_19, stp1_20); \
- stp2_20 = _mm_sub_epi16(stp1_19, stp1_20); \
- stp2_21 = _mm_sub_epi16(stp1_18, stp1_21); \
- stp2_22 = _mm_sub_epi16(stp1_17, stp1_22); \
- stp2_23 = _mm_sub_epi16(stp1_16, stp1_23); \
- \
- stp2_24 = _mm_sub_epi16(stp1_31, stp1_24); \
- stp2_25 = _mm_sub_epi16(stp1_30, stp1_25); \
- stp2_26 = _mm_sub_epi16(stp1_29, stp1_26); \
- stp2_27 = _mm_sub_epi16(stp1_28, stp1_27); \
- stp2_28 = _mm_add_epi16(stp1_27, stp1_28); \
- stp2_29 = _mm_add_epi16(stp1_26, stp1_29); \
- stp2_30 = _mm_add_epi16(stp1_25, stp1_30); \
- stp2_31 = _mm_add_epi16(stp1_24, stp1_31); \
-} \
-\
-/* Stage7 */ \
-{ \
- const __m128i lo_20_27 = _mm_unpacklo_epi16(stp2_20, stp2_27); \
- const __m128i hi_20_27 = _mm_unpackhi_epi16(stp2_20, stp2_27); \
- const __m128i lo_21_26 = _mm_unpacklo_epi16(stp2_21, stp2_26); \
- const __m128i hi_21_26 = _mm_unpackhi_epi16(stp2_21, stp2_26); \
- \
- const __m128i lo_22_25 = _mm_unpacklo_epi16(stp2_22, stp2_25); \
- const __m128i hi_22_25 = _mm_unpackhi_epi16(stp2_22, stp2_25); \
- const __m128i lo_23_24 = _mm_unpacklo_epi16(stp2_23, stp2_24); \
- const __m128i hi_23_24 = _mm_unpackhi_epi16(stp2_23, stp2_24); \
- \
- stp1_0 = _mm_add_epi16(stp2_0, stp2_15); \
- stp1_1 = _mm_add_epi16(stp2_1, stp2_14); \
- stp1_2 = _mm_add_epi16(stp2_2, stp2_13); \
- stp1_3 = _mm_add_epi16(stp2_3, stp2_12); \
- stp1_4 = _mm_add_epi16(stp2_4, stp2_11); \
- stp1_5 = _mm_add_epi16(stp2_5, stp2_10); \
- stp1_6 = _mm_add_epi16(stp2_6, stp2_9); \
- stp1_7 = _mm_add_epi16(stp2_7, stp2_8); \
- stp1_8 = _mm_sub_epi16(stp2_7, stp2_8); \
- stp1_9 = _mm_sub_epi16(stp2_6, stp2_9); \
- stp1_10 = _mm_sub_epi16(stp2_5, stp2_10); \
- stp1_11 = _mm_sub_epi16(stp2_4, stp2_11); \
- stp1_12 = _mm_sub_epi16(stp2_3, stp2_12); \
- stp1_13 = _mm_sub_epi16(stp2_2, stp2_13); \
- stp1_14 = _mm_sub_epi16(stp2_1, stp2_14); \
- stp1_15 = _mm_sub_epi16(stp2_0, stp2_15); \
- \
- stp1_16 = stp2_16; \
- stp1_17 = stp2_17; \
- stp1_18 = stp2_18; \
- stp1_19 = stp2_19; \
- \
- MULTIPLICATION_AND_ADD(lo_20_27, hi_20_27, lo_21_26, hi_21_26, stg6_0, \
- stg4_0, stg6_0, stg4_0, stp1_20, stp1_27, \
- stp1_21, stp1_26) \
- MULTIPLICATION_AND_ADD(lo_22_25, hi_22_25, lo_23_24, hi_23_24, stg6_0, \
- stg4_0, stg6_0, stg4_0, stp1_22, stp1_25, \
- stp1_23, stp1_24) \
- \
- stp1_28 = stp2_28; \
- stp1_29 = stp2_29; \
- stp1_30 = stp2_30; \
- stp1_31 = stp2_31; \
-}
-
-// Only upper-left 8x8 has non-zero coeff
-void vp10_idct32x32_34_add_sse2(const int16_t *input, uint8_t *dest,
- int stride) {
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i final_rounding = _mm_set1_epi16(1<<5);
-
- // vp10_idct constants for each stage
- const __m128i stg1_0 = pair_set_epi16(cospi_31_64, -cospi_1_64);
- const __m128i stg1_1 = pair_set_epi16(cospi_1_64, cospi_31_64);
- const __m128i stg1_6 = pair_set_epi16(cospi_7_64, -cospi_25_64);
- const __m128i stg1_7 = pair_set_epi16(cospi_25_64, cospi_7_64);
- const __m128i stg1_8 = pair_set_epi16(cospi_27_64, -cospi_5_64);
- const __m128i stg1_9 = pair_set_epi16(cospi_5_64, cospi_27_64);
- const __m128i stg1_14 = pair_set_epi16(cospi_3_64, -cospi_29_64);
- const __m128i stg1_15 = pair_set_epi16(cospi_29_64, cospi_3_64);
-
- const __m128i stg2_0 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i stg2_6 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i stg2_7 = pair_set_epi16(cospi_26_64, cospi_6_64);
-
- const __m128i stg3_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg3_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i stg3_4 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i stg3_5 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i stg3_6 = pair_set_epi16(-cospi_28_64, -cospi_4_64);
- const __m128i stg3_8 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i stg3_9 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i stg3_10 = pair_set_epi16(-cospi_12_64, -cospi_20_64);
-
- const __m128i stg4_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg4_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg4_4 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i stg4_5 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i stg4_6 = pair_set_epi16(-cospi_24_64, -cospi_8_64);
-
- const __m128i stg6_0 = pair_set_epi16(-cospi_16_64, cospi_16_64);
-
- __m128i in[32], col[32];
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6, stp1_7,
- stp1_8, stp1_9, stp1_10, stp1_11, stp1_12, stp1_13, stp1_14, stp1_15,
- stp1_16, stp1_17, stp1_18, stp1_19, stp1_20, stp1_21, stp1_22,
- stp1_23, stp1_24, stp1_25, stp1_26, stp1_27, stp1_28, stp1_29,
- stp1_30, stp1_31;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7,
- stp2_8, stp2_9, stp2_10, stp2_11, stp2_12, stp2_13, stp2_14, stp2_15,
- stp2_16, stp2_17, stp2_18, stp2_19, stp2_20, stp2_21, stp2_22,
- stp2_23, stp2_24, stp2_25, stp2_26, stp2_27, stp2_28, stp2_29,
- stp2_30, stp2_31;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
- int i;
-
- // Load input data. Only need to load the top left 8x8 block.
- in[0] = _mm_load_si128((const __m128i *)input);
- in[1] = _mm_load_si128((const __m128i *)(input + 32));
- in[2] = _mm_load_si128((const __m128i *)(input + 64));
- in[3] = _mm_load_si128((const __m128i *)(input + 96));
- in[4] = _mm_load_si128((const __m128i *)(input + 128));
- in[5] = _mm_load_si128((const __m128i *)(input + 160));
- in[6] = _mm_load_si128((const __m128i *)(input + 192));
- in[7] = _mm_load_si128((const __m128i *)(input + 224));
-
- for (i = 8; i < 32; ++i) {
- in[i] = _mm_setzero_si128();
- }
-
- array_transpose_8x8(in, in);
- // TODO(hkuang): Following transposes are unnecessary. But remove them will
- // lead to performance drop on some devices.
- array_transpose_8x8(in + 8, in + 8);
- array_transpose_8x8(in + 16, in + 16);
- array_transpose_8x8(in + 24, in + 24);
-
- IDCT32_34
-
- // 1_D: Store 32 intermediate results for each 8x32 block.
- col[0] = _mm_add_epi16(stp1_0, stp1_31);
- col[1] = _mm_add_epi16(stp1_1, stp1_30);
- col[2] = _mm_add_epi16(stp1_2, stp1_29);
- col[3] = _mm_add_epi16(stp1_3, stp1_28);
- col[4] = _mm_add_epi16(stp1_4, stp1_27);
- col[5] = _mm_add_epi16(stp1_5, stp1_26);
- col[6] = _mm_add_epi16(stp1_6, stp1_25);
- col[7] = _mm_add_epi16(stp1_7, stp1_24);
- col[8] = _mm_add_epi16(stp1_8, stp1_23);
- col[9] = _mm_add_epi16(stp1_9, stp1_22);
- col[10] = _mm_add_epi16(stp1_10, stp1_21);
- col[11] = _mm_add_epi16(stp1_11, stp1_20);
- col[12] = _mm_add_epi16(stp1_12, stp1_19);
- col[13] = _mm_add_epi16(stp1_13, stp1_18);
- col[14] = _mm_add_epi16(stp1_14, stp1_17);
- col[15] = _mm_add_epi16(stp1_15, stp1_16);
- col[16] = _mm_sub_epi16(stp1_15, stp1_16);
- col[17] = _mm_sub_epi16(stp1_14, stp1_17);
- col[18] = _mm_sub_epi16(stp1_13, stp1_18);
- col[19] = _mm_sub_epi16(stp1_12, stp1_19);
- col[20] = _mm_sub_epi16(stp1_11, stp1_20);
- col[21] = _mm_sub_epi16(stp1_10, stp1_21);
- col[22] = _mm_sub_epi16(stp1_9, stp1_22);
- col[23] = _mm_sub_epi16(stp1_8, stp1_23);
- col[24] = _mm_sub_epi16(stp1_7, stp1_24);
- col[25] = _mm_sub_epi16(stp1_6, stp1_25);
- col[26] = _mm_sub_epi16(stp1_5, stp1_26);
- col[27] = _mm_sub_epi16(stp1_4, stp1_27);
- col[28] = _mm_sub_epi16(stp1_3, stp1_28);
- col[29] = _mm_sub_epi16(stp1_2, stp1_29);
- col[30] = _mm_sub_epi16(stp1_1, stp1_30);
- col[31] = _mm_sub_epi16(stp1_0, stp1_31);
- for (i = 0; i < 4; i++) {
- int j;
- const __m128i zero = _mm_setzero_si128();
- // Transpose 32x8 block to 8x32 block
- array_transpose_8x8(col + i * 8, in);
- IDCT32_34
-
- // 2_D: Calculate the results and store them to destination.
- in[0] = _mm_add_epi16(stp1_0, stp1_31);
- in[1] = _mm_add_epi16(stp1_1, stp1_30);
- in[2] = _mm_add_epi16(stp1_2, stp1_29);
- in[3] = _mm_add_epi16(stp1_3, stp1_28);
- in[4] = _mm_add_epi16(stp1_4, stp1_27);
- in[5] = _mm_add_epi16(stp1_5, stp1_26);
- in[6] = _mm_add_epi16(stp1_6, stp1_25);
- in[7] = _mm_add_epi16(stp1_7, stp1_24);
- in[8] = _mm_add_epi16(stp1_8, stp1_23);
- in[9] = _mm_add_epi16(stp1_9, stp1_22);
- in[10] = _mm_add_epi16(stp1_10, stp1_21);
- in[11] = _mm_add_epi16(stp1_11, stp1_20);
- in[12] = _mm_add_epi16(stp1_12, stp1_19);
- in[13] = _mm_add_epi16(stp1_13, stp1_18);
- in[14] = _mm_add_epi16(stp1_14, stp1_17);
- in[15] = _mm_add_epi16(stp1_15, stp1_16);
- in[16] = _mm_sub_epi16(stp1_15, stp1_16);
- in[17] = _mm_sub_epi16(stp1_14, stp1_17);
- in[18] = _mm_sub_epi16(stp1_13, stp1_18);
- in[19] = _mm_sub_epi16(stp1_12, stp1_19);
- in[20] = _mm_sub_epi16(stp1_11, stp1_20);
- in[21] = _mm_sub_epi16(stp1_10, stp1_21);
- in[22] = _mm_sub_epi16(stp1_9, stp1_22);
- in[23] = _mm_sub_epi16(stp1_8, stp1_23);
- in[24] = _mm_sub_epi16(stp1_7, stp1_24);
- in[25] = _mm_sub_epi16(stp1_6, stp1_25);
- in[26] = _mm_sub_epi16(stp1_5, stp1_26);
- in[27] = _mm_sub_epi16(stp1_4, stp1_27);
- in[28] = _mm_sub_epi16(stp1_3, stp1_28);
- in[29] = _mm_sub_epi16(stp1_2, stp1_29);
- in[30] = _mm_sub_epi16(stp1_1, stp1_30);
- in[31] = _mm_sub_epi16(stp1_0, stp1_31);
-
- for (j = 0; j < 32; ++j) {
- // Final rounding and shift
- in[j] = _mm_adds_epi16(in[j], final_rounding);
- in[j] = _mm_srai_epi16(in[j], 6);
- RECON_AND_STORE(dest + j * stride, in[j]);
- }
-
- dest += 8;
- }
-}
-
-void vp10_idct32x32_1024_add_sse2(const int16_t *input, uint8_t *dest,
- int stride) {
- const __m128i rounding = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i final_rounding = _mm_set1_epi16(1 << 5);
- const __m128i zero = _mm_setzero_si128();
-
- // vp10_idct constants for each stage
- const __m128i stg1_0 = pair_set_epi16(cospi_31_64, -cospi_1_64);
- const __m128i stg1_1 = pair_set_epi16(cospi_1_64, cospi_31_64);
- const __m128i stg1_2 = pair_set_epi16(cospi_15_64, -cospi_17_64);
- const __m128i stg1_3 = pair_set_epi16(cospi_17_64, cospi_15_64);
- const __m128i stg1_4 = pair_set_epi16(cospi_23_64, -cospi_9_64);
- const __m128i stg1_5 = pair_set_epi16(cospi_9_64, cospi_23_64);
- const __m128i stg1_6 = pair_set_epi16(cospi_7_64, -cospi_25_64);
- const __m128i stg1_7 = pair_set_epi16(cospi_25_64, cospi_7_64);
- const __m128i stg1_8 = pair_set_epi16(cospi_27_64, -cospi_5_64);
- const __m128i stg1_9 = pair_set_epi16(cospi_5_64, cospi_27_64);
- const __m128i stg1_10 = pair_set_epi16(cospi_11_64, -cospi_21_64);
- const __m128i stg1_11 = pair_set_epi16(cospi_21_64, cospi_11_64);
- const __m128i stg1_12 = pair_set_epi16(cospi_19_64, -cospi_13_64);
- const __m128i stg1_13 = pair_set_epi16(cospi_13_64, cospi_19_64);
- const __m128i stg1_14 = pair_set_epi16(cospi_3_64, -cospi_29_64);
- const __m128i stg1_15 = pair_set_epi16(cospi_29_64, cospi_3_64);
-
- const __m128i stg2_0 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i stg2_1 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i stg2_2 = pair_set_epi16(cospi_14_64, -cospi_18_64);
- const __m128i stg2_3 = pair_set_epi16(cospi_18_64, cospi_14_64);
- const __m128i stg2_4 = pair_set_epi16(cospi_22_64, -cospi_10_64);
- const __m128i stg2_5 = pair_set_epi16(cospi_10_64, cospi_22_64);
- const __m128i stg2_6 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i stg2_7 = pair_set_epi16(cospi_26_64, cospi_6_64);
-
- const __m128i stg3_0 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i stg3_1 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i stg3_2 = pair_set_epi16(cospi_12_64, -cospi_20_64);
- const __m128i stg3_3 = pair_set_epi16(cospi_20_64, cospi_12_64);
- const __m128i stg3_4 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i stg3_5 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i stg3_6 = pair_set_epi16(-cospi_28_64, -cospi_4_64);
- const __m128i stg3_8 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i stg3_9 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i stg3_10 = pair_set_epi16(-cospi_12_64, -cospi_20_64);
-
- const __m128i stg4_0 = pair_set_epi16(cospi_16_64, cospi_16_64);
- const __m128i stg4_1 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i stg4_2 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i stg4_3 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i stg4_4 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i stg4_5 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i stg4_6 = pair_set_epi16(-cospi_24_64, -cospi_8_64);
-
- const __m128i stg6_0 = pair_set_epi16(-cospi_16_64, cospi_16_64);
-
- __m128i in[32], col[128], zero_idx[16];
- __m128i stp1_0, stp1_1, stp1_2, stp1_3, stp1_4, stp1_5, stp1_6, stp1_7,
- stp1_8, stp1_9, stp1_10, stp1_11, stp1_12, stp1_13, stp1_14, stp1_15,
- stp1_16, stp1_17, stp1_18, stp1_19, stp1_20, stp1_21, stp1_22,
- stp1_23, stp1_24, stp1_25, stp1_26, stp1_27, stp1_28, stp1_29,
- stp1_30, stp1_31;
- __m128i stp2_0, stp2_1, stp2_2, stp2_3, stp2_4, stp2_5, stp2_6, stp2_7,
- stp2_8, stp2_9, stp2_10, stp2_11, stp2_12, stp2_13, stp2_14, stp2_15,
- stp2_16, stp2_17, stp2_18, stp2_19, stp2_20, stp2_21, stp2_22,
- stp2_23, stp2_24, stp2_25, stp2_26, stp2_27, stp2_28, stp2_29,
- stp2_30, stp2_31;
- __m128i tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
- int i, j, i32;
-
- for (i = 0; i < 4; i++) {
- i32 = (i << 5);
- // First 1-D vp10_idct
- // Load input data.
- LOAD_DQCOEFF(in[0], input);
- LOAD_DQCOEFF(in[8], input);
- LOAD_DQCOEFF(in[16], input);
- LOAD_DQCOEFF(in[24], input);
- LOAD_DQCOEFF(in[1], input);
- LOAD_DQCOEFF(in[9], input);
- LOAD_DQCOEFF(in[17], input);
- LOAD_DQCOEFF(in[25], input);
- LOAD_DQCOEFF(in[2], input);
- LOAD_DQCOEFF(in[10], input);
- LOAD_DQCOEFF(in[18], input);
- LOAD_DQCOEFF(in[26], input);
- LOAD_DQCOEFF(in[3], input);
- LOAD_DQCOEFF(in[11], input);
- LOAD_DQCOEFF(in[19], input);
- LOAD_DQCOEFF(in[27], input);
-
- LOAD_DQCOEFF(in[4], input);
- LOAD_DQCOEFF(in[12], input);
- LOAD_DQCOEFF(in[20], input);
- LOAD_DQCOEFF(in[28], input);
- LOAD_DQCOEFF(in[5], input);
- LOAD_DQCOEFF(in[13], input);
- LOAD_DQCOEFF(in[21], input);
- LOAD_DQCOEFF(in[29], input);
- LOAD_DQCOEFF(in[6], input);
- LOAD_DQCOEFF(in[14], input);
- LOAD_DQCOEFF(in[22], input);
- LOAD_DQCOEFF(in[30], input);
- LOAD_DQCOEFF(in[7], input);
- LOAD_DQCOEFF(in[15], input);
- LOAD_DQCOEFF(in[23], input);
- LOAD_DQCOEFF(in[31], input);
-
- // checking if all entries are zero
- zero_idx[0] = _mm_or_si128(in[0], in[1]);
- zero_idx[1] = _mm_or_si128(in[2], in[3]);
- zero_idx[2] = _mm_or_si128(in[4], in[5]);
- zero_idx[3] = _mm_or_si128(in[6], in[7]);
- zero_idx[4] = _mm_or_si128(in[8], in[9]);
- zero_idx[5] = _mm_or_si128(in[10], in[11]);
- zero_idx[6] = _mm_or_si128(in[12], in[13]);
- zero_idx[7] = _mm_or_si128(in[14], in[15]);
- zero_idx[8] = _mm_or_si128(in[16], in[17]);
- zero_idx[9] = _mm_or_si128(in[18], in[19]);
- zero_idx[10] = _mm_or_si128(in[20], in[21]);
- zero_idx[11] = _mm_or_si128(in[22], in[23]);
- zero_idx[12] = _mm_or_si128(in[24], in[25]);
- zero_idx[13] = _mm_or_si128(in[26], in[27]);
- zero_idx[14] = _mm_or_si128(in[28], in[29]);
- zero_idx[15] = _mm_or_si128(in[30], in[31]);
-
- zero_idx[0] = _mm_or_si128(zero_idx[0], zero_idx[1]);
- zero_idx[1] = _mm_or_si128(zero_idx[2], zero_idx[3]);
- zero_idx[2] = _mm_or_si128(zero_idx[4], zero_idx[5]);
- zero_idx[3] = _mm_or_si128(zero_idx[6], zero_idx[7]);
- zero_idx[4] = _mm_or_si128(zero_idx[8], zero_idx[9]);
- zero_idx[5] = _mm_or_si128(zero_idx[10], zero_idx[11]);
- zero_idx[6] = _mm_or_si128(zero_idx[12], zero_idx[13]);
- zero_idx[7] = _mm_or_si128(zero_idx[14], zero_idx[15]);
-
- zero_idx[8] = _mm_or_si128(zero_idx[0], zero_idx[1]);
- zero_idx[9] = _mm_or_si128(zero_idx[2], zero_idx[3]);
- zero_idx[10] = _mm_or_si128(zero_idx[4], zero_idx[5]);
- zero_idx[11] = _mm_or_si128(zero_idx[6], zero_idx[7]);
- zero_idx[12] = _mm_or_si128(zero_idx[8], zero_idx[9]);
- zero_idx[13] = _mm_or_si128(zero_idx[10], zero_idx[11]);
- zero_idx[14] = _mm_or_si128(zero_idx[12], zero_idx[13]);
-
- if (_mm_movemask_epi8(_mm_cmpeq_epi32(zero_idx[14], zero)) == 0xFFFF) {
- col[i32 + 0] = _mm_setzero_si128();
- col[i32 + 1] = _mm_setzero_si128();
- col[i32 + 2] = _mm_setzero_si128();
- col[i32 + 3] = _mm_setzero_si128();
- col[i32 + 4] = _mm_setzero_si128();
- col[i32 + 5] = _mm_setzero_si128();
- col[i32 + 6] = _mm_setzero_si128();
- col[i32 + 7] = _mm_setzero_si128();
- col[i32 + 8] = _mm_setzero_si128();
- col[i32 + 9] = _mm_setzero_si128();
- col[i32 + 10] = _mm_setzero_si128();
- col[i32 + 11] = _mm_setzero_si128();
- col[i32 + 12] = _mm_setzero_si128();
- col[i32 + 13] = _mm_setzero_si128();
- col[i32 + 14] = _mm_setzero_si128();
- col[i32 + 15] = _mm_setzero_si128();
- col[i32 + 16] = _mm_setzero_si128();
- col[i32 + 17] = _mm_setzero_si128();
- col[i32 + 18] = _mm_setzero_si128();
- col[i32 + 19] = _mm_setzero_si128();
- col[i32 + 20] = _mm_setzero_si128();
- col[i32 + 21] = _mm_setzero_si128();
- col[i32 + 22] = _mm_setzero_si128();
- col[i32 + 23] = _mm_setzero_si128();
- col[i32 + 24] = _mm_setzero_si128();
- col[i32 + 25] = _mm_setzero_si128();
- col[i32 + 26] = _mm_setzero_si128();
- col[i32 + 27] = _mm_setzero_si128();
- col[i32 + 28] = _mm_setzero_si128();
- col[i32 + 29] = _mm_setzero_si128();
- col[i32 + 30] = _mm_setzero_si128();
- col[i32 + 31] = _mm_setzero_si128();
- continue;
- }
-
- // Transpose 32x8 block to 8x32 block
- array_transpose_8x8(in, in);
- array_transpose_8x8(in + 8, in + 8);
- array_transpose_8x8(in + 16, in + 16);
- array_transpose_8x8(in + 24, in + 24);
-
- IDCT32
-
- // 1_D: Store 32 intermediate results for each 8x32 block.
- col[i32 + 0] = _mm_add_epi16(stp1_0, stp1_31);
- col[i32 + 1] = _mm_add_epi16(stp1_1, stp1_30);
- col[i32 + 2] = _mm_add_epi16(stp1_2, stp1_29);
- col[i32 + 3] = _mm_add_epi16(stp1_3, stp1_28);
- col[i32 + 4] = _mm_add_epi16(stp1_4, stp1_27);
- col[i32 + 5] = _mm_add_epi16(stp1_5, stp1_26);
- col[i32 + 6] = _mm_add_epi16(stp1_6, stp1_25);
- col[i32 + 7] = _mm_add_epi16(stp1_7, stp1_24);
- col[i32 + 8] = _mm_add_epi16(stp1_8, stp1_23);
- col[i32 + 9] = _mm_add_epi16(stp1_9, stp1_22);
- col[i32 + 10] = _mm_add_epi16(stp1_10, stp1_21);
- col[i32 + 11] = _mm_add_epi16(stp1_11, stp1_20);
- col[i32 + 12] = _mm_add_epi16(stp1_12, stp1_19);
- col[i32 + 13] = _mm_add_epi16(stp1_13, stp1_18);
- col[i32 + 14] = _mm_add_epi16(stp1_14, stp1_17);
- col[i32 + 15] = _mm_add_epi16(stp1_15, stp1_16);
- col[i32 + 16] = _mm_sub_epi16(stp1_15, stp1_16);
- col[i32 + 17] = _mm_sub_epi16(stp1_14, stp1_17);
- col[i32 + 18] = _mm_sub_epi16(stp1_13, stp1_18);
- col[i32 + 19] = _mm_sub_epi16(stp1_12, stp1_19);
- col[i32 + 20] = _mm_sub_epi16(stp1_11, stp1_20);
- col[i32 + 21] = _mm_sub_epi16(stp1_10, stp1_21);
- col[i32 + 22] = _mm_sub_epi16(stp1_9, stp1_22);
- col[i32 + 23] = _mm_sub_epi16(stp1_8, stp1_23);
- col[i32 + 24] = _mm_sub_epi16(stp1_7, stp1_24);
- col[i32 + 25] = _mm_sub_epi16(stp1_6, stp1_25);
- col[i32 + 26] = _mm_sub_epi16(stp1_5, stp1_26);
- col[i32 + 27] = _mm_sub_epi16(stp1_4, stp1_27);
- col[i32 + 28] = _mm_sub_epi16(stp1_3, stp1_28);
- col[i32 + 29] = _mm_sub_epi16(stp1_2, stp1_29);
- col[i32 + 30] = _mm_sub_epi16(stp1_1, stp1_30);
- col[i32 + 31] = _mm_sub_epi16(stp1_0, stp1_31);
- }
- for (i = 0; i < 4; i++) {
- // Second 1-D vp10_idct
- j = i << 3;
-
- // Transpose 32x8 block to 8x32 block
- array_transpose_8x8(col + j, in);
- array_transpose_8x8(col + j + 32, in + 8);
- array_transpose_8x8(col + j + 64, in + 16);
- array_transpose_8x8(col + j + 96, in + 24);
-
- IDCT32
-
- // 2_D: Calculate the results and store them to destination.
- in[0] = _mm_add_epi16(stp1_0, stp1_31);
- in[1] = _mm_add_epi16(stp1_1, stp1_30);
- in[2] = _mm_add_epi16(stp1_2, stp1_29);
- in[3] = _mm_add_epi16(stp1_3, stp1_28);
- in[4] = _mm_add_epi16(stp1_4, stp1_27);
- in[5] = _mm_add_epi16(stp1_5, stp1_26);
- in[6] = _mm_add_epi16(stp1_6, stp1_25);
- in[7] = _mm_add_epi16(stp1_7, stp1_24);
- in[8] = _mm_add_epi16(stp1_8, stp1_23);
- in[9] = _mm_add_epi16(stp1_9, stp1_22);
- in[10] = _mm_add_epi16(stp1_10, stp1_21);
- in[11] = _mm_add_epi16(stp1_11, stp1_20);
- in[12] = _mm_add_epi16(stp1_12, stp1_19);
- in[13] = _mm_add_epi16(stp1_13, stp1_18);
- in[14] = _mm_add_epi16(stp1_14, stp1_17);
- in[15] = _mm_add_epi16(stp1_15, stp1_16);
- in[16] = _mm_sub_epi16(stp1_15, stp1_16);
- in[17] = _mm_sub_epi16(stp1_14, stp1_17);
- in[18] = _mm_sub_epi16(stp1_13, stp1_18);
- in[19] = _mm_sub_epi16(stp1_12, stp1_19);
- in[20] = _mm_sub_epi16(stp1_11, stp1_20);
- in[21] = _mm_sub_epi16(stp1_10, stp1_21);
- in[22] = _mm_sub_epi16(stp1_9, stp1_22);
- in[23] = _mm_sub_epi16(stp1_8, stp1_23);
- in[24] = _mm_sub_epi16(stp1_7, stp1_24);
- in[25] = _mm_sub_epi16(stp1_6, stp1_25);
- in[26] = _mm_sub_epi16(stp1_5, stp1_26);
- in[27] = _mm_sub_epi16(stp1_4, stp1_27);
- in[28] = _mm_sub_epi16(stp1_3, stp1_28);
- in[29] = _mm_sub_epi16(stp1_2, stp1_29);
- in[30] = _mm_sub_epi16(stp1_1, stp1_30);
- in[31] = _mm_sub_epi16(stp1_0, stp1_31);
-
- for (j = 0; j < 32; ++j) {
- // Final rounding and shift
- in[j] = _mm_adds_epi16(in[j], final_rounding);
- in[j] = _mm_srai_epi16(in[j], 6);
- RECON_AND_STORE(dest + j * stride, in[j]);
- }
-
- dest += 8;
- }
-}
-
-void vp10_idct32x32_1_add_sse2(const int16_t *input,
- uint8_t *dest,
- int stride) {
- __m128i dc_value;
- const __m128i zero = _mm_setzero_si128();
- int a, i;
-
- a = dct_const_round_shift(input[0] * cospi_16_64);
- a = dct_const_round_shift(a * cospi_16_64);
- a = ROUND_POWER_OF_TWO(a, 6);
-
- dc_value = _mm_set1_epi16(a);
-
- for (i = 0; i < 4; ++i) {
- int j;
- for (j = 0; j < 32; ++j) {
- RECON_AND_STORE(dest + j * stride, dc_value);
- }
- dest += 8;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static INLINE __m128i clamp_high_sse2(__m128i value, int bd) {
- __m128i ubounded, retval;
- const __m128i zero = _mm_set1_epi16(0);
- const __m128i one = _mm_set1_epi16(1);
- const __m128i max = _mm_subs_epi16(_mm_slli_epi16(one, bd), one);
- ubounded = _mm_cmpgt_epi16(value, max);
- retval = _mm_andnot_si128(ubounded, value);
- ubounded = _mm_and_si128(ubounded, max);
- retval = _mm_or_si128(retval, ubounded);
- retval = _mm_and_si128(retval, _mm_cmpgt_epi16(retval, zero));
- return retval;
-}
-
-void vp10_highbd_idct4x4_16_add_sse2(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[4 * 4];
- tran_low_t *outptr = out;
- int i, j;
- __m128i inptr[4];
- __m128i sign_bits[2];
- __m128i temp_mm, min_input, max_input;
- int test;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- int optimised_cols = 0;
- const __m128i zero = _mm_set1_epi16(0);
- const __m128i eight = _mm_set1_epi16(8);
- const __m128i max = _mm_set1_epi16(12043);
- const __m128i min = _mm_set1_epi16(-12043);
- // Load input into __m128i
- inptr[0] = _mm_loadu_si128((const __m128i *)input);
- inptr[1] = _mm_loadu_si128((const __m128i *)(input + 4));
- inptr[2] = _mm_loadu_si128((const __m128i *)(input + 8));
- inptr[3] = _mm_loadu_si128((const __m128i *)(input + 12));
-
- // Pack to 16 bits
- inptr[0] = _mm_packs_epi32(inptr[0], inptr[1]);
- inptr[1] = _mm_packs_epi32(inptr[2], inptr[3]);
-
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp_mm = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp_mm);
-
- if (!test) {
- // Do the row transform
- vp10_idct4_sse2(inptr);
-
- // Check the min & max values
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp_mm = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp_mm);
-
- if (test) {
- transpose_4x4(inptr);
- sign_bits[0] = _mm_cmplt_epi16(inptr[0], zero);
- sign_bits[1] = _mm_cmplt_epi16(inptr[1], zero);
- inptr[3] = _mm_unpackhi_epi16(inptr[1], sign_bits[1]);
- inptr[2] = _mm_unpacklo_epi16(inptr[1], sign_bits[1]);
- inptr[1] = _mm_unpackhi_epi16(inptr[0], sign_bits[0]);
- inptr[0] = _mm_unpacklo_epi16(inptr[0], sign_bits[0]);
- _mm_storeu_si128((__m128i *)outptr, inptr[0]);
- _mm_storeu_si128((__m128i *)(outptr + 4), inptr[1]);
- _mm_storeu_si128((__m128i *)(outptr + 8), inptr[2]);
- _mm_storeu_si128((__m128i *)(outptr + 12), inptr[3]);
- } else {
- // Set to use the optimised transform for the column
- optimised_cols = 1;
- }
- } else {
- // Run the un-optimised row transform
- for (i = 0; i < 4; ++i) {
- vp10_highbd_idct4_c(input, outptr, bd);
- input += 4;
- outptr += 4;
- }
- }
-
- if (optimised_cols) {
- vp10_idct4_sse2(inptr);
-
- // Final round and shift
- inptr[0] = _mm_add_epi16(inptr[0], eight);
- inptr[1] = _mm_add_epi16(inptr[1], eight);
-
- inptr[0] = _mm_srai_epi16(inptr[0], 4);
- inptr[1] = _mm_srai_epi16(inptr[1], 4);
-
- // Reconstruction and Store
- {
- __m128i d0 = _mm_loadl_epi64((const __m128i *)dest);
- __m128i d2 = _mm_loadl_epi64((const __m128i *)(dest + stride * 2));
- d0 = _mm_unpacklo_epi64(
- d0, _mm_loadl_epi64((const __m128i *)(dest + stride)));
- d2 = _mm_unpacklo_epi64(
- d2, _mm_loadl_epi64((const __m128i *)(dest + stride * 3)));
- d0 = clamp_high_sse2(_mm_adds_epi16(d0, inptr[0]), bd);
- d2 = clamp_high_sse2(_mm_adds_epi16(d2, inptr[1]), bd);
- // store input0
- _mm_storel_epi64((__m128i *)dest, d0);
- // store input1
- d0 = _mm_srli_si128(d0, 8);
- _mm_storel_epi64((__m128i *)(dest + stride), d0);
- // store input2
- _mm_storel_epi64((__m128i *)(dest + stride * 2), d2);
- // store input3
- d2 = _mm_srli_si128(d2, 8);
- _mm_storel_epi64((__m128i *)(dest + stride * 3), d2);
- }
- } else {
- // Run the un-optimised column transform
- tran_low_t temp_in[4], temp_out[4];
- // Columns
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j * 4 + i];
- vp10_highbd_idct4_c(temp_in, temp_out, bd);
- for (j = 0; j < 4; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 4), bd);
- }
- }
- }
-}
-
-void vp10_highbd_idct8x8_64_add_sse2(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[8 * 8];
- tran_low_t *outptr = out;
- int i, j, test;
- __m128i inptr[8];
- __m128i min_input, max_input, temp1, temp2, sign_bits;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- const __m128i zero = _mm_set1_epi16(0);
- const __m128i sixteen = _mm_set1_epi16(16);
- const __m128i max = _mm_set1_epi16(6201);
- const __m128i min = _mm_set1_epi16(-6201);
- int optimised_cols = 0;
-
- // Load input into __m128i & pack to 16 bits
- for (i = 0; i < 8; i++) {
- temp1 = _mm_loadu_si128((const __m128i *)(input + 8 * i));
- temp2 = _mm_loadu_si128((const __m128i *)(input + 8 * i + 4));
- inptr[i] = _mm_packs_epi32(temp1, temp2);
- }
-
- // Find the min & max for the row transform
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 8; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (!test) {
- // Do the row transform
- vp10_idct8_sse2(inptr);
-
- // Find the min & max for the column transform
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 8; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (test) {
- array_transpose_8x8(inptr, inptr);
- for (i = 0; i < 8; i++) {
- sign_bits = _mm_cmplt_epi16(inptr[i], zero);
- temp1 = _mm_unpackhi_epi16(inptr[i], sign_bits);
- temp2 = _mm_unpacklo_epi16(inptr[i], sign_bits);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (2 * i + 1)), temp1);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (2 * i)), temp2);
- }
- } else {
- // Set to use the optimised transform for the column
- optimised_cols = 1;
- }
- } else {
- // Run the un-optimised row transform
- for (i = 0; i < 8; ++i) {
- vp10_highbd_idct8_c(input, outptr, bd);
- input += 8;
- outptr += 8;
- }
- }
-
- if (optimised_cols) {
- vp10_idct8_sse2(inptr);
-
- // Final round & shift and Reconstruction and Store
- {
- __m128i d[8];
- for (i = 0; i < 8; i++) {
- inptr[i] = _mm_add_epi16(inptr[i], sixteen);
- d[i] = _mm_loadu_si128((const __m128i *)(dest + stride*i));
- inptr[i] = _mm_srai_epi16(inptr[i], 5);
- d[i] = clamp_high_sse2(_mm_adds_epi16(d[i], inptr[i]), bd);
- // Store
- _mm_storeu_si128((__m128i *)(dest + stride*i), d[i]);
- }
- }
- } else {
- // Run the un-optimised column transform
- tran_low_t temp_in[8], temp_out[8];
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- vp10_highbd_idct8_c(temp_in, temp_out, bd);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 5), bd);
- }
- }
- }
-}
-
-void vp10_highbd_idct8x8_10_add_sse2(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[8 * 8] = { 0 };
- tran_low_t *outptr = out;
- int i, j, test;
- __m128i inptr[8];
- __m128i min_input, max_input, temp1, temp2, sign_bits;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- const __m128i zero = _mm_set1_epi16(0);
- const __m128i sixteen = _mm_set1_epi16(16);
- const __m128i max = _mm_set1_epi16(6201);
- const __m128i min = _mm_set1_epi16(-6201);
- int optimised_cols = 0;
-
- // Load input into __m128i & pack to 16 bits
- for (i = 0; i < 8; i++) {
- temp1 = _mm_loadu_si128((const __m128i *)(input + 8 * i));
- temp2 = _mm_loadu_si128((const __m128i *)(input + 8 * i + 4));
- inptr[i] = _mm_packs_epi32(temp1, temp2);
- }
-
- // Find the min & max for the row transform
- // only first 4 row has non-zero coefs
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 4; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (!test) {
- // Do the row transform
- vp10_idct8_sse2(inptr);
-
- // Find the min & max for the column transform
- // N.B. Only first 4 cols contain non-zero coeffs
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 8; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (test) {
- // Use fact only first 4 rows contain non-zero coeffs
- array_transpose_4X8(inptr, inptr);
- for (i = 0; i < 4; i++) {
- sign_bits = _mm_cmplt_epi16(inptr[i], zero);
- temp1 = _mm_unpackhi_epi16(inptr[i], sign_bits);
- temp2 = _mm_unpacklo_epi16(inptr[i], sign_bits);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (2 * i + 1)), temp1);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (2 * i)), temp2);
- }
- } else {
- // Set to use the optimised transform for the column
- optimised_cols = 1;
- }
- } else {
- // Run the un-optimised row transform
- for (i = 0; i < 4; ++i) {
- vp10_highbd_idct8_c(input, outptr, bd);
- input += 8;
- outptr += 8;
- }
- }
-
- if (optimised_cols) {
- vp10_idct8_sse2(inptr);
-
- // Final round & shift and Reconstruction and Store
- {
- __m128i d[8];
- for (i = 0; i < 8; i++) {
- inptr[i] = _mm_add_epi16(inptr[i], sixteen);
- d[i] = _mm_loadu_si128((const __m128i *)(dest + stride*i));
- inptr[i] = _mm_srai_epi16(inptr[i], 5);
- d[i] = clamp_high_sse2(_mm_adds_epi16(d[i], inptr[i]), bd);
- // Store
- _mm_storeu_si128((__m128i *)(dest + stride*i), d[i]);
- }
- }
- } else {
- // Run the un-optimised column transform
- tran_low_t temp_in[8], temp_out[8];
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j * 8 + i];
- vp10_highbd_idct8_c(temp_in, temp_out, bd);
- for (j = 0; j < 8; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 5), bd);
- }
- }
- }
-}
-
-void vp10_highbd_idct16x16_256_add_sse2(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[16 * 16];
- tran_low_t *outptr = out;
- int i, j, test;
- __m128i inptr[32];
- __m128i min_input, max_input, temp1, temp2, sign_bits;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- const __m128i zero = _mm_set1_epi16(0);
- const __m128i rounding = _mm_set1_epi16(32);
- const __m128i max = _mm_set1_epi16(3155);
- const __m128i min = _mm_set1_epi16(-3155);
- int optimised_cols = 0;
-
- // Load input into __m128i & pack to 16 bits
- for (i = 0; i < 16; i++) {
- temp1 = _mm_loadu_si128((const __m128i *)(input + 16 * i));
- temp2 = _mm_loadu_si128((const __m128i *)(input + 16 * i + 4));
- inptr[i] = _mm_packs_epi32(temp1, temp2);
- temp1 = _mm_loadu_si128((const __m128i *)(input + 16 * i + 8));
- temp2 = _mm_loadu_si128((const __m128i *)(input + 16 * i + 12));
- inptr[i + 16] = _mm_packs_epi32(temp1, temp2);
- }
-
- // Find the min & max for the row transform
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 32; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (!test) {
- // Do the row transform
- vp10_idct16_sse2(inptr, inptr + 16);
-
- // Find the min & max for the column transform
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 32; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (test) {
- array_transpose_16x16(inptr, inptr + 16);
- for (i = 0; i < 16; i++) {
- sign_bits = _mm_cmplt_epi16(inptr[i], zero);
- temp1 = _mm_unpacklo_epi16(inptr[i], sign_bits);
- temp2 = _mm_unpackhi_epi16(inptr[i], sign_bits);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4)), temp1);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4 + 1)), temp2);
- sign_bits = _mm_cmplt_epi16(inptr[i + 16], zero);
- temp1 = _mm_unpacklo_epi16(inptr[i + 16], sign_bits);
- temp2 = _mm_unpackhi_epi16(inptr[i + 16], sign_bits);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4 + 2)), temp1);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4 + 3)), temp2);
- }
- } else {
- // Set to use the optimised transform for the column
- optimised_cols = 1;
- }
- } else {
- // Run the un-optimised row transform
- for (i = 0; i < 16; ++i) {
- vp10_highbd_idct16_c(input, outptr, bd);
- input += 16;
- outptr += 16;
- }
- }
-
- if (optimised_cols) {
- vp10_idct16_sse2(inptr, inptr + 16);
-
- // Final round & shift and Reconstruction and Store
- {
- __m128i d[2];
- for (i = 0; i < 16; i++) {
- inptr[i ] = _mm_add_epi16(inptr[i ], rounding);
- inptr[i+16] = _mm_add_epi16(inptr[i+16], rounding);
- d[0] = _mm_loadu_si128((const __m128i *)(dest + stride*i));
- d[1] = _mm_loadu_si128((const __m128i *)(dest + stride*i + 8));
- inptr[i ] = _mm_srai_epi16(inptr[i ], 6);
- inptr[i+16] = _mm_srai_epi16(inptr[i+16], 6);
- d[0] = clamp_high_sse2(_mm_add_epi16(d[0], inptr[i ]), bd);
- d[1] = clamp_high_sse2(_mm_add_epi16(d[1], inptr[i+16]), bd);
- // Store
- _mm_storeu_si128((__m128i *)(dest + stride*i), d[0]);
- _mm_storeu_si128((__m128i *)(dest + stride*i + 8), d[1]);
- }
- }
- } else {
- // Run the un-optimised column transform
- tran_low_t temp_in[16], temp_out[16];
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- vp10_highbd_idct16_c(temp_in, temp_out, bd);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 6), bd);
- }
- }
- }
-}
-
-void vp10_highbd_idct16x16_10_add_sse2(const tran_low_t *input, uint8_t *dest8,
- int stride, int bd) {
- tran_low_t out[16 * 16] = { 0 };
- tran_low_t *outptr = out;
- int i, j, test;
- __m128i inptr[32];
- __m128i min_input, max_input, temp1, temp2, sign_bits;
- uint16_t *dest = CONVERT_TO_SHORTPTR(dest8);
- const __m128i zero = _mm_set1_epi16(0);
- const __m128i rounding = _mm_set1_epi16(32);
- const __m128i max = _mm_set1_epi16(3155);
- const __m128i min = _mm_set1_epi16(-3155);
- int optimised_cols = 0;
-
- // Load input into __m128i & pack to 16 bits
- for (i = 0; i < 16; i++) {
- temp1 = _mm_loadu_si128((const __m128i *)(input + 16 * i));
- temp2 = _mm_loadu_si128((const __m128i *)(input + 16 * i + 4));
- inptr[i] = _mm_packs_epi32(temp1, temp2);
- temp1 = _mm_loadu_si128((const __m128i *)(input + 16 * i + 8));
- temp2 = _mm_loadu_si128((const __m128i *)(input + 16 * i + 12));
- inptr[i + 16] = _mm_packs_epi32(temp1, temp2);
- }
-
- // Find the min & max for the row transform
- // Since all non-zero dct coefficients are in upper-left 4x4 area,
- // we only need to consider first 4 rows here.
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 4; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (!test) {
- // Do the row transform (N.B. This transposes inptr)
- vp10_idct16_sse2(inptr, inptr + 16);
-
- // Find the min & max for the column transform
- // N.B. Only first 4 cols contain non-zero coeffs
- max_input = _mm_max_epi16(inptr[0], inptr[1]);
- min_input = _mm_min_epi16(inptr[0], inptr[1]);
- for (i = 2; i < 16; i++) {
- max_input = _mm_max_epi16(max_input, inptr[i]);
- min_input = _mm_min_epi16(min_input, inptr[i]);
- }
- max_input = _mm_cmpgt_epi16(max_input, max);
- min_input = _mm_cmplt_epi16(min_input, min);
- temp1 = _mm_or_si128(max_input, min_input);
- test = _mm_movemask_epi8(temp1);
-
- if (test) {
- // Use fact only first 4 rows contain non-zero coeffs
- array_transpose_8x8(inptr, inptr);
- array_transpose_8x8(inptr + 8, inptr + 16);
- for (i = 0; i < 4; i++) {
- sign_bits = _mm_cmplt_epi16(inptr[i], zero);
- temp1 = _mm_unpacklo_epi16(inptr[i], sign_bits);
- temp2 = _mm_unpackhi_epi16(inptr[i], sign_bits);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4)), temp1);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4 + 1)), temp2);
- sign_bits = _mm_cmplt_epi16(inptr[i + 16], zero);
- temp1 = _mm_unpacklo_epi16(inptr[i + 16], sign_bits);
- temp2 = _mm_unpackhi_epi16(inptr[i + 16], sign_bits);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4 + 2)), temp1);
- _mm_storeu_si128((__m128i *)(outptr + 4 * (i * 4 + 3)), temp2);
- }
- } else {
- // Set to use the optimised transform for the column
- optimised_cols = 1;
- }
- } else {
- // Run the un-optimised row transform
- for (i = 0; i < 4; ++i) {
- vp10_highbd_idct16_c(input, outptr, bd);
- input += 16;
- outptr += 16;
- }
- }
-
- if (optimised_cols) {
- vp10_idct16_sse2(inptr, inptr + 16);
-
- // Final round & shift and Reconstruction and Store
- {
- __m128i d[2];
- for (i = 0; i < 16; i++) {
- inptr[i ] = _mm_add_epi16(inptr[i ], rounding);
- inptr[i+16] = _mm_add_epi16(inptr[i+16], rounding);
- d[0] = _mm_loadu_si128((const __m128i *)(dest + stride*i));
- d[1] = _mm_loadu_si128((const __m128i *)(dest + stride*i + 8));
- inptr[i ] = _mm_srai_epi16(inptr[i ], 6);
- inptr[i+16] = _mm_srai_epi16(inptr[i+16], 6);
- d[0] = clamp_high_sse2(_mm_add_epi16(d[0], inptr[i ]), bd);
- d[1] = clamp_high_sse2(_mm_add_epi16(d[1], inptr[i+16]), bd);
- // Store
- _mm_storeu_si128((__m128i *)(dest + stride*i), d[0]);
- _mm_storeu_si128((__m128i *)(dest + stride*i + 8), d[1]);
- }
- }
- } else {
- // Run the un-optimised column transform
- tran_low_t temp_in[16], temp_out[16];
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j * 16 + i];
- vp10_highbd_idct16_c(temp_in, temp_out, bd);
- for (j = 0; j < 16; ++j) {
- dest[j * stride + i] = highbd_clip_pixel_add(
- dest[j * stride + i], ROUND_POWER_OF_TWO(temp_out[j], 6), bd);
- }
- }
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VPX_DSP_X86_INV_TXFM_SSE2_H_
-#define VPX_DSP_X86_INV_TXFM_SSE2_H_
-
-#include <emmintrin.h> // SSE2
-#include "./vpx_config.h"
-#include "vpx/vpx_integer.h"
-#include "vp10/common/vp10_inv_txfm.h"
-
-// perform 8x8 transpose
-static INLINE void array_transpose_8x8(__m128i *in, __m128i *res) {
- const __m128i tr0_0 = _mm_unpacklo_epi16(in[0], in[1]);
- const __m128i tr0_1 = _mm_unpacklo_epi16(in[2], in[3]);
- const __m128i tr0_2 = _mm_unpackhi_epi16(in[0], in[1]);
- const __m128i tr0_3 = _mm_unpackhi_epi16(in[2], in[3]);
- const __m128i tr0_4 = _mm_unpacklo_epi16(in[4], in[5]);
- const __m128i tr0_5 = _mm_unpacklo_epi16(in[6], in[7]);
- const __m128i tr0_6 = _mm_unpackhi_epi16(in[4], in[5]);
- const __m128i tr0_7 = _mm_unpackhi_epi16(in[6], in[7]);
-
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_4, tr0_5);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_2, tr0_3);
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_2, tr0_3);
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7);
-
- res[0] = _mm_unpacklo_epi64(tr1_0, tr1_1);
- res[1] = _mm_unpackhi_epi64(tr1_0, tr1_1);
- res[2] = _mm_unpacklo_epi64(tr1_2, tr1_3);
- res[3] = _mm_unpackhi_epi64(tr1_2, tr1_3);
- res[4] = _mm_unpacklo_epi64(tr1_4, tr1_5);
- res[5] = _mm_unpackhi_epi64(tr1_4, tr1_5);
- res[6] = _mm_unpacklo_epi64(tr1_6, tr1_7);
- res[7] = _mm_unpackhi_epi64(tr1_6, tr1_7);
-}
-
-#define TRANSPOSE_8X4(in0, in1, in2, in3, out0, out1) \
- { \
- const __m128i tr0_0 = _mm_unpacklo_epi16(in0, in1); \
- const __m128i tr0_1 = _mm_unpacklo_epi16(in2, in3); \
- \
- in0 = _mm_unpacklo_epi32(tr0_0, tr0_1); /* i1 i0 */ \
- in1 = _mm_unpackhi_epi32(tr0_0, tr0_1); /* i3 i2 */ \
- }
-
-static INLINE void array_transpose_4X8(__m128i *in, __m128i * out) {
- const __m128i tr0_0 = _mm_unpacklo_epi16(in[0], in[1]);
- const __m128i tr0_1 = _mm_unpacklo_epi16(in[2], in[3]);
- const __m128i tr0_4 = _mm_unpacklo_epi16(in[4], in[5]);
- const __m128i tr0_5 = _mm_unpacklo_epi16(in[6], in[7]);
-
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5);
-
- out[0] = _mm_unpacklo_epi64(tr1_0, tr1_4);
- out[1] = _mm_unpackhi_epi64(tr1_0, tr1_4);
- out[2] = _mm_unpacklo_epi64(tr1_2, tr1_6);
- out[3] = _mm_unpackhi_epi64(tr1_2, tr1_6);
-}
-
-static INLINE void array_transpose_16x16(__m128i *res0, __m128i *res1) {
- __m128i tbuf[8];
- array_transpose_8x8(res0, res0);
- array_transpose_8x8(res1, tbuf);
- array_transpose_8x8(res0 + 8, res1);
- array_transpose_8x8(res1 + 8, res1 + 8);
-
- res0[8] = tbuf[0];
- res0[9] = tbuf[1];
- res0[10] = tbuf[2];
- res0[11] = tbuf[3];
- res0[12] = tbuf[4];
- res0[13] = tbuf[5];
- res0[14] = tbuf[6];
- res0[15] = tbuf[7];
-}
-
-static INLINE void load_buffer_8x16(const int16_t *input, __m128i *in) {
- in[0] = _mm_load_si128((const __m128i *)(input + 0 * 16));
- in[1] = _mm_load_si128((const __m128i *)(input + 1 * 16));
- in[2] = _mm_load_si128((const __m128i *)(input + 2 * 16));
- in[3] = _mm_load_si128((const __m128i *)(input + 3 * 16));
- in[4] = _mm_load_si128((const __m128i *)(input + 4 * 16));
- in[5] = _mm_load_si128((const __m128i *)(input + 5 * 16));
- in[6] = _mm_load_si128((const __m128i *)(input + 6 * 16));
- in[7] = _mm_load_si128((const __m128i *)(input + 7 * 16));
-
- in[8] = _mm_load_si128((const __m128i *)(input + 8 * 16));
- in[9] = _mm_load_si128((const __m128i *)(input + 9 * 16));
- in[10] = _mm_load_si128((const __m128i *)(input + 10 * 16));
- in[11] = _mm_load_si128((const __m128i *)(input + 11 * 16));
- in[12] = _mm_load_si128((const __m128i *)(input + 12 * 16));
- in[13] = _mm_load_si128((const __m128i *)(input + 13 * 16));
- in[14] = _mm_load_si128((const __m128i *)(input + 14 * 16));
- in[15] = _mm_load_si128((const __m128i *)(input + 15 * 16));
-}
-
-#define RECON_AND_STORE(dest, in_x) \
- { \
- __m128i d0 = _mm_loadl_epi64((__m128i *)(dest)); \
- d0 = _mm_unpacklo_epi8(d0, zero); \
- d0 = _mm_add_epi16(in_x, d0); \
- d0 = _mm_packus_epi16(d0, d0); \
- _mm_storel_epi64((__m128i *)(dest), d0); \
- }
-
-static INLINE void write_buffer_8x16(uint8_t *dest, __m128i *in, int stride) {
- const __m128i final_rounding = _mm_set1_epi16(1<<5);
- const __m128i zero = _mm_setzero_si128();
- // Final rounding and shift
- in[0] = _mm_adds_epi16(in[0], final_rounding);
- in[1] = _mm_adds_epi16(in[1], final_rounding);
- in[2] = _mm_adds_epi16(in[2], final_rounding);
- in[3] = _mm_adds_epi16(in[3], final_rounding);
- in[4] = _mm_adds_epi16(in[4], final_rounding);
- in[5] = _mm_adds_epi16(in[5], final_rounding);
- in[6] = _mm_adds_epi16(in[6], final_rounding);
- in[7] = _mm_adds_epi16(in[7], final_rounding);
- in[8] = _mm_adds_epi16(in[8], final_rounding);
- in[9] = _mm_adds_epi16(in[9], final_rounding);
- in[10] = _mm_adds_epi16(in[10], final_rounding);
- in[11] = _mm_adds_epi16(in[11], final_rounding);
- in[12] = _mm_adds_epi16(in[12], final_rounding);
- in[13] = _mm_adds_epi16(in[13], final_rounding);
- in[14] = _mm_adds_epi16(in[14], final_rounding);
- in[15] = _mm_adds_epi16(in[15], final_rounding);
-
- in[0] = _mm_srai_epi16(in[0], 6);
- in[1] = _mm_srai_epi16(in[1], 6);
- in[2] = _mm_srai_epi16(in[2], 6);
- in[3] = _mm_srai_epi16(in[3], 6);
- in[4] = _mm_srai_epi16(in[4], 6);
- in[5] = _mm_srai_epi16(in[5], 6);
- in[6] = _mm_srai_epi16(in[6], 6);
- in[7] = _mm_srai_epi16(in[7], 6);
- in[8] = _mm_srai_epi16(in[8], 6);
- in[9] = _mm_srai_epi16(in[9], 6);
- in[10] = _mm_srai_epi16(in[10], 6);
- in[11] = _mm_srai_epi16(in[11], 6);
- in[12] = _mm_srai_epi16(in[12], 6);
- in[13] = _mm_srai_epi16(in[13], 6);
- in[14] = _mm_srai_epi16(in[14], 6);
- in[15] = _mm_srai_epi16(in[15], 6);
-
- RECON_AND_STORE(dest + 0 * stride, in[0]);
- RECON_AND_STORE(dest + 1 * stride, in[1]);
- RECON_AND_STORE(dest + 2 * stride, in[2]);
- RECON_AND_STORE(dest + 3 * stride, in[3]);
- RECON_AND_STORE(dest + 4 * stride, in[4]);
- RECON_AND_STORE(dest + 5 * stride, in[5]);
- RECON_AND_STORE(dest + 6 * stride, in[6]);
- RECON_AND_STORE(dest + 7 * stride, in[7]);
- RECON_AND_STORE(dest + 8 * stride, in[8]);
- RECON_AND_STORE(dest + 9 * stride, in[9]);
- RECON_AND_STORE(dest + 10 * stride, in[10]);
- RECON_AND_STORE(dest + 11 * stride, in[11]);
- RECON_AND_STORE(dest + 12 * stride, in[12]);
- RECON_AND_STORE(dest + 13 * stride, in[13]);
- RECON_AND_STORE(dest + 14 * stride, in[14]);
- RECON_AND_STORE(dest + 15 * stride, in[15]);
-}
-
-void idct4_sse2(__m128i *in);
-void idct8_sse2(__m128i *in);
-void idct16_sse2(__m128i *in0, __m128i *in1);
-void iadst4_sse2(__m128i *in);
-void iadst8_sse2(__m128i *in);
-void iadst16_sse2(__m128i *in0, __m128i *in1);
-
-#endif // VPX_DSP_X86_INV_TXFM_SSE2_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <stdlib.h> // qsort()
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "./vpx_scale_rtcd.h"
-
-#include "vpx_dsp/bitreader_buffer.h"
-#include "vpx_dsp/bitreader.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/mem_ops.h"
-#include "vpx_scale/vpx_scale.h"
-#include "vpx_util/vpx_thread.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/idct.h"
-#include "vp10/common/thread_common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconintra.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/seg_common.h"
-#include "vp10/common/tile_common.h"
-
-#include "vp10/decoder/decodeframe.h"
-#include "vp10/decoder/detokenize.h"
-#include "vp10/decoder/decodemv.h"
-#include "vp10/decoder/decoder.h"
-#include "vp10/decoder/dsubexp.h"
-
-#define MAX_VP9_HEADER_SIZE 80
-
-static int is_compound_reference_allowed(const VP10_COMMON *cm) {
- int i;
- if (frame_is_intra_only(cm))
- return 0;
- for (i = 1; i < REFS_PER_FRAME; ++i)
- if (cm->ref_frame_sign_bias[i + 1] != cm->ref_frame_sign_bias[1])
- return 1;
-
- return 0;
-}
-
-static void setup_compound_reference_mode(VP10_COMMON *cm) {
- if (cm->ref_frame_sign_bias[LAST_FRAME] ==
- cm->ref_frame_sign_bias[GOLDEN_FRAME]) {
- cm->comp_fixed_ref = ALTREF_FRAME;
- cm->comp_var_ref[0] = LAST_FRAME;
- cm->comp_var_ref[1] = GOLDEN_FRAME;
- } else if (cm->ref_frame_sign_bias[LAST_FRAME] ==
- cm->ref_frame_sign_bias[ALTREF_FRAME]) {
- cm->comp_fixed_ref = GOLDEN_FRAME;
- cm->comp_var_ref[0] = LAST_FRAME;
- cm->comp_var_ref[1] = ALTREF_FRAME;
- } else {
- cm->comp_fixed_ref = LAST_FRAME;
- cm->comp_var_ref[0] = GOLDEN_FRAME;
- cm->comp_var_ref[1] = ALTREF_FRAME;
- }
-}
-
-static int read_is_valid(const uint8_t *start, size_t len, const uint8_t *end) {
- return len != 0 && len <= (size_t)(end - start);
-}
-
-static int decode_unsigned_max(struct vpx_read_bit_buffer *rb, int max) {
- const int data = vpx_rb_read_literal(rb, get_unsigned_bits(max));
- return data > max ? max : data;
-}
-
-#if CONFIG_MISC_FIXES
-static TX_MODE read_tx_mode(struct vpx_read_bit_buffer *rb) {
- return vpx_rb_read_bit(rb) ? TX_MODE_SELECT : vpx_rb_read_literal(rb, 2);
-}
-#else
-static TX_MODE read_tx_mode(vpx_reader *r) {
- TX_MODE tx_mode = vpx_read_literal(r, 2);
- if (tx_mode == ALLOW_32X32)
- tx_mode += vpx_read_bit(r);
- return tx_mode;
-}
-#endif
-
-static void read_tx_mode_probs(struct tx_probs *tx_probs, vpx_reader *r) {
- int i, j;
-
- for (i = 0; i < TX_SIZE_CONTEXTS; ++i)
- for (j = 0; j < TX_SIZES - 3; ++j)
- vp10_diff_update_prob(r, &tx_probs->p8x8[i][j]);
-
- for (i = 0; i < TX_SIZE_CONTEXTS; ++i)
- for (j = 0; j < TX_SIZES - 2; ++j)
- vp10_diff_update_prob(r, &tx_probs->p16x16[i][j]);
-
- for (i = 0; i < TX_SIZE_CONTEXTS; ++i)
- for (j = 0; j < TX_SIZES - 1; ++j)
- vp10_diff_update_prob(r, &tx_probs->p32x32[i][j]);
-}
-
-static void read_switchable_interp_probs(FRAME_CONTEXT *fc, vpx_reader *r) {
- int i, j;
- for (j = 0; j < SWITCHABLE_FILTER_CONTEXTS; ++j)
- for (i = 0; i < SWITCHABLE_FILTERS - 1; ++i)
- vp10_diff_update_prob(r, &fc->switchable_interp_prob[j][i]);
-}
-
-static void read_inter_mode_probs(FRAME_CONTEXT *fc, vpx_reader *r) {
- int i, j;
- for (i = 0; i < INTER_MODE_CONTEXTS; ++i)
- for (j = 0; j < INTER_MODES - 1; ++j)
- vp10_diff_update_prob(r, &fc->inter_mode_probs[i][j]);
-}
-
-#if CONFIG_MISC_FIXES
-static REFERENCE_MODE read_frame_reference_mode(const VP10_COMMON *cm,
- struct vpx_read_bit_buffer *rb) {
- if (is_compound_reference_allowed(cm)) {
- return vpx_rb_read_bit(rb) ? REFERENCE_MODE_SELECT
- : (vpx_rb_read_bit(rb) ? COMPOUND_REFERENCE
- : SINGLE_REFERENCE);
- } else {
- return SINGLE_REFERENCE;
- }
-}
-#else
-static REFERENCE_MODE read_frame_reference_mode(const VP10_COMMON *cm,
- vpx_reader *r) {
- if (is_compound_reference_allowed(cm)) {
- return vpx_read_bit(r) ? (vpx_read_bit(r) ? REFERENCE_MODE_SELECT
- : COMPOUND_REFERENCE)
- : SINGLE_REFERENCE;
- } else {
- return SINGLE_REFERENCE;
- }
-}
-#endif
-
-static void read_frame_reference_mode_probs(VP10_COMMON *cm, vpx_reader *r) {
- FRAME_CONTEXT *const fc = cm->fc;
- int i;
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- for (i = 0; i < COMP_INTER_CONTEXTS; ++i)
- vp10_diff_update_prob(r, &fc->comp_inter_prob[i]);
-
- if (cm->reference_mode != COMPOUND_REFERENCE)
- for (i = 0; i < REF_CONTEXTS; ++i) {
- vp10_diff_update_prob(r, &fc->single_ref_prob[i][0]);
- vp10_diff_update_prob(r, &fc->single_ref_prob[i][1]);
- }
-
- if (cm->reference_mode != SINGLE_REFERENCE)
- for (i = 0; i < REF_CONTEXTS; ++i)
- vp10_diff_update_prob(r, &fc->comp_ref_prob[i]);
-}
-
-static void update_mv_probs(vpx_prob *p, int n, vpx_reader *r) {
- int i;
- for (i = 0; i < n; ++i)
-#if CONFIG_MISC_FIXES
- vp10_diff_update_prob(r, &p[i]);
-#else
- if (vpx_read(r, MV_UPDATE_PROB))
- p[i] = (vpx_read_literal(r, 7) << 1) | 1;
-#endif
-}
-
-static void read_mv_probs(nmv_context *ctx, int allow_hp, vpx_reader *r) {
- int i, j;
-
- update_mv_probs(ctx->joints, MV_JOINTS - 1, r);
-
- for (i = 0; i < 2; ++i) {
- nmv_component *const comp_ctx = &ctx->comps[i];
- update_mv_probs(&comp_ctx->sign, 1, r);
- update_mv_probs(comp_ctx->classes, MV_CLASSES - 1, r);
- update_mv_probs(comp_ctx->class0, CLASS0_SIZE - 1, r);
- update_mv_probs(comp_ctx->bits, MV_OFFSET_BITS, r);
- }
-
- for (i = 0; i < 2; ++i) {
- nmv_component *const comp_ctx = &ctx->comps[i];
- for (j = 0; j < CLASS0_SIZE; ++j)
- update_mv_probs(comp_ctx->class0_fp[j], MV_FP_SIZE - 1, r);
- update_mv_probs(comp_ctx->fp, 3, r);
- }
-
- if (allow_hp) {
- for (i = 0; i < 2; ++i) {
- nmv_component *const comp_ctx = &ctx->comps[i];
- update_mv_probs(&comp_ctx->class0_hp, 1, r);
- update_mv_probs(&comp_ctx->hp, 1, r);
- }
- }
-}
-
-static void inverse_transform_block_inter(MACROBLOCKD* xd, int plane,
- const TX_SIZE tx_size,
- uint8_t *dst, int stride,
- int eob, int block) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- TX_TYPE tx_type = get_tx_type(pd->plane_type, xd, block);
- const int seg_id = xd->mi[0]->mbmi.segment_id;
- if (eob > 0) {
- tran_low_t *const dqcoeff = pd->dqcoeff;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_4X4:
- vp10_highbd_inv_txfm_add_4x4(dqcoeff, dst, stride, eob, xd->bd,
- tx_type, xd->lossless[seg_id]);
- break;
- case TX_8X8:
- vp10_highbd_inv_txfm_add_8x8(dqcoeff, dst, stride, eob, xd->bd,
- tx_type);
- break;
- case TX_16X16:
- vp10_highbd_inv_txfm_add_16x16(dqcoeff, dst, stride, eob, xd->bd,
- tx_type);
- break;
- case TX_32X32:
- vp10_highbd_inv_txfm_add_32x32(dqcoeff, dst, stride, eob, xd->bd,
- tx_type);
- break;
- default:
- assert(0 && "Invalid transform size");
- return;
- }
- } else {
-#endif // CONFIG_VP9_HIGHBITDEPTH
- switch (tx_size) {
- case TX_4X4:
- vp10_inv_txfm_add_4x4(dqcoeff, dst, stride, eob, tx_type,
- xd->lossless[seg_id]);
- break;
- case TX_8X8:
- vp10_inv_txfm_add_8x8(dqcoeff, dst, stride, eob, tx_type);
- break;
- case TX_16X16:
- vp10_inv_txfm_add_16x16(dqcoeff, dst, stride, eob, tx_type);
- break;
- case TX_32X32:
- vp10_inv_txfm_add_32x32(dqcoeff, dst, stride, eob, tx_type);
- break;
- default:
- assert(0 && "Invalid transform size");
- return;
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- if (eob == 1) {
- dqcoeff[0] = 0;
- } else {
- if (tx_size <= TX_16X16 && eob <= 10)
- memset(dqcoeff, 0, 4 * (4 << tx_size) * sizeof(dqcoeff[0]));
- else if (tx_size == TX_32X32 && eob <= 34)
- memset(dqcoeff, 0, 256 * sizeof(dqcoeff[0]));
- else
- memset(dqcoeff, 0, (16 << (tx_size << 1)) * sizeof(dqcoeff[0]));
- }
- }
-}
-
-static void inverse_transform_block_intra(MACROBLOCKD* xd, int plane,
- const TX_TYPE tx_type,
- const TX_SIZE tx_size,
- uint8_t *dst, int stride,
- int eob) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- const int seg_id = xd->mi[0]->mbmi.segment_id;
- if (eob > 0) {
- tran_low_t *const dqcoeff = pd->dqcoeff;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_4X4:
- vp10_highbd_inv_txfm_add_4x4(dqcoeff, dst, stride, eob, xd->bd,
- tx_type, xd->lossless[seg_id]);
- break;
- case TX_8X8:
- vp10_highbd_inv_txfm_add_8x8(dqcoeff, dst, stride, eob, xd->bd,
- tx_type);
- break;
- case TX_16X16:
- vp10_highbd_inv_txfm_add_16x16(dqcoeff, dst, stride, eob, xd->bd,
- tx_type);
- break;
- case TX_32X32:
- vp10_highbd_inv_txfm_add_32x32(dqcoeff, dst, stride, eob, xd->bd,
- tx_type);
- break;
- default:
- assert(0 && "Invalid transform size");
- return;
- }
- } else {
-#endif // CONFIG_VP9_HIGHBITDEPTH
- switch (tx_size) {
- case TX_4X4:
- vp10_inv_txfm_add_4x4(dqcoeff, dst, stride, eob, tx_type,
- xd->lossless[seg_id]);
- break;
- case TX_8X8:
- vp10_inv_txfm_add_8x8(dqcoeff, dst, stride, eob, tx_type);
- break;
- case TX_16X16:
- vp10_inv_txfm_add_16x16(dqcoeff, dst, stride, eob, tx_type);
- break;
- case TX_32X32:
- vp10_inv_txfm_add_32x32(dqcoeff, dst, stride, eob, tx_type);
- break;
- default:
- assert(0 && "Invalid transform size");
- return;
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- if (eob == 1) {
- dqcoeff[0] = 0;
- } else {
- if (tx_type == DCT_DCT && tx_size <= TX_16X16 && eob <= 10)
- memset(dqcoeff, 0, 4 * (4 << tx_size) * sizeof(dqcoeff[0]));
- else if (tx_size == TX_32X32 && eob <= 34)
- memset(dqcoeff, 0, 256 * sizeof(dqcoeff[0]));
- else
- memset(dqcoeff, 0, (16 << (tx_size << 1)) * sizeof(dqcoeff[0]));
- }
- }
-}
-
-static void predict_and_reconstruct_intra_block(MACROBLOCKD *const xd,
- vpx_reader *r,
- MB_MODE_INFO *const mbmi,
- int plane,
- int row, int col,
- TX_SIZE tx_size) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- PREDICTION_MODE mode = (plane == 0) ? mbmi->mode : mbmi->uv_mode;
- PLANE_TYPE plane_type = (plane == 0) ? PLANE_TYPE_Y : PLANE_TYPE_UV;
- uint8_t *dst;
- int block_idx = (row << 1) + col;
- dst = &pd->dst.buf[4 * row * pd->dst.stride + 4 * col];
-
- if (mbmi->sb_type < BLOCK_8X8)
- if (plane == 0)
- mode = xd->mi[0]->bmi[(row << 1) + col].as_mode;
-
- vp10_predict_intra_block(xd, pd->n4_wl, pd->n4_hl, tx_size, mode,
- dst, pd->dst.stride, dst, pd->dst.stride,
- col, row, plane);
-
- if (!mbmi->skip) {
- TX_TYPE tx_type = get_tx_type(plane_type, xd, block_idx);
- const scan_order *sc = get_scan(tx_size, tx_type);
- const int eob = vp10_decode_block_tokens(xd, plane, sc, col, row, tx_size,
- r, mbmi->segment_id);
- inverse_transform_block_intra(xd, plane, tx_type, tx_size,
- dst, pd->dst.stride, eob);
- }
-}
-
-static int reconstruct_inter_block(MACROBLOCKD *const xd, vpx_reader *r,
- MB_MODE_INFO *const mbmi, int plane,
- int row, int col, TX_SIZE tx_size) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- PLANE_TYPE plane_type = (plane == 0) ? PLANE_TYPE_Y : PLANE_TYPE_UV;
- int block_idx = (row << 1) + col;
- TX_TYPE tx_type = get_tx_type(plane_type, xd, block_idx);
- const scan_order *sc = get_scan(tx_size, tx_type);
- const int eob = vp10_decode_block_tokens(xd, plane, sc, col, row, tx_size, r,
- mbmi->segment_id);
-
- inverse_transform_block_inter(xd, plane, tx_size,
- &pd->dst.buf[4 * row * pd->dst.stride + 4 * col],
- pd->dst.stride, eob, block_idx);
- return eob;
-}
-
-static void build_mc_border(const uint8_t *src, int src_stride,
- uint8_t *dst, int dst_stride,
- int x, int y, int b_w, int b_h, int w, int h) {
- // Get a pointer to the start of the real data for this row.
- const uint8_t *ref_row = src - x - y * src_stride;
-
- if (y >= h)
- ref_row += (h - 1) * src_stride;
- else if (y > 0)
- ref_row += y * src_stride;
-
- do {
- int right = 0, copy;
- int left = x < 0 ? -x : 0;
-
- if (left > b_w)
- left = b_w;
-
- if (x + b_w > w)
- right = x + b_w - w;
-
- if (right > b_w)
- right = b_w;
-
- copy = b_w - left - right;
-
- if (left)
- memset(dst, ref_row[0], left);
-
- if (copy)
- memcpy(dst + left, ref_row + x + left, copy);
-
- if (right)
- memset(dst + left + copy, ref_row[w - 1], right);
-
- dst += dst_stride;
- ++y;
-
- if (y > 0 && y < h)
- ref_row += src_stride;
- } while (--b_h);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void high_build_mc_border(const uint8_t *src8, int src_stride,
- uint16_t *dst, int dst_stride,
- int x, int y, int b_w, int b_h,
- int w, int h) {
- // Get a pointer to the start of the real data for this row.
- const uint16_t *src = CONVERT_TO_SHORTPTR(src8);
- const uint16_t *ref_row = src - x - y * src_stride;
-
- if (y >= h)
- ref_row += (h - 1) * src_stride;
- else if (y > 0)
- ref_row += y * src_stride;
-
- do {
- int right = 0, copy;
- int left = x < 0 ? -x : 0;
-
- if (left > b_w)
- left = b_w;
-
- if (x + b_w > w)
- right = x + b_w - w;
-
- if (right > b_w)
- right = b_w;
-
- copy = b_w - left - right;
-
- if (left)
- vpx_memset16(dst, ref_row[0], left);
-
- if (copy)
- memcpy(dst + left, ref_row + x + left, copy * sizeof(uint16_t));
-
- if (right)
- vpx_memset16(dst + left + copy, ref_row[w - 1], right);
-
- dst += dst_stride;
- ++y;
-
- if (y > 0 && y < h)
- ref_row += src_stride;
- } while (--b_h);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void extend_and_predict(const uint8_t *buf_ptr1, int pre_buf_stride,
- int x0, int y0, int b_w, int b_h,
- int frame_width, int frame_height,
- int border_offset,
- uint8_t *const dst, int dst_buf_stride,
- int subpel_x, int subpel_y,
- const InterpKernel *kernel,
- const struct scale_factors *sf,
- MACROBLOCKD *xd,
- int w, int h, int ref, int xs, int ys) {
- DECLARE_ALIGNED(16, uint16_t, mc_buf_high[80 * 2 * 80 * 2]);
- const uint8_t *buf_ptr;
-
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- high_build_mc_border(buf_ptr1, pre_buf_stride, mc_buf_high, b_w,
- x0, y0, b_w, b_h, frame_width, frame_height);
- buf_ptr = CONVERT_TO_BYTEPTR(mc_buf_high) + border_offset;
- } else {
- build_mc_border(buf_ptr1, pre_buf_stride, (uint8_t *)mc_buf_high, b_w,
- x0, y0, b_w, b_h, frame_width, frame_height);
- buf_ptr = ((uint8_t *)mc_buf_high) + border_offset;
- }
-
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- high_inter_predictor(buf_ptr, b_w, dst, dst_buf_stride, subpel_x,
- subpel_y, sf, w, h, ref, kernel, xs, ys, xd->bd);
- } else {
- inter_predictor(buf_ptr, b_w, dst, dst_buf_stride, subpel_x,
- subpel_y, sf, w, h, ref, kernel, xs, ys);
- }
-}
-#else
-static void extend_and_predict(const uint8_t *buf_ptr1, int pre_buf_stride,
- int x0, int y0, int b_w, int b_h,
- int frame_width, int frame_height,
- int border_offset,
- uint8_t *const dst, int dst_buf_stride,
- int subpel_x, int subpel_y,
- const InterpKernel *kernel,
- const struct scale_factors *sf,
- int w, int h, int ref, int xs, int ys) {
- DECLARE_ALIGNED(16, uint8_t, mc_buf[80 * 2 * 80 * 2]);
- const uint8_t *buf_ptr;
-
- build_mc_border(buf_ptr1, pre_buf_stride, mc_buf, b_w,
- x0, y0, b_w, b_h, frame_width, frame_height);
- buf_ptr = mc_buf + border_offset;
-
- inter_predictor(buf_ptr, b_w, dst, dst_buf_stride, subpel_x,
- subpel_y, sf, w, h, ref, kernel, xs, ys);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void dec_build_inter_predictors(VP10Decoder *const pbi, MACROBLOCKD *xd,
- int plane, int bw, int bh, int x,
- int y, int w, int h, int mi_x, int mi_y,
- const InterpKernel *kernel,
- const struct scale_factors *sf,
- struct buf_2d *pre_buf,
- struct buf_2d *dst_buf, const MV* mv,
- RefCntBuffer *ref_frame_buf,
- int is_scaled, int ref) {
- VP10_COMMON *const cm = &pbi->common;
- struct macroblockd_plane *const pd = &xd->plane[plane];
- uint8_t *const dst = dst_buf->buf + dst_buf->stride * y + x;
- MV32 scaled_mv;
- int xs, ys, x0, y0, x0_16, y0_16, frame_width, frame_height,
- buf_stride, subpel_x, subpel_y;
- uint8_t *ref_frame, *buf_ptr;
-
- // Get reference frame pointer, width and height.
- if (plane == 0) {
- frame_width = ref_frame_buf->buf.y_crop_width;
- frame_height = ref_frame_buf->buf.y_crop_height;
- ref_frame = ref_frame_buf->buf.y_buffer;
- } else {
- frame_width = ref_frame_buf->buf.uv_crop_width;
- frame_height = ref_frame_buf->buf.uv_crop_height;
- ref_frame = plane == 1 ? ref_frame_buf->buf.u_buffer
- : ref_frame_buf->buf.v_buffer;
- }
-
- if (is_scaled) {
- const MV mv_q4 = clamp_mv_to_umv_border_sb(xd, mv, bw, bh,
- pd->subsampling_x,
- pd->subsampling_y);
- // Co-ordinate of containing block to pixel precision.
- int x_start = (-xd->mb_to_left_edge >> (3 + pd->subsampling_x));
- int y_start = (-xd->mb_to_top_edge >> (3 + pd->subsampling_y));
-
- // Co-ordinate of the block to 1/16th pixel precision.
- x0_16 = (x_start + x) << SUBPEL_BITS;
- y0_16 = (y_start + y) << SUBPEL_BITS;
-
- // Co-ordinate of current block in reference frame
- // to 1/16th pixel precision.
- x0_16 = sf->scale_value_x(x0_16, sf);
- y0_16 = sf->scale_value_y(y0_16, sf);
-
- // Map the top left corner of the block into the reference frame.
- x0 = sf->scale_value_x(x_start + x, sf);
- y0 = sf->scale_value_y(y_start + y, sf);
-
- // Scale the MV and incorporate the sub-pixel offset of the block
- // in the reference frame.
- scaled_mv = vp10_scale_mv(&mv_q4, mi_x + x, mi_y + y, sf);
- xs = sf->x_step_q4;
- ys = sf->y_step_q4;
- } else {
- // Co-ordinate of containing block to pixel precision.
- x0 = (-xd->mb_to_left_edge >> (3 + pd->subsampling_x)) + x;
- y0 = (-xd->mb_to_top_edge >> (3 + pd->subsampling_y)) + y;
-
- // Co-ordinate of the block to 1/16th pixel precision.
- x0_16 = x0 << SUBPEL_BITS;
- y0_16 = y0 << SUBPEL_BITS;
-
- scaled_mv.row = mv->row * (1 << (1 - pd->subsampling_y));
- scaled_mv.col = mv->col * (1 << (1 - pd->subsampling_x));
- xs = ys = 16;
- }
- subpel_x = scaled_mv.col & SUBPEL_MASK;
- subpel_y = scaled_mv.row & SUBPEL_MASK;
-
- // Calculate the top left corner of the best matching block in the
- // reference frame.
- x0 += scaled_mv.col >> SUBPEL_BITS;
- y0 += scaled_mv.row >> SUBPEL_BITS;
- x0_16 += scaled_mv.col;
- y0_16 += scaled_mv.row;
-
- // Get reference block pointer.
- buf_ptr = ref_frame + y0 * pre_buf->stride + x0;
- buf_stride = pre_buf->stride;
-
- // Do border extension if there is motion or the
- // width/height is not a multiple of 8 pixels.
- if (is_scaled || scaled_mv.col || scaled_mv.row ||
- (frame_width & 0x7) || (frame_height & 0x7)) {
- int y1 = ((y0_16 + (h - 1) * ys) >> SUBPEL_BITS) + 1;
-
- // Get reference block bottom right horizontal coordinate.
- int x1 = ((x0_16 + (w - 1) * xs) >> SUBPEL_BITS) + 1;
- int x_pad = 0, y_pad = 0;
-
- if (subpel_x || (sf->x_step_q4 != SUBPEL_SHIFTS)) {
- x0 -= VP9_INTERP_EXTEND - 1;
- x1 += VP9_INTERP_EXTEND;
- x_pad = 1;
- }
-
- if (subpel_y || (sf->y_step_q4 != SUBPEL_SHIFTS)) {
- y0 -= VP9_INTERP_EXTEND - 1;
- y1 += VP9_INTERP_EXTEND;
- y_pad = 1;
- }
-
- // Wait until reference block is ready. Pad 7 more pixels as last 7
- // pixels of each superblock row can be changed by next superblock row.
- if (cm->frame_parallel_decode)
- vp10_frameworker_wait(pbi->frame_worker_owner, ref_frame_buf,
- VPXMAX(0, (y1 + 7)) << (plane == 0 ? 0 : 1));
-
- // Skip border extension if block is inside the frame.
- if (x0 < 0 || x0 > frame_width - 1 || x1 < 0 || x1 > frame_width - 1 ||
- y0 < 0 || y0 > frame_height - 1 || y1 < 0 || y1 > frame_height - 1) {
- // Extend the border.
- const uint8_t *const buf_ptr1 = ref_frame + y0 * buf_stride + x0;
- const int b_w = x1 - x0 + 1;
- const int b_h = y1 - y0 + 1;
- const int border_offset = y_pad * 3 * b_w + x_pad * 3;
-
- extend_and_predict(buf_ptr1, buf_stride, x0, y0, b_w, b_h,
- frame_width, frame_height, border_offset,
- dst, dst_buf->stride,
- subpel_x, subpel_y,
- kernel, sf,
-#if CONFIG_VP9_HIGHBITDEPTH
- xd,
-#endif
- w, h, ref, xs, ys);
- return;
- }
- } else {
- // Wait until reference block is ready. Pad 7 more pixels as last 7
- // pixels of each superblock row can be changed by next superblock row.
- if (cm->frame_parallel_decode) {
- const int y1 = (y0_16 + (h - 1) * ys) >> SUBPEL_BITS;
- vp10_frameworker_wait(pbi->frame_worker_owner, ref_frame_buf,
- VPXMAX(0, (y1 + 7)) << (plane == 0 ? 0 : 1));
- }
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- high_inter_predictor(buf_ptr, buf_stride, dst, dst_buf->stride, subpel_x,
- subpel_y, sf, w, h, ref, kernel, xs, ys, xd->bd);
- } else {
- inter_predictor(buf_ptr, buf_stride, dst, dst_buf->stride, subpel_x,
- subpel_y, sf, w, h, ref, kernel, xs, ys);
- }
-#else
- inter_predictor(buf_ptr, buf_stride, dst, dst_buf->stride, subpel_x,
- subpel_y, sf, w, h, ref, kernel, xs, ys);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-}
-
-static void dec_build_inter_predictors_sb(VP10Decoder *const pbi,
- MACROBLOCKD *xd,
- int mi_row, int mi_col) {
- int plane;
- const int mi_x = mi_col * MI_SIZE;
- const int mi_y = mi_row * MI_SIZE;
- const MODE_INFO *mi = xd->mi[0];
- const InterpKernel *kernel = vp10_filter_kernels[mi->mbmi.interp_filter];
- const BLOCK_SIZE sb_type = mi->mbmi.sb_type;
- const int is_compound = has_second_ref(&mi->mbmi);
-
- for (plane = 0; plane < MAX_MB_PLANE; ++plane) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- struct buf_2d *const dst_buf = &pd->dst;
- const int num_4x4_w = pd->n4_w;
- const int num_4x4_h = pd->n4_h;
-
- const int n4w_x4 = 4 * num_4x4_w;
- const int n4h_x4 = 4 * num_4x4_h;
- int ref;
-
- for (ref = 0; ref < 1 + is_compound; ++ref) {
- const struct scale_factors *const sf = &xd->block_refs[ref]->sf;
- struct buf_2d *const pre_buf = &pd->pre[ref];
- const int idx = xd->block_refs[ref]->idx;
- BufferPool *const pool = pbi->common.buffer_pool;
- RefCntBuffer *const ref_frame_buf = &pool->frame_bufs[idx];
- const int is_scaled = vp10_is_scaled(sf);
-
- if (sb_type < BLOCK_8X8) {
- const PARTITION_TYPE bp = BLOCK_8X8 - sb_type;
- const int have_vsplit = bp != PARTITION_HORZ;
- const int have_hsplit = bp != PARTITION_VERT;
- const int num_4x4_w = 2 >> ((!have_vsplit) | pd->subsampling_x);
- const int num_4x4_h = 2 >> ((!have_hsplit) | pd->subsampling_y);
- const int pw = 8 >> (have_vsplit | pd->subsampling_x);
- const int ph = 8 >> (have_hsplit | pd->subsampling_y);
- int x, y;
- for (y = 0; y < num_4x4_h; ++y) {
- for (x = 0; x < num_4x4_w; ++x) {
- const MV mv = average_split_mvs(pd, mi, ref, y * 2 + x);
- dec_build_inter_predictors(pbi, xd, plane, n4w_x4, n4h_x4,
- 4 * x, 4 * y, pw, ph, mi_x, mi_y, kernel,
- sf, pre_buf, dst_buf, &mv,
- ref_frame_buf, is_scaled, ref);
- }
- }
- } else {
- const MV mv = mi->mbmi.mv[ref].as_mv;
- dec_build_inter_predictors(pbi, xd, plane, n4w_x4, n4h_x4,
- 0, 0, n4w_x4, n4h_x4, mi_x, mi_y, kernel,
- sf, pre_buf, dst_buf, &mv, ref_frame_buf,
- is_scaled, ref);
- }
- }
- }
-}
-
-static INLINE TX_SIZE dec_get_uv_tx_size(const MB_MODE_INFO *mbmi,
- int n4_wl, int n4_hl) {
- // get minimum log2 num4x4s dimension
- const int x = VPXMIN(n4_wl, n4_hl);
- return VPXMIN(mbmi->tx_size, x);
-}
-
-static INLINE void dec_reset_skip_context(MACROBLOCKD *xd) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++) {
- struct macroblockd_plane *const pd = &xd->plane[i];
- memset(pd->above_context, 0, sizeof(ENTROPY_CONTEXT) * pd->n4_w);
- memset(pd->left_context, 0, sizeof(ENTROPY_CONTEXT) * pd->n4_h);
- }
-}
-
-static void set_plane_n4(MACROBLOCKD *const xd, int bw, int bh, int bwl,
- int bhl) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++) {
- xd->plane[i].n4_w = (bw << 1) >> xd->plane[i].subsampling_x;
- xd->plane[i].n4_h = (bh << 1) >> xd->plane[i].subsampling_y;
- xd->plane[i].n4_wl = bwl - xd->plane[i].subsampling_x;
- xd->plane[i].n4_hl = bhl - xd->plane[i].subsampling_y;
- }
-}
-
-static MB_MODE_INFO *set_offsets(VP10_COMMON *const cm, MACROBLOCKD *const xd,
- BLOCK_SIZE bsize, int mi_row, int mi_col,
- int bw, int bh, int x_mis, int y_mis,
- int bwl, int bhl) {
- const int offset = mi_row * cm->mi_stride + mi_col;
- int x, y;
- const TileInfo *const tile = &xd->tile;
-
- xd->mi = cm->mi_grid_visible + offset;
- xd->mi[0] = &cm->mi[offset];
- // TODO(slavarnway): Generate sb_type based on bwl and bhl, instead of
- // passing bsize from decode_partition().
- xd->mi[0]->mbmi.sb_type = bsize;
- for (y = 0; y < y_mis; ++y)
- for (x = !y; x < x_mis; ++x) {
- xd->mi[y * cm->mi_stride + x] = xd->mi[0];
- }
-
- set_plane_n4(xd, bw, bh, bwl, bhl);
-
- set_skip_context(xd, mi_row, mi_col);
-
- // Distance of Mb to the various image edges. These are specified to 8th pel
- // as they are always compared to values that are in 1/8th pel units
- set_mi_row_col(xd, tile, mi_row, bh, mi_col, bw, cm->mi_rows, cm->mi_cols);
-
- vp10_setup_dst_planes(xd->plane, get_frame_new_buffer(cm), mi_row, mi_col);
- return &xd->mi[0]->mbmi;
-}
-
-static void decode_block(VP10Decoder *const pbi, MACROBLOCKD *const xd,
- int mi_row, int mi_col,
- vpx_reader *r, BLOCK_SIZE bsize,
- int bwl, int bhl) {
- VP10_COMMON *const cm = &pbi->common;
- const int less8x8 = bsize < BLOCK_8X8;
- const int bw = 1 << (bwl - 1);
- const int bh = 1 << (bhl - 1);
- const int x_mis = VPXMIN(bw, cm->mi_cols - mi_col);
- const int y_mis = VPXMIN(bh, cm->mi_rows - mi_row);
-
- MB_MODE_INFO *mbmi = set_offsets(cm, xd, bsize, mi_row, mi_col,
- bw, bh, x_mis, y_mis, bwl, bhl);
-
- if (bsize >= BLOCK_8X8 && (cm->subsampling_x || cm->subsampling_y)) {
- const BLOCK_SIZE uv_subsize =
- ss_size_lookup[bsize][cm->subsampling_x][cm->subsampling_y];
- if (uv_subsize == BLOCK_INVALID)
- vpx_internal_error(xd->error_info,
- VPX_CODEC_CORRUPT_FRAME, "Invalid block size.");
- }
-
- vp10_read_mode_info(pbi, xd, mi_row, mi_col, r, x_mis, y_mis);
-
- if (mbmi->skip) {
- dec_reset_skip_context(xd);
- }
-
- if (!is_inter_block(mbmi)) {
- int plane;
- for (plane = 0; plane < MAX_MB_PLANE; ++plane) {
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- const TX_SIZE tx_size =
- plane ? dec_get_uv_tx_size(mbmi, pd->n4_wl, pd->n4_hl)
- : mbmi->tx_size;
- const int num_4x4_w = pd->n4_w;
- const int num_4x4_h = pd->n4_h;
- const int step = (1 << tx_size);
- int row, col;
- const int max_blocks_wide = num_4x4_w + (xd->mb_to_right_edge >= 0 ?
- 0 : xd->mb_to_right_edge >> (5 + pd->subsampling_x));
- const int max_blocks_high = num_4x4_h + (xd->mb_to_bottom_edge >= 0 ?
- 0 : xd->mb_to_bottom_edge >> (5 + pd->subsampling_y));
-
- if (plane <= 1 && mbmi->palette_mode_info.palette_size[plane])
- vp10_decode_palette_tokens(xd, plane, r);
-
- for (row = 0; row < max_blocks_high; row += step)
- for (col = 0; col < max_blocks_wide; col += step)
- predict_and_reconstruct_intra_block(xd, r, mbmi, plane,
- row, col, tx_size);
- }
- } else {
- // Prediction
- dec_build_inter_predictors_sb(pbi, xd, mi_row, mi_col);
-
- // Reconstruction
- if (!mbmi->skip) {
- int eobtotal = 0;
- int plane;
-
- for (plane = 0; plane < MAX_MB_PLANE; ++plane) {
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- const TX_SIZE tx_size =
- plane ? dec_get_uv_tx_size(mbmi, pd->n4_wl, pd->n4_hl)
- : mbmi->tx_size;
- const int num_4x4_w = pd->n4_w;
- const int num_4x4_h = pd->n4_h;
- const int step = (1 << tx_size);
- int row, col;
- const int max_blocks_wide = num_4x4_w + (xd->mb_to_right_edge >= 0 ?
- 0 : xd->mb_to_right_edge >> (5 + pd->subsampling_x));
- const int max_blocks_high = num_4x4_h + (xd->mb_to_bottom_edge >= 0 ?
- 0 : xd->mb_to_bottom_edge >> (5 + pd->subsampling_y));
-
- for (row = 0; row < max_blocks_high; row += step)
- for (col = 0; col < max_blocks_wide; col += step)
- eobtotal += reconstruct_inter_block(xd, r, mbmi, plane, row, col,
- tx_size);
- }
-
- if (!less8x8 && eobtotal == 0)
-#if CONFIG_MISC_FIXES
- mbmi->has_no_coeffs = 1; // skip loopfilter
-#else
- mbmi->skip = 1; // skip loopfilter
-#endif
- }
- }
-
- xd->corrupted |= vpx_reader_has_error(r);
-}
-
-static INLINE int dec_partition_plane_context(const MACROBLOCKD *xd,
- int mi_row, int mi_col,
- int bsl) {
- const PARTITION_CONTEXT *above_ctx = xd->above_seg_context + mi_col;
- const PARTITION_CONTEXT *left_ctx = xd->left_seg_context + (mi_row & MI_MASK);
- int above = (*above_ctx >> bsl) & 1 , left = (*left_ctx >> bsl) & 1;
-
-// assert(bsl >= 0);
-
- return (left * 2 + above) + bsl * PARTITION_PLOFFSET;
-}
-
-static INLINE void dec_update_partition_context(MACROBLOCKD *xd,
- int mi_row, int mi_col,
- BLOCK_SIZE subsize,
- int bw) {
- PARTITION_CONTEXT *const above_ctx = xd->above_seg_context + mi_col;
- PARTITION_CONTEXT *const left_ctx = xd->left_seg_context + (mi_row & MI_MASK);
-
- // update the partition context at the end notes. set partition bits
- // of block sizes larger than the current one to be one, and partition
- // bits of smaller block sizes to be zero.
- memset(above_ctx, partition_context_lookup[subsize].above, bw);
- memset(left_ctx, partition_context_lookup[subsize].left, bw);
-}
-
-static PARTITION_TYPE read_partition(VP10_COMMON *cm, MACROBLOCKD *xd,
- int mi_row, int mi_col, vpx_reader *r,
- int has_rows, int has_cols, int bsl) {
- const int ctx = dec_partition_plane_context(xd, mi_row, mi_col, bsl);
- const vpx_prob *const probs = cm->fc->partition_prob[ctx];
- FRAME_COUNTS *counts = xd->counts;
- PARTITION_TYPE p;
-
- if (has_rows && has_cols)
- p = (PARTITION_TYPE)vpx_read_tree(r, vp10_partition_tree, probs);
- else if (!has_rows && has_cols)
- p = vpx_read(r, probs[1]) ? PARTITION_SPLIT : PARTITION_HORZ;
- else if (has_rows && !has_cols)
- p = vpx_read(r, probs[2]) ? PARTITION_SPLIT : PARTITION_VERT;
- else
- p = PARTITION_SPLIT;
-
- if (counts)
- ++counts->partition[ctx][p];
-
- return p;
-}
-
-// TODO(slavarnway): eliminate bsize and subsize in future commits
-static void decode_partition(VP10Decoder *const pbi, MACROBLOCKD *const xd,
- int mi_row, int mi_col,
- vpx_reader* r, BLOCK_SIZE bsize, int n4x4_l2) {
- VP10_COMMON *const cm = &pbi->common;
- const int n8x8_l2 = n4x4_l2 - 1;
- const int num_8x8_wh = 1 << n8x8_l2;
- const int hbs = num_8x8_wh >> 1;
- PARTITION_TYPE partition;
- BLOCK_SIZE subsize;
- const int has_rows = (mi_row + hbs) < cm->mi_rows;
- const int has_cols = (mi_col + hbs) < cm->mi_cols;
-
- if (mi_row >= cm->mi_rows || mi_col >= cm->mi_cols)
- return;
-
- partition = read_partition(cm, xd, mi_row, mi_col, r, has_rows, has_cols,
- n8x8_l2);
- subsize = subsize_lookup[partition][bsize]; // get_subsize(bsize, partition);
- if (!hbs) {
- // calculate bmode block dimensions (log 2)
- xd->bmode_blocks_wl = 1 >> !!(partition & PARTITION_VERT);
- xd->bmode_blocks_hl = 1 >> !!(partition & PARTITION_HORZ);
- decode_block(pbi, xd, mi_row, mi_col, r, subsize, 1, 1);
- } else {
- switch (partition) {
- case PARTITION_NONE:
- decode_block(pbi, xd, mi_row, mi_col, r, subsize, n4x4_l2, n4x4_l2);
- break;
- case PARTITION_HORZ:
- decode_block(pbi, xd, mi_row, mi_col, r, subsize, n4x4_l2, n8x8_l2);
- if (has_rows)
- decode_block(pbi, xd, mi_row + hbs, mi_col, r, subsize, n4x4_l2,
- n8x8_l2);
- break;
- case PARTITION_VERT:
- decode_block(pbi, xd, mi_row, mi_col, r, subsize, n8x8_l2, n4x4_l2);
- if (has_cols)
- decode_block(pbi, xd, mi_row, mi_col + hbs, r, subsize, n8x8_l2,
- n4x4_l2);
- break;
- case PARTITION_SPLIT:
- decode_partition(pbi, xd, mi_row, mi_col, r, subsize, n8x8_l2);
- decode_partition(pbi, xd, mi_row, mi_col + hbs, r, subsize, n8x8_l2);
- decode_partition(pbi, xd, mi_row + hbs, mi_col, r, subsize, n8x8_l2);
- decode_partition(pbi, xd, mi_row + hbs, mi_col + hbs, r, subsize,
- n8x8_l2);
- break;
- default:
- assert(0 && "Invalid partition type");
- }
- }
-
- // update partition context
- if (bsize >= BLOCK_8X8 &&
- (bsize == BLOCK_8X8 || partition != PARTITION_SPLIT))
- dec_update_partition_context(xd, mi_row, mi_col, subsize, num_8x8_wh);
-}
-
-static void setup_token_decoder(const uint8_t *data,
- const uint8_t *data_end,
- size_t read_size,
- struct vpx_internal_error_info *error_info,
- vpx_reader *r,
- vpx_decrypt_cb decrypt_cb,
- void *decrypt_state) {
- // Validate the calculated partition length. If the buffer
- // described by the partition can't be fully read, then restrict
- // it to the portion that can be (for EC mode) or throw an error.
- if (!read_is_valid(data, read_size, data_end))
- vpx_internal_error(error_info, VPX_CODEC_CORRUPT_FRAME,
- "Truncated packet or corrupt tile length");
-
- if (vpx_reader_init(r, data, read_size, decrypt_cb, decrypt_state))
- vpx_internal_error(error_info, VPX_CODEC_MEM_ERROR,
- "Failed to allocate bool decoder %d", 1);
-}
-
-static void read_coef_probs_common(vp10_coeff_probs_model *coef_probs,
- vpx_reader *r) {
- int i, j, k, l, m;
-
- if (vpx_read_bit(r))
- for (i = 0; i < PLANE_TYPES; ++i)
- for (j = 0; j < REF_TYPES; ++j)
- for (k = 0; k < COEF_BANDS; ++k)
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l)
- for (m = 0; m < UNCONSTRAINED_NODES; ++m)
- vp10_diff_update_prob(r, &coef_probs[i][j][k][l][m]);
-}
-
-static void read_coef_probs(FRAME_CONTEXT *fc, TX_MODE tx_mode,
- vpx_reader *r) {
- const TX_SIZE max_tx_size = tx_mode_to_biggest_tx_size[tx_mode];
- TX_SIZE tx_size;
- for (tx_size = TX_4X4; tx_size <= max_tx_size; ++tx_size)
- read_coef_probs_common(fc->coef_probs[tx_size], r);
-}
-
-static void setup_segmentation(VP10_COMMON *const cm,
- struct vpx_read_bit_buffer *rb) {
- struct segmentation *const seg = &cm->seg;
-#if !CONFIG_MISC_FIXES
- struct segmentation_probs *const segp = &cm->segp;
-#endif
- int i, j;
-
- seg->update_map = 0;
- seg->update_data = 0;
-
- seg->enabled = vpx_rb_read_bit(rb);
- if (!seg->enabled)
- return;
-
- // Segmentation map update
- if (frame_is_intra_only(cm) || cm->error_resilient_mode) {
- seg->update_map = 1;
- } else {
- seg->update_map = vpx_rb_read_bit(rb);
- }
- if (seg->update_map) {
-#if !CONFIG_MISC_FIXES
- for (i = 0; i < SEG_TREE_PROBS; i++)
- segp->tree_probs[i] = vpx_rb_read_bit(rb) ? vpx_rb_read_literal(rb, 8)
- : MAX_PROB;
-#endif
- if (frame_is_intra_only(cm) || cm->error_resilient_mode) {
- seg->temporal_update = 0;
- } else {
- seg->temporal_update = vpx_rb_read_bit(rb);
- }
-#if !CONFIG_MISC_FIXES
- if (seg->temporal_update) {
- for (i = 0; i < PREDICTION_PROBS; i++)
- segp->pred_probs[i] = vpx_rb_read_bit(rb) ? vpx_rb_read_literal(rb, 8)
- : MAX_PROB;
- } else {
- for (i = 0; i < PREDICTION_PROBS; i++)
- segp->pred_probs[i] = MAX_PROB;
- }
-#endif
- }
-
- // Segmentation data update
- seg->update_data = vpx_rb_read_bit(rb);
- if (seg->update_data) {
- seg->abs_delta = vpx_rb_read_bit(rb);
-
- vp10_clearall_segfeatures(seg);
-
- for (i = 0; i < MAX_SEGMENTS; i++) {
- for (j = 0; j < SEG_LVL_MAX; j++) {
- int data = 0;
- const int feature_enabled = vpx_rb_read_bit(rb);
- if (feature_enabled) {
- vp10_enable_segfeature(seg, i, j);
- data = decode_unsigned_max(rb, vp10_seg_feature_data_max(j));
- if (vp10_is_segfeature_signed(j))
- data = vpx_rb_read_bit(rb) ? -data : data;
- }
- vp10_set_segdata(seg, i, j, data);
- }
- }
- }
-}
-
-static void setup_loopfilter(struct loopfilter *lf,
- struct vpx_read_bit_buffer *rb) {
- lf->filter_level = vpx_rb_read_literal(rb, 6);
- lf->sharpness_level = vpx_rb_read_literal(rb, 3);
-
- // Read in loop filter deltas applied at the MB level based on mode or ref
- // frame.
- lf->mode_ref_delta_update = 0;
-
- lf->mode_ref_delta_enabled = vpx_rb_read_bit(rb);
- if (lf->mode_ref_delta_enabled) {
- lf->mode_ref_delta_update = vpx_rb_read_bit(rb);
- if (lf->mode_ref_delta_update) {
- int i;
-
- for (i = 0; i < MAX_REF_FRAMES; i++)
- if (vpx_rb_read_bit(rb))
- lf->ref_deltas[i] = vpx_rb_read_inv_signed_literal(rb, 6);
-
- for (i = 0; i < MAX_MODE_LF_DELTAS; i++)
- if (vpx_rb_read_bit(rb))
- lf->mode_deltas[i] = vpx_rb_read_inv_signed_literal(rb, 6);
- }
- }
-}
-
-static INLINE int read_delta_q(struct vpx_read_bit_buffer *rb) {
- return vpx_rb_read_bit(rb) ?
- vpx_rb_read_inv_signed_literal(rb, CONFIG_MISC_FIXES ? 6 : 4) : 0;
-}
-
-static void setup_quantization(VP10_COMMON *const cm, MACROBLOCKD *const xd,
- struct vpx_read_bit_buffer *rb) {
- int i;
-
- cm->base_qindex = vpx_rb_read_literal(rb, QINDEX_BITS);
- cm->y_dc_delta_q = read_delta_q(rb);
- cm->uv_dc_delta_q = read_delta_q(rb);
- cm->uv_ac_delta_q = read_delta_q(rb);
- cm->dequant_bit_depth = cm->bit_depth;
- for (i = 0; i < (cm->seg.enabled ? MAX_SEGMENTS : 1); ++i) {
-#if CONFIG_MISC_FIXES
- const int qindex = vp10_get_qindex(&cm->seg, i, cm->base_qindex);
-#endif
- xd->lossless[i] = cm->y_dc_delta_q == 0 &&
-#if CONFIG_MISC_FIXES
- qindex == 0 &&
-#else
- cm->base_qindex == 0 &&
-#endif
- cm->uv_dc_delta_q == 0 &&
- cm->uv_ac_delta_q == 0;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- xd->bd = (int)cm->bit_depth;
-#endif
-}
-
-static void setup_segmentation_dequant(VP10_COMMON *const cm) {
- // Build y/uv dequant values based on segmentation.
- if (cm->seg.enabled) {
- int i;
- for (i = 0; i < MAX_SEGMENTS; ++i) {
- const int qindex = vp10_get_qindex(&cm->seg, i, cm->base_qindex);
- cm->y_dequant[i][0] = vp10_dc_quant(qindex, cm->y_dc_delta_q,
- cm->bit_depth);
- cm->y_dequant[i][1] = vp10_ac_quant(qindex, 0, cm->bit_depth);
- cm->uv_dequant[i][0] = vp10_dc_quant(qindex, cm->uv_dc_delta_q,
- cm->bit_depth);
- cm->uv_dequant[i][1] = vp10_ac_quant(qindex, cm->uv_ac_delta_q,
- cm->bit_depth);
- }
- } else {
- const int qindex = cm->base_qindex;
- // When segmentation is disabled, only the first value is used. The
- // remaining are don't cares.
- cm->y_dequant[0][0] = vp10_dc_quant(qindex, cm->y_dc_delta_q, cm->bit_depth);
- cm->y_dequant[0][1] = vp10_ac_quant(qindex, 0, cm->bit_depth);
- cm->uv_dequant[0][0] = vp10_dc_quant(qindex, cm->uv_dc_delta_q,
- cm->bit_depth);
- cm->uv_dequant[0][1] = vp10_ac_quant(qindex, cm->uv_ac_delta_q,
- cm->bit_depth);
- }
-}
-
-static INTERP_FILTER read_interp_filter(struct vpx_read_bit_buffer *rb) {
- return vpx_rb_read_bit(rb) ? SWITCHABLE : vpx_rb_read_literal(rb, 2);
-}
-
-static void setup_render_size(VP10_COMMON *cm,
- struct vpx_read_bit_buffer *rb) {
- cm->render_width = cm->width;
- cm->render_height = cm->height;
- if (vpx_rb_read_bit(rb))
- vp10_read_frame_size(rb, &cm->render_width, &cm->render_height);
-}
-
-static void resize_mv_buffer(VP10_COMMON *cm) {
- vpx_free(cm->cur_frame->mvs);
- cm->cur_frame->mi_rows = cm->mi_rows;
- cm->cur_frame->mi_cols = cm->mi_cols;
- cm->cur_frame->mvs = (MV_REF *)vpx_calloc(cm->mi_rows * cm->mi_cols,
- sizeof(*cm->cur_frame->mvs));
-}
-
-static void resize_context_buffers(VP10_COMMON *cm, int width, int height) {
-#if CONFIG_SIZE_LIMIT
- if (width > DECODE_WIDTH_LIMIT || height > DECODE_HEIGHT_LIMIT)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Dimensions of %dx%d beyond allowed size of %dx%d.",
- width, height, DECODE_WIDTH_LIMIT, DECODE_HEIGHT_LIMIT);
-#endif
- if (cm->width != width || cm->height != height) {
- const int new_mi_rows =
- ALIGN_POWER_OF_TWO(height, MI_SIZE_LOG2) >> MI_SIZE_LOG2;
- const int new_mi_cols =
- ALIGN_POWER_OF_TWO(width, MI_SIZE_LOG2) >> MI_SIZE_LOG2;
-
- // Allocations in vp10_alloc_context_buffers() depend on individual
- // dimensions as well as the overall size.
- if (new_mi_cols > cm->mi_cols || new_mi_rows > cm->mi_rows) {
- if (vp10_alloc_context_buffers(cm, width, height))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate context buffers");
- } else {
- vp10_set_mb_mi(cm, width, height);
- }
- vp10_init_context_buffers(cm);
- cm->width = width;
- cm->height = height;
- }
- if (cm->cur_frame->mvs == NULL || cm->mi_rows > cm->cur_frame->mi_rows ||
- cm->mi_cols > cm->cur_frame->mi_cols) {
- resize_mv_buffer(cm);
- }
-}
-
-static void setup_frame_size(VP10_COMMON *cm, struct vpx_read_bit_buffer *rb) {
- int width, height;
- BufferPool *const pool = cm->buffer_pool;
- vp10_read_frame_size(rb, &width, &height);
- resize_context_buffers(cm, width, height);
- setup_render_size(cm, rb);
-
- lock_buffer_pool(pool);
- if (vpx_realloc_frame_buffer(
- get_frame_new_buffer(cm), cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_DEC_BORDER_IN_PIXELS,
- cm->byte_alignment,
- &pool->frame_bufs[cm->new_fb_idx].raw_frame_buffer, pool->get_fb_cb,
- pool->cb_priv)) {
- unlock_buffer_pool(pool);
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate frame buffer");
- }
- unlock_buffer_pool(pool);
-
- pool->frame_bufs[cm->new_fb_idx].buf.subsampling_x = cm->subsampling_x;
- pool->frame_bufs[cm->new_fb_idx].buf.subsampling_y = cm->subsampling_y;
- pool->frame_bufs[cm->new_fb_idx].buf.bit_depth = (unsigned int)cm->bit_depth;
- pool->frame_bufs[cm->new_fb_idx].buf.color_space = cm->color_space;
- pool->frame_bufs[cm->new_fb_idx].buf.color_range = cm->color_range;
- pool->frame_bufs[cm->new_fb_idx].buf.render_width = cm->render_width;
- pool->frame_bufs[cm->new_fb_idx].buf.render_height = cm->render_height;
-}
-
-static INLINE int valid_ref_frame_img_fmt(vpx_bit_depth_t ref_bit_depth,
- int ref_xss, int ref_yss,
- vpx_bit_depth_t this_bit_depth,
- int this_xss, int this_yss) {
- return ref_bit_depth == this_bit_depth && ref_xss == this_xss &&
- ref_yss == this_yss;
-}
-
-static void setup_frame_size_with_refs(VP10_COMMON *cm,
- struct vpx_read_bit_buffer *rb) {
- int width, height;
- int found = 0, i;
- int has_valid_ref_frame = 0;
- BufferPool *const pool = cm->buffer_pool;
- for (i = 0; i < REFS_PER_FRAME; ++i) {
- if (vpx_rb_read_bit(rb)) {
- YV12_BUFFER_CONFIG *const buf = cm->frame_refs[i].buf;
- width = buf->y_crop_width;
- height = buf->y_crop_height;
-#if CONFIG_MISC_FIXES
- cm->render_width = buf->render_width;
- cm->render_height = buf->render_height;
-#endif
- found = 1;
- break;
- }
- }
-
- if (!found) {
- vp10_read_frame_size(rb, &width, &height);
-#if CONFIG_MISC_FIXES
- setup_render_size(cm, rb);
-#endif
- }
-
- if (width <= 0 || height <= 0)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Invalid frame size");
-
- // Check to make sure at least one of frames that this frame references
- // has valid dimensions.
- for (i = 0; i < REFS_PER_FRAME; ++i) {
- RefBuffer *const ref_frame = &cm->frame_refs[i];
- has_valid_ref_frame |= valid_ref_frame_size(ref_frame->buf->y_crop_width,
- ref_frame->buf->y_crop_height,
- width, height);
- }
- if (!has_valid_ref_frame)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Referenced frame has invalid size");
- for (i = 0; i < REFS_PER_FRAME; ++i) {
- RefBuffer *const ref_frame = &cm->frame_refs[i];
- if (!valid_ref_frame_img_fmt(
- ref_frame->buf->bit_depth,
- ref_frame->buf->subsampling_x,
- ref_frame->buf->subsampling_y,
- cm->bit_depth,
- cm->subsampling_x,
- cm->subsampling_y))
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Referenced frame has incompatible color format");
- }
-
- resize_context_buffers(cm, width, height);
-#if !CONFIG_MISC_FIXES
- setup_render_size(cm, rb);
-#endif
-
- lock_buffer_pool(pool);
- if (vpx_realloc_frame_buffer(
- get_frame_new_buffer(cm), cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_DEC_BORDER_IN_PIXELS,
- cm->byte_alignment,
- &pool->frame_bufs[cm->new_fb_idx].raw_frame_buffer, pool->get_fb_cb,
- pool->cb_priv)) {
- unlock_buffer_pool(pool);
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate frame buffer");
- }
- unlock_buffer_pool(pool);
-
- pool->frame_bufs[cm->new_fb_idx].buf.subsampling_x = cm->subsampling_x;
- pool->frame_bufs[cm->new_fb_idx].buf.subsampling_y = cm->subsampling_y;
- pool->frame_bufs[cm->new_fb_idx].buf.bit_depth = (unsigned int)cm->bit_depth;
- pool->frame_bufs[cm->new_fb_idx].buf.color_space = cm->color_space;
- pool->frame_bufs[cm->new_fb_idx].buf.color_range = cm->color_range;
- pool->frame_bufs[cm->new_fb_idx].buf.render_width = cm->render_width;
- pool->frame_bufs[cm->new_fb_idx].buf.render_height = cm->render_height;
-}
-
-static void setup_tile_info(VP10_COMMON *cm, struct vpx_read_bit_buffer *rb) {
- int min_log2_tile_cols, max_log2_tile_cols, max_ones;
- vp10_get_tile_n_bits(cm->mi_cols, &min_log2_tile_cols, &max_log2_tile_cols);
-
- // columns
- max_ones = max_log2_tile_cols - min_log2_tile_cols;
- cm->log2_tile_cols = min_log2_tile_cols;
- while (max_ones-- && vpx_rb_read_bit(rb))
- cm->log2_tile_cols++;
-
- if (cm->log2_tile_cols > 6)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Invalid number of tile columns");
-
- // rows
- cm->log2_tile_rows = vpx_rb_read_bit(rb);
- if (cm->log2_tile_rows)
- cm->log2_tile_rows += vpx_rb_read_bit(rb);
-
-#if CONFIG_MISC_FIXES
- // tile size magnitude
- if (cm->log2_tile_rows > 0 || cm->log2_tile_cols > 0) {
- cm->tile_sz_mag = vpx_rb_read_literal(rb, 2);
- }
-#else
- cm->tile_sz_mag = 3;
-#endif
-}
-
-typedef struct TileBuffer {
- const uint8_t *data;
- size_t size;
- int col; // only used with multi-threaded decoding
-} TileBuffer;
-
-static int mem_get_varsize(const uint8_t *data, const int mag) {
- switch (mag) {
- case 0:
- return data[0];
- case 1:
- return mem_get_le16(data);
- case 2:
- return mem_get_le24(data);
- case 3:
- return mem_get_le32(data);
- }
-
- assert("Invalid tile size marker value" && 0);
-
- return -1;
-}
-
-// Reads the next tile returning its size and adjusting '*data' accordingly
-// based on 'is_last'.
-static void get_tile_buffer(const uint8_t *const data_end,
- const int tile_sz_mag, int is_last,
- struct vpx_internal_error_info *error_info,
- const uint8_t **data,
- vpx_decrypt_cb decrypt_cb, void *decrypt_state,
- TileBuffer *buf) {
- size_t size;
-
- if (!is_last) {
- if (!read_is_valid(*data, 4, data_end))
- vpx_internal_error(error_info, VPX_CODEC_CORRUPT_FRAME,
- "Truncated packet or corrupt tile length");
-
- if (decrypt_cb) {
- uint8_t be_data[4];
- decrypt_cb(decrypt_state, *data, be_data, tile_sz_mag + 1);
- size = mem_get_varsize(be_data, tile_sz_mag) + CONFIG_MISC_FIXES;
- } else {
- size = mem_get_varsize(*data, tile_sz_mag) + CONFIG_MISC_FIXES;
- }
- *data += tile_sz_mag + 1;
-
- if (size > (size_t)(data_end - *data))
- vpx_internal_error(error_info, VPX_CODEC_CORRUPT_FRAME,
- "Truncated packet or corrupt tile size");
- } else {
- size = data_end - *data;
- }
-
- buf->data = *data;
- buf->size = size;
-
- *data += size;
-}
-
-static void get_tile_buffers(VP10Decoder *pbi,
- const uint8_t *data, const uint8_t *data_end,
- int tile_cols, int tile_rows,
- TileBuffer (*tile_buffers)[1 << 6]) {
- int r, c;
-
- for (r = 0; r < tile_rows; ++r) {
- for (c = 0; c < tile_cols; ++c) {
- const int is_last = (r == tile_rows - 1) && (c == tile_cols - 1);
- TileBuffer *const buf = &tile_buffers[r][c];
- buf->col = c;
- get_tile_buffer(data_end, pbi->common.tile_sz_mag,
- is_last, &pbi->common.error, &data,
- pbi->decrypt_cb, pbi->decrypt_state, buf);
- }
- }
-}
-
-static const uint8_t *decode_tiles(VP10Decoder *pbi,
- const uint8_t *data,
- const uint8_t *data_end) {
- VP10_COMMON *const cm = &pbi->common;
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- const int aligned_cols = mi_cols_aligned_to_sb(cm->mi_cols);
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int tile_rows = 1 << cm->log2_tile_rows;
- TileBuffer tile_buffers[4][1 << 6];
- int tile_row, tile_col;
- int mi_row, mi_col;
- TileData *tile_data = NULL;
-
- if (cm->lf.filter_level && !cm->skip_loop_filter &&
- pbi->lf_worker.data1 == NULL) {
- CHECK_MEM_ERROR(cm, pbi->lf_worker.data1,
- vpx_memalign(32, sizeof(LFWorkerData)));
- pbi->lf_worker.hook = (VPxWorkerHook)vp10_loop_filter_worker;
- if (pbi->max_threads > 1 && !winterface->reset(&pbi->lf_worker)) {
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Loop filter thread creation failed");
- }
- }
-
- if (cm->lf.filter_level && !cm->skip_loop_filter) {
- LFWorkerData *const lf_data = (LFWorkerData*)pbi->lf_worker.data1;
- // Be sure to sync as we might be resuming after a failed frame decode.
- winterface->sync(&pbi->lf_worker);
- vp10_loop_filter_data_reset(lf_data, get_frame_new_buffer(cm), cm,
- pbi->mb.plane);
- }
-
- assert(tile_rows <= 4);
- assert(tile_cols <= (1 << 6));
-
- // Note: this memset assumes above_context[0], [1] and [2]
- // are allocated as part of the same buffer.
- memset(cm->above_context, 0,
- sizeof(*cm->above_context) * MAX_MB_PLANE * 2 * aligned_cols);
-
- memset(cm->above_seg_context, 0,
- sizeof(*cm->above_seg_context) * aligned_cols);
-
- get_tile_buffers(pbi, data, data_end, tile_cols, tile_rows, tile_buffers);
-
- if (pbi->tile_data == NULL ||
- (tile_cols * tile_rows) != pbi->total_tiles) {
- vpx_free(pbi->tile_data);
- CHECK_MEM_ERROR(
- cm,
- pbi->tile_data,
- vpx_memalign(32, tile_cols * tile_rows * (sizeof(*pbi->tile_data))));
- pbi->total_tiles = tile_rows * tile_cols;
- }
-
- // Load all tile information into tile_data.
- for (tile_row = 0; tile_row < tile_rows; ++tile_row) {
- for (tile_col = 0; tile_col < tile_cols; ++tile_col) {
- const TileBuffer *const buf = &tile_buffers[tile_row][tile_col];
- tile_data = pbi->tile_data + tile_cols * tile_row + tile_col;
- tile_data->cm = cm;
- tile_data->xd = pbi->mb;
- tile_data->xd.corrupted = 0;
- tile_data->xd.counts =
- cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD ?
- &cm->counts : NULL;
- vp10_zero(tile_data->dqcoeff);
- vp10_tile_init(&tile_data->xd.tile, tile_data->cm, tile_row, tile_col);
- setup_token_decoder(buf->data, data_end, buf->size, &cm->error,
- &tile_data->bit_reader, pbi->decrypt_cb,
- pbi->decrypt_state);
- vp10_init_macroblockd(cm, &tile_data->xd, tile_data->dqcoeff);
- tile_data->xd.plane[0].color_index_map = tile_data->color_index_map[0];
- tile_data->xd.plane[1].color_index_map = tile_data->color_index_map[1];
- }
- }
-
- for (tile_row = 0; tile_row < tile_rows; ++tile_row) {
- TileInfo tile;
- vp10_tile_set_row(&tile, cm, tile_row);
- for (mi_row = tile.mi_row_start; mi_row < tile.mi_row_end;
- mi_row += MI_BLOCK_SIZE) {
- for (tile_col = 0; tile_col < tile_cols; ++tile_col) {
- const int col = pbi->inv_tile_order ?
- tile_cols - tile_col - 1 : tile_col;
- tile_data = pbi->tile_data + tile_cols * tile_row + col;
- vp10_tile_set_col(&tile, tile_data->cm, col);
- vp10_zero(tile_data->xd.left_context);
- vp10_zero(tile_data->xd.left_seg_context);
- for (mi_col = tile.mi_col_start; mi_col < tile.mi_col_end;
- mi_col += MI_BLOCK_SIZE) {
- decode_partition(pbi, &tile_data->xd, mi_row,
- mi_col, &tile_data->bit_reader, BLOCK_64X64, 4);
- }
- pbi->mb.corrupted |= tile_data->xd.corrupted;
- if (pbi->mb.corrupted)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Failed to decode tile data");
- }
- // Loopfilter one row.
- if (cm->lf.filter_level && !cm->skip_loop_filter) {
- const int lf_start = mi_row - MI_BLOCK_SIZE;
- LFWorkerData *const lf_data = (LFWorkerData*)pbi->lf_worker.data1;
-
- // delay the loopfilter by 1 macroblock row.
- if (lf_start < 0) continue;
-
- // decoding has completed: finish up the loop filter in this thread.
- if (mi_row + MI_BLOCK_SIZE >= cm->mi_rows) continue;
-
- winterface->sync(&pbi->lf_worker);
- lf_data->start = lf_start;
- lf_data->stop = mi_row;
- if (pbi->max_threads > 1) {
- winterface->launch(&pbi->lf_worker);
- } else {
- winterface->execute(&pbi->lf_worker);
- }
- }
- // After loopfiltering, the last 7 row pixels in each superblock row may
- // still be changed by the longest loopfilter of the next superblock
- // row.
- if (cm->frame_parallel_decode)
- vp10_frameworker_broadcast(pbi->cur_buf,
- mi_row << MI_BLOCK_SIZE_LOG2);
- }
- }
-
- // Loopfilter remaining rows in the frame.
- if (cm->lf.filter_level && !cm->skip_loop_filter) {
- LFWorkerData *const lf_data = (LFWorkerData*)pbi->lf_worker.data1;
- winterface->sync(&pbi->lf_worker);
- lf_data->start = lf_data->stop;
- lf_data->stop = cm->mi_rows;
- winterface->execute(&pbi->lf_worker);
- }
-
- // Get last tile data.
- tile_data = pbi->tile_data + tile_cols * tile_rows - 1;
-
- if (cm->frame_parallel_decode)
- vp10_frameworker_broadcast(pbi->cur_buf, INT_MAX);
- return vpx_reader_find_end(&tile_data->bit_reader);
-}
-
-static int tile_worker_hook(TileWorkerData *const tile_data,
- const TileInfo *const tile) {
- int mi_row, mi_col;
-
- if (setjmp(tile_data->error_info.jmp)) {
- tile_data->error_info.setjmp = 0;
- tile_data->xd.corrupted = 1;
- return 0;
- }
-
- tile_data->error_info.setjmp = 1;
- tile_data->xd.error_info = &tile_data->error_info;
-
- for (mi_row = tile->mi_row_start; mi_row < tile->mi_row_end;
- mi_row += MI_BLOCK_SIZE) {
- vp10_zero(tile_data->xd.left_context);
- vp10_zero(tile_data->xd.left_seg_context);
- for (mi_col = tile->mi_col_start; mi_col < tile->mi_col_end;
- mi_col += MI_BLOCK_SIZE) {
- decode_partition(tile_data->pbi, &tile_data->xd,
- mi_row, mi_col, &tile_data->bit_reader,
- BLOCK_64X64, 4);
- }
- }
- return !tile_data->xd.corrupted;
-}
-
-// sorts in descending order
-static int compare_tile_buffers(const void *a, const void *b) {
- const TileBuffer *const buf1 = (const TileBuffer*)a;
- const TileBuffer *const buf2 = (const TileBuffer*)b;
- return (int)(buf2->size - buf1->size);
-}
-
-static const uint8_t *decode_tiles_mt(VP10Decoder *pbi,
- const uint8_t *data,
- const uint8_t *data_end) {
- VP10_COMMON *const cm = &pbi->common;
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- const uint8_t *bit_reader_end = NULL;
- const int aligned_mi_cols = mi_cols_aligned_to_sb(cm->mi_cols);
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int tile_rows = 1 << cm->log2_tile_rows;
- const int num_workers = VPXMIN(pbi->max_threads & ~1, tile_cols);
- TileBuffer tile_buffers[1][1 << 6];
- int n;
- int final_worker = -1;
-
- assert(tile_cols <= (1 << 6));
- assert(tile_rows == 1);
- (void)tile_rows;
-
- // TODO(jzern): See if we can remove the restriction of passing in max
- // threads to the decoder.
- if (pbi->num_tile_workers == 0) {
- const int num_threads = pbi->max_threads & ~1;
- int i;
- CHECK_MEM_ERROR(cm, pbi->tile_workers,
- vpx_malloc(num_threads * sizeof(*pbi->tile_workers)));
- // Ensure tile data offsets will be properly aligned. This may fail on
- // platforms without DECLARE_ALIGNED().
- assert((sizeof(*pbi->tile_worker_data) % 16) == 0);
- CHECK_MEM_ERROR(cm, pbi->tile_worker_data,
- vpx_memalign(32, num_threads *
- sizeof(*pbi->tile_worker_data)));
- CHECK_MEM_ERROR(cm, pbi->tile_worker_info,
- vpx_malloc(num_threads * sizeof(*pbi->tile_worker_info)));
- for (i = 0; i < num_threads; ++i) {
- VPxWorker *const worker = &pbi->tile_workers[i];
- ++pbi->num_tile_workers;
-
- winterface->init(worker);
- if (i < num_threads - 1 && !winterface->reset(worker)) {
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Tile decoder thread creation failed");
- }
- }
- }
-
- // Reset tile decoding hook
- for (n = 0; n < num_workers; ++n) {
- VPxWorker *const worker = &pbi->tile_workers[n];
- winterface->sync(worker);
- worker->hook = (VPxWorkerHook)tile_worker_hook;
- worker->data1 = &pbi->tile_worker_data[n];
- worker->data2 = &pbi->tile_worker_info[n];
- }
-
- // Note: this memset assumes above_context[0], [1] and [2]
- // are allocated as part of the same buffer.
- memset(cm->above_context, 0,
- sizeof(*cm->above_context) * MAX_MB_PLANE * 2 * aligned_mi_cols);
- memset(cm->above_seg_context, 0,
- sizeof(*cm->above_seg_context) * aligned_mi_cols);
-
- // Load tile data into tile_buffers
- get_tile_buffers(pbi, data, data_end, tile_cols, tile_rows, tile_buffers);
-
- // Sort the buffers based on size in descending order.
- qsort(tile_buffers[0], tile_cols, sizeof(tile_buffers[0][0]),
- compare_tile_buffers);
-
- // Rearrange the tile buffers such that per-tile group the largest, and
- // presumably the most difficult, tile will be decoded in the main thread.
- // This should help minimize the number of instances where the main thread is
- // waiting for a worker to complete.
- {
- int group_start = 0;
- while (group_start < tile_cols) {
- const TileBuffer largest = tile_buffers[0][group_start];
- const int group_end = VPXMIN(group_start + num_workers, tile_cols) - 1;
- memmove(tile_buffers[0] + group_start, tile_buffers[0] + group_start + 1,
- (group_end - group_start) * sizeof(tile_buffers[0][0]));
- tile_buffers[0][group_end] = largest;
- group_start = group_end + 1;
- }
- }
-
- // Initialize thread frame counts.
- if (cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD) {
- int i;
-
- for (i = 0; i < num_workers; ++i) {
- TileWorkerData *const tile_data =
- (TileWorkerData*)pbi->tile_workers[i].data1;
- vp10_zero(tile_data->counts);
- }
- }
-
- n = 0;
- while (n < tile_cols) {
- int i;
- for (i = 0; i < num_workers && n < tile_cols; ++i) {
- VPxWorker *const worker = &pbi->tile_workers[i];
- TileWorkerData *const tile_data = (TileWorkerData*)worker->data1;
- TileInfo *const tile = (TileInfo*)worker->data2;
- TileBuffer *const buf = &tile_buffers[0][n];
-
- tile_data->pbi = pbi;
- tile_data->xd = pbi->mb;
- tile_data->xd.corrupted = 0;
- tile_data->xd.counts =
- cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD ?
- &tile_data->counts : NULL;
- vp10_zero(tile_data->dqcoeff);
- vp10_tile_init(tile, cm, 0, buf->col);
- vp10_tile_init(&tile_data->xd.tile, cm, 0, buf->col);
- setup_token_decoder(buf->data, data_end, buf->size, &cm->error,
- &tile_data->bit_reader, pbi->decrypt_cb,
- pbi->decrypt_state);
- vp10_init_macroblockd(cm, &tile_data->xd, tile_data->dqcoeff);
- tile_data->xd.plane[0].color_index_map = tile_data->color_index_map[0];
- tile_data->xd.plane[1].color_index_map = tile_data->color_index_map[1];
-
- worker->had_error = 0;
- if (i == num_workers - 1 || n == tile_cols - 1) {
- winterface->execute(worker);
- } else {
- winterface->launch(worker);
- }
-
- if (buf->col == tile_cols - 1) {
- final_worker = i;
- }
-
- ++n;
- }
-
- for (; i > 0; --i) {
- VPxWorker *const worker = &pbi->tile_workers[i - 1];
- // TODO(jzern): The tile may have specific error data associated with
- // its vpx_internal_error_info which could be propagated to the main info
- // in cm. Additionally once the threads have been synced and an error is
- // detected, there's no point in continuing to decode tiles.
- pbi->mb.corrupted |= !winterface->sync(worker);
- }
- if (final_worker > -1) {
- TileWorkerData *const tile_data =
- (TileWorkerData*)pbi->tile_workers[final_worker].data1;
- bit_reader_end = vpx_reader_find_end(&tile_data->bit_reader);
- final_worker = -1;
- }
-
- // Accumulate thread frame counts.
- if (n >= tile_cols &&
- cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD) {
- for (i = 0; i < num_workers; ++i) {
- TileWorkerData *const tile_data =
- (TileWorkerData*)pbi->tile_workers[i].data1;
- vp10_accumulate_frame_counts(cm, &tile_data->counts, 1);
- }
- }
- }
-
- return bit_reader_end;
-}
-
-static void error_handler(void *data) {
- VP10_COMMON *const cm = (VP10_COMMON *)data;
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME, "Truncated packet");
-}
-
-static void read_bitdepth_colorspace_sampling(
- VP10_COMMON *cm, struct vpx_read_bit_buffer *rb) {
- if (cm->profile >= PROFILE_2) {
- cm->bit_depth = vpx_rb_read_bit(rb) ? VPX_BITS_12 : VPX_BITS_10;
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth = 1;
-#endif
- } else {
- cm->bit_depth = VPX_BITS_8;
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth = 0;
-#endif
- }
- cm->color_space = vpx_rb_read_literal(rb, 3);
- if (cm->color_space != VPX_CS_SRGB) {
- // [16,235] (including xvycc) vs [0,255] range
- cm->color_range = vpx_rb_read_bit(rb);
- if (cm->profile == PROFILE_1 || cm->profile == PROFILE_3) {
- cm->subsampling_x = vpx_rb_read_bit(rb);
- cm->subsampling_y = vpx_rb_read_bit(rb);
- if (cm->subsampling_x == 1 && cm->subsampling_y == 1)
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "4:2:0 color not supported in profile 1 or 3");
- if (vpx_rb_read_bit(rb))
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Reserved bit set");
- } else {
- cm->subsampling_y = cm->subsampling_x = 1;
- }
- } else {
- if (cm->profile == PROFILE_1 || cm->profile == PROFILE_3) {
- // Note if colorspace is SRGB then 4:4:4 chroma sampling is assumed.
- // 4:2:2 or 4:4:0 chroma sampling is not allowed.
- cm->subsampling_y = cm->subsampling_x = 0;
- if (vpx_rb_read_bit(rb))
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Reserved bit set");
- } else {
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "4:4:4 color not supported in profile 0 or 2");
- }
- }
-}
-
-static size_t read_uncompressed_header(VP10Decoder *pbi,
- struct vpx_read_bit_buffer *rb) {
- VP10_COMMON *const cm = &pbi->common;
-#if CONFIG_MISC_FIXES
- MACROBLOCKD *const xd = &pbi->mb;
-#endif
- BufferPool *const pool = cm->buffer_pool;
- RefCntBuffer *const frame_bufs = pool->frame_bufs;
- int i, mask, ref_index = 0;
- size_t sz;
-
- cm->last_frame_type = cm->frame_type;
- cm->last_intra_only = cm->intra_only;
-
- if (vpx_rb_read_literal(rb, 2) != VP9_FRAME_MARKER)
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Invalid frame marker");
-
- cm->profile = vp10_read_profile(rb);
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->profile >= MAX_PROFILES)
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Unsupported bitstream profile");
-#else
- if (cm->profile >= PROFILE_2)
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Unsupported bitstream profile");
-#endif
-
- cm->show_existing_frame = vpx_rb_read_bit(rb);
- if (cm->show_existing_frame) {
- // Show an existing frame directly.
- const int frame_to_show = cm->ref_frame_map[vpx_rb_read_literal(rb, 3)];
- lock_buffer_pool(pool);
- if (frame_to_show < 0 || frame_bufs[frame_to_show].ref_count < 1) {
- unlock_buffer_pool(pool);
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Buffer %d does not contain a decoded frame",
- frame_to_show);
- }
-
- ref_cnt_fb(frame_bufs, &cm->new_fb_idx, frame_to_show);
- unlock_buffer_pool(pool);
- pbi->refresh_frame_flags = 0;
- cm->lf.filter_level = 0;
- cm->show_frame = 1;
-
- if (cm->frame_parallel_decode) {
- for (i = 0; i < REF_FRAMES; ++i)
- cm->next_ref_frame_map[i] = cm->ref_frame_map[i];
- }
- return 0;
- }
-
- cm->frame_type = (FRAME_TYPE) vpx_rb_read_bit(rb);
- cm->show_frame = vpx_rb_read_bit(rb);
- cm->error_resilient_mode = vpx_rb_read_bit(rb);
-
- if (cm->frame_type == KEY_FRAME) {
- if (!vp10_read_sync_code(rb))
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Invalid frame sync code");
-
- read_bitdepth_colorspace_sampling(cm, rb);
- pbi->refresh_frame_flags = (1 << REF_FRAMES) - 1;
-
- for (i = 0; i < REFS_PER_FRAME; ++i) {
- cm->frame_refs[i].idx = INVALID_IDX;
- cm->frame_refs[i].buf = NULL;
- }
-
- setup_frame_size(cm, rb);
- if (pbi->need_resync) {
- memset(&cm->ref_frame_map, -1, sizeof(cm->ref_frame_map));
- pbi->need_resync = 0;
- }
- if (frame_is_intra_only(cm))
- cm->allow_screen_content_tools = vpx_rb_read_bit(rb);
- } else {
- cm->intra_only = cm->show_frame ? 0 : vpx_rb_read_bit(rb);
-
- if (cm->error_resilient_mode) {
- cm->reset_frame_context = RESET_FRAME_CONTEXT_ALL;
- } else {
-#if CONFIG_MISC_FIXES
- if (cm->intra_only) {
- cm->reset_frame_context =
- vpx_rb_read_bit(rb) ? RESET_FRAME_CONTEXT_ALL
- : RESET_FRAME_CONTEXT_CURRENT;
- } else {
- cm->reset_frame_context =
- vpx_rb_read_bit(rb) ? RESET_FRAME_CONTEXT_CURRENT
- : RESET_FRAME_CONTEXT_NONE;
- if (cm->reset_frame_context == RESET_FRAME_CONTEXT_CURRENT)
- cm->reset_frame_context =
- vpx_rb_read_bit(rb) ? RESET_FRAME_CONTEXT_ALL
- : RESET_FRAME_CONTEXT_CURRENT;
- }
-#else
- static const RESET_FRAME_CONTEXT_MODE reset_frame_context_conv_tbl[4] = {
- RESET_FRAME_CONTEXT_NONE, RESET_FRAME_CONTEXT_NONE,
- RESET_FRAME_CONTEXT_CURRENT, RESET_FRAME_CONTEXT_ALL
- };
-
- cm->reset_frame_context =
- reset_frame_context_conv_tbl[vpx_rb_read_literal(rb, 2)];
-#endif
- }
-
- if (cm->intra_only) {
- if (!vp10_read_sync_code(rb))
- vpx_internal_error(&cm->error, VPX_CODEC_UNSUP_BITSTREAM,
- "Invalid frame sync code");
-#if CONFIG_MISC_FIXES
- read_bitdepth_colorspace_sampling(cm, rb);
-#else
- if (cm->profile > PROFILE_0) {
- read_bitdepth_colorspace_sampling(cm, rb);
- } else {
- // NOTE: The intra-only frame header does not include the specification
- // of either the color format or color sub-sampling in profile 0. VP9
- // specifies that the default color format should be YUV 4:2:0 in this
- // case (normative).
- cm->color_space = VPX_CS_BT_601;
- cm->color_range = 0;
- cm->subsampling_y = cm->subsampling_x = 1;
- cm->bit_depth = VPX_BITS_8;
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth = 0;
-#endif
- }
-#endif
-
- pbi->refresh_frame_flags = vpx_rb_read_literal(rb, REF_FRAMES);
- setup_frame_size(cm, rb);
- if (pbi->need_resync) {
- memset(&cm->ref_frame_map, -1, sizeof(cm->ref_frame_map));
- pbi->need_resync = 0;
- }
- } else if (pbi->need_resync != 1) { /* Skip if need resync */
- pbi->refresh_frame_flags = vpx_rb_read_literal(rb, REF_FRAMES);
- for (i = 0; i < REFS_PER_FRAME; ++i) {
- const int ref = vpx_rb_read_literal(rb, REF_FRAMES_LOG2);
- const int idx = cm->ref_frame_map[ref];
- RefBuffer *const ref_frame = &cm->frame_refs[i];
- ref_frame->idx = idx;
- ref_frame->buf = &frame_bufs[idx].buf;
- cm->ref_frame_sign_bias[LAST_FRAME + i] = vpx_rb_read_bit(rb);
- }
-
- setup_frame_size_with_refs(cm, rb);
-
- cm->allow_high_precision_mv = vpx_rb_read_bit(rb);
- cm->interp_filter = read_interp_filter(rb);
-
- for (i = 0; i < REFS_PER_FRAME; ++i) {
- RefBuffer *const ref_buf = &cm->frame_refs[i];
-#if CONFIG_VP9_HIGHBITDEPTH
- vp10_setup_scale_factors_for_frame(&ref_buf->sf,
- ref_buf->buf->y_crop_width,
- ref_buf->buf->y_crop_height,
- cm->width, cm->height,
- cm->use_highbitdepth);
-#else
- vp10_setup_scale_factors_for_frame(&ref_buf->sf,
- ref_buf->buf->y_crop_width,
- ref_buf->buf->y_crop_height,
- cm->width, cm->height);
-#endif
- }
- }
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- get_frame_new_buffer(cm)->bit_depth = cm->bit_depth;
-#endif
- get_frame_new_buffer(cm)->color_space = cm->color_space;
- get_frame_new_buffer(cm)->color_range = cm->color_range;
- get_frame_new_buffer(cm)->render_width = cm->render_width;
- get_frame_new_buffer(cm)->render_height = cm->render_height;
-
- if (pbi->need_resync) {
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Keyframe / intra-only frame required to reset decoder"
- " state");
- }
-
- if (!cm->error_resilient_mode) {
- cm->refresh_frame_context =
- vpx_rb_read_bit(rb) ? REFRESH_FRAME_CONTEXT_FORWARD
- : REFRESH_FRAME_CONTEXT_OFF;
- if (cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_FORWARD) {
- cm->refresh_frame_context =
- vpx_rb_read_bit(rb) ? REFRESH_FRAME_CONTEXT_FORWARD
- : REFRESH_FRAME_CONTEXT_BACKWARD;
-#if !CONFIG_MISC_FIXES
- } else {
- vpx_rb_read_bit(rb); // parallel decoding mode flag
-#endif
- }
- } else {
- cm->refresh_frame_context = REFRESH_FRAME_CONTEXT_OFF;
- }
-
- // This flag will be overridden by the call to vp10_setup_past_independence
- // below, forcing the use of context 0 for those frame types.
- cm->frame_context_idx = vpx_rb_read_literal(rb, FRAME_CONTEXTS_LOG2);
-
- // Generate next_ref_frame_map.
- lock_buffer_pool(pool);
- for (mask = pbi->refresh_frame_flags; mask; mask >>= 1) {
- if (mask & 1) {
- cm->next_ref_frame_map[ref_index] = cm->new_fb_idx;
- ++frame_bufs[cm->new_fb_idx].ref_count;
- } else {
- cm->next_ref_frame_map[ref_index] = cm->ref_frame_map[ref_index];
- }
- // Current thread holds the reference frame.
- if (cm->ref_frame_map[ref_index] >= 0)
- ++frame_bufs[cm->ref_frame_map[ref_index]].ref_count;
- ++ref_index;
- }
-
- for (; ref_index < REF_FRAMES; ++ref_index) {
- cm->next_ref_frame_map[ref_index] = cm->ref_frame_map[ref_index];
- // Current thread holds the reference frame.
- if (cm->ref_frame_map[ref_index] >= 0)
- ++frame_bufs[cm->ref_frame_map[ref_index]].ref_count;
- }
- unlock_buffer_pool(pool);
- pbi->hold_ref_buf = 1;
-
- if (frame_is_intra_only(cm) || cm->error_resilient_mode)
- vp10_setup_past_independence(cm);
-
- setup_loopfilter(&cm->lf, rb);
- setup_quantization(cm, &pbi->mb, rb);
- setup_segmentation(cm, rb);
- setup_segmentation_dequant(cm);
-#if CONFIG_MISC_FIXES
- cm->tx_mode = (!cm->seg.enabled && xd->lossless[0]) ? ONLY_4X4
- : read_tx_mode(rb);
- cm->reference_mode = read_frame_reference_mode(cm, rb);
-#endif
-
- setup_tile_info(cm, rb);
- sz = vpx_rb_read_literal(rb, 16);
-
- if (sz == 0)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Invalid header size");
-
- return sz;
-}
-
-static int read_compressed_header(VP10Decoder *pbi, const uint8_t *data,
- size_t partition_size) {
- VP10_COMMON *const cm = &pbi->common;
-#if !CONFIG_MISC_FIXES
- MACROBLOCKD *const xd = &pbi->mb;
-#endif
- FRAME_CONTEXT *const fc = cm->fc;
- vpx_reader r;
- int k, i, j;
-
- if (vpx_reader_init(&r, data, partition_size, pbi->decrypt_cb,
- pbi->decrypt_state))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate bool decoder 0");
-
-#if !CONFIG_MISC_FIXES
- cm->tx_mode = xd->lossless[0] ? ONLY_4X4 : read_tx_mode(&r);
-#endif
- if (cm->tx_mode == TX_MODE_SELECT)
- read_tx_mode_probs(&fc->tx_probs, &r);
- read_coef_probs(fc, cm->tx_mode, &r);
-
- for (k = 0; k < SKIP_CONTEXTS; ++k)
- vp10_diff_update_prob(&r, &fc->skip_probs[k]);
-
-#if CONFIG_MISC_FIXES
- if (cm->seg.enabled) {
- if (cm->seg.temporal_update) {
- for (k = 0; k < PREDICTION_PROBS; k++)
- vp10_diff_update_prob(&r, &cm->fc->seg.pred_probs[k]);
- }
- for (k = 0; k < MAX_SEGMENTS - 1; k++)
- vp10_diff_update_prob(&r, &cm->fc->seg.tree_probs[k]);
- }
-
- for (j = 0; j < INTRA_MODES; j++)
- for (i = 0; i < INTRA_MODES - 1; ++i)
- vp10_diff_update_prob(&r, &fc->uv_mode_prob[j][i]);
-
- for (j = 0; j < PARTITION_CONTEXTS; ++j)
- for (i = 0; i < PARTITION_TYPES - 1; ++i)
- vp10_diff_update_prob(&r, &fc->partition_prob[j][i]);
-#endif
-
- if (frame_is_intra_only(cm)) {
- vp10_copy(cm->kf_y_prob, vp10_kf_y_mode_prob);
-#if CONFIG_MISC_FIXES
- for (k = 0; k < INTRA_MODES; k++)
- for (j = 0; j < INTRA_MODES; j++)
- for (i = 0; i < INTRA_MODES - 1; ++i)
- vp10_diff_update_prob(&r, &cm->kf_y_prob[k][j][i]);
-#endif
- } else {
- nmv_context *const nmvc = &fc->nmvc;
-
- read_inter_mode_probs(fc, &r);
-
- if (cm->interp_filter == SWITCHABLE)
- read_switchable_interp_probs(fc, &r);
-
- for (i = 0; i < INTRA_INTER_CONTEXTS; i++)
- vp10_diff_update_prob(&r, &fc->intra_inter_prob[i]);
-
-#if !CONFIG_MISC_FIXES
- cm->reference_mode = read_frame_reference_mode(cm, &r);
-#endif
- if (cm->reference_mode != SINGLE_REFERENCE)
- setup_compound_reference_mode(cm);
- read_frame_reference_mode_probs(cm, &r);
-
- for (j = 0; j < BLOCK_SIZE_GROUPS; j++)
- for (i = 0; i < INTRA_MODES - 1; ++i)
- vp10_diff_update_prob(&r, &fc->y_mode_prob[j][i]);
-
-#if !CONFIG_MISC_FIXES
- for (j = 0; j < PARTITION_CONTEXTS; ++j)
- for (i = 0; i < PARTITION_TYPES - 1; ++i)
- vp10_diff_update_prob(&r, &fc->partition_prob[j][i]);
-#endif
-
- read_mv_probs(nmvc, cm->allow_high_precision_mv, &r);
- }
-
- return vpx_reader_has_error(&r);
-}
-
-#ifdef NDEBUG
-#define debug_check_frame_counts(cm) (void)0
-#else // !NDEBUG
-// Counts should only be incremented when frame_parallel_decoding_mode and
-// error_resilient_mode are disabled.
-static void debug_check_frame_counts(const VP10_COMMON *const cm) {
- FRAME_COUNTS zero_counts;
- vp10_zero(zero_counts);
- assert(cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_BACKWARD ||
- cm->error_resilient_mode);
- assert(!memcmp(cm->counts.y_mode, zero_counts.y_mode,
- sizeof(cm->counts.y_mode)));
- assert(!memcmp(cm->counts.uv_mode, zero_counts.uv_mode,
- sizeof(cm->counts.uv_mode)));
- assert(!memcmp(cm->counts.partition, zero_counts.partition,
- sizeof(cm->counts.partition)));
- assert(!memcmp(cm->counts.coef, zero_counts.coef,
- sizeof(cm->counts.coef)));
- assert(!memcmp(cm->counts.eob_branch, zero_counts.eob_branch,
- sizeof(cm->counts.eob_branch)));
- assert(!memcmp(cm->counts.switchable_interp, zero_counts.switchable_interp,
- sizeof(cm->counts.switchable_interp)));
- assert(!memcmp(cm->counts.inter_mode, zero_counts.inter_mode,
- sizeof(cm->counts.inter_mode)));
- assert(!memcmp(cm->counts.intra_inter, zero_counts.intra_inter,
- sizeof(cm->counts.intra_inter)));
- assert(!memcmp(cm->counts.comp_inter, zero_counts.comp_inter,
- sizeof(cm->counts.comp_inter)));
- assert(!memcmp(cm->counts.single_ref, zero_counts.single_ref,
- sizeof(cm->counts.single_ref)));
- assert(!memcmp(cm->counts.comp_ref, zero_counts.comp_ref,
- sizeof(cm->counts.comp_ref)));
- assert(!memcmp(&cm->counts.tx, &zero_counts.tx, sizeof(cm->counts.tx)));
- assert(!memcmp(cm->counts.skip, zero_counts.skip, sizeof(cm->counts.skip)));
- assert(!memcmp(&cm->counts.mv, &zero_counts.mv, sizeof(cm->counts.mv)));
-}
-#endif // NDEBUG
-
-static struct vpx_read_bit_buffer *init_read_bit_buffer(
- VP10Decoder *pbi,
- struct vpx_read_bit_buffer *rb,
- const uint8_t *data,
- const uint8_t *data_end,
- uint8_t clear_data[MAX_VP9_HEADER_SIZE]) {
- rb->bit_offset = 0;
- rb->error_handler = error_handler;
- rb->error_handler_data = &pbi->common;
- if (pbi->decrypt_cb) {
- const int n = (int)VPXMIN(MAX_VP9_HEADER_SIZE, data_end - data);
- pbi->decrypt_cb(pbi->decrypt_state, data, clear_data, n);
- rb->bit_buffer = clear_data;
- rb->bit_buffer_end = clear_data + n;
- } else {
- rb->bit_buffer = data;
- rb->bit_buffer_end = data_end;
- }
- return rb;
-}
-
-//------------------------------------------------------------------------------
-
-int vp10_read_sync_code(struct vpx_read_bit_buffer *const rb) {
- return vpx_rb_read_literal(rb, 8) == VP10_SYNC_CODE_0 &&
- vpx_rb_read_literal(rb, 8) == VP10_SYNC_CODE_1 &&
- vpx_rb_read_literal(rb, 8) == VP10_SYNC_CODE_2;
-}
-
-void vp10_read_frame_size(struct vpx_read_bit_buffer *rb,
- int *width, int *height) {
- *width = vpx_rb_read_literal(rb, 16) + 1;
- *height = vpx_rb_read_literal(rb, 16) + 1;
-}
-
-BITSTREAM_PROFILE vp10_read_profile(struct vpx_read_bit_buffer *rb) {
- int profile = vpx_rb_read_bit(rb);
- profile |= vpx_rb_read_bit(rb) << 1;
- if (profile > 2)
- profile += vpx_rb_read_bit(rb);
- return (BITSTREAM_PROFILE) profile;
-}
-
-void vp10_decode_frame(VP10Decoder *pbi,
- const uint8_t *data, const uint8_t *data_end,
- const uint8_t **p_data_end) {
- VP10_COMMON *const cm = &pbi->common;
- MACROBLOCKD *const xd = &pbi->mb;
- struct vpx_read_bit_buffer rb;
- int context_updated = 0;
- uint8_t clear_data[MAX_VP9_HEADER_SIZE];
- const size_t first_partition_size = read_uncompressed_header(pbi,
- init_read_bit_buffer(pbi, &rb, data, data_end, clear_data));
- const int tile_rows = 1 << cm->log2_tile_rows;
- const int tile_cols = 1 << cm->log2_tile_cols;
- YV12_BUFFER_CONFIG *const new_fb = get_frame_new_buffer(cm);
- xd->cur_buf = new_fb;
-
- if (!first_partition_size) {
- // showing a frame directly
- *p_data_end = data + (cm->profile <= PROFILE_2 ? 1 : 2);
- return;
- }
-
- data += vpx_rb_bytes_read(&rb);
- if (!read_is_valid(data, first_partition_size, data_end))
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Truncated packet or corrupt header length");
-
- cm->use_prev_frame_mvs = !cm->error_resilient_mode &&
- cm->width == cm->last_width &&
- cm->height == cm->last_height &&
- !cm->last_intra_only &&
- cm->last_show_frame &&
- (cm->last_frame_type != KEY_FRAME);
-
- vp10_setup_block_planes(xd, cm->subsampling_x, cm->subsampling_y);
-
- *cm->fc = cm->frame_contexts[cm->frame_context_idx];
- if (!cm->fc->initialized)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Uninitialized entropy context.");
-
- vp10_zero(cm->counts);
-
- xd->corrupted = 0;
- new_fb->corrupted = read_compressed_header(pbi, data, first_partition_size);
- if (new_fb->corrupted)
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Decode failed. Frame data header is corrupted.");
-
- if (cm->lf.filter_level && !cm->skip_loop_filter) {
- vp10_loop_filter_frame_init(cm, cm->lf.filter_level);
- }
-
- // If encoded in frame parallel mode, frame context is ready after decoding
- // the frame header.
- if (cm->frame_parallel_decode &&
- cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_BACKWARD) {
- VPxWorker *const worker = pbi->frame_worker_owner;
- FrameWorkerData *const frame_worker_data = worker->data1;
- if (cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_FORWARD) {
- context_updated = 1;
- cm->frame_contexts[cm->frame_context_idx] = *cm->fc;
- }
- vp10_frameworker_lock_stats(worker);
- pbi->cur_buf->row = -1;
- pbi->cur_buf->col = -1;
- frame_worker_data->frame_context_ready = 1;
- // Signal the main thread that context is ready.
- vp10_frameworker_signal_stats(worker);
- vp10_frameworker_unlock_stats(worker);
- }
-
- if (pbi->max_threads > 1 && tile_rows == 1 && tile_cols > 1) {
- // Multi-threaded tile decoder
- *p_data_end = decode_tiles_mt(pbi, data + first_partition_size, data_end);
- if (!xd->corrupted) {
- if (!cm->skip_loop_filter) {
- // If multiple threads are used to decode tiles, then we use those
- // threads to do parallel loopfiltering.
- vp10_loop_filter_frame_mt(new_fb, cm, pbi->mb.plane,
- cm->lf.filter_level, 0, 0, pbi->tile_workers,
- pbi->num_tile_workers, &pbi->lf_row_sync);
- }
- } else {
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Decode failed. Frame data is corrupted.");
-
- }
- } else {
- *p_data_end = decode_tiles(pbi, data + first_partition_size, data_end);
- }
-
- if (!xd->corrupted) {
- if (cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD) {
- vp10_adapt_coef_probs(cm);
-#if CONFIG_MISC_FIXES
- vp10_adapt_intra_frame_probs(cm);
-#endif
-
- if (!frame_is_intra_only(cm)) {
-#if !CONFIG_MISC_FIXES
- vp10_adapt_intra_frame_probs(cm);
-#endif
- vp10_adapt_inter_frame_probs(cm);
- vp10_adapt_mv_probs(cm, cm->allow_high_precision_mv);
- }
- } else {
- debug_check_frame_counts(cm);
- }
- } else {
- vpx_internal_error(&cm->error, VPX_CODEC_CORRUPT_FRAME,
- "Decode failed. Frame data is corrupted.");
- }
-
- // Non frame parallel update frame context here.
- if (cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_OFF &&
- !context_updated)
- cm->frame_contexts[cm->frame_context_idx] = *cm->fc;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_DECODER_DECODEFRAME_H_
-#define VP10_DECODER_DECODEFRAME_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10Decoder;
-struct vpx_read_bit_buffer;
-
-int vp10_read_sync_code(struct vpx_read_bit_buffer *const rb);
-void vp10_read_frame_size(struct vpx_read_bit_buffer *rb,
- int *width, int *height);
-BITSTREAM_PROFILE vp10_read_profile(struct vpx_read_bit_buffer *rb);
-
-void vp10_decode_frame(struct VP10Decoder *pbi,
- const uint8_t *data, const uint8_t *data_end,
- const uint8_t **p_data_end);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_DECODER_DECODEFRAME_H_
+++ /dev/null
-/*
- Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/entropymv.h"
-#include "vp10/common/mvref_common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/decoder/decodemv.h"
-#include "vp10/decoder/decodeframe.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-
-static INLINE int read_uniform(vpx_reader *r, int n) {
- int l = get_unsigned_bits(n);
- int m = (1 << l) - n;
- int v = vpx_read_literal(r, l-1);
-
- assert(l != 0);
-
- if (v < m)
- return v;
- else
- return (v << 1) - m + vpx_read_literal(r, 1);
-}
-
-static PREDICTION_MODE read_intra_mode(vpx_reader *r, const vpx_prob *p) {
- return (PREDICTION_MODE)vpx_read_tree(r, vp10_intra_mode_tree, p);
-}
-
-static PREDICTION_MODE read_intra_mode_y(VP10_COMMON *cm, MACROBLOCKD *xd,
- vpx_reader *r, int size_group) {
- const PREDICTION_MODE y_mode =
- read_intra_mode(r, cm->fc->y_mode_prob[size_group]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->y_mode[size_group][y_mode];
- return y_mode;
-}
-
-static PREDICTION_MODE read_intra_mode_uv(VP10_COMMON *cm, MACROBLOCKD *xd,
- vpx_reader *r,
- PREDICTION_MODE y_mode) {
- const PREDICTION_MODE uv_mode = read_intra_mode(r,
- cm->fc->uv_mode_prob[y_mode]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->uv_mode[y_mode][uv_mode];
- return uv_mode;
-}
-
-static PREDICTION_MODE read_inter_mode(VP10_COMMON *cm, MACROBLOCKD *xd,
- vpx_reader *r, int ctx) {
- const int mode = vpx_read_tree(r, vp10_inter_mode_tree,
- cm->fc->inter_mode_probs[ctx]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->inter_mode[ctx][mode];
-
- return NEARESTMV + mode;
-}
-
-static int read_segment_id(vpx_reader *r,
- const struct segmentation_probs *segp) {
- return vpx_read_tree(r, vp10_segment_tree, segp->tree_probs);
-}
-
-static TX_SIZE read_selected_tx_size(VP10_COMMON *cm, MACROBLOCKD *xd,
- TX_SIZE max_tx_size, vpx_reader *r) {
- FRAME_COUNTS *counts = xd->counts;
- const int ctx = get_tx_size_context(xd);
- const vpx_prob *tx_probs = get_tx_probs(max_tx_size, ctx, &cm->fc->tx_probs);
- int tx_size = vpx_read(r, tx_probs[0]);
- if (tx_size != TX_4X4 && max_tx_size >= TX_16X16) {
- tx_size += vpx_read(r, tx_probs[1]);
- if (tx_size != TX_8X8 && max_tx_size >= TX_32X32)
- tx_size += vpx_read(r, tx_probs[2]);
- }
-
- if (counts)
- ++get_tx_counts(max_tx_size, ctx, &counts->tx)[tx_size];
- return (TX_SIZE)tx_size;
-}
-
-static TX_SIZE read_tx_size(VP10_COMMON *cm, MACROBLOCKD *xd,
- int allow_select, vpx_reader *r) {
- TX_MODE tx_mode = cm->tx_mode;
- BLOCK_SIZE bsize = xd->mi[0]->mbmi.sb_type;
- const TX_SIZE max_tx_size = max_txsize_lookup[bsize];
- if (allow_select && tx_mode == TX_MODE_SELECT && bsize >= BLOCK_8X8)
- return read_selected_tx_size(cm, xd, max_tx_size, r);
- else
- return VPXMIN(max_tx_size, tx_mode_to_biggest_tx_size[tx_mode]);
-}
-
-static int dec_get_segment_id(const VP10_COMMON *cm, const uint8_t *segment_ids,
- int mi_offset, int x_mis, int y_mis) {
- int x, y, segment_id = INT_MAX;
-
- for (y = 0; y < y_mis; y++)
- for (x = 0; x < x_mis; x++)
- segment_id =
- VPXMIN(segment_id, segment_ids[mi_offset + y * cm->mi_cols + x]);
-
- assert(segment_id >= 0 && segment_id < MAX_SEGMENTS);
- return segment_id;
-}
-
-static void set_segment_id(VP10_COMMON *cm, int mi_offset,
- int x_mis, int y_mis, int segment_id) {
- int x, y;
-
- assert(segment_id >= 0 && segment_id < MAX_SEGMENTS);
-
- for (y = 0; y < y_mis; y++)
- for (x = 0; x < x_mis; x++)
- cm->current_frame_seg_map[mi_offset + y * cm->mi_cols + x] = segment_id;
-}
-
-static int read_intra_segment_id(VP10_COMMON *const cm, MACROBLOCKD *const xd,
- int mi_offset, int x_mis, int y_mis,
- vpx_reader *r) {
- struct segmentation *const seg = &cm->seg;
-#if CONFIG_MISC_FIXES
- FRAME_COUNTS *counts = xd->counts;
- struct segmentation_probs *const segp = &cm->fc->seg;
-#else
- struct segmentation_probs *const segp = &cm->segp;
-#endif
- int segment_id;
-
-#if !CONFIG_MISC_FIXES
- (void) xd;
-#endif
-
- if (!seg->enabled)
- return 0; // Default for disabled segmentation
-
- assert(seg->update_map && !seg->temporal_update);
-
- segment_id = read_segment_id(r, segp);
-#if CONFIG_MISC_FIXES
- if (counts)
- ++counts->seg.tree_total[segment_id];
-#endif
- set_segment_id(cm, mi_offset, x_mis, y_mis, segment_id);
- return segment_id;
-}
-
-static void copy_segment_id(const VP10_COMMON *cm,
- const uint8_t *last_segment_ids,
- uint8_t *current_segment_ids,
- int mi_offset, int x_mis, int y_mis) {
- int x, y;
-
- for (y = 0; y < y_mis; y++)
- for (x = 0; x < x_mis; x++)
- current_segment_ids[mi_offset + y * cm->mi_cols + x] = last_segment_ids ?
- last_segment_ids[mi_offset + y * cm->mi_cols + x] : 0;
-}
-
-static int read_inter_segment_id(VP10_COMMON *const cm, MACROBLOCKD *const xd,
- int mi_row, int mi_col, vpx_reader *r) {
- struct segmentation *const seg = &cm->seg;
-#if CONFIG_MISC_FIXES
- FRAME_COUNTS *counts = xd->counts;
- struct segmentation_probs *const segp = &cm->fc->seg;
-#else
- struct segmentation_probs *const segp = &cm->segp;
-#endif
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- int predicted_segment_id, segment_id;
- const int mi_offset = mi_row * cm->mi_cols + mi_col;
- const int bw = xd->plane[0].n4_w >> 1;
- const int bh = xd->plane[0].n4_h >> 1;
-
- // TODO(slavarnway): move x_mis, y_mis into xd ?????
- const int x_mis = VPXMIN(cm->mi_cols - mi_col, bw);
- const int y_mis = VPXMIN(cm->mi_rows - mi_row, bh);
-
- if (!seg->enabled)
- return 0; // Default for disabled segmentation
-
- predicted_segment_id = cm->last_frame_seg_map ?
- dec_get_segment_id(cm, cm->last_frame_seg_map, mi_offset, x_mis, y_mis) :
- 0;
-
- if (!seg->update_map) {
- copy_segment_id(cm, cm->last_frame_seg_map, cm->current_frame_seg_map,
- mi_offset, x_mis, y_mis);
- return predicted_segment_id;
- }
-
- if (seg->temporal_update) {
- const int ctx = vp10_get_pred_context_seg_id(xd);
- const vpx_prob pred_prob = segp->pred_probs[ctx];
- mbmi->seg_id_predicted = vpx_read(r, pred_prob);
-#if CONFIG_MISC_FIXES
- if (counts)
- ++counts->seg.pred[ctx][mbmi->seg_id_predicted];
-#endif
- if (mbmi->seg_id_predicted) {
- segment_id = predicted_segment_id;
- } else {
- segment_id = read_segment_id(r, segp);
-#if CONFIG_MISC_FIXES
- if (counts)
- ++counts->seg.tree_mispred[segment_id];
-#endif
- }
- } else {
- segment_id = read_segment_id(r, segp);
-#if CONFIG_MISC_FIXES
- if (counts)
- ++counts->seg.tree_total[segment_id];
-#endif
- }
- set_segment_id(cm, mi_offset, x_mis, y_mis, segment_id);
- return segment_id;
-}
-
-static int read_skip(VP10_COMMON *cm, const MACROBLOCKD *xd,
- int segment_id, vpx_reader *r) {
- if (segfeature_active(&cm->seg, segment_id, SEG_LVL_SKIP)) {
- return 1;
- } else {
- const int ctx = vp10_get_skip_context(xd);
- const int skip = vpx_read(r, cm->fc->skip_probs[ctx]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->skip[ctx][skip];
- return skip;
- }
-}
-
-static void read_palette_mode_info(VP10_COMMON *const cm,
- MACROBLOCKD *const xd,
- vpx_reader *r) {
- MODE_INFO *const mi = xd->mi[0];
- MB_MODE_INFO *const mbmi = &mi->mbmi;
- const MODE_INFO *above_mi = xd->above_mi;
- const MODE_INFO *left_mi = xd->left_mi;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- int i, palette_ctx = 0;
-
- if (above_mi)
- palette_ctx += (above_mi->mbmi.palette_mode_info.palette_size[0] > 0);
- if (left_mi)
- palette_ctx += (left_mi->mbmi.palette_mode_info.palette_size[0] > 0);
- if (vpx_read(r, vp10_default_palette_y_mode_prob[bsize - BLOCK_8X8]
- [palette_ctx])) {
- int n;
- PALETTE_MODE_INFO *pmi = &mbmi->palette_mode_info;
-
- pmi->palette_size[0] =
- vpx_read_tree(r, vp10_palette_size_tree,
- vp10_default_palette_y_size_prob[bsize - BLOCK_8X8]) + 2;
- n = pmi->palette_size[0];
-
- for (i = 0; i < n; ++i)
- pmi->palette_colors[i] = vpx_read_literal(r, cm->bit_depth);
-
- xd->plane[0].color_index_map[0] = read_uniform(r, n);
- assert(xd->plane[0].color_index_map[0] < n);
- }
-}
-
-static void read_intra_frame_mode_info(VP10_COMMON *const cm,
- MACROBLOCKD *const xd,
- int mi_row, int mi_col, vpx_reader *r) {
- MODE_INFO *const mi = xd->mi[0];
- MB_MODE_INFO *const mbmi = &mi->mbmi;
- const MODE_INFO *above_mi = xd->above_mi;
- const MODE_INFO *left_mi = xd->left_mi;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- int i;
- const int mi_offset = mi_row * cm->mi_cols + mi_col;
- const int bw = xd->plane[0].n4_w >> 1;
- const int bh = xd->plane[0].n4_h >> 1;
-
- // TODO(slavarnway): move x_mis, y_mis into xd ?????
- const int x_mis = VPXMIN(cm->mi_cols - mi_col, bw);
- const int y_mis = VPXMIN(cm->mi_rows - mi_row, bh);
-
- mbmi->segment_id = read_intra_segment_id(cm, xd, mi_offset, x_mis, y_mis, r);
- mbmi->skip = read_skip(cm, xd, mbmi->segment_id, r);
- mbmi->tx_size = read_tx_size(cm, xd, 1, r);
- mbmi->ref_frame[0] = INTRA_FRAME;
- mbmi->ref_frame[1] = NONE;
-
- switch (bsize) {
- case BLOCK_4X4:
- for (i = 0; i < 4; ++i)
- mi->bmi[i].as_mode =
- read_intra_mode(r, get_y_mode_probs(cm, mi, above_mi, left_mi, i));
- mbmi->mode = mi->bmi[3].as_mode;
- break;
- case BLOCK_4X8:
- mi->bmi[0].as_mode = mi->bmi[2].as_mode =
- read_intra_mode(r, get_y_mode_probs(cm, mi, above_mi, left_mi, 0));
- mi->bmi[1].as_mode = mi->bmi[3].as_mode = mbmi->mode =
- read_intra_mode(r, get_y_mode_probs(cm, mi, above_mi, left_mi, 1));
- break;
- case BLOCK_8X4:
- mi->bmi[0].as_mode = mi->bmi[1].as_mode =
- read_intra_mode(r, get_y_mode_probs(cm, mi, above_mi, left_mi, 0));
- mi->bmi[2].as_mode = mi->bmi[3].as_mode = mbmi->mode =
- read_intra_mode(r, get_y_mode_probs(cm, mi, above_mi, left_mi, 2));
- break;
- default:
- mbmi->mode = read_intra_mode(r,
- get_y_mode_probs(cm, mi, above_mi, left_mi, 0));
- }
-
- mbmi->uv_mode = read_intra_mode_uv(cm, xd, r, mbmi->mode);
-
- mbmi->palette_mode_info.palette_size[0] = 0;
- mbmi->palette_mode_info.palette_size[1] = 0;
- if (bsize >= BLOCK_8X8 && cm->allow_screen_content_tools &&
- mbmi->mode == DC_PRED)
- read_palette_mode_info(cm, xd, r);
-}
-
-static int read_mv_component(vpx_reader *r,
- const nmv_component *mvcomp, int usehp) {
- int mag, d, fr, hp;
- const int sign = vpx_read(r, mvcomp->sign);
- const int mv_class = vpx_read_tree(r, vp10_mv_class_tree, mvcomp->classes);
- const int class0 = mv_class == MV_CLASS_0;
-
- // Integer part
- if (class0) {
- d = vpx_read_tree(r, vp10_mv_class0_tree, mvcomp->class0);
- mag = 0;
- } else {
- int i;
- const int n = mv_class + CLASS0_BITS - 1; // number of bits
-
- d = 0;
- for (i = 0; i < n; ++i)
- d |= vpx_read(r, mvcomp->bits[i]) << i;
- mag = CLASS0_SIZE << (mv_class + 2);
- }
-
- // Fractional part
- fr = vpx_read_tree(r, vp10_mv_fp_tree, class0 ? mvcomp->class0_fp[d]
- : mvcomp->fp);
-
- // High precision part (if hp is not used, the default value of the hp is 1)
- hp = usehp ? vpx_read(r, class0 ? mvcomp->class0_hp : mvcomp->hp)
- : 1;
-
- // Result
- mag += ((d << 3) | (fr << 1) | hp) + 1;
- return sign ? -mag : mag;
-}
-
-static INLINE void read_mv(vpx_reader *r, MV *mv, const MV *ref,
- const nmv_context *ctx,
- nmv_context_counts *counts, int allow_hp) {
- const MV_JOINT_TYPE joint_type =
- (MV_JOINT_TYPE)vpx_read_tree(r, vp10_mv_joint_tree, ctx->joints);
- const int use_hp = allow_hp && vp10_use_mv_hp(ref);
- MV diff = {0, 0};
-
- if (mv_joint_vertical(joint_type))
- diff.row = read_mv_component(r, &ctx->comps[0], use_hp);
-
- if (mv_joint_horizontal(joint_type))
- diff.col = read_mv_component(r, &ctx->comps[1], use_hp);
-
- vp10_inc_mv(&diff, counts, use_hp);
-
- mv->row = ref->row + diff.row;
- mv->col = ref->col + diff.col;
-}
-
-static REFERENCE_MODE read_block_reference_mode(VP10_COMMON *cm,
- const MACROBLOCKD *xd,
- vpx_reader *r) {
- if (cm->reference_mode == REFERENCE_MODE_SELECT) {
- const int ctx = vp10_get_reference_mode_context(cm, xd);
- const REFERENCE_MODE mode =
- (REFERENCE_MODE)vpx_read(r, cm->fc->comp_inter_prob[ctx]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->comp_inter[ctx][mode];
- return mode; // SINGLE_REFERENCE or COMPOUND_REFERENCE
- } else {
- return cm->reference_mode;
- }
-}
-
-// Read the referncence frame
-static void read_ref_frames(VP10_COMMON *const cm, MACROBLOCKD *const xd,
- vpx_reader *r,
- int segment_id, MV_REFERENCE_FRAME ref_frame[2]) {
- FRAME_CONTEXT *const fc = cm->fc;
- FRAME_COUNTS *counts = xd->counts;
-
- if (segfeature_active(&cm->seg, segment_id, SEG_LVL_REF_FRAME)) {
- ref_frame[0] = (MV_REFERENCE_FRAME)get_segdata(&cm->seg, segment_id,
- SEG_LVL_REF_FRAME);
- ref_frame[1] = NONE;
- } else {
- const REFERENCE_MODE mode = read_block_reference_mode(cm, xd, r);
- // FIXME(rbultje) I'm pretty sure this breaks segmentation ref frame coding
- if (mode == COMPOUND_REFERENCE) {
- const int idx = cm->ref_frame_sign_bias[cm->comp_fixed_ref];
- const int ctx = vp10_get_pred_context_comp_ref_p(cm, xd);
- const int bit = vpx_read(r, fc->comp_ref_prob[ctx]);
- if (counts)
- ++counts->comp_ref[ctx][bit];
- ref_frame[idx] = cm->comp_fixed_ref;
- ref_frame[!idx] = cm->comp_var_ref[bit];
- } else if (mode == SINGLE_REFERENCE) {
- const int ctx0 = vp10_get_pred_context_single_ref_p1(xd);
- const int bit0 = vpx_read(r, fc->single_ref_prob[ctx0][0]);
- if (counts)
- ++counts->single_ref[ctx0][0][bit0];
- if (bit0) {
- const int ctx1 = vp10_get_pred_context_single_ref_p2(xd);
- const int bit1 = vpx_read(r, fc->single_ref_prob[ctx1][1]);
- if (counts)
- ++counts->single_ref[ctx1][1][bit1];
- ref_frame[0] = bit1 ? ALTREF_FRAME : GOLDEN_FRAME;
- } else {
- ref_frame[0] = LAST_FRAME;
- }
-
- ref_frame[1] = NONE;
- } else {
- assert(0 && "Invalid prediction mode.");
- }
- }
-}
-
-
-static INLINE INTERP_FILTER read_switchable_interp_filter(
- VP10_COMMON *const cm, MACROBLOCKD *const xd,
- vpx_reader *r) {
- const int ctx = vp10_get_pred_context_switchable_interp(xd);
- const INTERP_FILTER type =
- (INTERP_FILTER)vpx_read_tree(r, vp10_switchable_interp_tree,
- cm->fc->switchable_interp_prob[ctx]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->switchable_interp[ctx][type];
- return type;
-}
-
-static void read_intra_block_mode_info(VP10_COMMON *const cm,
- MACROBLOCKD *const xd, MODE_INFO *mi,
- vpx_reader *r) {
- MB_MODE_INFO *const mbmi = &mi->mbmi;
- const BLOCK_SIZE bsize = mi->mbmi.sb_type;
- int i;
-
- mbmi->ref_frame[0] = INTRA_FRAME;
- mbmi->ref_frame[1] = NONE;
-
- switch (bsize) {
- case BLOCK_4X4:
- for (i = 0; i < 4; ++i)
- mi->bmi[i].as_mode = read_intra_mode_y(cm, xd, r, 0);
- mbmi->mode = mi->bmi[3].as_mode;
- break;
- case BLOCK_4X8:
- mi->bmi[0].as_mode = mi->bmi[2].as_mode = read_intra_mode_y(cm, xd,
- r, 0);
- mi->bmi[1].as_mode = mi->bmi[3].as_mode = mbmi->mode =
- read_intra_mode_y(cm, xd, r, 0);
- break;
- case BLOCK_8X4:
- mi->bmi[0].as_mode = mi->bmi[1].as_mode = read_intra_mode_y(cm, xd,
- r, 0);
- mi->bmi[2].as_mode = mi->bmi[3].as_mode = mbmi->mode =
- read_intra_mode_y(cm, xd, r, 0);
- break;
- default:
- mbmi->mode = read_intra_mode_y(cm, xd, r, size_group_lookup[bsize]);
- }
-
- mbmi->uv_mode = read_intra_mode_uv(cm, xd, r, mbmi->mode);
-
- mbmi->palette_mode_info.palette_size[0] = 0;
- mbmi->palette_mode_info.palette_size[1] = 0;
-}
-
-static INLINE int is_mv_valid(const MV *mv) {
- return mv->row > MV_LOW && mv->row < MV_UPP &&
- mv->col > MV_LOW && mv->col < MV_UPP;
-}
-
-static INLINE int assign_mv(VP10_COMMON *cm, MACROBLOCKD *xd,
- PREDICTION_MODE mode,
- int_mv mv[2], int_mv ref_mv[2],
- int_mv nearest_mv[2], int_mv near_mv[2],
- int is_compound, int allow_hp, vpx_reader *r) {
- int i;
- int ret = 1;
-
- switch (mode) {
- case NEWMV: {
- FRAME_COUNTS *counts = xd->counts;
- nmv_context_counts *const mv_counts = counts ? &counts->mv : NULL;
- for (i = 0; i < 1 + is_compound; ++i) {
- read_mv(r, &mv[i].as_mv, &ref_mv[i].as_mv, &cm->fc->nmvc, mv_counts,
- allow_hp);
- ret = ret && is_mv_valid(&mv[i].as_mv);
- }
- break;
- }
- case NEARESTMV: {
- mv[0].as_int = nearest_mv[0].as_int;
- if (is_compound)
- mv[1].as_int = nearest_mv[1].as_int;
- break;
- }
- case NEARMV: {
- mv[0].as_int = near_mv[0].as_int;
- if (is_compound)
- mv[1].as_int = near_mv[1].as_int;
- break;
- }
- case ZEROMV: {
- mv[0].as_int = 0;
- if (is_compound)
- mv[1].as_int = 0;
- break;
- }
- default: {
- return 0;
- }
- }
- return ret;
-}
-
-static int read_is_inter_block(VP10_COMMON *const cm, MACROBLOCKD *const xd,
- int segment_id, vpx_reader *r) {
- if (segfeature_active(&cm->seg, segment_id, SEG_LVL_REF_FRAME)) {
- return get_segdata(&cm->seg, segment_id, SEG_LVL_REF_FRAME) != INTRA_FRAME;
- } else {
- const int ctx = vp10_get_intra_inter_context(xd);
- const int is_inter = vpx_read(r, cm->fc->intra_inter_prob[ctx]);
- FRAME_COUNTS *counts = xd->counts;
- if (counts)
- ++counts->intra_inter[ctx][is_inter];
- return is_inter;
- }
-}
-
-static void fpm_sync(void *const data, int mi_row) {
- VP10Decoder *const pbi = (VP10Decoder *)data;
- vp10_frameworker_wait(pbi->frame_worker_owner, pbi->common.prev_frame,
- mi_row << MI_BLOCK_SIZE_LOG2);
-}
-
-static void read_inter_block_mode_info(VP10Decoder *const pbi,
- MACROBLOCKD *const xd,
- MODE_INFO *const mi,
- int mi_row, int mi_col, vpx_reader *r) {
- VP10_COMMON *const cm = &pbi->common;
- MB_MODE_INFO *const mbmi = &mi->mbmi;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- const int allow_hp = cm->allow_high_precision_mv;
- int_mv nearestmv[2], nearmv[2];
- int_mv ref_mvs[MAX_REF_FRAMES][MAX_MV_REF_CANDIDATES];
- int ref, is_compound;
- uint8_t inter_mode_ctx[MAX_REF_FRAMES];
-
- read_ref_frames(cm, xd, r, mbmi->segment_id, mbmi->ref_frame);
- is_compound = has_second_ref(mbmi);
-
- for (ref = 0; ref < 1 + is_compound; ++ref) {
- const MV_REFERENCE_FRAME frame = mbmi->ref_frame[ref];
- RefBuffer *ref_buf = &cm->frame_refs[frame - LAST_FRAME];
-
- xd->block_refs[ref] = ref_buf;
- if ((!vp10_is_valid_scale(&ref_buf->sf)))
- vpx_internal_error(xd->error_info, VPX_CODEC_UNSUP_BITSTREAM,
- "Reference frame has invalid dimensions");
- vp10_setup_pre_planes(xd, ref, ref_buf->buf, mi_row, mi_col,
- &ref_buf->sf);
- vp10_find_mv_refs(cm, xd, mi, frame, ref_mvs[frame],
- mi_row, mi_col, fpm_sync, (void *)pbi, inter_mode_ctx);
- }
-
- if (segfeature_active(&cm->seg, mbmi->segment_id, SEG_LVL_SKIP)) {
- mbmi->mode = ZEROMV;
- if (bsize < BLOCK_8X8) {
- vpx_internal_error(xd->error_info, VPX_CODEC_UNSUP_BITSTREAM,
- "Invalid usage of segement feature on small blocks");
- return;
- }
- } else {
- if (bsize >= BLOCK_8X8)
- mbmi->mode = read_inter_mode(cm, xd, r,
- inter_mode_ctx[mbmi->ref_frame[0]]);
- }
-
- if (bsize < BLOCK_8X8 || mbmi->mode != ZEROMV) {
- for (ref = 0; ref < 1 + is_compound; ++ref) {
- vp10_find_best_ref_mvs(allow_hp, ref_mvs[mbmi->ref_frame[ref]],
- &nearestmv[ref], &nearmv[ref]);
- }
- }
-
- mbmi->interp_filter = (cm->interp_filter == SWITCHABLE)
- ? read_switchable_interp_filter(cm, xd, r)
- : cm->interp_filter;
-
- if (bsize < BLOCK_8X8) {
- const int num_4x4_w = 1 << xd->bmode_blocks_wl;
- const int num_4x4_h = 1 << xd->bmode_blocks_hl;
- int idx, idy;
- PREDICTION_MODE b_mode;
- int_mv nearest_sub8x8[2], near_sub8x8[2];
- for (idy = 0; idy < 2; idy += num_4x4_h) {
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- int_mv block[2];
- const int j = idy * 2 + idx;
- b_mode = read_inter_mode(cm, xd, r, inter_mode_ctx[mbmi->ref_frame[0]]);
-
- if (b_mode == NEARESTMV || b_mode == NEARMV) {
- uint8_t dummy_mode_ctx[MAX_REF_FRAMES];
- for (ref = 0; ref < 1 + is_compound; ++ref)
- vp10_append_sub8x8_mvs_for_idx(cm, xd, j, ref, mi_row, mi_col,
- &nearest_sub8x8[ref],
- &near_sub8x8[ref],
- dummy_mode_ctx);
- }
-
- if (!assign_mv(cm, xd, b_mode, block, nearestmv,
- nearest_sub8x8, near_sub8x8,
- is_compound, allow_hp, r)) {
- xd->corrupted |= 1;
- break;
- };
-
- mi->bmi[j].as_mv[0].as_int = block[0].as_int;
- if (is_compound)
- mi->bmi[j].as_mv[1].as_int = block[1].as_int;
-
- if (num_4x4_h == 2)
- mi->bmi[j + 2] = mi->bmi[j];
- if (num_4x4_w == 2)
- mi->bmi[j + 1] = mi->bmi[j];
- }
- }
-
- mi->mbmi.mode = b_mode;
-
- mbmi->mv[0].as_int = mi->bmi[3].as_mv[0].as_int;
- mbmi->mv[1].as_int = mi->bmi[3].as_mv[1].as_int;
- } else {
- xd->corrupted |= !assign_mv(cm, xd, mbmi->mode, mbmi->mv, nearestmv,
- nearestmv, nearmv, is_compound, allow_hp, r);
- }
-}
-
-static void read_inter_frame_mode_info(VP10Decoder *const pbi,
- MACROBLOCKD *const xd,
- int mi_row, int mi_col, vpx_reader *r) {
- VP10_COMMON *const cm = &pbi->common;
- MODE_INFO *const mi = xd->mi[0];
- MB_MODE_INFO *const mbmi = &mi->mbmi;
- int inter_block;
-
- mbmi->mv[0].as_int = 0;
- mbmi->mv[1].as_int = 0;
- mbmi->segment_id = read_inter_segment_id(cm, xd, mi_row, mi_col, r);
- mbmi->skip = read_skip(cm, xd, mbmi->segment_id, r);
- inter_block = read_is_inter_block(cm, xd, mbmi->segment_id, r);
- mbmi->tx_size = read_tx_size(cm, xd, !mbmi->skip || !inter_block, r);
-
- if (inter_block)
- read_inter_block_mode_info(pbi, xd, mi, mi_row, mi_col, r);
- else
- read_intra_block_mode_info(cm, xd, mi, r);
-}
-
-void vp10_read_mode_info(VP10Decoder *const pbi, MACROBLOCKD *xd,
- int mi_row, int mi_col, vpx_reader *r,
- int x_mis, int y_mis) {
- VP10_COMMON *const cm = &pbi->common;
- MODE_INFO *const mi = xd->mi[0];
- MV_REF* frame_mvs = cm->cur_frame->mvs + mi_row * cm->mi_cols + mi_col;
- int w, h;
-
- if (frame_is_intra_only(cm)) {
- read_intra_frame_mode_info(cm, xd, mi_row, mi_col, r);
- } else {
- read_inter_frame_mode_info(pbi, xd, mi_row, mi_col, r);
-
- for (h = 0; h < y_mis; ++h) {
- MV_REF *const frame_mv = frame_mvs + h * cm->mi_cols;
- for (w = 0; w < x_mis; ++w) {
- MV_REF *const mv = frame_mv + w;
- mv->ref_frame[0] = mi->mbmi.ref_frame[0];
- mv->ref_frame[1] = mi->mbmi.ref_frame[1];
- mv->mv[0].as_int = mi->mbmi.mv[0].as_int;
- mv->mv[1].as_int = mi->mbmi.mv[1].as_int;
- }
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_DECODER_DECODEMV_H_
-#define VP10_DECODER_DECODEMV_H_
-
-#include "vpx_dsp/bitreader.h"
-
-#include "vp10/decoder/decoder.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_read_mode_info(VP10Decoder *const pbi, MACROBLOCKD *xd,
- int mi_row, int mi_col, vpx_reader *r,
- int x_mis, int y_mis);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_DECODER_DECODEMV_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <limits.h>
-#include <stdio.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "./vpx_scale_rtcd.h"
-
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/system_state.h"
-#include "vpx_ports/vpx_once.h"
-#include "vpx_ports/vpx_timer.h"
-#include "vpx_scale/vpx_scale.h"
-#include "vpx_util/vpx_thread.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/loopfilter.h"
-#include "vp10/common/onyxc_int.h"
-#if CONFIG_VP9_POSTPROC
-#include "vp10/common/postproc.h"
-#endif
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconintra.h"
-
-#include "vp10/decoder/decodeframe.h"
-#include "vp10/decoder/decoder.h"
-#include "vp10/decoder/detokenize.h"
-
-static void initialize_dec(void) {
- static volatile int init_done = 0;
-
- if (!init_done) {
- vp10_rtcd();
- vpx_dsp_rtcd();
- vpx_scale_rtcd();
- vp10_init_intra_predictors();
- init_done = 1;
- }
-}
-
-static void vp10_dec_setup_mi(VP10_COMMON *cm) {
- cm->mi = cm->mip + cm->mi_stride + 1;
- cm->mi_grid_visible = cm->mi_grid_base + cm->mi_stride + 1;
- memset(cm->mi_grid_base, 0,
- cm->mi_stride * (cm->mi_rows + 1) * sizeof(*cm->mi_grid_base));
-}
-
-static int vp10_dec_alloc_mi(VP10_COMMON *cm, int mi_size) {
- cm->mip = vpx_calloc(mi_size, sizeof(*cm->mip));
- if (!cm->mip)
- return 1;
- cm->mi_alloc_size = mi_size;
- cm->mi_grid_base = (MODE_INFO **)vpx_calloc(mi_size, sizeof(MODE_INFO*));
- if (!cm->mi_grid_base)
- return 1;
- return 0;
-}
-
-static void vp10_dec_free_mi(VP10_COMMON *cm) {
- vpx_free(cm->mip);
- cm->mip = NULL;
- vpx_free(cm->mi_grid_base);
- cm->mi_grid_base = NULL;
-}
-
-VP10Decoder *vp10_decoder_create(BufferPool *const pool) {
- VP10Decoder *volatile const pbi = vpx_memalign(32, sizeof(*pbi));
- VP10_COMMON *volatile const cm = pbi ? &pbi->common : NULL;
-
- if (!cm)
- return NULL;
-
- vp10_zero(*pbi);
-
- if (setjmp(cm->error.jmp)) {
- cm->error.setjmp = 0;
- vp10_decoder_remove(pbi);
- return NULL;
- }
-
- cm->error.setjmp = 1;
-
- CHECK_MEM_ERROR(cm, cm->fc,
- (FRAME_CONTEXT *)vpx_calloc(1, sizeof(*cm->fc)));
- CHECK_MEM_ERROR(cm, cm->frame_contexts,
- (FRAME_CONTEXT *)vpx_calloc(FRAME_CONTEXTS,
- sizeof(*cm->frame_contexts)));
-
- pbi->need_resync = 1;
- once(initialize_dec);
-
- // Initialize the references to not point to any frame buffers.
- memset(&cm->ref_frame_map, -1, sizeof(cm->ref_frame_map));
- memset(&cm->next_ref_frame_map, -1, sizeof(cm->next_ref_frame_map));
-
- cm->current_video_frame = 0;
- pbi->ready_for_new_data = 1;
- pbi->common.buffer_pool = pool;
-
- cm->bit_depth = VPX_BITS_8;
- cm->dequant_bit_depth = VPX_BITS_8;
-
- cm->alloc_mi = vp10_dec_alloc_mi;
- cm->free_mi = vp10_dec_free_mi;
- cm->setup_mi = vp10_dec_setup_mi;
-
- vp10_loop_filter_init(cm);
-
- cm->error.setjmp = 0;
-
- vpx_get_worker_interface()->init(&pbi->lf_worker);
-
- return pbi;
-}
-
-void vp10_decoder_remove(VP10Decoder *pbi) {
- int i;
-
- if (!pbi)
- return;
-
- vpx_get_worker_interface()->end(&pbi->lf_worker);
- vpx_free(pbi->lf_worker.data1);
- vpx_free(pbi->tile_data);
- for (i = 0; i < pbi->num_tile_workers; ++i) {
- VPxWorker *const worker = &pbi->tile_workers[i];
- vpx_get_worker_interface()->end(worker);
- }
- vpx_free(pbi->tile_worker_data);
- vpx_free(pbi->tile_worker_info);
- vpx_free(pbi->tile_workers);
-
- if (pbi->num_tile_workers > 0) {
- vp10_loop_filter_dealloc(&pbi->lf_row_sync);
- }
-
- vpx_free(pbi);
-}
-
-static int equal_dimensions(const YV12_BUFFER_CONFIG *a,
- const YV12_BUFFER_CONFIG *b) {
- return a->y_height == b->y_height && a->y_width == b->y_width &&
- a->uv_height == b->uv_height && a->uv_width == b->uv_width;
-}
-
-vpx_codec_err_t vp10_copy_reference_dec(VP10Decoder *pbi,
- VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd) {
- VP10_COMMON *cm = &pbi->common;
-
- /* TODO(jkoleszar): The decoder doesn't have any real knowledge of what the
- * encoder is using the frame buffers for. This is just a stub to keep the
- * vpxenc --test-decode functionality working, and will be replaced in a
- * later commit that adds VP9-specific controls for this functionality.
- */
- if (ref_frame_flag == VP9_LAST_FLAG) {
- const YV12_BUFFER_CONFIG *const cfg = get_ref_frame(cm, 0);
- if (cfg == NULL) {
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "No 'last' reference frame");
- return VPX_CODEC_ERROR;
- }
- if (!equal_dimensions(cfg, sd))
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Incorrect buffer dimensions");
- else
- vp8_yv12_copy_frame(cfg, sd);
- } else {
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Invalid reference frame");
- }
-
- return cm->error.error_code;
-}
-
-
-vpx_codec_err_t vp10_set_reference_dec(VP10_COMMON *cm,
- VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd) {
- RefBuffer *ref_buf = NULL;
- RefCntBuffer *const frame_bufs = cm->buffer_pool->frame_bufs;
-
- // TODO(jkoleszar): The decoder doesn't have any real knowledge of what the
- // encoder is using the frame buffers for. This is just a stub to keep the
- // vpxenc --test-decode functionality working, and will be replaced in a
- // later commit that adds VP9-specific controls for this functionality.
- if (ref_frame_flag == VP9_LAST_FLAG) {
- ref_buf = &cm->frame_refs[0];
- } else if (ref_frame_flag == VP9_GOLD_FLAG) {
- ref_buf = &cm->frame_refs[1];
- } else if (ref_frame_flag == VP9_ALT_FLAG) {
- ref_buf = &cm->frame_refs[2];
- } else {
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Invalid reference frame");
- return cm->error.error_code;
- }
-
- if (!equal_dimensions(ref_buf->buf, sd)) {
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Incorrect buffer dimensions");
- } else {
- int *ref_fb_ptr = &ref_buf->idx;
-
- // Find an empty frame buffer.
- const int free_fb = get_free_fb(cm);
- if (cm->new_fb_idx == INVALID_IDX)
- return VPX_CODEC_MEM_ERROR;
-
- // Decrease ref_count since it will be increased again in
- // ref_cnt_fb() below.
- --frame_bufs[free_fb].ref_count;
-
- // Manage the reference counters and copy image.
- ref_cnt_fb(frame_bufs, ref_fb_ptr, free_fb);
- ref_buf->buf = &frame_bufs[*ref_fb_ptr].buf;
- vp8_yv12_copy_frame(sd, ref_buf->buf);
- }
-
- return cm->error.error_code;
-}
-
-/* If any buffer updating is signaled it should be done here. */
-static void swap_frame_buffers(VP10Decoder *pbi) {
- int ref_index = 0, mask;
- VP10_COMMON *const cm = &pbi->common;
- BufferPool *const pool = cm->buffer_pool;
- RefCntBuffer *const frame_bufs = cm->buffer_pool->frame_bufs;
-
- lock_buffer_pool(pool);
- for (mask = pbi->refresh_frame_flags; mask; mask >>= 1) {
- const int old_idx = cm->ref_frame_map[ref_index];
- // Current thread releases the holding of reference frame.
- decrease_ref_count(old_idx, frame_bufs, pool);
-
- // Release the reference frame in reference map.
- if ((mask & 1) && old_idx >= 0) {
- decrease_ref_count(old_idx, frame_bufs, pool);
- }
- cm->ref_frame_map[ref_index] = cm->next_ref_frame_map[ref_index];
- ++ref_index;
- }
-
- // Current thread releases the holding of reference frame.
- for (; ref_index < REF_FRAMES && !cm->show_existing_frame; ++ref_index) {
- const int old_idx = cm->ref_frame_map[ref_index];
- decrease_ref_count(old_idx, frame_bufs, pool);
- cm->ref_frame_map[ref_index] = cm->next_ref_frame_map[ref_index];
- }
- unlock_buffer_pool(pool);
- pbi->hold_ref_buf = 0;
- cm->frame_to_show = get_frame_new_buffer(cm);
-
- if (!cm->frame_parallel_decode || !cm->show_frame) {
- lock_buffer_pool(pool);
- --frame_bufs[cm->new_fb_idx].ref_count;
- unlock_buffer_pool(pool);
- }
-
- // Invalidate these references until the next frame starts.
- for (ref_index = 0; ref_index < 3; ref_index++)
- cm->frame_refs[ref_index].idx = -1;
-}
-
-int vp10_receive_compressed_data(VP10Decoder *pbi,
- size_t size, const uint8_t **psource) {
- VP10_COMMON *volatile const cm = &pbi->common;
- BufferPool *volatile const pool = cm->buffer_pool;
- RefCntBuffer *volatile const frame_bufs = cm->buffer_pool->frame_bufs;
- const uint8_t *source = *psource;
- int retcode = 0;
- cm->error.error_code = VPX_CODEC_OK;
-
- if (size == 0) {
- // This is used to signal that we are missing frames.
- // We do not know if the missing frame(s) was supposed to update
- // any of the reference buffers, but we act conservative and
- // mark only the last buffer as corrupted.
- //
- // TODO(jkoleszar): Error concealment is undefined and non-normative
- // at this point, but if it becomes so, [0] may not always be the correct
- // thing to do here.
- if (cm->frame_refs[0].idx > 0) {
- assert(cm->frame_refs[0].buf != NULL);
- cm->frame_refs[0].buf->corrupted = 1;
- }
- }
-
- pbi->ready_for_new_data = 0;
-
- // Check if the previous frame was a frame without any references to it.
- // Release frame buffer if not decoding in frame parallel mode.
- if (!cm->frame_parallel_decode && cm->new_fb_idx >= 0
- && frame_bufs[cm->new_fb_idx].ref_count == 0)
- pool->release_fb_cb(pool->cb_priv,
- &frame_bufs[cm->new_fb_idx].raw_frame_buffer);
- // Find a free frame buffer. Return error if can not find any.
- cm->new_fb_idx = get_free_fb(cm);
- if (cm->new_fb_idx == INVALID_IDX)
- return VPX_CODEC_MEM_ERROR;
-
- // Assign a MV array to the frame buffer.
- cm->cur_frame = &pool->frame_bufs[cm->new_fb_idx];
-
- pbi->hold_ref_buf = 0;
- if (cm->frame_parallel_decode) {
- VPxWorker *const worker = pbi->frame_worker_owner;
- vp10_frameworker_lock_stats(worker);
- frame_bufs[cm->new_fb_idx].frame_worker_owner = worker;
- // Reset decoding progress.
- pbi->cur_buf = &frame_bufs[cm->new_fb_idx];
- pbi->cur_buf->row = -1;
- pbi->cur_buf->col = -1;
- vp10_frameworker_unlock_stats(worker);
- } else {
- pbi->cur_buf = &frame_bufs[cm->new_fb_idx];
- }
-
-
- if (setjmp(cm->error.jmp)) {
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- int i;
-
- cm->error.setjmp = 0;
- pbi->ready_for_new_data = 1;
-
- // Synchronize all threads immediately as a subsequent decode call may
- // cause a resize invalidating some allocations.
- winterface->sync(&pbi->lf_worker);
- for (i = 0; i < pbi->num_tile_workers; ++i) {
- winterface->sync(&pbi->tile_workers[i]);
- }
-
- lock_buffer_pool(pool);
- // Release all the reference buffers if worker thread is holding them.
- if (pbi->hold_ref_buf == 1) {
- int ref_index = 0, mask;
- for (mask = pbi->refresh_frame_flags; mask; mask >>= 1) {
- const int old_idx = cm->ref_frame_map[ref_index];
- // Current thread releases the holding of reference frame.
- decrease_ref_count(old_idx, frame_bufs, pool);
-
- // Release the reference frame in reference map.
- if ((mask & 1) && old_idx >= 0) {
- decrease_ref_count(old_idx, frame_bufs, pool);
- }
- ++ref_index;
- }
-
- // Current thread releases the holding of reference frame.
- for (; ref_index < REF_FRAMES && !cm->show_existing_frame; ++ref_index) {
- const int old_idx = cm->ref_frame_map[ref_index];
- decrease_ref_count(old_idx, frame_bufs, pool);
- }
- pbi->hold_ref_buf = 0;
- }
- // Release current frame.
- decrease_ref_count(cm->new_fb_idx, frame_bufs, pool);
- unlock_buffer_pool(pool);
-
- vpx_clear_system_state();
- return -1;
- }
-
- cm->error.setjmp = 1;
- vp10_decode_frame(pbi, source, source + size, psource);
-
- swap_frame_buffers(pbi);
-
- vpx_clear_system_state();
-
- if (!cm->show_existing_frame) {
- cm->last_show_frame = cm->show_frame;
- cm->prev_frame = cm->cur_frame;
- if (cm->seg.enabled && !cm->frame_parallel_decode)
- vp10_swap_current_and_last_seg_map(cm);
- }
-
- // Update progress in frame parallel decode.
- if (cm->frame_parallel_decode) {
- // Need to lock the mutex here as another thread may
- // be accessing this buffer.
- VPxWorker *const worker = pbi->frame_worker_owner;
- FrameWorkerData *const frame_worker_data = worker->data1;
- vp10_frameworker_lock_stats(worker);
-
- if (cm->show_frame) {
- cm->current_video_frame++;
- }
- frame_worker_data->frame_decoded = 1;
- frame_worker_data->frame_context_ready = 1;
- vp10_frameworker_signal_stats(worker);
- vp10_frameworker_unlock_stats(worker);
- } else {
- cm->last_width = cm->width;
- cm->last_height = cm->height;
- if (cm->show_frame) {
- cm->current_video_frame++;
- }
- }
-
- cm->error.setjmp = 0;
- return retcode;
-}
-
-int vp10_get_raw_frame(VP10Decoder *pbi, YV12_BUFFER_CONFIG *sd,
- vp10_ppflags_t *flags) {
- VP10_COMMON *const cm = &pbi->common;
- int ret = -1;
-#if !CONFIG_VP9_POSTPROC
- (void)*flags;
-#endif
-
- if (pbi->ready_for_new_data == 1)
- return ret;
-
- pbi->ready_for_new_data = 1;
-
- /* no raw frame to show!!! */
- if (!cm->show_frame)
- return ret;
-
- pbi->ready_for_new_data = 1;
-
-#if CONFIG_VP9_POSTPROC
- if (!cm->show_existing_frame) {
- ret = vp10_post_proc_frame(cm, sd, flags);
- } else {
- *sd = *cm->frame_to_show;
- ret = 0;
- }
-#else
- *sd = *cm->frame_to_show;
- ret = 0;
-#endif /*!CONFIG_POSTPROC*/
- vpx_clear_system_state();
- return ret;
-}
-
-vpx_codec_err_t vp10_parse_superframe_index(const uint8_t *data,
- size_t data_sz,
- uint32_t sizes[8], int *count,
- vpx_decrypt_cb decrypt_cb,
- void *decrypt_state) {
- // A chunk ending with a byte matching 0xc0 is an invalid chunk unless
- // it is a super frame index. If the last byte of real video compression
- // data is 0xc0 the encoder must add a 0 byte. If we have the marker but
- // not the associated matching marker byte at the front of the index we have
- // an invalid bitstream and need to return an error.
-
- uint8_t marker;
-#if CONFIG_MISC_FIXES
- size_t frame_sz_sum = 0;
-#endif
-
- assert(data_sz);
- marker = read_marker(decrypt_cb, decrypt_state, data + data_sz - 1);
- *count = 0;
-
- if ((marker & 0xe0) == 0xc0) {
- const uint32_t frames = (marker & 0x7) + 1;
- const uint32_t mag = ((marker >> 3) & 0x3) + 1;
- const size_t index_sz = 2 + mag * (frames - CONFIG_MISC_FIXES);
-
- // This chunk is marked as having a superframe index but doesn't have
- // enough data for it, thus it's an invalid superframe index.
- if (data_sz < index_sz)
- return VPX_CODEC_CORRUPT_FRAME;
-
- {
- const uint8_t marker2 = read_marker(decrypt_cb, decrypt_state,
- data + data_sz - index_sz);
-
- // This chunk is marked as having a superframe index but doesn't have
- // the matching marker byte at the front of the index therefore it's an
- // invalid chunk.
- if (marker != marker2)
- return VPX_CODEC_CORRUPT_FRAME;
- }
-
- {
- // Found a valid superframe index.
- uint32_t i, j;
- const uint8_t *x = &data[data_sz - index_sz + 1];
-
- // Frames has a maximum of 8 and mag has a maximum of 4.
- uint8_t clear_buffer[32];
- assert(sizeof(clear_buffer) >= frames * mag);
- if (decrypt_cb) {
- decrypt_cb(decrypt_state, x, clear_buffer, frames * mag);
- x = clear_buffer;
- }
-
- for (i = 0; i < frames - CONFIG_MISC_FIXES; ++i) {
- uint32_t this_sz = 0;
-
- for (j = 0; j < mag; ++j)
- this_sz |= (*x++) << (j * 8);
- this_sz += CONFIG_MISC_FIXES;
- sizes[i] = this_sz;
-#if CONFIG_MISC_FIXES
- frame_sz_sum += this_sz;
-#endif
- }
-#if CONFIG_MISC_FIXES
- sizes[i] = data_sz - index_sz - frame_sz_sum;
-#endif
- *count = frames;
- }
- }
- return VPX_CODEC_OK;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_DECODER_DECODER_H_
-#define VP10_DECODER_DECODER_H_
-
-#include "./vpx_config.h"
-
-#include "vpx/vpx_codec.h"
-#include "vpx_dsp/bitreader.h"
-#include "vpx_scale/yv12config.h"
-#include "vpx_util/vpx_thread.h"
-
-#include "vp10/common/thread_common.h"
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/ppflags.h"
-#include "vp10/decoder/dthread.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// TODO(hkuang): combine this with TileWorkerData.
-typedef struct TileData {
- VP10_COMMON *cm;
- vpx_reader bit_reader;
- DECLARE_ALIGNED(16, MACROBLOCKD, xd);
- /* dqcoeff are shared by all the planes. So planes must be decoded serially */
- DECLARE_ALIGNED(16, tran_low_t, dqcoeff[32 * 32]);
- DECLARE_ALIGNED(16, uint8_t, color_index_map[2][64 * 64]);
-} TileData;
-
-typedef struct TileWorkerData {
- struct VP10Decoder *pbi;
- vpx_reader bit_reader;
- FRAME_COUNTS counts;
- DECLARE_ALIGNED(16, MACROBLOCKD, xd);
- /* dqcoeff are shared by all the planes. So planes must be decoded serially */
- DECLARE_ALIGNED(16, tran_low_t, dqcoeff[32 * 32]);
- DECLARE_ALIGNED(16, uint8_t, color_index_map[2][64 * 64]);
- struct vpx_internal_error_info error_info;
-} TileWorkerData;
-
-typedef struct VP10Decoder {
- DECLARE_ALIGNED(16, MACROBLOCKD, mb);
-
- DECLARE_ALIGNED(16, VP10_COMMON, common);
-
- int ready_for_new_data;
-
- int refresh_frame_flags;
-
- // TODO(hkuang): Combine this with cur_buf in macroblockd as they are
- // the same.
- RefCntBuffer *cur_buf; // Current decoding frame buffer.
-
- VPxWorker *frame_worker_owner; // frame_worker that owns this pbi.
- VPxWorker lf_worker;
- VPxWorker *tile_workers;
- TileWorkerData *tile_worker_data;
- TileInfo *tile_worker_info;
- int num_tile_workers;
-
- TileData *tile_data;
- int total_tiles;
-
- VP9LfSync lf_row_sync;
-
- vpx_decrypt_cb decrypt_cb;
- void *decrypt_state;
-
- int max_threads;
- int inv_tile_order;
- int need_resync; // wait for key/intra-only frame.
- int hold_ref_buf; // hold the reference buffer.
-} VP10Decoder;
-
-int vp10_receive_compressed_data(struct VP10Decoder *pbi,
- size_t size, const uint8_t **dest);
-
-int vp10_get_raw_frame(struct VP10Decoder *pbi, YV12_BUFFER_CONFIG *sd,
- vp10_ppflags_t *flags);
-
-vpx_codec_err_t vp10_copy_reference_dec(struct VP10Decoder *pbi,
- VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd);
-
-vpx_codec_err_t vp10_set_reference_dec(VP10_COMMON *cm,
- VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd);
-
-static INLINE uint8_t read_marker(vpx_decrypt_cb decrypt_cb,
- void *decrypt_state,
- const uint8_t *data) {
- if (decrypt_cb) {
- uint8_t marker;
- decrypt_cb(decrypt_state, data, &marker, 1);
- return marker;
- }
- return *data;
-}
-
-// This function is exposed for use in tests, as well as the inlined function
-// "read_marker".
-vpx_codec_err_t vp10_parse_superframe_index(const uint8_t *data,
- size_t data_sz,
- uint32_t sizes[8], int *count,
- vpx_decrypt_cb decrypt_cb,
- void *decrypt_state);
-
-struct VP10Decoder *vp10_decoder_create(BufferPool *const pool);
-
-void vp10_decoder_remove(struct VP10Decoder *pbi);
-
-static INLINE void decrease_ref_count(int idx, RefCntBuffer *const frame_bufs,
- BufferPool *const pool) {
- if (idx >= 0) {
- --frame_bufs[idx].ref_count;
- // A worker may only get a free framebuffer index when calling get_free_fb.
- // But the private buffer is not set up until finish decoding header.
- // So any error happens during decoding header, the frame_bufs will not
- // have valid priv buffer.
- if (frame_bufs[idx].ref_count == 0 &&
- frame_bufs[idx].raw_frame_buffer.priv) {
- pool->release_fb_cb(pool->cb_priv, &frame_bufs[idx].raw_frame_buffer);
- }
- }
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_DECODER_DECODER_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#if CONFIG_COEFFICIENT_RANGE_CHECKING
-#include "vp10/common/idct.h"
-#endif
-
-#include "vp10/decoder/detokenize.h"
-
-#define EOB_CONTEXT_NODE 0
-#define ZERO_CONTEXT_NODE 1
-#define ONE_CONTEXT_NODE 2
-#define LOW_VAL_CONTEXT_NODE 0
-#define TWO_CONTEXT_NODE 1
-#define THREE_CONTEXT_NODE 2
-#define HIGH_LOW_CONTEXT_NODE 3
-#define CAT_ONE_CONTEXT_NODE 4
-#define CAT_THREEFOUR_CONTEXT_NODE 5
-#define CAT_THREE_CONTEXT_NODE 6
-#define CAT_FIVE_CONTEXT_NODE 7
-
-#define INCREMENT_COUNT(token) \
- do { \
- if (counts) \
- ++coef_counts[band][ctx][token]; \
- } while (0)
-
-static INLINE int read_coeff(const vpx_prob *probs, int n, vpx_reader *r) {
- int i, val = 0;
- for (i = 0; i < n; ++i)
- val = (val << 1) | vpx_read(r, probs[i]);
- return val;
-}
-
-static int decode_coefs(const MACROBLOCKD *xd,
- PLANE_TYPE type,
- tran_low_t *dqcoeff, TX_SIZE tx_size, const int16_t *dq,
- int ctx, const int16_t *scan, const int16_t *nb,
- vpx_reader *r) {
- FRAME_COUNTS *counts = xd->counts;
- const int max_eob = 16 << (tx_size << 1);
- const FRAME_CONTEXT *const fc = xd->fc;
- const int ref = is_inter_block(&xd->mi[0]->mbmi);
- int band, c = 0;
- const vpx_prob (*coef_probs)[COEFF_CONTEXTS][UNCONSTRAINED_NODES] =
- fc->coef_probs[tx_size][type][ref];
- const vpx_prob *prob;
- unsigned int (*coef_counts)[COEFF_CONTEXTS][UNCONSTRAINED_NODES + 1];
- unsigned int (*eob_branch_count)[COEFF_CONTEXTS];
- uint8_t token_cache[32 * 32];
- const uint8_t *band_translate = get_band_translate(tx_size);
- const int dq_shift = (tx_size == TX_32X32);
- int v, token;
- int16_t dqv = dq[0];
- const uint8_t *cat1_prob;
- const uint8_t *cat2_prob;
- const uint8_t *cat3_prob;
- const uint8_t *cat4_prob;
- const uint8_t *cat5_prob;
- const uint8_t *cat6_prob;
-
- if (counts) {
- coef_counts = counts->coef[tx_size][type][ref];
- eob_branch_count = counts->eob_branch[tx_size][type][ref];
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->bd > VPX_BITS_8) {
- if (xd->bd == VPX_BITS_10) {
- cat1_prob = vp10_cat1_prob_high10;
- cat2_prob = vp10_cat2_prob_high10;
- cat3_prob = vp10_cat3_prob_high10;
- cat4_prob = vp10_cat4_prob_high10;
- cat5_prob = vp10_cat5_prob_high10;
- cat6_prob = vp10_cat6_prob_high10;
- } else {
- cat1_prob = vp10_cat1_prob_high12;
- cat2_prob = vp10_cat2_prob_high12;
- cat3_prob = vp10_cat3_prob_high12;
- cat4_prob = vp10_cat4_prob_high12;
- cat5_prob = vp10_cat5_prob_high12;
- cat6_prob = vp10_cat6_prob_high12;
- }
- } else {
- cat1_prob = vp10_cat1_prob;
- cat2_prob = vp10_cat2_prob;
- cat3_prob = vp10_cat3_prob;
- cat4_prob = vp10_cat4_prob;
- cat5_prob = vp10_cat5_prob;
- cat6_prob = vp10_cat6_prob;
- }
-#else
- cat1_prob = vp10_cat1_prob;
- cat2_prob = vp10_cat2_prob;
- cat3_prob = vp10_cat3_prob;
- cat4_prob = vp10_cat4_prob;
- cat5_prob = vp10_cat5_prob;
- cat6_prob = vp10_cat6_prob;
-#endif
-
- while (c < max_eob) {
- int val = -1;
- band = *band_translate++;
- prob = coef_probs[band][ctx];
- if (counts)
- ++eob_branch_count[band][ctx];
- if (!vpx_read(r, prob[EOB_CONTEXT_NODE])) {
- INCREMENT_COUNT(EOB_MODEL_TOKEN);
- break;
- }
-
- while (!vpx_read(r, prob[ZERO_CONTEXT_NODE])) {
- INCREMENT_COUNT(ZERO_TOKEN);
- dqv = dq[1];
- token_cache[scan[c]] = 0;
- ++c;
- if (c >= max_eob)
- return c; // zero tokens at the end (no eob token)
- ctx = get_coef_context(nb, token_cache, c);
- band = *band_translate++;
- prob = coef_probs[band][ctx];
- }
-
- if (!vpx_read(r, prob[ONE_CONTEXT_NODE])) {
- INCREMENT_COUNT(ONE_TOKEN);
- token = ONE_TOKEN;
- val = 1;
- } else {
- INCREMENT_COUNT(TWO_TOKEN);
- token = vpx_read_tree(r, vp10_coef_con_tree,
- vp10_pareto8_full[prob[PIVOT_NODE] - 1]);
- switch (token) {
- case TWO_TOKEN:
- case THREE_TOKEN:
- case FOUR_TOKEN:
- val = token;
- break;
- case CATEGORY1_TOKEN:
- val = CAT1_MIN_VAL + read_coeff(cat1_prob, 1, r);
- break;
- case CATEGORY2_TOKEN:
- val = CAT2_MIN_VAL + read_coeff(cat2_prob, 2, r);
- break;
- case CATEGORY3_TOKEN:
- val = CAT3_MIN_VAL + read_coeff(cat3_prob, 3, r);
- break;
- case CATEGORY4_TOKEN:
- val = CAT4_MIN_VAL + read_coeff(cat4_prob, 4, r);
- break;
- case CATEGORY5_TOKEN:
- val = CAT5_MIN_VAL + read_coeff(cat5_prob, 5, r);
- break;
- case CATEGORY6_TOKEN: {
-#if CONFIG_MISC_FIXES
- const int skip_bits = TX_SIZES - 1 - tx_size;
-#else
- const int skip_bits = 0;
-#endif
- const uint8_t *cat6p = cat6_prob + skip_bits;
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (xd->bd) {
- case VPX_BITS_8:
- val = CAT6_MIN_VAL + read_coeff(cat6p, 14 - skip_bits, r);
- break;
- case VPX_BITS_10:
- val = CAT6_MIN_VAL + read_coeff(cat6p, 16 - skip_bits, r);
- break;
- case VPX_BITS_12:
- val = CAT6_MIN_VAL + read_coeff(cat6p, 18 - skip_bits, r);
- break;
- default:
- assert(0);
- return -1;
- }
-#else
- val = CAT6_MIN_VAL + read_coeff(cat6p, 14 - skip_bits, r);
-#endif
- break;
- }
- }
- }
- v = (val * dqv) >> dq_shift;
-#if CONFIG_COEFFICIENT_RANGE_CHECKING
-#if CONFIG_VP9_HIGHBITDEPTH
- dqcoeff[scan[c]] = highbd_check_range((vpx_read_bit(r) ? -v : v),
- xd->bd);
-#else
- dqcoeff[scan[c]] = check_range(vpx_read_bit(r) ? -v : v);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-#else
- dqcoeff[scan[c]] = vpx_read_bit(r) ? -v : v;
-#endif // CONFIG_COEFFICIENT_RANGE_CHECKING
- token_cache[scan[c]] = vp10_pt_energy_class[token];
- ++c;
- ctx = get_coef_context(nb, token_cache, c);
- dqv = dq[1];
- }
-
- return c;
-}
-
-// TODO(slavarnway): Decode version of vp10_set_context. Modify vp10_set_context
-// after testing is complete, then delete this version.
-static
-void dec_set_contexts(const MACROBLOCKD *xd, struct macroblockd_plane *pd,
- TX_SIZE tx_size, int has_eob,
- int aoff, int loff) {
- ENTROPY_CONTEXT *const a = pd->above_context + aoff;
- ENTROPY_CONTEXT *const l = pd->left_context + loff;
- const int tx_size_in_blocks = 1 << tx_size;
-
- // above
- if (has_eob && xd->mb_to_right_edge < 0) {
- int i;
- const int blocks_wide = pd->n4_w +
- (xd->mb_to_right_edge >> (5 + pd->subsampling_x));
- int above_contexts = tx_size_in_blocks;
- if (above_contexts + aoff > blocks_wide)
- above_contexts = blocks_wide - aoff;
-
- for (i = 0; i < above_contexts; ++i)
- a[i] = has_eob;
- for (i = above_contexts; i < tx_size_in_blocks; ++i)
- a[i] = 0;
- } else {
- memset(a, has_eob, sizeof(ENTROPY_CONTEXT) * tx_size_in_blocks);
- }
-
- // left
- if (has_eob && xd->mb_to_bottom_edge < 0) {
- int i;
- const int blocks_high = pd->n4_h +
- (xd->mb_to_bottom_edge >> (5 + pd->subsampling_y));
- int left_contexts = tx_size_in_blocks;
- if (left_contexts + loff > blocks_high)
- left_contexts = blocks_high - loff;
-
- for (i = 0; i < left_contexts; ++i)
- l[i] = has_eob;
- for (i = left_contexts; i < tx_size_in_blocks; ++i)
- l[i] = 0;
- } else {
- memset(l, has_eob, sizeof(ENTROPY_CONTEXT) * tx_size_in_blocks);
- }
-}
-
-void vp10_decode_palette_tokens(MACROBLOCKD *const xd, int plane,
- vpx_reader *r) {
- MODE_INFO *const mi = xd->mi[0];
- MB_MODE_INFO *const mbmi = &mi->mbmi;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- int rows = 4 * num_4x4_blocks_high_lookup[bsize];
- int cols = 4 * num_4x4_blocks_wide_lookup[bsize];
- int color_idx, color_ctx, color_order[PALETTE_MAX_SIZE];
- int n = mbmi->palette_mode_info.palette_size[plane != 0];
- int i, j;
- uint8_t *color_map = xd->plane[plane].color_index_map;
- const vpx_prob (* prob)[PALETTE_COLOR_CONTEXTS][PALETTE_COLORS - 1] =
- plane ? vp10_default_palette_uv_color_prob :
- vp10_default_palette_y_color_prob;
-
- for (i = 0; i < rows; ++i) {
- for (j = (i == 0 ? 1 : 0); j < cols; ++j) {
- color_ctx = vp10_get_palette_color_context(color_map, cols, i, j, n,
- color_order);
- color_idx = vpx_read_tree(r, vp10_palette_color_tree[n - 2],
- prob[n - 2][color_ctx]);
- assert(color_idx >= 0 && color_idx < n);
- color_map[i * cols + j] = color_order[color_idx];
- }
- }
-}
-
-int vp10_decode_block_tokens(MACROBLOCKD *xd,
- int plane, const scan_order *sc,
- int x, int y,
- TX_SIZE tx_size, vpx_reader *r,
- int seg_id) {
- struct macroblockd_plane *const pd = &xd->plane[plane];
- const int16_t *const dequant = pd->seg_dequant[seg_id];
- const int ctx = get_entropy_context(tx_size, pd->above_context + x,
- pd->left_context + y);
- const int eob = decode_coefs(xd, pd->plane_type,
- pd->dqcoeff, tx_size,
- dequant, ctx, sc->scan, sc->neighbors, r);
- dec_set_contexts(xd, pd, tx_size, eob > 0, x, y);
- return eob;
-}
-
-
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_DECODER_DETOKENIZE_H_
-#define VP10_DECODER_DETOKENIZE_H_
-
-#include "vpx_dsp/bitreader.h"
-#include "vp10/decoder/decoder.h"
-#include "vp10/common/scan.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_decode_palette_tokens(MACROBLOCKD *const xd, int plane,
- vpx_reader *r);
-int vp10_decode_block_tokens(MACROBLOCKD *xd,
- int plane, const scan_order *sc,
- int x, int y,
- TX_SIZE tx_size, vpx_reader *r,
- int seg_id);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_DECODER_DETOKENIZE_H_
+++ /dev/null
-/*
- Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/entropy.h"
-
-#include "vp10/decoder/dsubexp.h"
-
-static int inv_recenter_nonneg(int v, int m) {
- if (v > 2 * m)
- return v;
-
- return (v & 1) ? m - ((v + 1) >> 1) : m + (v >> 1);
-}
-
-static int decode_uniform(vpx_reader *r) {
- const int l = 8;
- const int m = (1 << l) - 191;
- const int v = vpx_read_literal(r, l - 1);
- return v < m ? v : (v << 1) - m + vpx_read_bit(r);
-}
-
-static int inv_remap_prob(int v, int m) {
- static int inv_map_table[MAX_PROB] = {
- 7, 20, 33, 46, 59, 72, 85, 98, 111, 124, 137, 150, 163, 176, 189,
- 202, 215, 228, 241, 254, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11,
- 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27,
- 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43,
- 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 60,
- 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 73, 74, 75, 76,
- 77, 78, 79, 80, 81, 82, 83, 84, 86, 87, 88, 89, 90, 91, 92,
- 93, 94, 95, 96, 97, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108,
- 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 125,
- 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 138, 139, 140, 141,
- 142, 143, 144, 145, 146, 147, 148, 149, 151, 152, 153, 154, 155, 156, 157,
- 158, 159, 160, 161, 162, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173,
- 174, 175, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 190,
- 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 203, 204, 205, 206,
- 207, 208, 209, 210, 211, 212, 213, 214, 216, 217, 218, 219, 220, 221, 222,
- 223, 224, 225, 226, 227, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238,
- 239, 240, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 253
- };
- assert(v < (int)(sizeof(inv_map_table) / sizeof(inv_map_table[0])));
- v = inv_map_table[v];
- m--;
- if ((m << 1) <= MAX_PROB) {
- return 1 + inv_recenter_nonneg(v, m);
- } else {
- return MAX_PROB - inv_recenter_nonneg(v, MAX_PROB - 1 - m);
- }
-}
-
-static int decode_term_subexp(vpx_reader *r) {
- if (!vpx_read_bit(r))
- return vpx_read_literal(r, 4);
- if (!vpx_read_bit(r))
- return vpx_read_literal(r, 4) + 16;
- if (!vpx_read_bit(r))
- return vpx_read_literal(r, 5) + 32;
- return decode_uniform(r) + 64;
-}
-
-void vp10_diff_update_prob(vpx_reader *r, vpx_prob* p) {
- if (vpx_read(r, DIFF_UPDATE_PROB)) {
- const int delp = decode_term_subexp(r);
- *p = (vpx_prob)inv_remap_prob(delp, *p);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_DECODER_DSUBEXP_H_
-#define VP10_DECODER_DSUBEXP_H_
-
-#include "vpx_dsp/bitreader.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_diff_update_prob(vpx_reader *r, vpx_prob* p);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_DECODER_DSUBEXP_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vpx_config.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/decoder/dthread.h"
-#include "vp10/decoder/decoder.h"
-
-// #define DEBUG_THREAD
-
-// TODO(hkuang): Clean up all the #ifdef in this file.
-void vp10_frameworker_lock_stats(VPxWorker *const worker) {
-#if CONFIG_MULTITHREAD
- FrameWorkerData *const worker_data = worker->data1;
- pthread_mutex_lock(&worker_data->stats_mutex);
-#else
- (void)worker;
-#endif
-}
-
-void vp10_frameworker_unlock_stats(VPxWorker *const worker) {
-#if CONFIG_MULTITHREAD
- FrameWorkerData *const worker_data = worker->data1;
- pthread_mutex_unlock(&worker_data->stats_mutex);
-#else
- (void)worker;
-#endif
-}
-
-void vp10_frameworker_signal_stats(VPxWorker *const worker) {
-#if CONFIG_MULTITHREAD
- FrameWorkerData *const worker_data = worker->data1;
-
-// TODO(hkuang): Fix the pthread_cond_broadcast in windows wrapper.
-#if defined(_WIN32) && !HAVE_PTHREAD_H
- pthread_cond_signal(&worker_data->stats_cond);
-#else
- pthread_cond_broadcast(&worker_data->stats_cond);
-#endif
-
-#else
- (void)worker;
-#endif
-}
-
-// This macro prevents thread_sanitizer from reporting known concurrent writes.
-#if defined(__has_feature)
-#if __has_feature(thread_sanitizer)
-#define BUILDING_WITH_TSAN
-#endif
-#endif
-
-// TODO(hkuang): Remove worker parameter as it is only used in debug code.
-void vp10_frameworker_wait(VPxWorker *const worker, RefCntBuffer *const ref_buf,
- int row) {
-#if CONFIG_MULTITHREAD
- if (!ref_buf)
- return;
-
-#ifndef BUILDING_WITH_TSAN
- // The following line of code will get harmless tsan error but it is the key
- // to get best performance.
- if (ref_buf->row >= row && ref_buf->buf.corrupted != 1) return;
-#endif
-
- {
- // Find the worker thread that owns the reference frame. If the reference
- // frame has been fully decoded, it may not have owner.
- VPxWorker *const ref_worker = ref_buf->frame_worker_owner;
- FrameWorkerData *const ref_worker_data =
- (FrameWorkerData *)ref_worker->data1;
- const VP10Decoder *const pbi = ref_worker_data->pbi;
-
-#ifdef DEBUG_THREAD
- {
- FrameWorkerData *const worker_data = (FrameWorkerData *)worker->data1;
- printf("%d %p worker is waiting for %d %p worker (%d) ref %d \r\n",
- worker_data->worker_id, worker, ref_worker_data->worker_id,
- ref_buf->frame_worker_owner, row, ref_buf->row);
- }
-#endif
-
- vp10_frameworker_lock_stats(ref_worker);
- while (ref_buf->row < row && pbi->cur_buf == ref_buf &&
- ref_buf->buf.corrupted != 1) {
- pthread_cond_wait(&ref_worker_data->stats_cond,
- &ref_worker_data->stats_mutex);
- }
-
- if (ref_buf->buf.corrupted == 1) {
- FrameWorkerData *const worker_data = (FrameWorkerData *)worker->data1;
- vp10_frameworker_unlock_stats(ref_worker);
- vpx_internal_error(&worker_data->pbi->common.error,
- VPX_CODEC_CORRUPT_FRAME,
- "Worker %p failed to decode frame", worker);
- }
- vp10_frameworker_unlock_stats(ref_worker);
- }
-#else
- (void)worker;
- (void)ref_buf;
- (void)row;
- (void)ref_buf;
-#endif // CONFIG_MULTITHREAD
-}
-
-void vp10_frameworker_broadcast(RefCntBuffer *const buf, int row) {
-#if CONFIG_MULTITHREAD
- VPxWorker *worker = buf->frame_worker_owner;
-
-#ifdef DEBUG_THREAD
- {
- FrameWorkerData *const worker_data = (FrameWorkerData *)worker->data1;
- printf("%d %p worker decode to (%d) \r\n", worker_data->worker_id,
- buf->frame_worker_owner, row);
- }
-#endif
-
- vp10_frameworker_lock_stats(worker);
- buf->row = row;
- vp10_frameworker_signal_stats(worker);
- vp10_frameworker_unlock_stats(worker);
-#else
- (void)buf;
- (void)row;
-#endif // CONFIG_MULTITHREAD
-}
-
-void vp10_frameworker_copy_context(VPxWorker *const dst_worker,
- VPxWorker *const src_worker) {
-#if CONFIG_MULTITHREAD
- FrameWorkerData *const src_worker_data = (FrameWorkerData *)src_worker->data1;
- FrameWorkerData *const dst_worker_data = (FrameWorkerData *)dst_worker->data1;
- VP10_COMMON *const src_cm = &src_worker_data->pbi->common;
- VP10_COMMON *const dst_cm = &dst_worker_data->pbi->common;
- int i;
-
- // Wait until source frame's context is ready.
- vp10_frameworker_lock_stats(src_worker);
- while (!src_worker_data->frame_context_ready) {
- pthread_cond_wait(&src_worker_data->stats_cond,
- &src_worker_data->stats_mutex);
- }
-
- dst_cm->last_frame_seg_map = src_cm->seg.enabled ?
- src_cm->current_frame_seg_map : src_cm->last_frame_seg_map;
- dst_worker_data->pbi->need_resync = src_worker_data->pbi->need_resync;
- vp10_frameworker_unlock_stats(src_worker);
-
- dst_cm->bit_depth = src_cm->bit_depth;
-#if CONFIG_VP9_HIGHBITDEPTH
- dst_cm->use_highbitdepth = src_cm->use_highbitdepth;
-#endif
- dst_cm->prev_frame = src_cm->show_existing_frame ?
- src_cm->prev_frame : src_cm->cur_frame;
- dst_cm->last_width = !src_cm->show_existing_frame ?
- src_cm->width : src_cm->last_width;
- dst_cm->last_height = !src_cm->show_existing_frame ?
- src_cm->height : src_cm->last_height;
- dst_cm->subsampling_x = src_cm->subsampling_x;
- dst_cm->subsampling_y = src_cm->subsampling_y;
- dst_cm->frame_type = src_cm->frame_type;
- dst_cm->last_show_frame = !src_cm->show_existing_frame ?
- src_cm->show_frame : src_cm->last_show_frame;
- for (i = 0; i < REF_FRAMES; ++i)
- dst_cm->ref_frame_map[i] = src_cm->next_ref_frame_map[i];
-
- memcpy(dst_cm->lf_info.lfthr, src_cm->lf_info.lfthr,
- (MAX_LOOP_FILTER + 1) * sizeof(loop_filter_thresh));
- dst_cm->lf.last_sharpness_level = src_cm->lf.sharpness_level;
- dst_cm->lf.filter_level = src_cm->lf.filter_level;
- memcpy(dst_cm->lf.ref_deltas, src_cm->lf.ref_deltas, MAX_REF_FRAMES);
- memcpy(dst_cm->lf.mode_deltas, src_cm->lf.mode_deltas, MAX_MODE_LF_DELTAS);
- dst_cm->seg = src_cm->seg;
- memcpy(dst_cm->frame_contexts, src_cm->frame_contexts,
- FRAME_CONTEXTS * sizeof(dst_cm->frame_contexts[0]));
-#else
- (void) dst_worker;
- (void) src_worker;
-#endif // CONFIG_MULTITHREAD
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_DECODER_DTHREAD_H_
-#define VP10_DECODER_DTHREAD_H_
-
-#include "./vpx_config.h"
-#include "vpx_util/vpx_thread.h"
-#include "vpx/internal/vpx_codec_internal.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10Common;
-struct VP10Decoder;
-
-// WorkerData for the FrameWorker thread. It contains all the information of
-// the worker and decode structures for decoding a frame.
-typedef struct FrameWorkerData {
- struct VP10Decoder *pbi;
- const uint8_t *data;
- const uint8_t *data_end;
- size_t data_size;
- void *user_priv;
- int result;
- int worker_id;
- int received_frame;
-
- // scratch_buffer is used in frame parallel mode only.
- // It is used to make a copy of the compressed data.
- uint8_t *scratch_buffer;
- size_t scratch_buffer_size;
-
-#if CONFIG_MULTITHREAD
- pthread_mutex_t stats_mutex;
- pthread_cond_t stats_cond;
-#endif
-
- int frame_context_ready; // Current frame's context is ready to read.
- int frame_decoded; // Finished decoding current frame.
-} FrameWorkerData;
-
-void vp10_frameworker_lock_stats(VPxWorker *const worker);
-void vp10_frameworker_unlock_stats(VPxWorker *const worker);
-void vp10_frameworker_signal_stats(VPxWorker *const worker);
-
-// Wait until ref_buf has been decoded to row in real pixel unit.
-// Note: worker may already finish decoding ref_buf and release it in order to
-// start decoding next frame. So need to check whether worker is still decoding
-// ref_buf.
-void vp10_frameworker_wait(VPxWorker *const worker, RefCntBuffer *const ref_buf,
- int row);
-
-// FrameWorker broadcasts its decoding progress so other workers that are
-// waiting on it can resume decoding.
-void vp10_frameworker_broadcast(RefCntBuffer *const buf, int row);
-
-// Copy necessary decoding context from src worker to dst worker.
-void vp10_frameworker_copy_context(VPxWorker *const dst_worker,
- VPxWorker *const src_worker);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_DECODER_DTHREAD_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-#include <math.h>
-
-#include "vp10/encoder/aq_complexity.h"
-#include "vp10/encoder/aq_variance.h"
-#include "vp10/encoder/encodeframe.h"
-#include "vp10/common/seg_common.h"
-#include "vp10/encoder/segmentation.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_ports/system_state.h"
-
-#define AQ_C_SEGMENTS 5
-#define DEFAULT_AQ2_SEG 3 // Neutral Q segment
-#define AQ_C_STRENGTHS 3
-static const double aq_c_q_adj_factor[AQ_C_STRENGTHS][AQ_C_SEGMENTS] =
- { {1.75, 1.25, 1.05, 1.00, 0.90},
- {2.00, 1.50, 1.15, 1.00, 0.85},
- {2.50, 1.75, 1.25, 1.00, 0.80} };
-static const double aq_c_transitions[AQ_C_STRENGTHS][AQ_C_SEGMENTS] =
- { {0.15, 0.30, 0.55, 2.00, 100.0},
- {0.20, 0.40, 0.65, 2.00, 100.0},
- {0.25, 0.50, 0.75, 2.00, 100.0} };
-static const double aq_c_var_thresholds[AQ_C_STRENGTHS][AQ_C_SEGMENTS] =
- { {-4.0, -3.0, -2.0, 100.00, 100.0},
- {-3.5, -2.5, -1.5, 100.00, 100.0},
- {-3.0, -2.0, -1.0, 100.00, 100.0} };
-
-#define DEFAULT_COMPLEXITY 64
-
-
-static int get_aq_c_strength(int q_index, vpx_bit_depth_t bit_depth) {
- // Approximate base quatizer (truncated to int)
- const int base_quant = vp10_ac_quant(q_index, 0, bit_depth) / 4;
- return (base_quant > 10) + (base_quant > 25);
-}
-
-void vp10_setup_in_frame_q_adj(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- struct segmentation *const seg = &cm->seg;
-
- // Make SURE use of floating point in this function is safe.
- vpx_clear_system_state();
-
- if (cm->frame_type == KEY_FRAME ||
- cpi->refresh_alt_ref_frame ||
- (cpi->refresh_golden_frame && !cpi->rc.is_src_frame_alt_ref)) {
- int segment;
- const int aq_strength = get_aq_c_strength(cm->base_qindex, cm->bit_depth);
-
- // Clear down the segment map.
- memset(cpi->segmentation_map, DEFAULT_AQ2_SEG, cm->mi_rows * cm->mi_cols);
-
- vp10_clearall_segfeatures(seg);
-
- // Segmentation only makes sense if the target bits per SB is above a
- // threshold. Below this the overheads will usually outweigh any benefit.
- if (cpi->rc.sb64_target_rate < 256) {
- vp10_disable_segmentation(seg);
- return;
- }
-
- vp10_enable_segmentation(seg);
-
- // Select delta coding method.
- seg->abs_delta = SEGMENT_DELTADATA;
-
- // Default segment "Q" feature is disabled so it defaults to the baseline Q.
- vp10_disable_segfeature(seg, DEFAULT_AQ2_SEG, SEG_LVL_ALT_Q);
-
- // Use some of the segments for in frame Q adjustment.
- for (segment = 0; segment < AQ_C_SEGMENTS; ++segment) {
- int qindex_delta;
-
- if (segment == DEFAULT_AQ2_SEG)
- continue;
-
- qindex_delta =
- vp10_compute_qdelta_by_rate(&cpi->rc, cm->frame_type, cm->base_qindex,
- aq_c_q_adj_factor[aq_strength][segment],
- cm->bit_depth);
-
-
- // For AQ complexity mode, we dont allow Q0 in a segment if the base
- // Q is not 0. Q0 (lossless) implies 4x4 only and in AQ mode 2 a segment
- // Q delta is sometimes applied without going back around the rd loop.
- // This could lead to an illegal combination of partition size and q.
- if ((cm->base_qindex != 0) && ((cm->base_qindex + qindex_delta) == 0)) {
- qindex_delta = -cm->base_qindex + 1;
- }
- if ((cm->base_qindex + qindex_delta) > 0) {
- vp10_enable_segfeature(seg, segment, SEG_LVL_ALT_Q);
- vp10_set_segdata(seg, segment, SEG_LVL_ALT_Q, qindex_delta);
- }
- }
- }
-}
-
-#define DEFAULT_LV_THRESH 10.0
-#define MIN_DEFAULT_LV_THRESH 8.0
-#define VAR_STRENGTH_STEP 0.25
-// Select a segment for the current block.
-// The choice of segment for a block depends on the ratio of the projected
-// bits for the block vs a target average and its spatial complexity.
-void vp10_caq_select_segment(VP10_COMP *cpi, MACROBLOCK *mb, BLOCK_SIZE bs,
- int mi_row, int mi_col, int projected_rate) {
- VP10_COMMON *const cm = &cpi->common;
-
- const int mi_offset = mi_row * cm->mi_cols + mi_col;
- const int bw = num_8x8_blocks_wide_lookup[BLOCK_64X64];
- const int bh = num_8x8_blocks_high_lookup[BLOCK_64X64];
- const int xmis = VPXMIN(cm->mi_cols - mi_col, num_8x8_blocks_wide_lookup[bs]);
- const int ymis = VPXMIN(cm->mi_rows - mi_row, num_8x8_blocks_high_lookup[bs]);
- int x, y;
- int i;
- unsigned char segment;
-
- if (0) {
- segment = DEFAULT_AQ2_SEG;
- } else {
- // Rate depends on fraction of a SB64 in frame (xmis * ymis / bw * bh).
- // It is converted to bits * 256 units.
- const int target_rate = (cpi->rc.sb64_target_rate * xmis * ymis * 256) /
- (bw * bh);
- double logvar;
- double low_var_thresh;
- const int aq_strength = get_aq_c_strength(cm->base_qindex, cm->bit_depth);
-
- vpx_clear_system_state();
- low_var_thresh = (cpi->oxcf.pass == 2)
- ? VPXMAX(cpi->twopass.mb_av_energy, MIN_DEFAULT_LV_THRESH)
- : DEFAULT_LV_THRESH;
-
- vp10_setup_src_planes(mb, cpi->Source, mi_row, mi_col);
- logvar = vp10_log_block_var(cpi, mb, bs);
-
- segment = AQ_C_SEGMENTS - 1; // Just in case no break out below.
- for (i = 0; i < AQ_C_SEGMENTS; ++i) {
- // Test rate against a threshold value and variance against a threshold.
- // Increasing segment number (higher variance and complexity) = higher Q.
- if ((projected_rate <
- target_rate * aq_c_transitions[aq_strength][i]) &&
- (logvar < (low_var_thresh + aq_c_var_thresholds[aq_strength][i]))) {
- segment = i;
- break;
- }
- }
- }
-
- // Fill in the entires in the segment map corresponding to this SB64.
- for (y = 0; y < ymis; y++) {
- for (x = 0; x < xmis; x++) {
- cpi->segmentation_map[mi_offset + y * cm->mi_cols + x] = segment;
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_AQ_COMPLEXITY_H_
-#define VP10_ENCODER_AQ_COMPLEXITY_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#include "vp10/common/enums.h"
-
-struct VP10_COMP;
-struct macroblock;
-
-// Select a segment for the current Block.
-void vp10_caq_select_segment(struct VP10_COMP *cpi, struct macroblock *,
- BLOCK_SIZE bs,
- int mi_row, int mi_col, int projected_rate);
-
-// This function sets up a set of segments with delta Q values around
-// the baseline frame quantizer.
-void vp10_setup_in_frame_q_adj(struct VP10_COMP *cpi);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_AQ_COMPLEXITY_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-#include <math.h>
-
-#include "vp10/common/seg_common.h"
-#include "vp10/encoder/aq_cyclicrefresh.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/segmentation.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_ports/system_state.h"
-
-struct CYCLIC_REFRESH {
- // Percentage of blocks per frame that are targeted as candidates
- // for cyclic refresh.
- int percent_refresh;
- // Maximum q-delta as percentage of base q.
- int max_qdelta_perc;
- // Superblock starting index for cycling through the frame.
- int sb_index;
- // Controls how long block will need to wait to be refreshed again, in
- // excess of the cycle time, i.e., in the case of all zero motion, block
- // will be refreshed every (100/percent_refresh + time_for_refresh) frames.
- int time_for_refresh;
- // Target number of (8x8) blocks that are set for delta-q.
- int target_num_seg_blocks;
- // Actual number of (8x8) blocks that were applied delta-q.
- int actual_num_seg1_blocks;
- int actual_num_seg2_blocks;
- // RD mult. parameters for segment 1.
- int rdmult;
- // Cyclic refresh map.
- signed char *map;
- // Map of the last q a block was coded at.
- uint8_t *last_coded_q_map;
- // Thresholds applied to the projected rate/distortion of the coding block,
- // when deciding whether block should be refreshed.
- int64_t thresh_rate_sb;
- int64_t thresh_dist_sb;
- // Threshold applied to the motion vector (in units of 1/8 pel) of the
- // coding block, when deciding whether block should be refreshed.
- int16_t motion_thresh;
- // Rate target ratio to set q delta.
- double rate_ratio_qdelta;
- // Boost factor for rate target ratio, for segment CR_SEGMENT_ID_BOOST2.
- int rate_boost_fac;
- double low_content_avg;
- int qindex_delta[3];
-};
-
-CYCLIC_REFRESH *vp10_cyclic_refresh_alloc(int mi_rows, int mi_cols) {
- size_t last_coded_q_map_size;
- CYCLIC_REFRESH *const cr = vpx_calloc(1, sizeof(*cr));
- if (cr == NULL)
- return NULL;
-
- cr->map = vpx_calloc(mi_rows * mi_cols, sizeof(*cr->map));
- if (cr->map == NULL) {
- vpx_free(cr);
- return NULL;
- }
- last_coded_q_map_size = mi_rows * mi_cols * sizeof(*cr->last_coded_q_map);
- cr->last_coded_q_map = vpx_malloc(last_coded_q_map_size);
- if (cr->last_coded_q_map == NULL) {
- vpx_free(cr);
- return NULL;
- }
- assert(MAXQ <= 255);
- memset(cr->last_coded_q_map, MAXQ, last_coded_q_map_size);
-
- return cr;
-}
-
-void vp10_cyclic_refresh_free(CYCLIC_REFRESH *cr) {
- vpx_free(cr->map);
- vpx_free(cr->last_coded_q_map);
- vpx_free(cr);
-}
-
-// Check if we should turn off cyclic refresh based on bitrate condition.
-static int apply_cyclic_refresh_bitrate(const VP10_COMMON *cm,
- const RATE_CONTROL *rc) {
- // Turn off cyclic refresh if bits available per frame is not sufficiently
- // larger than bit cost of segmentation. Segment map bit cost should scale
- // with number of seg blocks, so compare available bits to number of blocks.
- // Average bits available per frame = avg_frame_bandwidth
- // Number of (8x8) blocks in frame = mi_rows * mi_cols;
- const float factor = 0.25;
- const int number_blocks = cm->mi_rows * cm->mi_cols;
- // The condition below corresponds to turning off at target bitrates:
- // (at 30fps), ~12kbps for CIF, 36kbps for VGA, 100kps for HD/720p.
- // Also turn off at very small frame sizes, to avoid too large fraction of
- // superblocks to be refreshed per frame. Threshold below is less than QCIF.
- if (rc->avg_frame_bandwidth < factor * number_blocks ||
- number_blocks / 64 < 5)
- return 0;
- else
- return 1;
-}
-
-// Check if this coding block, of size bsize, should be considered for refresh
-// (lower-qp coding). Decision can be based on various factors, such as
-// size of the coding block (i.e., below min_block size rejected), coding
-// mode, and rate/distortion.
-static int candidate_refresh_aq(const CYCLIC_REFRESH *cr,
- const MB_MODE_INFO *mbmi,
- int64_t rate,
- int64_t dist,
- int bsize) {
- MV mv = mbmi->mv[0].as_mv;
- // Reject the block for lower-qp coding if projected distortion
- // is above the threshold, and any of the following is true:
- // 1) mode uses large mv
- // 2) mode is an intra-mode
- // Otherwise accept for refresh.
- if (dist > cr->thresh_dist_sb &&
- (mv.row > cr->motion_thresh || mv.row < -cr->motion_thresh ||
- mv.col > cr->motion_thresh || mv.col < -cr->motion_thresh ||
- !is_inter_block(mbmi)))
- return CR_SEGMENT_ID_BASE;
- else if (bsize >= BLOCK_16X16 &&
- rate < cr->thresh_rate_sb &&
- is_inter_block(mbmi) &&
- mbmi->mv[0].as_int == 0 &&
- cr->rate_boost_fac > 10)
- // More aggressive delta-q for bigger blocks with zero motion.
- return CR_SEGMENT_ID_BOOST2;
- else
- return CR_SEGMENT_ID_BOOST1;
-}
-
-// Compute delta-q for the segment.
-static int compute_deltaq(const VP10_COMP *cpi, int q, double rate_factor) {
- const CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- const RATE_CONTROL *const rc = &cpi->rc;
- int deltaq = vp10_compute_qdelta_by_rate(rc, cpi->common.frame_type,
- q, rate_factor,
- cpi->common.bit_depth);
- if ((-deltaq) > cr->max_qdelta_perc * q / 100) {
- deltaq = -cr->max_qdelta_perc * q / 100;
- }
- return deltaq;
-}
-
-// For the just encoded frame, estimate the bits, incorporating the delta-q
-// from non-base segment. For now ignore effect of multiple segments
-// (with different delta-q). Note this function is called in the postencode
-// (called from rc_update_rate_correction_factors()).
-int vp10_cyclic_refresh_estimate_bits_at_q(const VP10_COMP *cpi,
- double correction_factor) {
- const VP10_COMMON *const cm = &cpi->common;
- const CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- int estimated_bits;
- int mbs = cm->MBs;
- int num8x8bl = mbs << 2;
- // Weight for non-base segments: use actual number of blocks refreshed in
- // previous/just encoded frame. Note number of blocks here is in 8x8 units.
- double weight_segment1 = (double)cr->actual_num_seg1_blocks / num8x8bl;
- double weight_segment2 = (double)cr->actual_num_seg2_blocks / num8x8bl;
- // Take segment weighted average for estimated bits.
- estimated_bits = (int)((1.0 - weight_segment1 - weight_segment2) *
- vp10_estimate_bits_at_q(cm->frame_type, cm->base_qindex, mbs,
- correction_factor, cm->bit_depth) +
- weight_segment1 *
- vp10_estimate_bits_at_q(cm->frame_type,
- cm->base_qindex + cr->qindex_delta[1], mbs,
- correction_factor, cm->bit_depth) +
- weight_segment2 *
- vp10_estimate_bits_at_q(cm->frame_type,
- cm->base_qindex + cr->qindex_delta[2], mbs,
- correction_factor, cm->bit_depth));
- return estimated_bits;
-}
-
-// Prior to encoding the frame, estimate the bits per mb, for a given q = i and
-// a corresponding delta-q (for segment 1). This function is called in the
-// rc_regulate_q() to set the base qp index.
-// Note: the segment map is set to either 0/CR_SEGMENT_ID_BASE (no refresh) or
-// to 1/CR_SEGMENT_ID_BOOST1 (refresh) for each superblock, prior to encoding.
-int vp10_cyclic_refresh_rc_bits_per_mb(const VP10_COMP *cpi, int i,
- double correction_factor) {
- const VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- int bits_per_mb;
- int num8x8bl = cm->MBs << 2;
- // Weight for segment prior to encoding: take the average of the target
- // number for the frame to be encoded and the actual from the previous frame.
- double weight_segment = (double)((cr->target_num_seg_blocks +
- cr->actual_num_seg1_blocks + cr->actual_num_seg2_blocks) >> 1) /
- num8x8bl;
- // Compute delta-q corresponding to qindex i.
- int deltaq = compute_deltaq(cpi, i, cr->rate_ratio_qdelta);
- // Take segment weighted average for bits per mb.
- bits_per_mb = (int)((1.0 - weight_segment) *
- vp10_rc_bits_per_mb(cm->frame_type, i, correction_factor, cm->bit_depth) +
- weight_segment *
- vp10_rc_bits_per_mb(cm->frame_type, i + deltaq, correction_factor,
- cm->bit_depth));
- return bits_per_mb;
-}
-
-// Prior to coding a given prediction block, of size bsize at (mi_row, mi_col),
-// check if we should reset the segment_id, and update the cyclic_refresh map
-// and segmentation map.
-void vp10_cyclic_refresh_update_segment(VP10_COMP *const cpi,
- MB_MODE_INFO *const mbmi,
- int mi_row, int mi_col,
- BLOCK_SIZE bsize,
- int64_t rate,
- int64_t dist,
- int skip) {
- const VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- const int bw = num_8x8_blocks_wide_lookup[bsize];
- const int bh = num_8x8_blocks_high_lookup[bsize];
- const int xmis = VPXMIN(cm->mi_cols - mi_col, bw);
- const int ymis = VPXMIN(cm->mi_rows - mi_row, bh);
- const int block_index = mi_row * cm->mi_cols + mi_col;
- const int refresh_this_block = candidate_refresh_aq(cr, mbmi, rate, dist,
- bsize);
- // Default is to not update the refresh map.
- int new_map_value = cr->map[block_index];
- int x = 0; int y = 0;
-
- // If this block is labeled for refresh, check if we should reset the
- // segment_id.
- if (cyclic_refresh_segment_id_boosted(mbmi->segment_id)) {
- mbmi->segment_id = refresh_this_block;
- // Reset segment_id if will be skipped.
- if (skip)
- mbmi->segment_id = CR_SEGMENT_ID_BASE;
- }
-
- // Update the cyclic refresh map, to be used for setting segmentation map
- // for the next frame. If the block will be refreshed this frame, mark it
- // as clean. The magnitude of the -ve influences how long before we consider
- // it for refresh again.
- if (cyclic_refresh_segment_id_boosted(mbmi->segment_id)) {
- new_map_value = -cr->time_for_refresh;
- } else if (refresh_this_block) {
- // Else if it is accepted as candidate for refresh, and has not already
- // been refreshed (marked as 1) then mark it as a candidate for cleanup
- // for future time (marked as 0), otherwise don't update it.
- if (cr->map[block_index] == 1)
- new_map_value = 0;
- } else {
- // Leave it marked as block that is not candidate for refresh.
- new_map_value = 1;
- }
-
- // Update entries in the cyclic refresh map with new_map_value, and
- // copy mbmi->segment_id into global segmentation map.
- for (y = 0; y < ymis; y++)
- for (x = 0; x < xmis; x++) {
- int map_offset = block_index + y * cm->mi_cols + x;
- cr->map[map_offset] = new_map_value;
- cpi->segmentation_map[map_offset] = mbmi->segment_id;
- // Inter skip blocks were clearly not coded at the current qindex, so
- // don't update the map for them. For cases where motion is non-zero or
- // the reference frame isn't the previous frame, the previous value in
- // the map for this spatial location is not entirely correct.
- if (!is_inter_block(mbmi) || !skip)
- cr->last_coded_q_map[map_offset] = clamp(
- cm->base_qindex + cr->qindex_delta[mbmi->segment_id], 0, MAXQ);
- }
-}
-
-// Update the actual number of blocks that were applied the segment delta q.
-void vp10_cyclic_refresh_postencode(VP10_COMP *const cpi) {
- VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- unsigned char *const seg_map = cpi->segmentation_map;
- int mi_row, mi_col;
- cr->actual_num_seg1_blocks = 0;
- cr->actual_num_seg2_blocks = 0;
- for (mi_row = 0; mi_row < cm->mi_rows; mi_row++)
- for (mi_col = 0; mi_col < cm->mi_cols; mi_col++) {
- if (cyclic_refresh_segment_id(
- seg_map[mi_row * cm->mi_cols + mi_col]) == CR_SEGMENT_ID_BOOST1)
- cr->actual_num_seg1_blocks++;
- else if (cyclic_refresh_segment_id(
- seg_map[mi_row * cm->mi_cols + mi_col]) == CR_SEGMENT_ID_BOOST2)
- cr->actual_num_seg2_blocks++;
- }
-}
-
-// Set golden frame update interval, for 1 pass CBR mode.
-void vp10_cyclic_refresh_set_golden_update(VP10_COMP *const cpi) {
- RATE_CONTROL *const rc = &cpi->rc;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- // Set minimum gf_interval for GF update to a multiple (== 2) of refresh
- // period. Depending on past encoding stats, GF flag may be reset and update
- // may not occur until next baseline_gf_interval.
- if (cr->percent_refresh > 0)
- rc->baseline_gf_interval = 4 * (100 / cr->percent_refresh);
- else
- rc->baseline_gf_interval = 40;
-}
-
-// Update some encoding stats (from the just encoded frame). If this frame's
-// background has high motion, refresh the golden frame. Otherwise, if the
-// golden reference is to be updated check if we should NOT update the golden
-// ref.
-void vp10_cyclic_refresh_check_golden_update(VP10_COMP *const cpi) {
- VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- int mi_row, mi_col;
- double fraction_low = 0.0;
- int low_content_frame = 0;
-
- MODE_INFO **mi = cm->mi_grid_visible;
- RATE_CONTROL *const rc = &cpi->rc;
- const int rows = cm->mi_rows, cols = cm->mi_cols;
- int cnt1 = 0, cnt2 = 0;
- int force_gf_refresh = 0;
-
- for (mi_row = 0; mi_row < rows; mi_row++) {
- for (mi_col = 0; mi_col < cols; mi_col++) {
- int16_t abs_mvr = mi[0]->mbmi.mv[0].as_mv.row >= 0 ?
- mi[0]->mbmi.mv[0].as_mv.row : -1 * mi[0]->mbmi.mv[0].as_mv.row;
- int16_t abs_mvc = mi[0]->mbmi.mv[0].as_mv.col >= 0 ?
- mi[0]->mbmi.mv[0].as_mv.col : -1 * mi[0]->mbmi.mv[0].as_mv.col;
-
- // Calculate the motion of the background.
- if (abs_mvr <= 16 && abs_mvc <= 16) {
- cnt1++;
- if (abs_mvr == 0 && abs_mvc == 0)
- cnt2++;
- }
- mi++;
-
- // Accumulate low_content_frame.
- if (cr->map[mi_row * cols + mi_col] < 1)
- low_content_frame++;
- }
- mi += 8;
- }
-
- // For video conference clips, if the background has high motion in current
- // frame because of the camera movement, set this frame as the golden frame.
- // Use 70% and 5% as the thresholds for golden frame refreshing.
- // Also, force this frame as a golden update frame if this frame will change
- // the resolution (resize_pending != 0).
- if (cpi->resize_pending != 0 ||
- (cnt1 * 10 > (70 * rows * cols) && cnt2 * 20 < cnt1)) {
- vp10_cyclic_refresh_set_golden_update(cpi);
- rc->frames_till_gf_update_due = rc->baseline_gf_interval;
-
- if (rc->frames_till_gf_update_due > rc->frames_to_key)
- rc->frames_till_gf_update_due = rc->frames_to_key;
- cpi->refresh_golden_frame = 1;
- force_gf_refresh = 1;
- }
-
- fraction_low =
- (double)low_content_frame / (rows * cols);
- // Update average.
- cr->low_content_avg = (fraction_low + 3 * cr->low_content_avg) / 4;
- if (!force_gf_refresh && cpi->refresh_golden_frame == 1) {
- // Don't update golden reference if the amount of low_content for the
- // current encoded frame is small, or if the recursive average of the
- // low_content over the update interval window falls below threshold.
- if (fraction_low < 0.8 || cr->low_content_avg < 0.7)
- cpi->refresh_golden_frame = 0;
- // Reset for next internal.
- cr->low_content_avg = fraction_low;
- }
-}
-
-// Update the segmentation map, and related quantities: cyclic refresh map,
-// refresh sb_index, and target number of blocks to be refreshed.
-// The map is set to either 0/CR_SEGMENT_ID_BASE (no refresh) or to
-// 1/CR_SEGMENT_ID_BOOST1 (refresh) for each superblock.
-// Blocks labeled as BOOST1 may later get set to BOOST2 (during the
-// encoding of the superblock).
-static void cyclic_refresh_update_map(VP10_COMP *const cpi) {
- VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- unsigned char *const seg_map = cpi->segmentation_map;
- int i, block_count, bl_index, sb_rows, sb_cols, sbs_in_frame;
- int xmis, ymis, x, y;
- memset(seg_map, CR_SEGMENT_ID_BASE, cm->mi_rows * cm->mi_cols);
- sb_cols = (cm->mi_cols + MI_BLOCK_SIZE - 1) / MI_BLOCK_SIZE;
- sb_rows = (cm->mi_rows + MI_BLOCK_SIZE - 1) / MI_BLOCK_SIZE;
- sbs_in_frame = sb_cols * sb_rows;
- // Number of target blocks to get the q delta (segment 1).
- block_count = cr->percent_refresh * cm->mi_rows * cm->mi_cols / 100;
- // Set the segmentation map: cycle through the superblocks, starting at
- // cr->mb_index, and stopping when either block_count blocks have been found
- // to be refreshed, or we have passed through whole frame.
- assert(cr->sb_index < sbs_in_frame);
- i = cr->sb_index;
- cr->target_num_seg_blocks = 0;
- do {
- int sum_map = 0;
- // Get the mi_row/mi_col corresponding to superblock index i.
- int sb_row_index = (i / sb_cols);
- int sb_col_index = i - sb_row_index * sb_cols;
- int mi_row = sb_row_index * MI_BLOCK_SIZE;
- int mi_col = sb_col_index * MI_BLOCK_SIZE;
- int qindex_thresh =
- cpi->oxcf.content == VP9E_CONTENT_SCREEN
- ? vp10_get_qindex(&cm->seg, CR_SEGMENT_ID_BOOST2, cm->base_qindex)
- : 0;
- assert(mi_row >= 0 && mi_row < cm->mi_rows);
- assert(mi_col >= 0 && mi_col < cm->mi_cols);
- bl_index = mi_row * cm->mi_cols + mi_col;
- // Loop through all 8x8 blocks in superblock and update map.
- xmis =
- VPXMIN(cm->mi_cols - mi_col, num_8x8_blocks_wide_lookup[BLOCK_64X64]);
- ymis =
- VPXMIN(cm->mi_rows - mi_row, num_8x8_blocks_high_lookup[BLOCK_64X64]);
- for (y = 0; y < ymis; y++) {
- for (x = 0; x < xmis; x++) {
- const int bl_index2 = bl_index + y * cm->mi_cols + x;
- // If the block is as a candidate for clean up then mark it
- // for possible boost/refresh (segment 1). The segment id may get
- // reset to 0 later if block gets coded anything other than ZEROMV.
- if (cr->map[bl_index2] == 0) {
- if (cr->last_coded_q_map[bl_index2] > qindex_thresh)
- sum_map++;
- } else if (cr->map[bl_index2] < 0) {
- cr->map[bl_index2]++;
- }
- }
- }
- // Enforce constant segment over superblock.
- // If segment is at least half of superblock, set to 1.
- if (sum_map >= xmis * ymis / 2) {
- for (y = 0; y < ymis; y++)
- for (x = 0; x < xmis; x++) {
- seg_map[bl_index + y * cm->mi_cols + x] = CR_SEGMENT_ID_BOOST1;
- }
- cr->target_num_seg_blocks += xmis * ymis;
- }
- i++;
- if (i == sbs_in_frame) {
- i = 0;
- }
- } while (cr->target_num_seg_blocks < block_count && i != cr->sb_index);
- cr->sb_index = i;
-}
-
-// Set cyclic refresh parameters.
-void vp10_cyclic_refresh_update_parameters(VP10_COMP *const cpi) {
- const RATE_CONTROL *const rc = &cpi->rc;
- const VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- cr->percent_refresh = 10;
- cr->max_qdelta_perc = 50;
- cr->time_for_refresh = 0;
- // Use larger delta-qp (increase rate_ratio_qdelta) for first few (~4)
- // periods of the refresh cycle, after a key frame.
- if (rc->frames_since_key < 4 * cr->percent_refresh)
- cr->rate_ratio_qdelta = 3.0;
- else
- cr->rate_ratio_qdelta = 2.0;
- // Adjust some parameters for low resolutions at low bitrates.
- if (cm->width <= 352 &&
- cm->height <= 288 &&
- rc->avg_frame_bandwidth < 3400) {
- cr->motion_thresh = 4;
- cr->rate_boost_fac = 10;
- } else {
- cr->motion_thresh = 32;
- cr->rate_boost_fac = 17;
- }
-}
-
-// Setup cyclic background refresh: set delta q and segmentation map.
-void vp10_cyclic_refresh_setup(VP10_COMP *const cpi) {
- VP10_COMMON *const cm = &cpi->common;
- const RATE_CONTROL *const rc = &cpi->rc;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- struct segmentation *const seg = &cm->seg;
- const int apply_cyclic_refresh = apply_cyclic_refresh_bitrate(cm, rc);
- if (cm->current_video_frame == 0)
- cr->low_content_avg = 0.0;
- // Don't apply refresh on key frame or enhancement layer frames.
- if (!apply_cyclic_refresh || cm->frame_type == KEY_FRAME) {
- // Set segmentation map to 0 and disable.
- unsigned char *const seg_map = cpi->segmentation_map;
- memset(seg_map, 0, cm->mi_rows * cm->mi_cols);
- vp10_disable_segmentation(&cm->seg);
- if (cm->frame_type == KEY_FRAME) {
- memset(cr->last_coded_q_map, MAXQ,
- cm->mi_rows * cm->mi_cols * sizeof(*cr->last_coded_q_map));
- cr->sb_index = 0;
- }
- return;
- } else {
- int qindex_delta = 0;
- int qindex2;
- const double q = vp10_convert_qindex_to_q(cm->base_qindex, cm->bit_depth);
- vpx_clear_system_state();
- // Set rate threshold to some multiple (set to 2 for now) of the target
- // rate (target is given by sb64_target_rate and scaled by 256).
- cr->thresh_rate_sb = ((int64_t)(rc->sb64_target_rate) << 8) << 2;
- // Distortion threshold, quadratic in Q, scale factor to be adjusted.
- // q will not exceed 457, so (q * q) is within 32bit; see:
- // vp10_convert_qindex_to_q(), vp10_ac_quant(), ac_qlookup*[].
- cr->thresh_dist_sb = ((int64_t)(q * q)) << 2;
-
- // Set up segmentation.
- // Clear down the segment map.
- vp10_enable_segmentation(&cm->seg);
- vp10_clearall_segfeatures(seg);
- // Select delta coding method.
- seg->abs_delta = SEGMENT_DELTADATA;
-
- // Note: setting temporal_update has no effect, as the seg-map coding method
- // (temporal or spatial) is determined in vp10_choose_segmap_coding_method(),
- // based on the coding cost of each method. For error_resilient mode on the
- // last_frame_seg_map is set to 0, so if temporal coding is used, it is
- // relative to 0 previous map.
- // seg->temporal_update = 0;
-
- // Segment BASE "Q" feature is disabled so it defaults to the baseline Q.
- vp10_disable_segfeature(seg, CR_SEGMENT_ID_BASE, SEG_LVL_ALT_Q);
- // Use segment BOOST1 for in-frame Q adjustment.
- vp10_enable_segfeature(seg, CR_SEGMENT_ID_BOOST1, SEG_LVL_ALT_Q);
- // Use segment BOOST2 for more aggressive in-frame Q adjustment.
- vp10_enable_segfeature(seg, CR_SEGMENT_ID_BOOST2, SEG_LVL_ALT_Q);
-
- // Set the q delta for segment BOOST1.
- qindex_delta = compute_deltaq(cpi, cm->base_qindex, cr->rate_ratio_qdelta);
- cr->qindex_delta[1] = qindex_delta;
-
- // Compute rd-mult for segment BOOST1.
- qindex2 = clamp(cm->base_qindex + cm->y_dc_delta_q + qindex_delta, 0, MAXQ);
-
- cr->rdmult = vp10_compute_rd_mult(cpi, qindex2);
-
- vp10_set_segdata(seg, CR_SEGMENT_ID_BOOST1, SEG_LVL_ALT_Q, qindex_delta);
-
- // Set a more aggressive (higher) q delta for segment BOOST2.
- qindex_delta = compute_deltaq(
- cpi, cm->base_qindex,
- VPXMIN(CR_MAX_RATE_TARGET_RATIO,
- 0.1 * cr->rate_boost_fac * cr->rate_ratio_qdelta));
- cr->qindex_delta[2] = qindex_delta;
- vp10_set_segdata(seg, CR_SEGMENT_ID_BOOST2, SEG_LVL_ALT_Q, qindex_delta);
-
- // Update the segmentation and refresh map.
- cyclic_refresh_update_map(cpi);
- }
-}
-
-int vp10_cyclic_refresh_get_rdmult(const CYCLIC_REFRESH *cr) {
- return cr->rdmult;
-}
-
-void vp10_cyclic_refresh_reset_resize(VP10_COMP *const cpi) {
- const VP10_COMMON *const cm = &cpi->common;
- CYCLIC_REFRESH *const cr = cpi->cyclic_refresh;
- memset(cr->map, 0, cm->mi_rows * cm->mi_cols);
- cr->sb_index = 0;
- cpi->refresh_golden_frame = 1;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_AQ_CYCLICREFRESH_H_
-#define VP10_ENCODER_AQ_CYCLICREFRESH_H_
-
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// The segment ids used in cyclic refresh: from base (no boost) to increasing
-// boost (higher delta-qp).
-#define CR_SEGMENT_ID_BASE 0
-#define CR_SEGMENT_ID_BOOST1 1
-#define CR_SEGMENT_ID_BOOST2 2
-
-// Maximum rate target ratio for setting segment delta-qp.
-#define CR_MAX_RATE_TARGET_RATIO 4.0
-
-struct VP10_COMP;
-
-struct CYCLIC_REFRESH;
-typedef struct CYCLIC_REFRESH CYCLIC_REFRESH;
-
-CYCLIC_REFRESH *vp10_cyclic_refresh_alloc(int mi_rows, int mi_cols);
-
-void vp10_cyclic_refresh_free(CYCLIC_REFRESH *cr);
-
-// Estimate the bits, incorporating the delta-q from segment 1, after encoding
-// the frame.
-int vp10_cyclic_refresh_estimate_bits_at_q(const struct VP10_COMP *cpi,
- double correction_factor);
-
-// Estimate the bits per mb, for a given q = i and a corresponding delta-q
-// (for segment 1), prior to encoding the frame.
-int vp10_cyclic_refresh_rc_bits_per_mb(const struct VP10_COMP *cpi, int i,
- double correction_factor);
-
-// Prior to coding a given prediction block, of size bsize at (mi_row, mi_col),
-// check if we should reset the segment_id, and update the cyclic_refresh map
-// and segmentation map.
-void vp10_cyclic_refresh_update_segment(struct VP10_COMP *const cpi,
- MB_MODE_INFO *const mbmi,
- int mi_row, int mi_col, BLOCK_SIZE bsize,
- int64_t rate, int64_t dist, int skip);
-
-// Update the segmentation map, and related quantities: cyclic refresh map,
-// refresh sb_index, and target number of blocks to be refreshed.
-void vp10_cyclic_refresh_update__map(struct VP10_COMP *const cpi);
-
-// Update the actual number of blocks that were applied the segment delta q.
-void vp10_cyclic_refresh_postencode(struct VP10_COMP *const cpi);
-
-// Set golden frame update interval, for 1 pass CBR mode.
-void vp10_cyclic_refresh_set_golden_update(struct VP10_COMP *const cpi);
-
-// Check if we should not update golden reference, based on past refresh stats.
-void vp10_cyclic_refresh_check_golden_update(struct VP10_COMP *const cpi);
-
-// Set/update global/frame level refresh parameters.
-void vp10_cyclic_refresh_update_parameters(struct VP10_COMP *const cpi);
-
-// Setup cyclic background refresh: set delta q and segmentation map.
-void vp10_cyclic_refresh_setup(struct VP10_COMP *const cpi);
-
-int vp10_cyclic_refresh_get_rdmult(const CYCLIC_REFRESH *cr);
-
-void vp10_cyclic_refresh_reset_resize(struct VP10_COMP *const cpi);
-
-static INLINE int cyclic_refresh_segment_id_boosted(int segment_id) {
- return segment_id == CR_SEGMENT_ID_BOOST1 ||
- segment_id == CR_SEGMENT_ID_BOOST2;
-}
-
-static INLINE int cyclic_refresh_segment_id(int segment_id) {
- if (segment_id == CR_SEGMENT_ID_BOOST1)
- return CR_SEGMENT_ID_BOOST1;
- else if (segment_id == CR_SEGMENT_ID_BOOST2)
- return CR_SEGMENT_ID_BOOST2;
- else
- return CR_SEGMENT_ID_BASE;
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_AQ_CYCLICREFRESH_H_
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-
-#include "vpx_ports/mem.h"
-
-#include "vp10/encoder/aq_variance.h"
-
-#include "vp10/common/seg_common.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/segmentation.h"
-#include "vpx_ports/system_state.h"
-
-#define ENERGY_MIN (-4)
-#define ENERGY_MAX (1)
-#define ENERGY_SPAN (ENERGY_MAX - ENERGY_MIN + 1)
-#define ENERGY_IN_BOUNDS(energy)\
- assert((energy) >= ENERGY_MIN && (energy) <= ENERGY_MAX)
-
-static const double rate_ratio[MAX_SEGMENTS] =
- {2.5, 2.0, 1.5, 1.0, 0.75, 1.0, 1.0, 1.0};
-static const int segment_id[ENERGY_SPAN] = {0, 1, 1, 2, 3, 4};
-
-#define SEGMENT_ID(i) segment_id[(i) - ENERGY_MIN]
-
-DECLARE_ALIGNED(16, static const uint8_t, vp10_64_zeros[64]) = {0};
-#if CONFIG_VP9_HIGHBITDEPTH
-DECLARE_ALIGNED(16, static const uint16_t, vp10_highbd_64_zeros[64]) = {0};
-#endif
-
-unsigned int vp10_vaq_segment_id(int energy) {
- ENERGY_IN_BOUNDS(energy);
- return SEGMENT_ID(energy);
-}
-
-void vp10_vaq_frame_setup(VP10_COMP *cpi) {
- VP10_COMMON *cm = &cpi->common;
- struct segmentation *seg = &cm->seg;
- int i;
-
- if (cm->frame_type == KEY_FRAME ||
- cpi->refresh_alt_ref_frame ||
- (cpi->refresh_golden_frame && !cpi->rc.is_src_frame_alt_ref)) {
- vp10_enable_segmentation(seg);
- vp10_clearall_segfeatures(seg);
-
- seg->abs_delta = SEGMENT_DELTADATA;
-
- vpx_clear_system_state();
-
- for (i = 0; i < MAX_SEGMENTS; ++i) {
- int qindex_delta =
- vp10_compute_qdelta_by_rate(&cpi->rc, cm->frame_type, cm->base_qindex,
- rate_ratio[i], cm->bit_depth);
-
- // We don't allow qindex 0 in a segment if the base value is not 0.
- // Q index 0 (lossless) implies 4x4 encoding only and in AQ mode a segment
- // Q delta is sometimes applied without going back around the rd loop.
- // This could lead to an illegal combination of partition size and q.
- if ((cm->base_qindex != 0) && ((cm->base_qindex + qindex_delta) == 0)) {
- qindex_delta = -cm->base_qindex + 1;
- }
-
- // No need to enable SEG_LVL_ALT_Q for this segment.
- if (rate_ratio[i] == 1.0) {
- continue;
- }
-
- vp10_set_segdata(seg, i, SEG_LVL_ALT_Q, qindex_delta);
- vp10_enable_segfeature(seg, i, SEG_LVL_ALT_Q);
- }
- }
-}
-
-/* TODO(agrange, paulwilkins): The block_variance calls the unoptimized versions
- * of variance() and highbd_8_variance(). It should not.
- */
-static void aq_variance(const uint8_t *a, int a_stride,
- const uint8_t *b, int b_stride,
- int w, int h, unsigned int *sse, int *sum) {
- int i, j;
-
- *sum = 0;
- *sse = 0;
-
- for (i = 0; i < h; i++) {
- for (j = 0; j < w; j++) {
- const int diff = a[j] - b[j];
- *sum += diff;
- *sse += diff * diff;
- }
-
- a += a_stride;
- b += b_stride;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void aq_highbd_variance64(const uint8_t *a8, int a_stride,
- const uint8_t *b8, int b_stride,
- int w, int h, uint64_t *sse, uint64_t *sum) {
- int i, j;
-
- uint16_t *a = CONVERT_TO_SHORTPTR(a8);
- uint16_t *b = CONVERT_TO_SHORTPTR(b8);
- *sum = 0;
- *sse = 0;
-
- for (i = 0; i < h; i++) {
- for (j = 0; j < w; j++) {
- const int diff = a[j] - b[j];
- *sum += diff;
- *sse += diff * diff;
- }
- a += a_stride;
- b += b_stride;
- }
-}
-
-static void aq_highbd_8_variance(const uint8_t *a8, int a_stride,
- const uint8_t *b8, int b_stride,
- int w, int h, unsigned int *sse, int *sum) {
- uint64_t sse_long = 0;
- uint64_t sum_long = 0;
- aq_highbd_variance64(a8, a_stride, b8, b_stride, w, h, &sse_long, &sum_long);
- *sse = (unsigned int)sse_long;
- *sum = (int)sum_long;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static unsigned int block_variance(VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bs) {
- MACROBLOCKD *xd = &x->e_mbd;
- unsigned int var, sse;
- int right_overflow = (xd->mb_to_right_edge < 0) ?
- ((-xd->mb_to_right_edge) >> 3) : 0;
- int bottom_overflow = (xd->mb_to_bottom_edge < 0) ?
- ((-xd->mb_to_bottom_edge) >> 3) : 0;
-
- if (right_overflow || bottom_overflow) {
- const int bw = 8 * num_8x8_blocks_wide_lookup[bs] - right_overflow;
- const int bh = 8 * num_8x8_blocks_high_lookup[bs] - bottom_overflow;
- int avg;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- aq_highbd_8_variance(x->plane[0].src.buf, x->plane[0].src.stride,
- CONVERT_TO_BYTEPTR(vp10_highbd_64_zeros), 0, bw, bh,
- &sse, &avg);
- sse >>= 2 * (xd->bd - 8);
- avg >>= (xd->bd - 8);
- } else {
- aq_variance(x->plane[0].src.buf, x->plane[0].src.stride,
- vp10_64_zeros, 0, bw, bh, &sse, &avg);
- }
-#else
- aq_variance(x->plane[0].src.buf, x->plane[0].src.stride,
- vp10_64_zeros, 0, bw, bh, &sse, &avg);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- var = sse - (((int64_t)avg * avg) / (bw * bh));
- return (256 * var) / (bw * bh);
- } else {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- var = cpi->fn_ptr[bs].vf(x->plane[0].src.buf,
- x->plane[0].src.stride,
- CONVERT_TO_BYTEPTR(vp10_highbd_64_zeros),
- 0, &sse);
- } else {
- var = cpi->fn_ptr[bs].vf(x->plane[0].src.buf,
- x->plane[0].src.stride,
- vp10_64_zeros, 0, &sse);
- }
-#else
- var = cpi->fn_ptr[bs].vf(x->plane[0].src.buf,
- x->plane[0].src.stride,
- vp10_64_zeros, 0, &sse);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- return (256 * var) >> num_pels_log2_lookup[bs];
- }
-}
-
-double vp10_log_block_var(VP10_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bs) {
- unsigned int var = block_variance(cpi, x, bs);
- vpx_clear_system_state();
- return log(var + 1.0);
-}
-
-#define DEFAULT_E_MIDPOINT 10.0
-int vp10_block_energy(VP10_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bs) {
- double energy;
- double energy_midpoint;
- vpx_clear_system_state();
- energy_midpoint =
- (cpi->oxcf.pass == 2) ? cpi->twopass.mb_av_energy : DEFAULT_E_MIDPOINT;
- energy = vp10_log_block_var(cpi, x, bs) - energy_midpoint;
- return clamp((int)round(energy), ENERGY_MIN, ENERGY_MAX);
-}
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_AQ_VARIANCE_H_
-#define VP10_ENCODER_AQ_VARIANCE_H_
-
-#include "vp10/encoder/encoder.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-unsigned int vp10_vaq_segment_id(int energy);
-void vp10_vaq_frame_setup(VP10_COMP *cpi);
-
-int vp10_block_energy(VP10_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bs);
-double vp10_log_block_var(VP10_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bs);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_AQ_VARIANCE_H_
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <arm_neon.h>
-#include <assert.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-
-#include "vpx/vpx_integer.h"
-
-static INLINE unsigned int horizontal_add_u16x8(const uint16x8_t v_16x8) {
- const uint32x4_t a = vpaddlq_u16(v_16x8);
- const uint64x2_t b = vpaddlq_u32(a);
- const uint32x2_t c = vadd_u32(vreinterpret_u32_u64(vget_low_u64(b)),
- vreinterpret_u32_u64(vget_high_u64(b)));
- return vget_lane_u32(c, 0);
-}
-
-unsigned int vp10_avg_8x8_neon(const uint8_t *s, int p) {
- uint8x8_t v_s0 = vld1_u8(s);
- const uint8x8_t v_s1 = vld1_u8(s + p);
- uint16x8_t v_sum = vaddl_u8(v_s0, v_s1);
-
- v_s0 = vld1_u8(s + 2 * p);
- v_sum = vaddw_u8(v_sum, v_s0);
-
- v_s0 = vld1_u8(s + 3 * p);
- v_sum = vaddw_u8(v_sum, v_s0);
-
- v_s0 = vld1_u8(s + 4 * p);
- v_sum = vaddw_u8(v_sum, v_s0);
-
- v_s0 = vld1_u8(s + 5 * p);
- v_sum = vaddw_u8(v_sum, v_s0);
-
- v_s0 = vld1_u8(s + 6 * p);
- v_sum = vaddw_u8(v_sum, v_s0);
-
- v_s0 = vld1_u8(s + 7 * p);
- v_sum = vaddw_u8(v_sum, v_s0);
-
- return (horizontal_add_u16x8(v_sum) + 32) >> 6;
-}
-
-void vp10_int_pro_row_neon(int16_t hbuf[16], uint8_t const *ref,
- const int ref_stride, const int height) {
- int i;
- uint16x8_t vec_sum_lo = vdupq_n_u16(0);
- uint16x8_t vec_sum_hi = vdupq_n_u16(0);
- const int shift_factor = ((height >> 5) + 3) * -1;
- const int16x8_t vec_shift = vdupq_n_s16(shift_factor);
-
- for (i = 0; i < height; i += 8) {
- const uint8x16_t vec_row1 = vld1q_u8(ref);
- const uint8x16_t vec_row2 = vld1q_u8(ref + ref_stride);
- const uint8x16_t vec_row3 = vld1q_u8(ref + ref_stride * 2);
- const uint8x16_t vec_row4 = vld1q_u8(ref + ref_stride * 3);
- const uint8x16_t vec_row5 = vld1q_u8(ref + ref_stride * 4);
- const uint8x16_t vec_row6 = vld1q_u8(ref + ref_stride * 5);
- const uint8x16_t vec_row7 = vld1q_u8(ref + ref_stride * 6);
- const uint8x16_t vec_row8 = vld1q_u8(ref + ref_stride * 7);
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row1));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row1));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row2));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row2));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row3));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row3));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row4));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row4));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row5));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row5));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row6));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row6));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row7));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row7));
-
- vec_sum_lo = vaddw_u8(vec_sum_lo, vget_low_u8(vec_row8));
- vec_sum_hi = vaddw_u8(vec_sum_hi, vget_high_u8(vec_row8));
-
- ref += ref_stride * 8;
- }
-
- vec_sum_lo = vshlq_u16(vec_sum_lo, vec_shift);
- vec_sum_hi = vshlq_u16(vec_sum_hi, vec_shift);
-
- vst1q_s16(hbuf, vreinterpretq_s16_u16(vec_sum_lo));
- hbuf += 8;
- vst1q_s16(hbuf, vreinterpretq_s16_u16(vec_sum_hi));
-}
-
-int16_t vp10_int_pro_col_neon(uint8_t const *ref, const int width) {
- int i;
- uint16x8_t vec_sum = vdupq_n_u16(0);
-
- for (i = 0; i < width; i += 16) {
- const uint8x16_t vec_row = vld1q_u8(ref);
- vec_sum = vaddw_u8(vec_sum, vget_low_u8(vec_row));
- vec_sum = vaddw_u8(vec_sum, vget_high_u8(vec_row));
- ref += 16;
- }
-
- return horizontal_add_u16x8(vec_sum);
-}
-
-// ref, src = [0, 510] - max diff = 16-bits
-// bwl = {2, 3, 4}, width = {16, 32, 64}
-int vp10_vector_var_neon(int16_t const *ref, int16_t const *src, const int bwl) {
- int width = 4 << bwl;
- int32x4_t sse = vdupq_n_s32(0);
- int16x8_t total = vdupq_n_s16(0);
-
- assert(width >= 8);
- assert((width % 8) == 0);
-
- do {
- const int16x8_t r = vld1q_s16(ref);
- const int16x8_t s = vld1q_s16(src);
- const int16x8_t diff = vsubq_s16(r, s); // [-510, 510], 10 bits.
- const int16x4_t diff_lo = vget_low_s16(diff);
- const int16x4_t diff_hi = vget_high_s16(diff);
- sse = vmlal_s16(sse, diff_lo, diff_lo); // dynamic range 26 bits.
- sse = vmlal_s16(sse, diff_hi, diff_hi);
- total = vaddq_s16(total, diff); // dynamic range 16 bits.
-
- ref += 8;
- src += 8;
- width -= 8;
- } while (width != 0);
-
- {
- // Note: 'total''s pairwise addition could be implemented similarly to
- // horizontal_add_u16x8(), but one less vpaddl with 'total' when paired
- // with the summation of 'sse' performed better on a Cortex-A15.
- const int32x4_t t0 = vpaddlq_s16(total); // cascading summation of 'total'
- const int32x2_t t1 = vadd_s32(vget_low_s32(t0), vget_high_s32(t0));
- const int32x2_t t2 = vpadd_s32(t1, t1);
- const int t = vget_lane_s32(t2, 0);
- const int64x2_t s0 = vpaddlq_s32(sse); // cascading summation of 'sse'.
- const int32x2_t s1 = vadd_s32(vreinterpret_s32_s64(vget_low_s64(s0)),
- vreinterpret_s32_s64(vget_high_s64(s0)));
- const int s = vget_lane_s32(s1, 0);
- const int shift_factor = bwl + 2;
- return s - ((t * t) >> shift_factor);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <arm_neon.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-
-#include "vp10/common/blockd.h"
-#include "vpx_dsp/txfm_common.h"
-
-void vp10_fdct8x8_quant_neon(const int16_t *input, int stride,
- int16_t* coeff_ptr, intptr_t n_coeffs,
- int skip_block, const int16_t* zbin_ptr,
- const int16_t* round_ptr, const int16_t* quant_ptr,
- const int16_t* quant_shift_ptr,
- int16_t* qcoeff_ptr, int16_t* dqcoeff_ptr,
- const int16_t* dequant_ptr, uint16_t* eob_ptr,
- const int16_t* scan_ptr,
- const int16_t* iscan_ptr) {
- int16_t temp_buffer[64];
- (void)coeff_ptr;
-
- vpx_fdct8x8_neon(input, temp_buffer, stride);
- vp10_quantize_fp_neon(temp_buffer, n_coeffs, skip_block, zbin_ptr, round_ptr,
- quant_ptr, quant_shift_ptr, qcoeff_ptr, dqcoeff_ptr,
- dequant_ptr, eob_ptr, scan_ptr, iscan_ptr);
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <arm_neon.h>
-#include <assert.h>
-
-#include "./vp10_rtcd.h"
-
-int64_t vp10_block_error_fp_neon(const int16_t *coeff, const int16_t *dqcoeff,
- int block_size) {
- int64x2_t error = vdupq_n_s64(0);
-
- assert(block_size >= 8);
- assert((block_size % 8) == 0);
-
- do {
- const int16x8_t c = vld1q_s16(coeff);
- const int16x8_t d = vld1q_s16(dqcoeff);
- const int16x8_t diff = vsubq_s16(c, d);
- const int16x4_t diff_lo = vget_low_s16(diff);
- const int16x4_t diff_hi = vget_high_s16(diff);
- // diff is 15-bits, the squares 30, so we can store 2 in 31-bits before
- // accumulating them in 64-bits.
- const int32x4_t err0 = vmull_s16(diff_lo, diff_lo);
- const int32x4_t err1 = vmlal_s16(err0, diff_hi, diff_hi);
- const int64x2_t err2 = vaddl_s32(vget_low_s32(err1), vget_high_s32(err1));
- error = vaddq_s64(error, err2);
- coeff += 8;
- dqcoeff += 8;
- block_size -= 8;
- } while (block_size != 0);
-
- return vgetq_lane_s64(error, 0) + vgetq_lane_s64(error, 1);
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <arm_neon.h>
-
-#include <math.h>
-
-#include "vpx_mem/vpx_mem.h"
-
-#include "vp10/common/quant_common.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/rd.h"
-
-void vp10_quantize_fp_neon(const int16_t *coeff_ptr, intptr_t count,
- int skip_block, const int16_t *zbin_ptr,
- const int16_t *round_ptr, const int16_t *quant_ptr,
- const int16_t *quant_shift_ptr, int16_t *qcoeff_ptr,
- int16_t *dqcoeff_ptr, const int16_t *dequant_ptr,
- uint16_t *eob_ptr,
- const int16_t *scan, const int16_t *iscan) {
- // TODO(jingning) Decide the need of these arguments after the
- // quantization process is completed.
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)scan;
-
- if (!skip_block) {
- // Quantization pass: All coefficients with index >= zero_flag are
- // skippable. Note: zero_flag can be zero.
- int i;
- const int16x8_t v_zero = vdupq_n_s16(0);
- const int16x8_t v_one = vdupq_n_s16(1);
- int16x8_t v_eobmax_76543210 = vdupq_n_s16(-1);
- int16x8_t v_round = vmovq_n_s16(round_ptr[1]);
- int16x8_t v_quant = vmovq_n_s16(quant_ptr[1]);
- int16x8_t v_dequant = vmovq_n_s16(dequant_ptr[1]);
- // adjust for dc
- v_round = vsetq_lane_s16(round_ptr[0], v_round, 0);
- v_quant = vsetq_lane_s16(quant_ptr[0], v_quant, 0);
- v_dequant = vsetq_lane_s16(dequant_ptr[0], v_dequant, 0);
- // process dc and the first seven ac coeffs
- {
- const int16x8_t v_iscan = vld1q_s16(&iscan[0]);
- const int16x8_t v_coeff = vld1q_s16(&coeff_ptr[0]);
- const int16x8_t v_coeff_sign = vshrq_n_s16(v_coeff, 15);
- const int16x8_t v_tmp = vabaq_s16(v_round, v_coeff, v_zero);
- const int32x4_t v_tmp_lo = vmull_s16(vget_low_s16(v_tmp),
- vget_low_s16(v_quant));
- const int32x4_t v_tmp_hi = vmull_s16(vget_high_s16(v_tmp),
- vget_high_s16(v_quant));
- const int16x8_t v_tmp2 = vcombine_s16(vshrn_n_s32(v_tmp_lo, 16),
- vshrn_n_s32(v_tmp_hi, 16));
- const uint16x8_t v_nz_mask = vceqq_s16(v_tmp2, v_zero);
- const int16x8_t v_iscan_plus1 = vaddq_s16(v_iscan, v_one);
- const int16x8_t v_nz_iscan = vbslq_s16(v_nz_mask, v_zero, v_iscan_plus1);
- const int16x8_t v_qcoeff_a = veorq_s16(v_tmp2, v_coeff_sign);
- const int16x8_t v_qcoeff = vsubq_s16(v_qcoeff_a, v_coeff_sign);
- const int16x8_t v_dqcoeff = vmulq_s16(v_qcoeff, v_dequant);
- v_eobmax_76543210 = vmaxq_s16(v_eobmax_76543210, v_nz_iscan);
- vst1q_s16(&qcoeff_ptr[0], v_qcoeff);
- vst1q_s16(&dqcoeff_ptr[0], v_dqcoeff);
- v_round = vmovq_n_s16(round_ptr[1]);
- v_quant = vmovq_n_s16(quant_ptr[1]);
- v_dequant = vmovq_n_s16(dequant_ptr[1]);
- }
- // now process the rest of the ac coeffs
- for (i = 8; i < count; i += 8) {
- const int16x8_t v_iscan = vld1q_s16(&iscan[i]);
- const int16x8_t v_coeff = vld1q_s16(&coeff_ptr[i]);
- const int16x8_t v_coeff_sign = vshrq_n_s16(v_coeff, 15);
- const int16x8_t v_tmp = vabaq_s16(v_round, v_coeff, v_zero);
- const int32x4_t v_tmp_lo = vmull_s16(vget_low_s16(v_tmp),
- vget_low_s16(v_quant));
- const int32x4_t v_tmp_hi = vmull_s16(vget_high_s16(v_tmp),
- vget_high_s16(v_quant));
- const int16x8_t v_tmp2 = vcombine_s16(vshrn_n_s32(v_tmp_lo, 16),
- vshrn_n_s32(v_tmp_hi, 16));
- const uint16x8_t v_nz_mask = vceqq_s16(v_tmp2, v_zero);
- const int16x8_t v_iscan_plus1 = vaddq_s16(v_iscan, v_one);
- const int16x8_t v_nz_iscan = vbslq_s16(v_nz_mask, v_zero, v_iscan_plus1);
- const int16x8_t v_qcoeff_a = veorq_s16(v_tmp2, v_coeff_sign);
- const int16x8_t v_qcoeff = vsubq_s16(v_qcoeff_a, v_coeff_sign);
- const int16x8_t v_dqcoeff = vmulq_s16(v_qcoeff, v_dequant);
- v_eobmax_76543210 = vmaxq_s16(v_eobmax_76543210, v_nz_iscan);
- vst1q_s16(&qcoeff_ptr[i], v_qcoeff);
- vst1q_s16(&dqcoeff_ptr[i], v_dqcoeff);
- }
- {
- const int16x4_t v_eobmax_3210 =
- vmax_s16(vget_low_s16(v_eobmax_76543210),
- vget_high_s16(v_eobmax_76543210));
- const int64x1_t v_eobmax_xx32 =
- vshr_n_s64(vreinterpret_s64_s16(v_eobmax_3210), 32);
- const int16x4_t v_eobmax_tmp =
- vmax_s16(v_eobmax_3210, vreinterpret_s16_s64(v_eobmax_xx32));
- const int64x1_t v_eobmax_xxx3 =
- vshr_n_s64(vreinterpret_s64_s16(v_eobmax_tmp), 16);
- const int16x4_t v_eobmax_final =
- vmax_s16(v_eobmax_tmp, vreinterpret_s16_s64(v_eobmax_xxx3));
-
- *eob_ptr = (uint16_t)vget_lane_s16(v_eobmax_final, 0);
- }
- } else {
- memset(qcoeff_ptr, 0, count * sizeof(int16_t));
- memset(dqcoeff_ptr, 0, count * sizeof(int16_t));
- *eob_ptr = 0;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#include "./vp10_rtcd.h"
-#include "vp10/common/common.h"
-#include "vpx_ports/mem.h"
-
-unsigned int vp10_avg_8x8_c(const uint8_t *s, int p) {
- int i, j;
- int sum = 0;
- for (i = 0; i < 8; ++i, s+=p)
- for (j = 0; j < 8; sum += s[j], ++j) {}
-
- return (sum + 32) >> 6;
-}
-
-unsigned int vp10_avg_4x4_c(const uint8_t *s, int p) {
- int i, j;
- int sum = 0;
- for (i = 0; i < 4; ++i, s+=p)
- for (j = 0; j < 4; sum += s[j], ++j) {}
-
- return (sum + 8) >> 4;
-}
-
-// src_diff: first pass, 9 bit, dynamic range [-255, 255]
-// second pass, 12 bit, dynamic range [-2040, 2040]
-static void hadamard_col8(const int16_t *src_diff, int src_stride,
- int16_t *coeff) {
- int16_t b0 = src_diff[0 * src_stride] + src_diff[1 * src_stride];
- int16_t b1 = src_diff[0 * src_stride] - src_diff[1 * src_stride];
- int16_t b2 = src_diff[2 * src_stride] + src_diff[3 * src_stride];
- int16_t b3 = src_diff[2 * src_stride] - src_diff[3 * src_stride];
- int16_t b4 = src_diff[4 * src_stride] + src_diff[5 * src_stride];
- int16_t b5 = src_diff[4 * src_stride] - src_diff[5 * src_stride];
- int16_t b6 = src_diff[6 * src_stride] + src_diff[7 * src_stride];
- int16_t b7 = src_diff[6 * src_stride] - src_diff[7 * src_stride];
-
- int16_t c0 = b0 + b2;
- int16_t c1 = b1 + b3;
- int16_t c2 = b0 - b2;
- int16_t c3 = b1 - b3;
- int16_t c4 = b4 + b6;
- int16_t c5 = b5 + b7;
- int16_t c6 = b4 - b6;
- int16_t c7 = b5 - b7;
-
- coeff[0] = c0 + c4;
- coeff[7] = c1 + c5;
- coeff[3] = c2 + c6;
- coeff[4] = c3 + c7;
- coeff[2] = c0 - c4;
- coeff[6] = c1 - c5;
- coeff[1] = c2 - c6;
- coeff[5] = c3 - c7;
-}
-
-void vp10_hadamard_8x8_c(int16_t const *src_diff, int src_stride,
- int16_t *coeff) {
- int idx;
- int16_t buffer[64];
- int16_t *tmp_buf = &buffer[0];
- for (idx = 0; idx < 8; ++idx) {
- hadamard_col8(src_diff, src_stride, tmp_buf); // src_diff: 9 bit
- // dynamic range [-255, 255]
- tmp_buf += 8;
- ++src_diff;
- }
-
- tmp_buf = &buffer[0];
- for (idx = 0; idx < 8; ++idx) {
- hadamard_col8(tmp_buf, 8, coeff); // tmp_buf: 12 bit
- // dynamic range [-2040, 2040]
- coeff += 8; // coeff: 15 bit
- // dynamic range [-16320, 16320]
- ++tmp_buf;
- }
-}
-
-// In place 16x16 2D Hadamard transform
-void vp10_hadamard_16x16_c(int16_t const *src_diff, int src_stride,
- int16_t *coeff) {
- int idx;
- for (idx = 0; idx < 4; ++idx) {
- // src_diff: 9 bit, dynamic range [-255, 255]
- int16_t const *src_ptr = src_diff + (idx >> 1) * 8 * src_stride
- + (idx & 0x01) * 8;
- vp10_hadamard_8x8_c(src_ptr, src_stride, coeff + idx * 64);
- }
-
- // coeff: 15 bit, dynamic range [-16320, 16320]
- for (idx = 0; idx < 64; ++idx) {
- int16_t a0 = coeff[0];
- int16_t a1 = coeff[64];
- int16_t a2 = coeff[128];
- int16_t a3 = coeff[192];
-
- int16_t b0 = (a0 + a1) >> 1; // (a0 + a1): 16 bit, [-32640, 32640]
- int16_t b1 = (a0 - a1) >> 1; // b0-b3: 15 bit, dynamic range
- int16_t b2 = (a2 + a3) >> 1; // [-16320, 16320]
- int16_t b3 = (a2 - a3) >> 1;
-
- coeff[0] = b0 + b2; // 16 bit, [-32640, 32640]
- coeff[64] = b1 + b3;
- coeff[128] = b0 - b2;
- coeff[192] = b1 - b3;
-
- ++coeff;
- }
-}
-
-// coeff: 16 bits, dynamic range [-32640, 32640].
-// length: value range {16, 64, 256, 1024}.
-int16_t vp10_satd_c(const int16_t *coeff, int length) {
- int i;
- int satd = 0;
- for (i = 0; i < length; ++i)
- satd += abs(coeff[i]);
-
- // satd: 26 bits, dynamic range [-32640 * 1024, 32640 * 1024]
- return (int16_t)satd;
-}
-
-// Integer projection onto row vectors.
-// height: value range {16, 32, 64}.
-void vp10_int_pro_row_c(int16_t hbuf[16], uint8_t const *ref,
- const int ref_stride, const int height) {
- int idx;
- const int norm_factor = height >> 1;
- for (idx = 0; idx < 16; ++idx) {
- int i;
- hbuf[idx] = 0;
- // hbuf[idx]: 14 bit, dynamic range [0, 16320].
- for (i = 0; i < height; ++i)
- hbuf[idx] += ref[i * ref_stride];
- // hbuf[idx]: 9 bit, dynamic range [0, 510].
- hbuf[idx] /= norm_factor;
- ++ref;
- }
-}
-
-// width: value range {16, 32, 64}.
-int16_t vp10_int_pro_col_c(uint8_t const *ref, const int width) {
- int idx;
- int16_t sum = 0;
- // sum: 14 bit, dynamic range [0, 16320]
- for (idx = 0; idx < width; ++idx)
- sum += ref[idx];
- return sum;
-}
-
-// ref: [0 - 510]
-// src: [0 - 510]
-// bwl: {2, 3, 4}
-int vp10_vector_var_c(int16_t const *ref, int16_t const *src,
- const int bwl) {
- int i;
- int width = 4 << bwl;
- int sse = 0, mean = 0, var;
-
- for (i = 0; i < width; ++i) {
- int diff = ref[i] - src[i]; // diff: dynamic range [-510, 510], 10 bits.
- mean += diff; // mean: dynamic range 16 bits.
- sse += diff * diff; // sse: dynamic range 26 bits.
- }
-
- // (mean * mean): dynamic range 31 bits.
- var = sse - ((mean * mean) >> (bwl + 2));
- return var;
-}
-
-void vp10_minmax_8x8_c(const uint8_t *s, int p, const uint8_t *d, int dp,
- int *min, int *max) {
- int i, j;
- *min = 255;
- *max = 0;
- for (i = 0; i < 8; ++i, s += p, d += dp) {
- for (j = 0; j < 8; ++j) {
- int diff = abs(s[j]-d[j]);
- *min = diff < *min ? diff : *min;
- *max = diff > *max ? diff : *max;
- }
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-unsigned int vp10_highbd_avg_8x8_c(const uint8_t *s8, int p) {
- int i, j;
- int sum = 0;
- const uint16_t* s = CONVERT_TO_SHORTPTR(s8);
- for (i = 0; i < 8; ++i, s+=p)
- for (j = 0; j < 8; sum += s[j], ++j) {}
-
- return (sum + 32) >> 6;
-}
-
-unsigned int vp10_highbd_avg_4x4_c(const uint8_t *s8, int p) {
- int i, j;
- int sum = 0;
- const uint16_t* s = CONVERT_TO_SHORTPTR(s8);
- for (i = 0; i < 4; ++i, s+=p)
- for (j = 0; j < 4; sum += s[j], ++j) {}
-
- return (sum + 8) >> 4;
-}
-
-void vp10_highbd_minmax_8x8_c(const uint8_t *s8, int p, const uint8_t *d8,
- int dp, int *min, int *max) {
- int i, j;
- const uint16_t* s = CONVERT_TO_SHORTPTR(s8);
- const uint16_t* d = CONVERT_TO_SHORTPTR(d8);
- *min = 255;
- *max = 0;
- for (i = 0; i < 8; ++i, s += p, d += dp) {
- for (j = 0; j < 8; ++j) {
- int diff = abs(s[j]-d[j]);
- *min = diff < *min ? diff : *min;
- *max = diff > *max ? diff : *max;
- }
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <stdio.h>
-#include <limits.h>
-
-#include "vpx/vpx_encoder.h"
-#include "vpx_dsp/bitwriter_buffer.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem_ops.h"
-#include "vpx_ports/system_state.h"
-
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/entropymv.h"
-#include "vp10/common/mvref_common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/seg_common.h"
-#include "vp10/common/tile_common.h"
-
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/bitstream.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/encoder/segmentation.h"
-#include "vp10/encoder/subexp.h"
-#include "vp10/encoder/tokenize.h"
-
-static const struct vp10_token intra_mode_encodings[INTRA_MODES] = {
- {0, 1}, {6, 3}, {28, 5}, {30, 5}, {58, 6}, {59, 6}, {126, 7}, {127, 7},
- {62, 6}, {2, 2}};
-static const struct vp10_token switchable_interp_encodings[SWITCHABLE_FILTERS] =
- {{0, 1}, {2, 2}, {3, 2}};
-static const struct vp10_token partition_encodings[PARTITION_TYPES] =
- {{0, 1}, {2, 2}, {6, 3}, {7, 3}};
-static const struct vp10_token inter_mode_encodings[INTER_MODES] =
- {{2, 2}, {6, 3}, {0, 1}, {7, 3}};
-static const struct vp10_token palette_size_encodings[] = {
- {0, 1}, {2, 2}, {6, 3}, {14, 4}, {30, 5}, {62, 6}, {63, 6},
-};
-static const struct vp10_token
-palette_color_encodings[PALETTE_MAX_SIZE - 1][PALETTE_MAX_SIZE] = {
- {{0, 1}, {1, 1}}, // 2 colors
- {{0, 1}, {2, 2}, {3, 2}}, // 3 colors
- {{0, 1}, {2, 2}, {6, 3}, {7, 3}}, // 4 colors
- {{0, 1}, {2, 2}, {6, 3}, {14, 4}, {15, 4}}, // 5 colors
- {{0, 1}, {2, 2}, {6, 3}, {14, 4}, {30, 5}, {31, 5}}, // 6 colors
- {{0, 1}, {2, 2}, {6, 3}, {14, 4}, {30, 5}, {62, 6}, {63, 6}}, // 7 colors
- {{0, 1}, {2, 2}, {6, 3}, {14, 4},
- {30, 5}, {62, 6}, {126, 7}, {127, 7}}, // 8 colors
-};
-
-static INLINE void write_uniform(vpx_writer *w, int n, int v) {
- int l = get_unsigned_bits(n);
- int m = (1 << l) - n;
- if (l == 0)
- return;
- if (v < m) {
- vpx_write_literal(w, v, l - 1);
- } else {
- vpx_write_literal(w, m + ((v - m) >> 1), l - 1);
- vpx_write_literal(w, (v - m) & 1, 1);
- }
-}
-
-static void write_intra_mode(vpx_writer *w, PREDICTION_MODE mode,
- const vpx_prob *probs) {
- vp10_write_token(w, vp10_intra_mode_tree, probs, &intra_mode_encodings[mode]);
-}
-
-static void write_inter_mode(vpx_writer *w, PREDICTION_MODE mode,
- const vpx_prob *probs) {
- assert(is_inter_mode(mode));
- vp10_write_token(w, vp10_inter_mode_tree, probs,
- &inter_mode_encodings[INTER_OFFSET(mode)]);
-}
-
-static void encode_unsigned_max(struct vpx_write_bit_buffer *wb,
- int data, int max) {
- vpx_wb_write_literal(wb, data, get_unsigned_bits(max));
-}
-
-static void prob_diff_update(const vpx_tree_index *tree,
- vpx_prob probs[/*n - 1*/],
- const unsigned int counts[/*n - 1*/],
- int n, vpx_writer *w) {
- int i;
- unsigned int branch_ct[32][2];
-
- // Assuming max number of probabilities <= 32
- assert(n <= 32);
-
- vp10_tree_probs_from_distribution(tree, branch_ct, counts);
- for (i = 0; i < n - 1; ++i)
- vp10_cond_prob_diff_update(w, &probs[i], branch_ct[i]);
-}
-
-static void write_selected_tx_size(const VP10_COMMON *cm,
- const MACROBLOCKD *xd, vpx_writer *w) {
- TX_SIZE tx_size = xd->mi[0]->mbmi.tx_size;
- BLOCK_SIZE bsize = xd->mi[0]->mbmi.sb_type;
- const TX_SIZE max_tx_size = max_txsize_lookup[bsize];
- const vpx_prob *const tx_probs = get_tx_probs2(max_tx_size, xd,
- &cm->fc->tx_probs);
- vpx_write(w, tx_size != TX_4X4, tx_probs[0]);
- if (tx_size != TX_4X4 && max_tx_size >= TX_16X16) {
- vpx_write(w, tx_size != TX_8X8, tx_probs[1]);
- if (tx_size != TX_8X8 && max_tx_size >= TX_32X32)
- vpx_write(w, tx_size != TX_16X16, tx_probs[2]);
- }
-}
-
-static int write_skip(const VP10_COMMON *cm, const MACROBLOCKD *xd,
- int segment_id, const MODE_INFO *mi, vpx_writer *w) {
- if (segfeature_active(&cm->seg, segment_id, SEG_LVL_SKIP)) {
- return 1;
- } else {
- const int skip = mi->mbmi.skip;
- vpx_write(w, skip, vp10_get_skip_prob(cm, xd));
- return skip;
- }
-}
-
-static void update_skip_probs(VP10_COMMON *cm, vpx_writer *w,
- FRAME_COUNTS *counts) {
- int k;
-
- for (k = 0; k < SKIP_CONTEXTS; ++k)
- vp10_cond_prob_diff_update(w, &cm->fc->skip_probs[k], counts->skip[k]);
-}
-
-static void update_switchable_interp_probs(VP10_COMMON *cm, vpx_writer *w,
- FRAME_COUNTS *counts) {
- int j;
- for (j = 0; j < SWITCHABLE_FILTER_CONTEXTS; ++j)
- prob_diff_update(vp10_switchable_interp_tree,
- cm->fc->switchable_interp_prob[j],
- counts->switchable_interp[j], SWITCHABLE_FILTERS, w);
-}
-
-static void pack_palette_tokens(vpx_writer *w, TOKENEXTRA **tp,
- BLOCK_SIZE bsize, int n) {
- int rows = 4 * num_4x4_blocks_high_lookup[bsize];
- int cols = 4 * num_4x4_blocks_wide_lookup[bsize];
- int i;
- TOKENEXTRA *p = *tp;
-
- for (i = 0; i < rows * cols -1; ++i) {
- vp10_write_token(w, vp10_palette_color_tree[n - 2], p->context_tree,
- &palette_color_encodings[n - 2][p->token]);
- ++p;
- }
-
- *tp = p;
-}
-
-static void pack_mb_tokens(vpx_writer *w,
- TOKENEXTRA **tp, const TOKENEXTRA *const stop,
- vpx_bit_depth_t bit_depth, const TX_SIZE tx) {
- TOKENEXTRA *p = *tp;
-#if !CONFIG_MISC_FIXES
- (void) tx;
-#endif
-
- while (p < stop && p->token != EOSB_TOKEN) {
- const int t = p->token;
- const struct vp10_token *const a = &vp10_coef_encodings[t];
- int i = 0;
- int v = a->value;
- int n = a->len;
-#if CONFIG_VP9_HIGHBITDEPTH
- const vp10_extra_bit *b;
- if (bit_depth == VPX_BITS_12)
- b = &vp10_extra_bits_high12[t];
- else if (bit_depth == VPX_BITS_10)
- b = &vp10_extra_bits_high10[t];
- else
- b = &vp10_extra_bits[t];
-#else
- const vp10_extra_bit *const b = &vp10_extra_bits[t];
- (void) bit_depth;
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- /* skip one or two nodes */
- if (p->skip_eob_node) {
- n -= p->skip_eob_node;
- i = 2 * p->skip_eob_node;
- }
-
- // TODO(jbb): expanding this can lead to big gains. It allows
- // much better branch prediction and would enable us to avoid numerous
- // lookups and compares.
-
- // If we have a token that's in the constrained set, the coefficient tree
- // is split into two treed writes. The first treed write takes care of the
- // unconstrained nodes. The second treed write takes care of the
- // constrained nodes.
- if (t >= TWO_TOKEN && t < EOB_TOKEN) {
- int len = UNCONSTRAINED_NODES - p->skip_eob_node;
- int bits = v >> (n - len);
- vp10_write_tree(w, vp10_coef_tree, p->context_tree, bits, len, i);
- vp10_write_tree(w, vp10_coef_con_tree,
- vp10_pareto8_full[p->context_tree[PIVOT_NODE] - 1],
- v, n - len, 0);
- } else {
- vp10_write_tree(w, vp10_coef_tree, p->context_tree, v, n, i);
- }
-
- if (b->base_val) {
- const int e = p->extra, l = b->len;
-#if CONFIG_MISC_FIXES
- int skip_bits =
- (b->base_val == CAT6_MIN_VAL) ? TX_SIZES - 1 - tx : 0;
-#else
- int skip_bits = 0;
-#endif
-
- if (l) {
- const unsigned char *pb = b->prob;
- int v = e >> 1;
- int n = l; /* number of bits in v, assumed nonzero */
- int i = 0;
-
- do {
- const int bb = (v >> --n) & 1;
- if (skip_bits) {
- skip_bits--;
- assert(!bb);
- } else {
- vpx_write(w, bb, pb[i >> 1]);
- }
- i = b->tree[i + bb];
- } while (n);
- }
-
- vpx_write_bit(w, e & 1);
- }
- ++p;
- }
-
- *tp = p;
-}
-
-static void write_segment_id(vpx_writer *w, const struct segmentation *seg,
- const struct segmentation_probs *segp,
- int segment_id) {
- if (seg->enabled && seg->update_map)
- vp10_write_tree(w, vp10_segment_tree, segp->tree_probs, segment_id, 3, 0);
-}
-
-// This function encodes the reference frame
-static void write_ref_frames(const VP10_COMMON *cm, const MACROBLOCKD *xd,
- vpx_writer *w) {
- const MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- const int is_compound = has_second_ref(mbmi);
- const int segment_id = mbmi->segment_id;
-
- // If segment level coding of this signal is disabled...
- // or the segment allows multiple reference frame options
- if (segfeature_active(&cm->seg, segment_id, SEG_LVL_REF_FRAME)) {
- assert(!is_compound);
- assert(mbmi->ref_frame[0] ==
- get_segdata(&cm->seg, segment_id, SEG_LVL_REF_FRAME));
- } else {
- // does the feature use compound prediction or not
- // (if not specified at the frame/segment level)
- if (cm->reference_mode == REFERENCE_MODE_SELECT) {
- vpx_write(w, is_compound, vp10_get_reference_mode_prob(cm, xd));
- } else {
- assert(!is_compound == (cm->reference_mode == SINGLE_REFERENCE));
- }
-
- if (is_compound) {
- vpx_write(w, mbmi->ref_frame[0] == GOLDEN_FRAME,
- vp10_get_pred_prob_comp_ref_p(cm, xd));
- } else {
- const int bit0 = mbmi->ref_frame[0] != LAST_FRAME;
- vpx_write(w, bit0, vp10_get_pred_prob_single_ref_p1(cm, xd));
- if (bit0) {
- const int bit1 = mbmi->ref_frame[0] != GOLDEN_FRAME;
- vpx_write(w, bit1, vp10_get_pred_prob_single_ref_p2(cm, xd));
- }
- }
- }
-}
-
-static void pack_inter_mode_mvs(VP10_COMP *cpi, const MODE_INFO *mi,
- vpx_writer *w) {
- VP10_COMMON *const cm = &cpi->common;
- const nmv_context *nmvc = &cm->fc->nmvc;
- const MACROBLOCK *const x = &cpi->td.mb;
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct segmentation *const seg = &cm->seg;
-#if CONFIG_MISC_FIXES
- const struct segmentation_probs *const segp = &cm->fc->seg;
-#else
- const struct segmentation_probs *const segp = &cm->segp;
-#endif
- const MB_MODE_INFO *const mbmi = &mi->mbmi;
- const MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
- const PREDICTION_MODE mode = mbmi->mode;
- const int segment_id = mbmi->segment_id;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- const int allow_hp = cm->allow_high_precision_mv;
- const int is_inter = is_inter_block(mbmi);
- const int is_compound = has_second_ref(mbmi);
- int skip, ref;
-
- if (seg->update_map) {
- if (seg->temporal_update) {
- const int pred_flag = mbmi->seg_id_predicted;
- vpx_prob pred_prob = vp10_get_pred_prob_seg_id(segp, xd);
- vpx_write(w, pred_flag, pred_prob);
- if (!pred_flag)
- write_segment_id(w, seg, segp, segment_id);
- } else {
- write_segment_id(w, seg, segp, segment_id);
- }
- }
-
- skip = write_skip(cm, xd, segment_id, mi, w);
-
- if (!segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME))
- vpx_write(w, is_inter, vp10_get_intra_inter_prob(cm, xd));
-
- if (bsize >= BLOCK_8X8 && cm->tx_mode == TX_MODE_SELECT &&
- !(is_inter && skip)) {
- write_selected_tx_size(cm, xd, w);
- }
-
- if (!is_inter) {
- if (bsize >= BLOCK_8X8) {
- write_intra_mode(w, mode, cm->fc->y_mode_prob[size_group_lookup[bsize]]);
- } else {
- int idx, idy;
- const int num_4x4_w = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[bsize];
- for (idy = 0; idy < 2; idy += num_4x4_h) {
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- const PREDICTION_MODE b_mode = mi->bmi[idy * 2 + idx].as_mode;
- write_intra_mode(w, b_mode, cm->fc->y_mode_prob[0]);
- }
- }
- }
- write_intra_mode(w, mbmi->uv_mode, cm->fc->uv_mode_prob[mode]);
- } else {
- const int mode_ctx = mbmi_ext->mode_context[mbmi->ref_frame[0]];
- const vpx_prob *const inter_probs = cm->fc->inter_mode_probs[mode_ctx];
- write_ref_frames(cm, xd, w);
-
- // If segment skip is not enabled code the mode.
- if (!segfeature_active(seg, segment_id, SEG_LVL_SKIP)) {
- if (bsize >= BLOCK_8X8) {
- write_inter_mode(w, mode, inter_probs);
- }
- }
-
- if (cm->interp_filter == SWITCHABLE) {
- const int ctx = vp10_get_pred_context_switchable_interp(xd);
- vp10_write_token(w, vp10_switchable_interp_tree,
- cm->fc->switchable_interp_prob[ctx],
- &switchable_interp_encodings[mbmi->interp_filter]);
- ++cpi->interp_filter_selected[0][mbmi->interp_filter];
- } else {
- assert(mbmi->interp_filter == cm->interp_filter);
- }
-
- if (bsize < BLOCK_8X8) {
- const int num_4x4_w = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[bsize];
- int idx, idy;
- for (idy = 0; idy < 2; idy += num_4x4_h) {
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- const int j = idy * 2 + idx;
- const PREDICTION_MODE b_mode = mi->bmi[j].as_mode;
- write_inter_mode(w, b_mode, inter_probs);
- if (b_mode == NEWMV) {
- for (ref = 0; ref < 1 + is_compound; ++ref)
- vp10_encode_mv(cpi, w, &mi->bmi[j].as_mv[ref].as_mv,
- &mbmi_ext->ref_mvs[mbmi->ref_frame[ref]][0].as_mv,
- nmvc, allow_hp);
- }
- }
- }
- } else {
- if (mode == NEWMV) {
- for (ref = 0; ref < 1 + is_compound; ++ref)
- vp10_encode_mv(cpi, w, &mbmi->mv[ref].as_mv,
- &mbmi_ext->ref_mvs[mbmi->ref_frame[ref]][0].as_mv, nmvc,
- allow_hp);
- }
- }
- }
-}
-
-static void write_palette_mode_info(const VP10_COMMON *cm,
- const MACROBLOCKD *xd,
- const MODE_INFO *const mi,
- vpx_writer *w) {
- const MB_MODE_INFO *const mbmi = &mi->mbmi;
- const MODE_INFO *const above_mi = xd->above_mi;
- const MODE_INFO *const left_mi = xd->left_mi;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- const PALETTE_MODE_INFO *pmi = &mbmi->palette_mode_info;
- int palette_ctx = 0;
- int n, i;
-
- n = pmi->palette_size[0];
- if (above_mi)
- palette_ctx += (above_mi->mbmi.palette_mode_info.palette_size[0] > 0);
- if (left_mi)
- palette_ctx += (left_mi->mbmi.palette_mode_info.palette_size[0] > 0);
- vpx_write(w, n > 0,
- vp10_default_palette_y_mode_prob[bsize - BLOCK_8X8][palette_ctx]);
- if (n > 0) {
- vp10_write_token(w, vp10_palette_size_tree,
- vp10_default_palette_y_size_prob[bsize - BLOCK_8X8],
- &palette_size_encodings[n - 2]);
- for (i = 0; i < n; ++i)
- vpx_write_literal(w, pmi->palette_colors[i],
- cm->bit_depth);
- write_uniform(w, n, pmi->palette_first_color_idx[0]);
- }
-}
-
-static void write_mb_modes_kf(const VP10_COMMON *cm, const MACROBLOCKD *xd,
- MODE_INFO **mi_8x8, vpx_writer *w) {
- const struct segmentation *const seg = &cm->seg;
-#if CONFIG_MISC_FIXES
- const struct segmentation_probs *const segp = &cm->fc->seg;
-#else
- const struct segmentation_probs *const segp = &cm->segp;
-#endif
- const MODE_INFO *const mi = mi_8x8[0];
- const MODE_INFO *const above_mi = xd->above_mi;
- const MODE_INFO *const left_mi = xd->left_mi;
- const MB_MODE_INFO *const mbmi = &mi->mbmi;
- const BLOCK_SIZE bsize = mbmi->sb_type;
-
- if (seg->update_map)
- write_segment_id(w, seg, segp, mbmi->segment_id);
-
- write_skip(cm, xd, mbmi->segment_id, mi, w);
-
- if (bsize >= BLOCK_8X8 && cm->tx_mode == TX_MODE_SELECT)
- write_selected_tx_size(cm, xd, w);
-
- if (bsize >= BLOCK_8X8) {
- write_intra_mode(w, mbmi->mode,
- get_y_mode_probs(cm, mi, above_mi, left_mi, 0));
- } else {
- const int num_4x4_w = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[bsize];
- int idx, idy;
-
- for (idy = 0; idy < 2; idy += num_4x4_h) {
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- const int block = idy * 2 + idx;
- write_intra_mode(w, mi->bmi[block].as_mode,
- get_y_mode_probs(cm, mi, above_mi, left_mi, block));
- }
- }
- }
-
- write_intra_mode(w, mbmi->uv_mode, cm->fc->uv_mode_prob[mbmi->mode]);
-
- if (bsize >= BLOCK_8X8 && cm->allow_screen_content_tools &&
- mbmi->mode == DC_PRED)
- write_palette_mode_info(cm, xd, mi, w);
-}
-
-static void write_modes_b(VP10_COMP *cpi, const TileInfo *const tile,
- vpx_writer *w, TOKENEXTRA **tok,
- const TOKENEXTRA *const tok_end,
- int mi_row, int mi_col) {
- const VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &cpi->td.mb.e_mbd;
- MODE_INFO *m;
- int plane;
-
- xd->mi = cm->mi_grid_visible + (mi_row * cm->mi_stride + mi_col);
- m = xd->mi[0];
-
- cpi->td.mb.mbmi_ext = cpi->mbmi_ext_base + (mi_row * cm->mi_cols + mi_col);
-
- set_mi_row_col(xd, tile,
- mi_row, num_8x8_blocks_high_lookup[m->mbmi.sb_type],
- mi_col, num_8x8_blocks_wide_lookup[m->mbmi.sb_type],
- cm->mi_rows, cm->mi_cols);
- if (frame_is_intra_only(cm)) {
- write_mb_modes_kf(cm, xd, xd->mi, w);
- } else {
- pack_inter_mode_mvs(cpi, m, w);
- }
-
- if (m->mbmi.palette_mode_info.palette_size[0] > 0) {
- assert(*tok < tok_end);
- pack_palette_tokens(w, tok, m->mbmi.sb_type,
- m->mbmi.palette_mode_info.palette_size[0]);
- assert(*tok < tok_end);
- }
-
- if (!m->mbmi.skip) {
- assert(*tok < tok_end);
- for (plane = 0; plane < MAX_MB_PLANE; ++plane) {
- TX_SIZE tx = plane ? get_uv_tx_size(&m->mbmi, &xd->plane[plane])
- : m->mbmi.tx_size;
- pack_mb_tokens(w, tok, tok_end, cm->bit_depth, tx);
- assert(*tok < tok_end && (*tok)->token == EOSB_TOKEN);
- (*tok)++;
- }
- }
-}
-
-static void write_partition(const VP10_COMMON *const cm,
- const MACROBLOCKD *const xd,
- int hbs, int mi_row, int mi_col,
- PARTITION_TYPE p, BLOCK_SIZE bsize, vpx_writer *w) {
- const int ctx = partition_plane_context(xd, mi_row, mi_col, bsize);
- const vpx_prob *const probs = cm->fc->partition_prob[ctx];
- const int has_rows = (mi_row + hbs) < cm->mi_rows;
- const int has_cols = (mi_col + hbs) < cm->mi_cols;
-
- if (has_rows && has_cols) {
- vp10_write_token(w, vp10_partition_tree, probs, &partition_encodings[p]);
- } else if (!has_rows && has_cols) {
- assert(p == PARTITION_SPLIT || p == PARTITION_HORZ);
- vpx_write(w, p == PARTITION_SPLIT, probs[1]);
- } else if (has_rows && !has_cols) {
- assert(p == PARTITION_SPLIT || p == PARTITION_VERT);
- vpx_write(w, p == PARTITION_SPLIT, probs[2]);
- } else {
- assert(p == PARTITION_SPLIT);
- }
-}
-
-static void write_modes_sb(VP10_COMP *cpi,
- const TileInfo *const tile, vpx_writer *w,
- TOKENEXTRA **tok, const TOKENEXTRA *const tok_end,
- int mi_row, int mi_col, BLOCK_SIZE bsize) {
- const VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &cpi->td.mb.e_mbd;
-
- const int bsl = b_width_log2_lookup[bsize];
- const int bs = (1 << bsl) / 4;
- PARTITION_TYPE partition;
- BLOCK_SIZE subsize;
- const MODE_INFO *m = NULL;
-
- if (mi_row >= cm->mi_rows || mi_col >= cm->mi_cols)
- return;
-
- m = cm->mi_grid_visible[mi_row * cm->mi_stride + mi_col];
-
- partition = partition_lookup[bsl][m->mbmi.sb_type];
- write_partition(cm, xd, bs, mi_row, mi_col, partition, bsize, w);
- subsize = get_subsize(bsize, partition);
- if (subsize < BLOCK_8X8) {
- write_modes_b(cpi, tile, w, tok, tok_end, mi_row, mi_col);
- } else {
- switch (partition) {
- case PARTITION_NONE:
- write_modes_b(cpi, tile, w, tok, tok_end, mi_row, mi_col);
- break;
- case PARTITION_HORZ:
- write_modes_b(cpi, tile, w, tok, tok_end, mi_row, mi_col);
- if (mi_row + bs < cm->mi_rows)
- write_modes_b(cpi, tile, w, tok, tok_end, mi_row + bs, mi_col);
- break;
- case PARTITION_VERT:
- write_modes_b(cpi, tile, w, tok, tok_end, mi_row, mi_col);
- if (mi_col + bs < cm->mi_cols)
- write_modes_b(cpi, tile, w, tok, tok_end, mi_row, mi_col + bs);
- break;
- case PARTITION_SPLIT:
- write_modes_sb(cpi, tile, w, tok, tok_end, mi_row, mi_col, subsize);
- write_modes_sb(cpi, tile, w, tok, tok_end, mi_row, mi_col + bs,
- subsize);
- write_modes_sb(cpi, tile, w, tok, tok_end, mi_row + bs, mi_col,
- subsize);
- write_modes_sb(cpi, tile, w, tok, tok_end, mi_row + bs, mi_col + bs,
- subsize);
- break;
- default:
- assert(0);
- }
- }
-
- // update partition context
- if (bsize >= BLOCK_8X8 &&
- (bsize == BLOCK_8X8 || partition != PARTITION_SPLIT))
- update_partition_context(xd, mi_row, mi_col, subsize, bsize);
-}
-
-static void write_modes(VP10_COMP *cpi,
- const TileInfo *const tile, vpx_writer *w,
- TOKENEXTRA **tok, const TOKENEXTRA *const tok_end) {
- MACROBLOCKD *const xd = &cpi->td.mb.e_mbd;
- int mi_row, mi_col;
-
- for (mi_row = tile->mi_row_start; mi_row < tile->mi_row_end;
- mi_row += MI_BLOCK_SIZE) {
- vp10_zero(xd->left_seg_context);
- for (mi_col = tile->mi_col_start; mi_col < tile->mi_col_end;
- mi_col += MI_BLOCK_SIZE)
- write_modes_sb(cpi, tile, w, tok, tok_end, mi_row, mi_col,
- BLOCK_64X64);
- }
-}
-
-static void build_tree_distribution(VP10_COMP *cpi, TX_SIZE tx_size,
- vp10_coeff_stats *coef_branch_ct,
- vp10_coeff_probs_model *coef_probs) {
- vp10_coeff_count *coef_counts = cpi->td.rd_counts.coef_counts[tx_size];
- unsigned int (*eob_branch_ct)[REF_TYPES][COEF_BANDS][COEFF_CONTEXTS] =
- cpi->common.counts.eob_branch[tx_size];
- int i, j, k, l, m;
-
- for (i = 0; i < PLANE_TYPES; ++i) {
- for (j = 0; j < REF_TYPES; ++j) {
- for (k = 0; k < COEF_BANDS; ++k) {
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l) {
- vp10_tree_probs_from_distribution(vp10_coef_tree,
- coef_branch_ct[i][j][k][l],
- coef_counts[i][j][k][l]);
- coef_branch_ct[i][j][k][l][0][1] = eob_branch_ct[i][j][k][l] -
- coef_branch_ct[i][j][k][l][0][0];
- for (m = 0; m < UNCONSTRAINED_NODES; ++m)
- coef_probs[i][j][k][l][m] = get_binary_prob(
- coef_branch_ct[i][j][k][l][m][0],
- coef_branch_ct[i][j][k][l][m][1]);
- }
- }
- }
- }
-}
-
-static void update_coef_probs_common(vpx_writer* const bc, VP10_COMP *cpi,
- TX_SIZE tx_size,
- vp10_coeff_stats *frame_branch_ct,
- vp10_coeff_probs_model *new_coef_probs) {
- vp10_coeff_probs_model *old_coef_probs = cpi->common.fc->coef_probs[tx_size];
- const vpx_prob upd = DIFF_UPDATE_PROB;
- const int entropy_nodes_update = UNCONSTRAINED_NODES;
- int i, j, k, l, t;
- int stepsize = cpi->sf.coeff_prob_appx_step;
-
- switch (cpi->sf.use_fast_coef_updates) {
- case TWO_LOOP: {
- /* dry run to see if there is any update at all needed */
- int savings = 0;
- int update[2] = {0, 0};
- for (i = 0; i < PLANE_TYPES; ++i) {
- for (j = 0; j < REF_TYPES; ++j) {
- for (k = 0; k < COEF_BANDS; ++k) {
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l) {
- for (t = 0; t < entropy_nodes_update; ++t) {
- vpx_prob newp = new_coef_probs[i][j][k][l][t];
- const vpx_prob oldp = old_coef_probs[i][j][k][l][t];
- int s;
- int u = 0;
- if (t == PIVOT_NODE)
- s = vp10_prob_diff_update_savings_search_model(
- frame_branch_ct[i][j][k][l][0],
- old_coef_probs[i][j][k][l], &newp, upd, stepsize);
- else
- s = vp10_prob_diff_update_savings_search(
- frame_branch_ct[i][j][k][l][t], oldp, &newp, upd);
- if (s > 0 && newp != oldp)
- u = 1;
- if (u)
- savings += s - (int)(vp10_cost_zero(upd));
- else
- savings -= (int)(vp10_cost_zero(upd));
- update[u]++;
- }
- }
- }
- }
- }
-
- // printf("Update %d %d, savings %d\n", update[0], update[1], savings);
- /* Is coef updated at all */
- if (update[1] == 0 || savings < 0) {
- vpx_write_bit(bc, 0);
- return;
- }
- vpx_write_bit(bc, 1);
- for (i = 0; i < PLANE_TYPES; ++i) {
- for (j = 0; j < REF_TYPES; ++j) {
- for (k = 0; k < COEF_BANDS; ++k) {
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l) {
- // calc probs and branch cts for this frame only
- for (t = 0; t < entropy_nodes_update; ++t) {
- vpx_prob newp = new_coef_probs[i][j][k][l][t];
- vpx_prob *oldp = old_coef_probs[i][j][k][l] + t;
- const vpx_prob upd = DIFF_UPDATE_PROB;
- int s;
- int u = 0;
- if (t == PIVOT_NODE)
- s = vp10_prob_diff_update_savings_search_model(
- frame_branch_ct[i][j][k][l][0],
- old_coef_probs[i][j][k][l], &newp, upd, stepsize);
- else
- s = vp10_prob_diff_update_savings_search(
- frame_branch_ct[i][j][k][l][t],
- *oldp, &newp, upd);
- if (s > 0 && newp != *oldp)
- u = 1;
- vpx_write(bc, u, upd);
- if (u) {
- /* send/use new probability */
- vp10_write_prob_diff_update(bc, newp, *oldp);
- *oldp = newp;
- }
- }
- }
- }
- }
- }
- return;
- }
-
- case ONE_LOOP_REDUCED: {
- int updates = 0;
- int noupdates_before_first = 0;
- for (i = 0; i < PLANE_TYPES; ++i) {
- for (j = 0; j < REF_TYPES; ++j) {
- for (k = 0; k < COEF_BANDS; ++k) {
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l) {
- // calc probs and branch cts for this frame only
- for (t = 0; t < entropy_nodes_update; ++t) {
- vpx_prob newp = new_coef_probs[i][j][k][l][t];
- vpx_prob *oldp = old_coef_probs[i][j][k][l] + t;
- int s;
- int u = 0;
-
- if (t == PIVOT_NODE) {
- s = vp10_prob_diff_update_savings_search_model(
- frame_branch_ct[i][j][k][l][0],
- old_coef_probs[i][j][k][l], &newp, upd, stepsize);
- } else {
- s = vp10_prob_diff_update_savings_search(
- frame_branch_ct[i][j][k][l][t],
- *oldp, &newp, upd);
- }
-
- if (s > 0 && newp != *oldp)
- u = 1;
- updates += u;
- if (u == 0 && updates == 0) {
- noupdates_before_first++;
- continue;
- }
- if (u == 1 && updates == 1) {
- int v;
- // first update
- vpx_write_bit(bc, 1);
- for (v = 0; v < noupdates_before_first; ++v)
- vpx_write(bc, 0, upd);
- }
- vpx_write(bc, u, upd);
- if (u) {
- /* send/use new probability */
- vp10_write_prob_diff_update(bc, newp, *oldp);
- *oldp = newp;
- }
- }
- }
- }
- }
- }
- if (updates == 0) {
- vpx_write_bit(bc, 0); // no updates
- }
- return;
- }
- default:
- assert(0);
- }
-}
-
-static void update_coef_probs(VP10_COMP *cpi, vpx_writer* w) {
- const TX_MODE tx_mode = cpi->common.tx_mode;
- const TX_SIZE max_tx_size = tx_mode_to_biggest_tx_size[tx_mode];
- TX_SIZE tx_size;
- for (tx_size = TX_4X4; tx_size <= max_tx_size; ++tx_size) {
- vp10_coeff_stats frame_branch_ct[PLANE_TYPES];
- vp10_coeff_probs_model frame_coef_probs[PLANE_TYPES];
- if (cpi->td.counts->tx.tx_totals[tx_size] <= 20 ||
- (tx_size >= TX_16X16 && cpi->sf.tx_size_search_method == USE_TX_8X8)) {
- vpx_write_bit(w, 0);
- } else {
- build_tree_distribution(cpi, tx_size, frame_branch_ct,
- frame_coef_probs);
- update_coef_probs_common(w, cpi, tx_size, frame_branch_ct,
- frame_coef_probs);
- }
- }
-}
-
-static void encode_loopfilter(struct loopfilter *lf,
- struct vpx_write_bit_buffer *wb) {
- int i;
-
- // Encode the loop filter level and type
- vpx_wb_write_literal(wb, lf->filter_level, 6);
- vpx_wb_write_literal(wb, lf->sharpness_level, 3);
-
- // Write out loop filter deltas applied at the MB level based on mode or
- // ref frame (if they are enabled).
- vpx_wb_write_bit(wb, lf->mode_ref_delta_enabled);
-
- if (lf->mode_ref_delta_enabled) {
- vpx_wb_write_bit(wb, lf->mode_ref_delta_update);
- if (lf->mode_ref_delta_update) {
- for (i = 0; i < MAX_REF_FRAMES; i++) {
- const int delta = lf->ref_deltas[i];
- const int changed = delta != lf->last_ref_deltas[i];
- vpx_wb_write_bit(wb, changed);
- if (changed) {
- lf->last_ref_deltas[i] = delta;
- vpx_wb_write_inv_signed_literal(wb, delta, 6);
- }
- }
-
- for (i = 0; i < MAX_MODE_LF_DELTAS; i++) {
- const int delta = lf->mode_deltas[i];
- const int changed = delta != lf->last_mode_deltas[i];
- vpx_wb_write_bit(wb, changed);
- if (changed) {
- lf->last_mode_deltas[i] = delta;
- vpx_wb_write_inv_signed_literal(wb, delta, 6);
- }
- }
- }
- }
-}
-
-static void write_delta_q(struct vpx_write_bit_buffer *wb, int delta_q) {
- if (delta_q != 0) {
- vpx_wb_write_bit(wb, 1);
- vpx_wb_write_inv_signed_literal(wb, delta_q, CONFIG_MISC_FIXES ? 6 : 4);
- } else {
- vpx_wb_write_bit(wb, 0);
- }
-}
-
-static void encode_quantization(const VP10_COMMON *const cm,
- struct vpx_write_bit_buffer *wb) {
- vpx_wb_write_literal(wb, cm->base_qindex, QINDEX_BITS);
- write_delta_q(wb, cm->y_dc_delta_q);
- write_delta_q(wb, cm->uv_dc_delta_q);
- write_delta_q(wb, cm->uv_ac_delta_q);
-}
-
-static void encode_segmentation(VP10_COMMON *cm, MACROBLOCKD *xd,
- struct vpx_write_bit_buffer *wb) {
- int i, j;
-
- const struct segmentation *seg = &cm->seg;
-#if !CONFIG_MISC_FIXES
- const struct segmentation_probs *segp = &cm->segp;
-#endif
-
- vpx_wb_write_bit(wb, seg->enabled);
- if (!seg->enabled)
- return;
-
- // Segmentation map
- if (!frame_is_intra_only(cm) && !cm->error_resilient_mode) {
- vpx_wb_write_bit(wb, seg->update_map);
- } else {
- assert(seg->update_map == 1);
- }
- if (seg->update_map) {
- // Select the coding strategy (temporal or spatial)
- vp10_choose_segmap_coding_method(cm, xd);
-#if !CONFIG_MISC_FIXES
- // Write out probabilities used to decode unpredicted macro-block segments
- for (i = 0; i < SEG_TREE_PROBS; i++) {
- const int prob = segp->tree_probs[i];
- const int update = prob != MAX_PROB;
- vpx_wb_write_bit(wb, update);
- if (update)
- vpx_wb_write_literal(wb, prob, 8);
- }
-#endif
-
- // Write out the chosen coding method.
- if (!frame_is_intra_only(cm) && !cm->error_resilient_mode) {
- vpx_wb_write_bit(wb, seg->temporal_update);
- } else {
- assert(seg->temporal_update == 0);
- }
-
-#if !CONFIG_MISC_FIXES
- if (seg->temporal_update) {
- for (i = 0; i < PREDICTION_PROBS; i++) {
- const int prob = segp->pred_probs[i];
- const int update = prob != MAX_PROB;
- vpx_wb_write_bit(wb, update);
- if (update)
- vpx_wb_write_literal(wb, prob, 8);
- }
- }
-#endif
- }
-
- // Segmentation data
- vpx_wb_write_bit(wb, seg->update_data);
- if (seg->update_data) {
- vpx_wb_write_bit(wb, seg->abs_delta);
-
- for (i = 0; i < MAX_SEGMENTS; i++) {
- for (j = 0; j < SEG_LVL_MAX; j++) {
- const int active = segfeature_active(seg, i, j);
- vpx_wb_write_bit(wb, active);
- if (active) {
- const int data = get_segdata(seg, i, j);
- const int data_max = vp10_seg_feature_data_max(j);
-
- if (vp10_is_segfeature_signed(j)) {
- encode_unsigned_max(wb, abs(data), data_max);
- vpx_wb_write_bit(wb, data < 0);
- } else {
- encode_unsigned_max(wb, data, data_max);
- }
- }
- }
- }
- }
-}
-
-#if CONFIG_MISC_FIXES
-static void update_seg_probs(VP10_COMP *cpi, vpx_writer *w) {
- VP10_COMMON *cm = &cpi->common;
-
- if (!cpi->common.seg.enabled)
- return;
-
- if (cpi->common.seg.temporal_update) {
- int i;
-
- for (i = 0; i < PREDICTION_PROBS; i++)
- vp10_cond_prob_diff_update(w, &cm->fc->seg.pred_probs[i],
- cm->counts.seg.pred[i]);
-
- prob_diff_update(vp10_segment_tree, cm->fc->seg.tree_probs,
- cm->counts.seg.tree_mispred, MAX_SEGMENTS, w);
- } else {
- prob_diff_update(vp10_segment_tree, cm->fc->seg.tree_probs,
- cm->counts.seg.tree_total, MAX_SEGMENTS, w);
- }
-}
-
-static void write_txfm_mode(TX_MODE mode, struct vpx_write_bit_buffer *wb) {
- vpx_wb_write_bit(wb, mode == TX_MODE_SELECT);
- if (mode != TX_MODE_SELECT)
- vpx_wb_write_literal(wb, mode, 2);
-}
-#endif
-
-static void update_txfm_probs(VP10_COMMON *cm, vpx_writer *w,
- FRAME_COUNTS *counts) {
-#if !CONFIG_MISC_FIXES
- // Mode
- vpx_write_literal(w, VPXMIN(cm->tx_mode, ALLOW_32X32), 2);
- if (cm->tx_mode >= ALLOW_32X32)
- vpx_write_bit(w, cm->tx_mode == TX_MODE_SELECT);
-
- // Probabilities
-#endif
-
- if (cm->tx_mode == TX_MODE_SELECT) {
- int i, j;
- unsigned int ct_8x8p[TX_SIZES - 3][2];
- unsigned int ct_16x16p[TX_SIZES - 2][2];
- unsigned int ct_32x32p[TX_SIZES - 1][2];
-
-
- for (i = 0; i < TX_SIZE_CONTEXTS; i++) {
- vp10_tx_counts_to_branch_counts_8x8(counts->tx.p8x8[i], ct_8x8p);
- for (j = 0; j < TX_SIZES - 3; j++)
- vp10_cond_prob_diff_update(w, &cm->fc->tx_probs.p8x8[i][j], ct_8x8p[j]);
- }
-
- for (i = 0; i < TX_SIZE_CONTEXTS; i++) {
- vp10_tx_counts_to_branch_counts_16x16(counts->tx.p16x16[i], ct_16x16p);
- for (j = 0; j < TX_SIZES - 2; j++)
- vp10_cond_prob_diff_update(w, &cm->fc->tx_probs.p16x16[i][j],
- ct_16x16p[j]);
- }
-
- for (i = 0; i < TX_SIZE_CONTEXTS; i++) {
- vp10_tx_counts_to_branch_counts_32x32(counts->tx.p32x32[i], ct_32x32p);
- for (j = 0; j < TX_SIZES - 1; j++)
- vp10_cond_prob_diff_update(w, &cm->fc->tx_probs.p32x32[i][j],
- ct_32x32p[j]);
- }
- }
-}
-
-static void write_interp_filter(INTERP_FILTER filter,
- struct vpx_write_bit_buffer *wb) {
- vpx_wb_write_bit(wb, filter == SWITCHABLE);
- if (filter != SWITCHABLE)
- vpx_wb_write_literal(wb, filter, 2);
-}
-
-static void fix_interp_filter(VP10_COMMON *cm, FRAME_COUNTS *counts) {
- if (cm->interp_filter == SWITCHABLE) {
- // Check to see if only one of the filters is actually used
- int count[SWITCHABLE_FILTERS];
- int i, j, c = 0;
- for (i = 0; i < SWITCHABLE_FILTERS; ++i) {
- count[i] = 0;
- for (j = 0; j < SWITCHABLE_FILTER_CONTEXTS; ++j)
- count[i] += counts->switchable_interp[j][i];
- c += (count[i] > 0);
- }
- if (c == 1) {
- // Only one filter is used. So set the filter at frame level
- for (i = 0; i < SWITCHABLE_FILTERS; ++i) {
- if (count[i]) {
- cm->interp_filter = i;
- break;
- }
- }
- }
- }
-}
-
-static void write_tile_info(const VP10_COMMON *const cm,
- struct vpx_write_bit_buffer *wb) {
- int min_log2_tile_cols, max_log2_tile_cols, ones;
- vp10_get_tile_n_bits(cm->mi_cols, &min_log2_tile_cols, &max_log2_tile_cols);
-
- // columns
- ones = cm->log2_tile_cols - min_log2_tile_cols;
- while (ones--)
- vpx_wb_write_bit(wb, 1);
-
- if (cm->log2_tile_cols < max_log2_tile_cols)
- vpx_wb_write_bit(wb, 0);
-
- // rows
- vpx_wb_write_bit(wb, cm->log2_tile_rows != 0);
- if (cm->log2_tile_rows != 0)
- vpx_wb_write_bit(wb, cm->log2_tile_rows != 1);
-}
-
-static int get_refresh_mask(VP10_COMP *cpi) {
- if (vp10_preserve_existing_gf(cpi)) {
- // We have decided to preserve the previously existing golden frame as our
- // new ARF frame. However, in the short term we leave it in the GF slot and,
- // if we're updating the GF with the current decoded frame, we save it
- // instead to the ARF slot.
- // Later, in the function vp10_encoder.c:vp10_update_reference_frames() we
- // will swap gld_fb_idx and alt_fb_idx to achieve our objective. We do it
- // there so that it can be done outside of the recode loop.
- // Note: This is highly specific to the use of ARF as a forward reference,
- // and this needs to be generalized as other uses are implemented
- // (like RTC/temporal scalability).
- return (cpi->refresh_last_frame << cpi->lst_fb_idx) |
- (cpi->refresh_golden_frame << cpi->alt_fb_idx);
- } else {
- int arf_idx = cpi->alt_fb_idx;
- if ((cpi->oxcf.pass == 2) && cpi->multi_arf_allowed) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- arf_idx = gf_group->arf_update_idx[gf_group->index];
- }
- return (cpi->refresh_last_frame << cpi->lst_fb_idx) |
- (cpi->refresh_golden_frame << cpi->gld_fb_idx) |
- (cpi->refresh_alt_ref_frame << arf_idx);
- }
-}
-
-static size_t encode_tiles(VP10_COMP *cpi, uint8_t *data_ptr,
- unsigned int *max_tile_sz) {
- VP10_COMMON *const cm = &cpi->common;
- vpx_writer residual_bc;
- int tile_row, tile_col;
- TOKENEXTRA *tok_end;
- size_t total_size = 0;
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int tile_rows = 1 << cm->log2_tile_rows;
- unsigned int max_tile = 0;
-
- memset(cm->above_seg_context, 0,
- sizeof(*cm->above_seg_context) * mi_cols_aligned_to_sb(cm->mi_cols));
-
- for (tile_row = 0; tile_row < tile_rows; tile_row++) {
- for (tile_col = 0; tile_col < tile_cols; tile_col++) {
- int tile_idx = tile_row * tile_cols + tile_col;
- TOKENEXTRA *tok = cpi->tile_tok[tile_row][tile_col];
-
- tok_end = cpi->tile_tok[tile_row][tile_col] +
- cpi->tok_count[tile_row][tile_col];
-
- if (tile_col < tile_cols - 1 || tile_row < tile_rows - 1)
- vpx_start_encode(&residual_bc, data_ptr + total_size + 4);
- else
- vpx_start_encode(&residual_bc, data_ptr + total_size);
-
- write_modes(cpi, &cpi->tile_data[tile_idx].tile_info,
- &residual_bc, &tok, tok_end);
- assert(tok == tok_end);
- vpx_stop_encode(&residual_bc);
- if (tile_col < tile_cols - 1 || tile_row < tile_rows - 1) {
- unsigned int tile_sz;
-
- // size of this tile
- assert(residual_bc.pos > 0);
- tile_sz = residual_bc.pos - CONFIG_MISC_FIXES;
- mem_put_le32(data_ptr + total_size, tile_sz);
- max_tile = max_tile > tile_sz ? max_tile : tile_sz;
- total_size += 4;
- }
-
- total_size += residual_bc.pos;
- }
- }
- *max_tile_sz = max_tile;
-
- return total_size;
-}
-
-static void write_render_size(const VP10_COMMON *cm,
- struct vpx_write_bit_buffer *wb) {
- const int scaling_active = cm->width != cm->render_width ||
- cm->height != cm->render_height;
- vpx_wb_write_bit(wb, scaling_active);
- if (scaling_active) {
- vpx_wb_write_literal(wb, cm->render_width - 1, 16);
- vpx_wb_write_literal(wb, cm->render_height - 1, 16);
- }
-}
-
-static void write_frame_size(const VP10_COMMON *cm,
- struct vpx_write_bit_buffer *wb) {
- vpx_wb_write_literal(wb, cm->width - 1, 16);
- vpx_wb_write_literal(wb, cm->height - 1, 16);
-
- write_render_size(cm, wb);
-}
-
-static void write_frame_size_with_refs(VP10_COMP *cpi,
- struct vpx_write_bit_buffer *wb) {
- VP10_COMMON *const cm = &cpi->common;
- int found = 0;
-
- MV_REFERENCE_FRAME ref_frame;
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ++ref_frame) {
- YV12_BUFFER_CONFIG *cfg = get_ref_frame_buffer(cpi, ref_frame);
-
- if (cfg != NULL) {
- found = cm->width == cfg->y_crop_width &&
- cm->height == cfg->y_crop_height;
-#if CONFIG_MISC_FIXES
- found &= cm->render_width == cfg->render_width &&
- cm->render_height == cfg->render_height;
-#endif
- }
- vpx_wb_write_bit(wb, found);
- if (found) {
- break;
- }
- }
-
- if (!found) {
- vpx_wb_write_literal(wb, cm->width - 1, 16);
- vpx_wb_write_literal(wb, cm->height - 1, 16);
-
-#if CONFIG_MISC_FIXES
- write_render_size(cm, wb);
-#endif
- }
-
-#if !CONFIG_MISC_FIXES
- write_render_size(cm, wb);
-#endif
-}
-
-static void write_sync_code(struct vpx_write_bit_buffer *wb) {
- vpx_wb_write_literal(wb, VP10_SYNC_CODE_0, 8);
- vpx_wb_write_literal(wb, VP10_SYNC_CODE_1, 8);
- vpx_wb_write_literal(wb, VP10_SYNC_CODE_2, 8);
-}
-
-static void write_profile(BITSTREAM_PROFILE profile,
- struct vpx_write_bit_buffer *wb) {
- switch (profile) {
- case PROFILE_0:
- vpx_wb_write_literal(wb, 0, 2);
- break;
- case PROFILE_1:
- vpx_wb_write_literal(wb, 2, 2);
- break;
- case PROFILE_2:
- vpx_wb_write_literal(wb, 1, 2);
- break;
- case PROFILE_3:
- vpx_wb_write_literal(wb, 6, 3);
- break;
- default:
- assert(0);
- }
-}
-
-static void write_bitdepth_colorspace_sampling(
- VP10_COMMON *const cm, struct vpx_write_bit_buffer *wb) {
- if (cm->profile >= PROFILE_2) {
- assert(cm->bit_depth > VPX_BITS_8);
- vpx_wb_write_bit(wb, cm->bit_depth == VPX_BITS_10 ? 0 : 1);
- }
- vpx_wb_write_literal(wb, cm->color_space, 3);
- if (cm->color_space != VPX_CS_SRGB) {
- // 0: [16, 235] (i.e. xvYCC), 1: [0, 255]
- vpx_wb_write_bit(wb, cm->color_range);
- if (cm->profile == PROFILE_1 || cm->profile == PROFILE_3) {
- assert(cm->subsampling_x != 1 || cm->subsampling_y != 1);
- vpx_wb_write_bit(wb, cm->subsampling_x);
- vpx_wb_write_bit(wb, cm->subsampling_y);
- vpx_wb_write_bit(wb, 0); // unused
- } else {
- assert(cm->subsampling_x == 1 && cm->subsampling_y == 1);
- }
- } else {
- assert(cm->profile == PROFILE_1 || cm->profile == PROFILE_3);
- vpx_wb_write_bit(wb, 0); // unused
- }
-}
-
-static void write_uncompressed_header(VP10_COMP *cpi,
- struct vpx_write_bit_buffer *wb) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &cpi->td.mb.e_mbd;
-
- vpx_wb_write_literal(wb, VP9_FRAME_MARKER, 2);
-
- write_profile(cm->profile, wb);
-
- vpx_wb_write_bit(wb, 0); // show_existing_frame
- vpx_wb_write_bit(wb, cm->frame_type);
- vpx_wb_write_bit(wb, cm->show_frame);
- vpx_wb_write_bit(wb, cm->error_resilient_mode);
-
- if (cm->frame_type == KEY_FRAME) {
- write_sync_code(wb);
- write_bitdepth_colorspace_sampling(cm, wb);
- write_frame_size(cm, wb);
- if (frame_is_intra_only(cm))
- vpx_wb_write_bit(wb, cm->allow_screen_content_tools);
- } else {
- if (!cm->show_frame)
- vpx_wb_write_bit(wb, cm->intra_only);
-
- if (!cm->error_resilient_mode) {
-#if CONFIG_MISC_FIXES
- if (cm->intra_only) {
- vpx_wb_write_bit(wb,
- cm->reset_frame_context == RESET_FRAME_CONTEXT_ALL);
- } else {
- vpx_wb_write_bit(wb,
- cm->reset_frame_context != RESET_FRAME_CONTEXT_NONE);
- if (cm->reset_frame_context != RESET_FRAME_CONTEXT_NONE)
- vpx_wb_write_bit(wb,
- cm->reset_frame_context == RESET_FRAME_CONTEXT_ALL);
- }
-#else
- static const int reset_frame_context_conv_tbl[3] = { 0, 2, 3 };
-
- vpx_wb_write_literal(wb,
- reset_frame_context_conv_tbl[cm->reset_frame_context], 2);
-#endif
- }
-
- if (cm->intra_only) {
- write_sync_code(wb);
-
-#if CONFIG_MISC_FIXES
- write_bitdepth_colorspace_sampling(cm, wb);
-#else
- // Note for profile 0, 420 8bpp is assumed.
- if (cm->profile > PROFILE_0) {
- write_bitdepth_colorspace_sampling(cm, wb);
- }
-#endif
-
- vpx_wb_write_literal(wb, get_refresh_mask(cpi), REF_FRAMES);
- write_frame_size(cm, wb);
- } else {
- MV_REFERENCE_FRAME ref_frame;
- vpx_wb_write_literal(wb, get_refresh_mask(cpi), REF_FRAMES);
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ++ref_frame) {
- assert(get_ref_frame_map_idx(cpi, ref_frame) != INVALID_IDX);
- vpx_wb_write_literal(wb, get_ref_frame_map_idx(cpi, ref_frame),
- REF_FRAMES_LOG2);
- vpx_wb_write_bit(wb, cm->ref_frame_sign_bias[ref_frame]);
- }
-
- write_frame_size_with_refs(cpi, wb);
-
- vpx_wb_write_bit(wb, cm->allow_high_precision_mv);
-
- fix_interp_filter(cm, cpi->td.counts);
- write_interp_filter(cm->interp_filter, wb);
- }
- }
-
- if (!cm->error_resilient_mode) {
- vpx_wb_write_bit(wb,
- cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_OFF);
-#if CONFIG_MISC_FIXES
- if (cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_OFF)
-#endif
- vpx_wb_write_bit(wb, cm->refresh_frame_context !=
- REFRESH_FRAME_CONTEXT_BACKWARD);
- }
-
- vpx_wb_write_literal(wb, cm->frame_context_idx, FRAME_CONTEXTS_LOG2);
-
- encode_loopfilter(&cm->lf, wb);
- encode_quantization(cm, wb);
- encode_segmentation(cm, xd, wb);
-#if CONFIG_MISC_FIXES
- if (!cm->seg.enabled && xd->lossless[0])
- cm->tx_mode = TX_4X4;
- else
- write_txfm_mode(cm->tx_mode, wb);
- if (cpi->allow_comp_inter_inter) {
- const int use_hybrid_pred = cm->reference_mode == REFERENCE_MODE_SELECT;
- const int use_compound_pred = cm->reference_mode != SINGLE_REFERENCE;
-
- vpx_wb_write_bit(wb, use_hybrid_pred);
- if (!use_hybrid_pred)
- vpx_wb_write_bit(wb, use_compound_pred);
- }
-#endif
-
- write_tile_info(cm, wb);
-}
-
-static size_t write_compressed_header(VP10_COMP *cpi, uint8_t *data) {
- VP10_COMMON *const cm = &cpi->common;
- FRAME_CONTEXT *const fc = cm->fc;
- FRAME_COUNTS *counts = cpi->td.counts;
- vpx_writer header_bc;
- int i;
-#if CONFIG_MISC_FIXES
- int j;
-#endif
-
- vpx_start_encode(&header_bc, data);
-
-#if !CONFIG_MISC_FIXES
- if (cpi->td.mb.e_mbd.lossless[0])
- cm->tx_mode = TX_4X4;
- else
- update_txfm_probs(cm, &header_bc, counts);
-#else
- update_txfm_probs(cm, &header_bc, counts);
-#endif
- update_coef_probs(cpi, &header_bc);
- update_skip_probs(cm, &header_bc, counts);
-#if CONFIG_MISC_FIXES
- update_seg_probs(cpi, &header_bc);
-
- for (i = 0; i < INTRA_MODES; ++i)
- prob_diff_update(vp10_intra_mode_tree, fc->uv_mode_prob[i],
- counts->uv_mode[i], INTRA_MODES, &header_bc);
-
- for (i = 0; i < PARTITION_CONTEXTS; ++i)
- prob_diff_update(vp10_partition_tree, fc->partition_prob[i],
- counts->partition[i], PARTITION_TYPES, &header_bc);
-#endif
-
- if (frame_is_intra_only(cm)) {
- vp10_copy(cm->kf_y_prob, vp10_kf_y_mode_prob);
-#if CONFIG_MISC_FIXES
- for (i = 0; i < INTRA_MODES; ++i)
- for (j = 0; j < INTRA_MODES; ++j)
- prob_diff_update(vp10_intra_mode_tree, cm->kf_y_prob[i][j],
- counts->kf_y_mode[i][j], INTRA_MODES, &header_bc);
-#endif
- } else {
- for (i = 0; i < INTER_MODE_CONTEXTS; ++i)
- prob_diff_update(vp10_inter_mode_tree, cm->fc->inter_mode_probs[i],
- counts->inter_mode[i], INTER_MODES, &header_bc);
-
- if (cm->interp_filter == SWITCHABLE)
- update_switchable_interp_probs(cm, &header_bc, counts);
-
- for (i = 0; i < INTRA_INTER_CONTEXTS; i++)
- vp10_cond_prob_diff_update(&header_bc, &fc->intra_inter_prob[i],
- counts->intra_inter[i]);
-
- if (cpi->allow_comp_inter_inter) {
- const int use_hybrid_pred = cm->reference_mode == REFERENCE_MODE_SELECT;
-#if !CONFIG_MISC_FIXES
- const int use_compound_pred = cm->reference_mode != SINGLE_REFERENCE;
-
- vpx_write_bit(&header_bc, use_compound_pred);
- if (use_compound_pred) {
- vpx_write_bit(&header_bc, use_hybrid_pred);
- if (use_hybrid_pred)
- for (i = 0; i < COMP_INTER_CONTEXTS; i++)
- vp10_cond_prob_diff_update(&header_bc, &fc->comp_inter_prob[i],
- counts->comp_inter[i]);
- }
-#else
- if (use_hybrid_pred)
- for (i = 0; i < COMP_INTER_CONTEXTS; i++)
- vp10_cond_prob_diff_update(&header_bc, &fc->comp_inter_prob[i],
- counts->comp_inter[i]);
-#endif
- }
-
- if (cm->reference_mode != COMPOUND_REFERENCE) {
- for (i = 0; i < REF_CONTEXTS; i++) {
- vp10_cond_prob_diff_update(&header_bc, &fc->single_ref_prob[i][0],
- counts->single_ref[i][0]);
- vp10_cond_prob_diff_update(&header_bc, &fc->single_ref_prob[i][1],
- counts->single_ref[i][1]);
- }
- }
-
- if (cm->reference_mode != SINGLE_REFERENCE)
- for (i = 0; i < REF_CONTEXTS; i++)
- vp10_cond_prob_diff_update(&header_bc, &fc->comp_ref_prob[i],
- counts->comp_ref[i]);
-
- for (i = 0; i < BLOCK_SIZE_GROUPS; ++i)
- prob_diff_update(vp10_intra_mode_tree, cm->fc->y_mode_prob[i],
- counts->y_mode[i], INTRA_MODES, &header_bc);
-
-#if !CONFIG_MISC_FIXES
- for (i = 0; i < PARTITION_CONTEXTS; ++i)
- prob_diff_update(vp10_partition_tree, fc->partition_prob[i],
- counts->partition[i], PARTITION_TYPES, &header_bc);
-#endif
-
- vp10_write_nmv_probs(cm, cm->allow_high_precision_mv, &header_bc,
- &counts->mv);
- }
-
- vpx_stop_encode(&header_bc);
- assert(header_bc.pos <= 0xffff);
-
- return header_bc.pos;
-}
-
-#if CONFIG_MISC_FIXES
-static int remux_tiles(uint8_t *dest, const int sz,
- const int n_tiles, const int mag) {
- int rpos = 0, wpos = 0, n;
-
- for (n = 0; n < n_tiles; n++) {
- int tile_sz;
-
- if (n == n_tiles - 1) {
- tile_sz = sz - rpos;
- } else {
- tile_sz = mem_get_le32(&dest[rpos]) + 1;
- rpos += 4;
- switch (mag) {
- case 0:
- dest[wpos] = tile_sz - 1;
- break;
- case 1:
- mem_put_le16(&dest[wpos], tile_sz - 1);
- break;
- case 2:
- mem_put_le24(&dest[wpos], tile_sz - 1);
- break;
- case 3: // remuxing should only happen if mag < 3
- default:
- assert("Invalid value for tile size magnitude" && 0);
- }
- wpos += mag + 1;
- }
-
- memmove(&dest[wpos], &dest[rpos], tile_sz);
- wpos += tile_sz;
- rpos += tile_sz;
- }
-
- assert(rpos > wpos);
- assert(rpos == sz);
-
- return wpos;
-}
-#endif
-
-void vp10_pack_bitstream(VP10_COMP *const cpi, uint8_t *dest, size_t *size) {
- uint8_t *data = dest;
- size_t first_part_size, uncompressed_hdr_size, data_sz;
- struct vpx_write_bit_buffer wb = {data, 0};
- struct vpx_write_bit_buffer saved_wb;
- unsigned int max_tile;
-#if CONFIG_MISC_FIXES
- VP10_COMMON *const cm = &cpi->common;
- const int n_log2_tiles = cm->log2_tile_rows + cm->log2_tile_cols;
- const int have_tiles = n_log2_tiles > 0;
-#else
- const int have_tiles = 0; // we have tiles, but we don't want to write a
- // tile size marker in the header
-#endif
-
- write_uncompressed_header(cpi, &wb);
- saved_wb = wb;
- // don't know in advance first part. size
- vpx_wb_write_literal(&wb, 0, 16 + have_tiles * 2);
-
- uncompressed_hdr_size = vpx_wb_bytes_written(&wb);
- data += uncompressed_hdr_size;
-
- vpx_clear_system_state();
-
- first_part_size = write_compressed_header(cpi, data);
- data += first_part_size;
-
- data_sz = encode_tiles(cpi, data, &max_tile);
-#if CONFIG_MISC_FIXES
- if (max_tile > 0) {
- int mag;
- unsigned int mask;
-
- // Choose the (tile size) magnitude
- for (mag = 0, mask = 0xff; mag < 4; mag++) {
- if (max_tile <= mask)
- break;
- mask <<= 8;
- mask |= 0xff;
- }
- assert(n_log2_tiles > 0);
- vpx_wb_write_literal(&saved_wb, mag, 2);
- if (mag < 3)
- data_sz = remux_tiles(data, data_sz, 1 << n_log2_tiles, mag);
- } else {
- assert(n_log2_tiles == 0);
- }
-#endif
- data += data_sz;
-
- // TODO(jbb): Figure out what to do if first_part_size > 16 bits.
- vpx_wb_write_literal(&saved_wb, (int)first_part_size, 16);
-
- *size = data - dest;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_BITSTREAM_H_
-#define VP10_ENCODER_BITSTREAM_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#include "vp10/encoder/encoder.h"
-
-void vp10_pack_bitstream(VP10_COMP *const cpi, uint8_t *dest, size_t *size);
-
-static INLINE int vp10_preserve_existing_gf(VP10_COMP *cpi) {
- return !cpi->multi_arf_allowed && cpi->refresh_golden_frame &&
- cpi->rc.is_src_frame_alt_ref;
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_BITSTREAM_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_BLOCK_H_
-#define VP10_ENCODER_BLOCK_H_
-
-#include "vp10/common/entropymv.h"
-#include "vp10/common/entropy.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct {
- unsigned int sse;
- int sum;
- unsigned int var;
-} diff;
-
-struct macroblock_plane {
- DECLARE_ALIGNED(16, int16_t, src_diff[64 * 64]);
- tran_low_t *qcoeff;
- tran_low_t *coeff;
- uint16_t *eobs;
- struct buf_2d src;
-
- // Quantizer setings
- int16_t *quant_fp;
- int16_t *round_fp;
- int16_t *quant;
- int16_t *quant_shift;
- int16_t *zbin;
- int16_t *round;
-
- int64_t quant_thred[2];
-};
-
-/* The [2] dimension is for whether we skip the EOB node (i.e. if previous
- * coefficient in this block was zero) or not. */
-typedef unsigned int vp10_coeff_cost[PLANE_TYPES][REF_TYPES][COEF_BANDS][2]
- [COEFF_CONTEXTS][ENTROPY_TOKENS];
-
-typedef struct {
- int_mv ref_mvs[MAX_REF_FRAMES][MAX_MV_REF_CANDIDATES];
- uint8_t mode_context[MAX_REF_FRAMES];
-} MB_MODE_INFO_EXT;
-
-typedef struct {
- uint8_t best_palette_color_map[4096];
- double kmeans_data_buf[4096];
- uint8_t kmeans_indices_buf[4096];
- uint8_t kmeans_pre_indices_buf[4096];
-} PALETTE_BUFFER;
-
-typedef struct macroblock MACROBLOCK;
-struct macroblock {
- struct macroblock_plane plane[MAX_MB_PLANE];
-
- MACROBLOCKD e_mbd;
- MB_MODE_INFO_EXT *mbmi_ext;
- int skip_block;
- int select_tx_size;
- int skip_recode;
- int skip_optimize;
- int q_index;
-
- int errorperbit;
- int sadperbit16;
- int sadperbit4;
- int rddiv;
- int rdmult;
- int mb_energy;
-
- // These are set to their default values at the beginning, and then adjusted
- // further in the encoding process.
- BLOCK_SIZE min_partition_size;
- BLOCK_SIZE max_partition_size;
-
- int mv_best_ref_index[MAX_REF_FRAMES];
- unsigned int max_mv_context[MAX_REF_FRAMES];
- unsigned int source_variance;
- unsigned int pred_sse[MAX_REF_FRAMES];
- int pred_mv_sad[MAX_REF_FRAMES];
-
- int nmvjointcost[MV_JOINTS];
- int *nmvcost[2];
- int *nmvcost_hp[2];
- int **mvcost;
-
- int nmvjointsadcost[MV_JOINTS];
- int *nmvsadcost[2];
- int *nmvsadcost_hp[2];
- int **mvsadcost;
-
- PALETTE_BUFFER *palette_buffer;
-
- // These define limits to motion vector components to prevent them
- // from extending outside the UMV borders
- int mv_col_min;
- int mv_col_max;
- int mv_row_min;
- int mv_row_max;
-
- // Notes transform blocks where no coefficents are coded.
- // Set during mode selection. Read during block encoding.
- uint8_t zcoeff_blk[TX_SIZES][256];
-
- int skip;
-
- int encode_breakout;
-
- // note that token_costs is the cost when eob node is skipped
- vp10_coeff_cost token_costs[TX_SIZES];
-
- int optimize;
-
- // indicate if it is in the rd search loop or encoding process
- int use_lp32x32fdct;
-
- // use fast quantization process
- int quant_fp;
-
- // skip forward transform and quantization
- uint8_t skip_txfm[MAX_MB_PLANE << 2];
- #define SKIP_TXFM_NONE 0
- #define SKIP_TXFM_AC_DC 1
- #define SKIP_TXFM_AC_ONLY 2
-
- int64_t bsse[MAX_MB_PLANE << 2];
-
- // Used to store sub partition's choices.
- MV pred_mv[MAX_REF_FRAMES];
-
- // Strong color activity detection. Used in RTC coding mode to enhance
- // the visual quality at the boundary of moving color objects.
- uint8_t color_sensitivity[2];
-};
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_BLOCK_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-#include "vp10/common/common.h"
-#include "vp10/common/filter.h"
-#include "vpx/vpx_integer.h"
-#include "vpx_dsp/vpx_convolve.h"
-#include "vpx_dsp/vpx_filter.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-
-static int horizontal_filter(const uint8_t *s) {
- return (s[1] - s[-2]) * 2 + (s[-1] - s[0]) * 6;
-}
-
-static int vertical_filter(const uint8_t *s, int p) {
- return (s[p] - s[-2 * p]) * 2 + (s[-p] - s[0]) * 6;
-}
-
-static int variance(int sum, int sum_squared, int size) {
- return sum_squared / size - (sum / size) * (sum / size);
-}
-// Calculate a blockiness level for a vertical block edge.
-// This function returns a new blockiness metric that's defined as
-
-// p0 p1 p2 p3
-// q0 q1 q2 q3
-// block edge ->
-// r0 r1 r2 r3
-// s0 s1 s2 s3
-
-// blockiness = p0*-2+q0*6+r0*-6+s0*2 +
-// p1*-2+q1*6+r1*-6+s1*2 +
-// p2*-2+q2*6+r2*-6+s2*2 +
-// p3*-2+q3*6+r3*-6+s3*2 ;
-
-// reconstructed_blockiness = abs(blockiness from reconstructed buffer -
-// blockiness from source buffer,0)
-//
-// I make the assumption that flat blocks are much more visible than high
-// contrast blocks. As such, I scale the result of the blockiness calc
-// by dividing the blockiness by the variance of the pixels on either side
-// of the edge as follows:
-// var_0 = (q0^2+q1^2+q2^2+q3^2) - ((q0 + q1 + q2 + q3) / 4 )^2
-// var_1 = (r0^2+r1^2+r2^2+r3^2) - ((r0 + r1 + r2 + r3) / 4 )^2
-// The returned blockiness is the scaled value
-// Reconstructed blockiness / ( 1 + var_0 + var_1 ) ;
-static int blockiness_vertical(const uint8_t *s, int sp, const uint8_t *r,
- int rp, int size) {
- int s_blockiness = 0;
- int r_blockiness = 0;
- int sum_0 = 0;
- int sum_sq_0 = 0;
- int sum_1 = 0;
- int sum_sq_1 = 0;
- int i;
- int var_0;
- int var_1;
- for (i = 0; i < size; ++i, s += sp, r += rp) {
- s_blockiness += horizontal_filter(s);
- r_blockiness += horizontal_filter(r);
- sum_0 += s[0];
- sum_sq_0 += s[0]*s[0];
- sum_1 += s[-1];
- sum_sq_1 += s[-1]*s[-1];
- }
- var_0 = variance(sum_0, sum_sq_0, size);
- var_1 = variance(sum_1, sum_sq_1, size);
- r_blockiness = abs(r_blockiness);
- s_blockiness = abs(s_blockiness);
-
- if (r_blockiness > s_blockiness)
- return (r_blockiness - s_blockiness) / (1 + var_0 + var_1);
- else
- return 0;
-}
-
-// Calculate a blockiness level for a horizontal block edge
-// same as above.
-static int blockiness_horizontal(const uint8_t *s, int sp, const uint8_t *r,
- int rp, int size) {
- int s_blockiness = 0;
- int r_blockiness = 0;
- int sum_0 = 0;
- int sum_sq_0 = 0;
- int sum_1 = 0;
- int sum_sq_1 = 0;
- int i;
- int var_0;
- int var_1;
- for (i = 0; i < size; ++i, ++s, ++r) {
- s_blockiness += vertical_filter(s, sp);
- r_blockiness += vertical_filter(r, rp);
- sum_0 += s[0];
- sum_sq_0 += s[0] * s[0];
- sum_1 += s[-sp];
- sum_sq_1 += s[-sp] * s[-sp];
- }
- var_0 = variance(sum_0, sum_sq_0, size);
- var_1 = variance(sum_1, sum_sq_1, size);
- r_blockiness = abs(r_blockiness);
- s_blockiness = abs(s_blockiness);
-
- if (r_blockiness > s_blockiness)
- return (r_blockiness - s_blockiness) / (1 + var_0 + var_1);
- else
- return 0;
-}
-
-// This function returns the blockiness for the entire frame currently by
-// looking at all borders in steps of 4.
-double vp10_get_blockiness(const unsigned char *img1, int img1_pitch,
- const unsigned char *img2, int img2_pitch,
- int width, int height ) {
- double blockiness = 0;
- int i, j;
- vpx_clear_system_state();
- for (i = 0; i < height; i += 4, img1 += img1_pitch * 4,
- img2 += img2_pitch * 4) {
- for (j = 0; j < width; j += 4) {
- if (i > 0 && i < height && j > 0 && j < width) {
- blockiness += blockiness_vertical(img1 + j, img1_pitch,
- img2 + j, img2_pitch, 4);
- blockiness += blockiness_horizontal(img1 + j, img1_pitch,
- img2 + j, img2_pitch, 4);
- }
- }
- }
- blockiness /= width * height / 16;
- return blockiness;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/encoder/context_tree.h"
-#include "vp10/encoder/encoder.h"
-
-static const BLOCK_SIZE square[] = {
- BLOCK_8X8,
- BLOCK_16X16,
- BLOCK_32X32,
- BLOCK_64X64,
-};
-
-static void alloc_mode_context(VP10_COMMON *cm, int num_4x4_blk,
- PICK_MODE_CONTEXT *ctx) {
- const int num_blk = (num_4x4_blk < 4 ? 4 : num_4x4_blk);
- const int num_pix = num_blk << 4;
- int i, k;
- ctx->num_4x4_blk = num_blk;
-
- CHECK_MEM_ERROR(cm, ctx->zcoeff_blk,
- vpx_calloc(num_blk, sizeof(uint8_t)));
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- for (k = 0; k < 3; ++k) {
- CHECK_MEM_ERROR(cm, ctx->coeff[i][k],
- vpx_memalign(32, num_pix * sizeof(*ctx->coeff[i][k])));
- CHECK_MEM_ERROR(cm, ctx->qcoeff[i][k],
- vpx_memalign(32, num_pix * sizeof(*ctx->qcoeff[i][k])));
- CHECK_MEM_ERROR(cm, ctx->dqcoeff[i][k],
- vpx_memalign(32, num_pix * sizeof(*ctx->dqcoeff[i][k])));
- CHECK_MEM_ERROR(cm, ctx->eobs[i][k],
- vpx_memalign(32, num_blk * sizeof(*ctx->eobs[i][k])));
- ctx->coeff_pbuf[i][k] = ctx->coeff[i][k];
- ctx->qcoeff_pbuf[i][k] = ctx->qcoeff[i][k];
- ctx->dqcoeff_pbuf[i][k] = ctx->dqcoeff[i][k];
- ctx->eobs_pbuf[i][k] = ctx->eobs[i][k];
- }
- }
-}
-
-static void free_mode_context(PICK_MODE_CONTEXT *ctx) {
- int i, k;
- vpx_free(ctx->zcoeff_blk);
- ctx->zcoeff_blk = 0;
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- for (k = 0; k < 3; ++k) {
- vpx_free(ctx->coeff[i][k]);
- ctx->coeff[i][k] = 0;
- vpx_free(ctx->qcoeff[i][k]);
- ctx->qcoeff[i][k] = 0;
- vpx_free(ctx->dqcoeff[i][k]);
- ctx->dqcoeff[i][k] = 0;
- vpx_free(ctx->eobs[i][k]);
- ctx->eobs[i][k] = 0;
- }
- }
-
- for (i = 0; i < 2; ++i) {
- vpx_free(ctx->color_index_map[i]);
- ctx->color_index_map[i] = 0;
- }
-}
-
-static void alloc_tree_contexts(VP10_COMMON *cm, PC_TREE *tree,
- int num_4x4_blk) {
- alloc_mode_context(cm, num_4x4_blk, &tree->none);
- alloc_mode_context(cm, num_4x4_blk/2, &tree->horizontal[0]);
- alloc_mode_context(cm, num_4x4_blk/2, &tree->vertical[0]);
-
- if (num_4x4_blk > 4) {
- alloc_mode_context(cm, num_4x4_blk/2, &tree->horizontal[1]);
- alloc_mode_context(cm, num_4x4_blk/2, &tree->vertical[1]);
- } else {
- memset(&tree->horizontal[1], 0, sizeof(tree->horizontal[1]));
- memset(&tree->vertical[1], 0, sizeof(tree->vertical[1]));
- }
-}
-
-static void free_tree_contexts(PC_TREE *tree) {
- free_mode_context(&tree->none);
- free_mode_context(&tree->horizontal[0]);
- free_mode_context(&tree->horizontal[1]);
- free_mode_context(&tree->vertical[0]);
- free_mode_context(&tree->vertical[1]);
-}
-
-// This function sets up a tree of contexts such that at each square
-// partition level. There are contexts for none, horizontal, vertical, and
-// split. Along with a block_size value and a selected block_size which
-// represents the state of our search.
-void vp10_setup_pc_tree(VP10_COMMON *cm, ThreadData *td) {
- int i, j;
- const int leaf_nodes = 64;
- const int tree_nodes = 64 + 16 + 4 + 1;
- int pc_tree_index = 0;
- PC_TREE *this_pc;
- PICK_MODE_CONTEXT *this_leaf;
- int square_index = 1;
- int nodes;
-
- vpx_free(td->leaf_tree);
- CHECK_MEM_ERROR(cm, td->leaf_tree, vpx_calloc(leaf_nodes,
- sizeof(*td->leaf_tree)));
- vpx_free(td->pc_tree);
- CHECK_MEM_ERROR(cm, td->pc_tree, vpx_calloc(tree_nodes,
- sizeof(*td->pc_tree)));
-
- this_pc = &td->pc_tree[0];
- this_leaf = &td->leaf_tree[0];
-
- // 4x4 blocks smaller than 8x8 but in the same 8x8 block share the same
- // context so we only need to allocate 1 for each 8x8 block.
- for (i = 0; i < leaf_nodes; ++i)
- alloc_mode_context(cm, 1, &td->leaf_tree[i]);
-
- // Sets up all the leaf nodes in the tree.
- for (pc_tree_index = 0; pc_tree_index < leaf_nodes; ++pc_tree_index) {
- PC_TREE *const tree = &td->pc_tree[pc_tree_index];
- tree->block_size = square[0];
- alloc_tree_contexts(cm, tree, 4);
- tree->leaf_split[0] = this_leaf++;
- for (j = 1; j < 4; j++)
- tree->leaf_split[j] = tree->leaf_split[0];
- }
-
- // Each node has 4 leaf nodes, fill each block_size level of the tree
- // from leafs to the root.
- for (nodes = 16; nodes > 0; nodes >>= 2) {
- for (i = 0; i < nodes; ++i) {
- PC_TREE *const tree = &td->pc_tree[pc_tree_index];
- alloc_tree_contexts(cm, tree, 4 << (2 * square_index));
- tree->block_size = square[square_index];
- for (j = 0; j < 4; j++)
- tree->split[j] = this_pc++;
- ++pc_tree_index;
- }
- ++square_index;
- }
- td->pc_root = &td->pc_tree[tree_nodes - 1];
- td->pc_root[0].none.best_mode_index = 2;
-}
-
-void vp10_free_pc_tree(ThreadData *td) {
- const int tree_nodes = 64 + 16 + 4 + 1;
- int i;
-
- // Set up all 4x4 mode contexts
- for (i = 0; i < 64; ++i)
- free_mode_context(&td->leaf_tree[i]);
-
- // Sets up all the leaf nodes in the tree.
- for (i = 0; i < tree_nodes; ++i)
- free_tree_contexts(&td->pc_tree[i]);
-
- vpx_free(td->pc_tree);
- td->pc_tree = NULL;
- vpx_free(td->leaf_tree);
- td->leaf_tree = NULL;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_CONTEXT_TREE_H_
-#define VP10_ENCODER_CONTEXT_TREE_H_
-
-#include "vp10/common/blockd.h"
-#include "vp10/encoder/block.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10_COMP;
-struct VP10Common;
-struct ThreadData;
-
-// Structure to hold snapshot of coding context during the mode picking process
-typedef struct {
- MODE_INFO mic;
- MB_MODE_INFO_EXT mbmi_ext;
- uint8_t *zcoeff_blk;
- uint8_t *color_index_map[2];
- tran_low_t *coeff[MAX_MB_PLANE][3];
- tran_low_t *qcoeff[MAX_MB_PLANE][3];
- tran_low_t *dqcoeff[MAX_MB_PLANE][3];
- uint16_t *eobs[MAX_MB_PLANE][3];
-
- // dual buffer pointers, 0: in use, 1: best in store
- tran_low_t *coeff_pbuf[MAX_MB_PLANE][3];
- tran_low_t *qcoeff_pbuf[MAX_MB_PLANE][3];
- tran_low_t *dqcoeff_pbuf[MAX_MB_PLANE][3];
- uint16_t *eobs_pbuf[MAX_MB_PLANE][3];
-
- int is_coded;
- int num_4x4_blk;
- int skip;
- int pred_pixel_ready;
- // For current partition, only if all Y, U, and V transform blocks'
- // coefficients are quantized to 0, skippable is set to 0.
- int skippable;
- uint8_t skip_txfm[MAX_MB_PLANE << 2];
- int best_mode_index;
- int hybrid_pred_diff;
- int comp_pred_diff;
- int single_pred_diff;
- int64_t best_filter_diff[SWITCHABLE_FILTER_CONTEXTS];
-
- // TODO(jingning) Use RD_COST struct here instead. This involves a boarder
- // scope of refactoring.
- int rate;
- int64_t dist;
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
- unsigned int newmv_sse;
- unsigned int zeromv_sse;
- PREDICTION_MODE best_sse_inter_mode;
- int_mv best_sse_mv;
- MV_REFERENCE_FRAME best_reference_frame;
- MV_REFERENCE_FRAME best_zeromv_reference_frame;
-#endif
-
- // motion vector cache for adaptive motion search control in partition
- // search loop
- MV pred_mv[MAX_REF_FRAMES];
- INTERP_FILTER pred_interp_filter;
-} PICK_MODE_CONTEXT;
-
-typedef struct PC_TREE {
- int index;
- PARTITION_TYPE partitioning;
- BLOCK_SIZE block_size;
- PICK_MODE_CONTEXT none;
- PICK_MODE_CONTEXT horizontal[2];
- PICK_MODE_CONTEXT vertical[2];
- union {
- struct PC_TREE *split[4];
- PICK_MODE_CONTEXT *leaf_split[4];
- };
-} PC_TREE;
-
-void vp10_setup_pc_tree(struct VP10Common *cm, struct ThreadData *td);
-void vp10_free_pc_tree(struct ThreadData *td);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif /* VP10_ENCODER_CONTEXT_TREE_H_ */
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#include <assert.h>
-
-#include "vp10/encoder/cost.h"
-
-const unsigned int vp10_prob_cost[256] = {
- 2047, 2047, 1791, 1641, 1535, 1452, 1385, 1328, 1279, 1235, 1196, 1161,
- 1129, 1099, 1072, 1046, 1023, 1000, 979, 959, 940, 922, 905, 889,
- 873, 858, 843, 829, 816, 803, 790, 778, 767, 755, 744, 733,
- 723, 713, 703, 693, 684, 675, 666, 657, 649, 641, 633, 625,
- 617, 609, 602, 594, 587, 580, 573, 567, 560, 553, 547, 541,
- 534, 528, 522, 516, 511, 505, 499, 494, 488, 483, 477, 472,
- 467, 462, 457, 452, 447, 442, 437, 433, 428, 424, 419, 415,
- 410, 406, 401, 397, 393, 389, 385, 381, 377, 373, 369, 365,
- 361, 357, 353, 349, 346, 342, 338, 335, 331, 328, 324, 321,
- 317, 314, 311, 307, 304, 301, 297, 294, 291, 288, 285, 281,
- 278, 275, 272, 269, 266, 263, 260, 257, 255, 252, 249, 246,
- 243, 240, 238, 235, 232, 229, 227, 224, 221, 219, 216, 214,
- 211, 208, 206, 203, 201, 198, 196, 194, 191, 189, 186, 184,
- 181, 179, 177, 174, 172, 170, 168, 165, 163, 161, 159, 156,
- 154, 152, 150, 148, 145, 143, 141, 139, 137, 135, 133, 131,
- 129, 127, 125, 123, 121, 119, 117, 115, 113, 111, 109, 107,
- 105, 103, 101, 99, 97, 95, 93, 92, 90, 88, 86, 84,
- 82, 81, 79, 77, 75, 73, 72, 70, 68, 66, 65, 63,
- 61, 60, 58, 56, 55, 53, 51, 50, 48, 46, 45, 43,
- 41, 40, 38, 37, 35, 33, 32, 30, 29, 27, 25, 24,
- 22, 21, 19, 18, 16, 15, 13, 12, 10, 9, 7, 6,
- 4, 3, 1, 1};
-
-static void cost(int *costs, vpx_tree tree, const vpx_prob *probs,
- int i, int c) {
- const vpx_prob prob = probs[i / 2];
- int b;
-
- for (b = 0; b <= 1; ++b) {
- const int cc = c + vp10_cost_bit(prob, b);
- const vpx_tree_index ii = tree[i + b];
-
- if (ii <= 0)
- costs[-ii] = cc;
- else
- cost(costs, tree, probs, ii, cc);
- }
-}
-
-void vp10_cost_tokens(int *costs, const vpx_prob *probs, vpx_tree tree) {
- cost(costs, tree, probs, 0, 0);
-}
-
-void vp10_cost_tokens_skip(int *costs, const vpx_prob *probs, vpx_tree tree) {
- assert(tree[0] <= 0 && tree[1] > 0);
-
- costs[-tree[0]] = vp10_cost_bit(probs[0], 0);
- cost(costs, tree, probs, 2, 0);
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_COST_H_
-#define VP10_ENCODER_COST_H_
-
-#include "vpx_dsp/prob.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-extern const unsigned int vp10_prob_cost[256];
-
-#define vp10_cost_zero(prob) (vp10_prob_cost[prob])
-
-#define vp10_cost_one(prob) vp10_cost_zero(vpx_complement(prob))
-
-#define vp10_cost_bit(prob, bit) vp10_cost_zero((bit) ? vpx_complement(prob) \
- : (prob))
-
-static INLINE unsigned int cost_branch256(const unsigned int ct[2],
- vpx_prob p) {
- return ct[0] * vp10_cost_zero(p) + ct[1] * vp10_cost_one(p);
-}
-
-static INLINE int treed_cost(vpx_tree tree, const vpx_prob *probs,
- int bits, int len) {
- int cost = 0;
- vpx_tree_index i = 0;
-
- do {
- const int bit = (bits >> --len) & 1;
- cost += vp10_cost_bit(probs[i >> 1], bit);
- i = tree[i + bit];
- } while (len);
-
- return cost;
-}
-
-void vp10_cost_tokens(int *costs, const vpx_prob *probs, vpx_tree tree);
-void vp10_cost_tokens_skip(int *costs, const vpx_prob *probs, vpx_tree tree);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_COST_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <math.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-
-#include "vp10/common/blockd.h"
-#include "vp10/common/idct.h"
-#include "vpx_dsp/fwd_txfm.h"
-#include "vpx_ports/mem.h"
-
-static INLINE void range_check(const tran_low_t *input, const int size,
- const int bit) {
-#if CONFIG_COEFFICIENT_RANGE_CHECKING
- int i;
- for (i = 0; i < size; ++i) {
- assert(abs(input[i]) < (1 << bit));
- }
-#else
- (void)input;
- (void)size;
- (void)bit;
-#endif
-}
-
-static void fdct4(const tran_low_t *input, tran_low_t *output) {
- tran_high_t temp;
- tran_low_t step[4];
-
- // stage 0
- range_check(input, 4, 11);
-
- // stage 1
- output[0] = input[0] + input[3];
- output[1] = input[1] + input[2];
- output[2] = input[1] - input[2];
- output[3] = input[0] - input[3];
-
- range_check(output, 4, 12);
-
- // stage 2
- temp = output[0] * cospi_16_64 + output[1] * cospi_16_64;
- step[0] = (tran_low_t)fdct_round_shift(temp);
- temp = output[1] * -cospi_16_64 + output[0] * cospi_16_64;
- step[1] = (tran_low_t)fdct_round_shift(temp);
- temp = output[2] * cospi_24_64 + output[3] * cospi_8_64;
- step[2] = (tran_low_t)fdct_round_shift(temp);
- temp = output[3] * cospi_24_64 + output[2] * -cospi_8_64;
- step[3] = (tran_low_t)fdct_round_shift(temp);
-
- range_check(step, 4, 13);
-
- // stage 3
- output[0] = step[0];
- output[1] = step[2];
- output[2] = step[1];
- output[3] = step[3];
-
- range_check(output, 4, 13);
-}
-
-static void fdct8(const tran_low_t *input, tran_low_t *output) {
- tran_high_t temp;
- tran_low_t step[8];
-
- // stage 0
- range_check(input, 8, 12);
-
- // stage 1
- output[0] = input[0] + input[7];
- output[1] = input[1] + input[6];
- output[2] = input[2] + input[5];
- output[3] = input[3] + input[4];
- output[4] = input[3] - input[4];
- output[5] = input[2] - input[5];
- output[6] = input[1] - input[6];
- output[7] = input[0] - input[7];
-
- range_check(output, 8, 13);
-
- // stage 2
- step[0] = output[0] + output[3];
- step[1] = output[1] + output[2];
- step[2] = output[1] - output[2];
- step[3] = output[0] - output[3];
- step[4] = output[4];
- temp = output[5] * -cospi_16_64 + output[6] * cospi_16_64;
- step[5] = (tran_low_t)fdct_round_shift(temp);
- temp = output[6] * cospi_16_64 + output[5] * cospi_16_64;
- step[6] = (tran_low_t)fdct_round_shift(temp);
- step[7] = output[7];
-
- range_check(step, 8, 14);
-
- // stage 3
- temp = step[0] * cospi_16_64 + step[1] * cospi_16_64;
- output[0] = (tran_low_t)fdct_round_shift(temp);
- temp = step[1] * -cospi_16_64 + step[0] * cospi_16_64;
- output[1] = (tran_low_t)fdct_round_shift(temp);
- temp = step[2] * cospi_24_64 + step[3] * cospi_8_64;
- output[2] = (tran_low_t)fdct_round_shift(temp);
- temp = step[3] * cospi_24_64 + step[2] * -cospi_8_64;
- output[3] = (tran_low_t)fdct_round_shift(temp);
- output[4] = step[4] + step[5];
- output[5] = step[4] - step[5];
- output[6] = step[7] - step[6];
- output[7] = step[7] + step[6];
-
- range_check(output, 8, 14);
-
- // stage 4
- step[0] = output[0];
- step[1] = output[1];
- step[2] = output[2];
- step[3] = output[3];
- temp = output[4] * cospi_28_64 + output[7] * cospi_4_64;
- step[4] = (tran_low_t)fdct_round_shift(temp);
- temp = output[5] * cospi_12_64 + output[6] * cospi_20_64;
- step[5] = (tran_low_t)fdct_round_shift(temp);
- temp = output[6] * cospi_12_64 + output[5] * -cospi_20_64;
- step[6] = (tran_low_t)fdct_round_shift(temp);
- temp = output[7] * cospi_28_64 + output[4] * -cospi_4_64;
- step[7] = (tran_low_t)fdct_round_shift(temp);
-
- range_check(step, 8, 14);
-
- // stage 5
- output[0] = step[0];
- output[1] = step[4];
- output[2] = step[2];
- output[3] = step[6];
- output[4] = step[1];
- output[5] = step[5];
- output[6] = step[3];
- output[7] = step[7];
-
- range_check(output, 8, 14);
-}
-
-static void fdct16(const tran_low_t *input, tran_low_t *output) {
- tran_high_t temp;
- tran_low_t step[16];
-
- // stage 0
- range_check(input, 16, 13);
-
- // stage 1
- output[0] = input[0] + input[15];
- output[1] = input[1] + input[14];
- output[2] = input[2] + input[13];
- output[3] = input[3] + input[12];
- output[4] = input[4] + input[11];
- output[5] = input[5] + input[10];
- output[6] = input[6] + input[9];
- output[7] = input[7] + input[8];
- output[8] = input[7] - input[8];
- output[9] = input[6] - input[9];
- output[10] = input[5] - input[10];
- output[11] = input[4] - input[11];
- output[12] = input[3] - input[12];
- output[13] = input[2] - input[13];
- output[14] = input[1] - input[14];
- output[15] = input[0] - input[15];
-
- range_check(output, 16, 14);
-
- // stage 2
- step[0] = output[0] + output[7];
- step[1] = output[1] + output[6];
- step[2] = output[2] + output[5];
- step[3] = output[3] + output[4];
- step[4] = output[3] - output[4];
- step[5] = output[2] - output[5];
- step[6] = output[1] - output[6];
- step[7] = output[0] - output[7];
- step[8] = output[8];
- step[9] = output[9];
- temp = output[10] * -cospi_16_64 + output[13] * cospi_16_64;
- step[10] = (tran_low_t)fdct_round_shift(temp);
- temp = output[11] * -cospi_16_64 + output[12] * cospi_16_64;
- step[11] = (tran_low_t)fdct_round_shift(temp);
- temp = output[12] * cospi_16_64 + output[11] * cospi_16_64;
- step[12] = (tran_low_t)fdct_round_shift(temp);
- temp = output[13] * cospi_16_64 + output[10] * cospi_16_64;
- step[13] = (tran_low_t)fdct_round_shift(temp);
- step[14] = output[14];
- step[15] = output[15];
-
- range_check(step, 16, 15);
-
- // stage 3
- output[0] = step[0] + step[3];
- output[1] = step[1] + step[2];
- output[2] = step[1] - step[2];
- output[3] = step[0] - step[3];
- output[4] = step[4];
- temp = step[5] * -cospi_16_64 + step[6] * cospi_16_64;
- output[5] = (tran_low_t)fdct_round_shift(temp);
- temp = step[6] * cospi_16_64 + step[5] * cospi_16_64;
- output[6] = (tran_low_t)fdct_round_shift(temp);
- output[7] = step[7];
- output[8] = step[8] + step[11];
- output[9] = step[9] + step[10];
- output[10] = step[9] - step[10];
- output[11] = step[8] - step[11];
- output[12] = step[15] - step[12];
- output[13] = step[14] - step[13];
- output[14] = step[14] + step[13];
- output[15] = step[15] + step[12];
-
- range_check(output, 16, 16);
-
- // stage 4
- temp = output[0] * cospi_16_64 + output[1] * cospi_16_64;
- step[0] = (tran_low_t)fdct_round_shift(temp);
- temp = output[1] * -cospi_16_64 + output[0] * cospi_16_64;
- step[1] = (tran_low_t)fdct_round_shift(temp);
- temp = output[2] * cospi_24_64 + output[3] * cospi_8_64;
- step[2] = (tran_low_t)fdct_round_shift(temp);
- temp = output[3] * cospi_24_64 + output[2] * -cospi_8_64;
- step[3] = (tran_low_t)fdct_round_shift(temp);
- step[4] = output[4] + output[5];
- step[5] = output[4] - output[5];
- step[6] = output[7] - output[6];
- step[7] = output[7] + output[6];
- step[8] = output[8];
- temp = output[9] * -cospi_8_64 + output[14] * cospi_24_64;
- step[9] = (tran_low_t)fdct_round_shift(temp);
- temp = output[10] * -cospi_24_64 + output[13] * -cospi_8_64;
- step[10] = (tran_low_t)fdct_round_shift(temp);
- step[11] = output[11];
- step[12] = output[12];
- temp = output[13] * cospi_24_64 + output[10] * -cospi_8_64;
- step[13] = (tran_low_t)fdct_round_shift(temp);
- temp = output[14] * cospi_8_64 + output[9] * cospi_24_64;
- step[14] = (tran_low_t)fdct_round_shift(temp);
- step[15] = output[15];
-
- range_check(step, 16, 16);
-
- // stage 5
- output[0] = step[0];
- output[1] = step[1];
- output[2] = step[2];
- output[3] = step[3];
- temp = step[4] * cospi_28_64 + step[7] * cospi_4_64;
- output[4] = (tran_low_t)fdct_round_shift(temp);
- temp = step[5] * cospi_12_64 + step[6] * cospi_20_64;
- output[5] = (tran_low_t)fdct_round_shift(temp);
- temp = step[6] * cospi_12_64 + step[5] * -cospi_20_64;
- output[6] = (tran_low_t)fdct_round_shift(temp);
- temp = step[7] * cospi_28_64 + step[4] * -cospi_4_64;
- output[7] = (tran_low_t)fdct_round_shift(temp);
- output[8] = step[8] + step[9];
- output[9] = step[8] - step[9];
- output[10] = step[11] - step[10];
- output[11] = step[11] + step[10];
- output[12] = step[12] + step[13];
- output[13] = step[12] - step[13];
- output[14] = step[15] - step[14];
- output[15] = step[15] + step[14];
-
- range_check(output, 16, 16);
-
- // stage 6
- step[0] = output[0];
- step[1] = output[1];
- step[2] = output[2];
- step[3] = output[3];
- step[4] = output[4];
- step[5] = output[5];
- step[6] = output[6];
- step[7] = output[7];
- temp = output[8] * cospi_30_64 + output[15] * cospi_2_64;
- step[8] = (tran_low_t)fdct_round_shift(temp);
- temp = output[9] * cospi_14_64 + output[14] * cospi_18_64;
- step[9] = (tran_low_t)fdct_round_shift(temp);
- temp = output[10] * cospi_22_64 + output[13] * cospi_10_64;
- step[10] = (tran_low_t)fdct_round_shift(temp);
- temp = output[11] * cospi_6_64 + output[12] * cospi_26_64;
- step[11] = (tran_low_t)fdct_round_shift(temp);
- temp = output[12] * cospi_6_64 + output[11] * -cospi_26_64;
- step[12] = (tran_low_t)fdct_round_shift(temp);
- temp = output[13] * cospi_22_64 + output[10] * -cospi_10_64;
- step[13] = (tran_low_t)fdct_round_shift(temp);
- temp = output[14] * cospi_14_64 + output[9] * -cospi_18_64;
- step[14] = (tran_low_t)fdct_round_shift(temp);
- temp = output[15] * cospi_30_64 + output[8] * -cospi_2_64;
- step[15] = (tran_low_t)fdct_round_shift(temp);
-
- range_check(step, 16, 16);
-
- // stage 7
- output[0] = step[0];
- output[1] = step[8];
- output[2] = step[4];
- output[3] = step[12];
- output[4] = step[2];
- output[5] = step[10];
- output[6] = step[6];
- output[7] = step[14];
- output[8] = step[1];
- output[9] = step[9];
- output[10] = step[5];
- output[11] = step[13];
- output[12] = step[3];
- output[13] = step[11];
- output[14] = step[7];
- output[15] = step[15];
-
- range_check(output, 16, 16);
-}
-
-/* #TODO(angiebird): Unify this with vp10_fwd_txfm.c: vp10_fdct32
-static void fdct32(const tran_low_t *input, tran_low_t *output) {
- tran_high_t temp;
- tran_low_t step[32];
-
- // stage 0
- range_check(input, 32, 14);
-
- // stage 1
- output[0] = input[0] + input[31];
- output[1] = input[1] + input[30];
- output[2] = input[2] + input[29];
- output[3] = input[3] + input[28];
- output[4] = input[4] + input[27];
- output[5] = input[5] + input[26];
- output[6] = input[6] + input[25];
- output[7] = input[7] + input[24];
- output[8] = input[8] + input[23];
- output[9] = input[9] + input[22];
- output[10] = input[10] + input[21];
- output[11] = input[11] + input[20];
- output[12] = input[12] + input[19];
- output[13] = input[13] + input[18];
- output[14] = input[14] + input[17];
- output[15] = input[15] + input[16];
- output[16] = input[15] - input[16];
- output[17] = input[14] - input[17];
- output[18] = input[13] - input[18];
- output[19] = input[12] - input[19];
- output[20] = input[11] - input[20];
- output[21] = input[10] - input[21];
- output[22] = input[9] - input[22];
- output[23] = input[8] - input[23];
- output[24] = input[7] - input[24];
- output[25] = input[6] - input[25];
- output[26] = input[5] - input[26];
- output[27] = input[4] - input[27];
- output[28] = input[3] - input[28];
- output[29] = input[2] - input[29];
- output[30] = input[1] - input[30];
- output[31] = input[0] - input[31];
-
- range_check(output, 32, 15);
-
- // stage 2
- step[0] = output[0] + output[15];
- step[1] = output[1] + output[14];
- step[2] = output[2] + output[13];
- step[3] = output[3] + output[12];
- step[4] = output[4] + output[11];
- step[5] = output[5] + output[10];
- step[6] = output[6] + output[9];
- step[7] = output[7] + output[8];
- step[8] = output[7] - output[8];
- step[9] = output[6] - output[9];
- step[10] = output[5] - output[10];
- step[11] = output[4] - output[11];
- step[12] = output[3] - output[12];
- step[13] = output[2] - output[13];
- step[14] = output[1] - output[14];
- step[15] = output[0] - output[15];
- step[16] = output[16];
- step[17] = output[17];
- step[18] = output[18];
- step[19] = output[19];
- temp = output[20] * -cospi_16_64 + output[27] * cospi_16_64;
- step[20] = (tran_low_t)fdct_round_shift(temp);
- temp = output[21] * -cospi_16_64 + output[26] * cospi_16_64;
- step[21] = (tran_low_t)fdct_round_shift(temp);
- temp = output[22] * -cospi_16_64 + output[25] * cospi_16_64;
- step[22] = (tran_low_t)fdct_round_shift(temp);
- temp = output[23] * -cospi_16_64 + output[24] * cospi_16_64;
- step[23] = (tran_low_t)fdct_round_shift(temp);
- temp = output[24] * cospi_16_64 + output[23] * cospi_16_64;
- step[24] = (tran_low_t)fdct_round_shift(temp);
- temp = output[25] * cospi_16_64 + output[22] * cospi_16_64;
- step[25] = (tran_low_t)fdct_round_shift(temp);
- temp = output[26] * cospi_16_64 + output[21] * cospi_16_64;
- step[26] = (tran_low_t)fdct_round_shift(temp);
- temp = output[27] * cospi_16_64 + output[20] * cospi_16_64;
- step[27] = (tran_low_t)fdct_round_shift(temp);
- step[28] = output[28];
- step[29] = output[29];
- step[30] = output[30];
- step[31] = output[31];
-
- range_check(step, 32, 16);
-
- // stage 3
- output[0] = step[0] + step[7];
- output[1] = step[1] + step[6];
- output[2] = step[2] + step[5];
- output[3] = step[3] + step[4];
- output[4] = step[3] - step[4];
- output[5] = step[2] - step[5];
- output[6] = step[1] - step[6];
- output[7] = step[0] - step[7];
- output[8] = step[8];
- output[9] = step[9];
- temp = step[10] * -cospi_16_64 + step[13] * cospi_16_64;
- output[10] = (tran_low_t)fdct_round_shift(temp);
- temp = step[11] * -cospi_16_64 + step[12] * cospi_16_64;
- output[11] = (tran_low_t)fdct_round_shift(temp);
- temp = step[12] * cospi_16_64 + step[11] * cospi_16_64;
- output[12] = (tran_low_t)fdct_round_shift(temp);
- temp = step[13] * cospi_16_64 + step[10] * cospi_16_64;
- output[13] = (tran_low_t)fdct_round_shift(temp);
- output[14] = step[14];
- output[15] = step[15];
- output[16] = step[16] + step[23];
- output[17] = step[17] + step[22];
- output[18] = step[18] + step[21];
- output[19] = step[19] + step[20];
- output[20] = step[19] - step[20];
- output[21] = step[18] - step[21];
- output[22] = step[17] - step[22];
- output[23] = step[16] - step[23];
- output[24] = step[31] - step[24];
- output[25] = step[30] - step[25];
- output[26] = step[29] - step[26];
- output[27] = step[28] - step[27];
- output[28] = step[28] + step[27];
- output[29] = step[29] + step[26];
- output[30] = step[30] + step[25];
- output[31] = step[31] + step[24];
-
- range_check(output, 32, 17);
-
- // stage 4
- step[0] = output[0] + output[3];
- step[1] = output[1] + output[2];
- step[2] = output[1] - output[2];
- step[3] = output[0] - output[3];
- step[4] = output[4];
- temp = output[5] * -cospi_16_64 + output[6] * cospi_16_64;
- step[5] = (tran_low_t)fdct_round_shift(temp);
- temp = output[6] * cospi_16_64 + output[5] * cospi_16_64;
- step[6] = (tran_low_t)fdct_round_shift(temp);
- step[7] = output[7];
- step[8] = output[8] + output[11];
- step[9] = output[9] + output[10];
- step[10] = output[9] - output[10];
- step[11] = output[8] - output[11];
- step[12] = output[15] - output[12];
- step[13] = output[14] - output[13];
- step[14] = output[14] + output[13];
- step[15] = output[15] + output[12];
- step[16] = output[16];
- step[17] = output[17];
- temp = output[18] * -cospi_8_64 + output[29] * cospi_24_64;
- step[18] = (tran_low_t)fdct_round_shift(temp);
- temp = output[19] * -cospi_8_64 + output[28] * cospi_24_64;
- step[19] = (tran_low_t)fdct_round_shift(temp);
- temp = output[20] * -cospi_24_64 + output[27] * -cospi_8_64;
- step[20] = (tran_low_t)fdct_round_shift(temp);
- temp = output[21] * -cospi_24_64 + output[26] * -cospi_8_64;
- step[21] = (tran_low_t)fdct_round_shift(temp);
- step[22] = output[22];
- step[23] = output[23];
- step[24] = output[24];
- step[25] = output[25];
- temp = output[26] * cospi_24_64 + output[21] * -cospi_8_64;
- step[26] = (tran_low_t)fdct_round_shift(temp);
- temp = output[27] * cospi_24_64 + output[20] * -cospi_8_64;
- step[27] = (tran_low_t)fdct_round_shift(temp);
- temp = output[28] * cospi_8_64 + output[19] * cospi_24_64;
- step[28] = (tran_low_t)fdct_round_shift(temp);
- temp = output[29] * cospi_8_64 + output[18] * cospi_24_64;
- step[29] = (tran_low_t)fdct_round_shift(temp);
- step[30] = output[30];
- step[31] = output[31];
-
- range_check(step, 32, 18);
-
- // stage 5
- temp = step[0] * cospi_16_64 + step[1] * cospi_16_64;
- output[0] = (tran_low_t)fdct_round_shift(temp);
- temp = step[1] * -cospi_16_64 + step[0] * cospi_16_64;
- output[1] = (tran_low_t)fdct_round_shift(temp);
- temp = step[2] * cospi_24_64 + step[3] * cospi_8_64;
- output[2] = (tran_low_t)fdct_round_shift(temp);
- temp = step[3] * cospi_24_64 + step[2] * -cospi_8_64;
- output[3] = (tran_low_t)fdct_round_shift(temp);
- output[4] = step[4] + step[5];
- output[5] = step[4] - step[5];
- output[6] = step[7] - step[6];
- output[7] = step[7] + step[6];
- output[8] = step[8];
- temp = step[9] * -cospi_8_64 + step[14] * cospi_24_64;
- output[9] = (tran_low_t)fdct_round_shift(temp);
- temp = step[10] * -cospi_24_64 + step[13] * -cospi_8_64;
- output[10] = (tran_low_t)fdct_round_shift(temp);
- output[11] = step[11];
- output[12] = step[12];
- temp = step[13] * cospi_24_64 + step[10] * -cospi_8_64;
- output[13] = (tran_low_t)fdct_round_shift(temp);
- temp = step[14] * cospi_8_64 + step[9] * cospi_24_64;
- output[14] = (tran_low_t)fdct_round_shift(temp);
- output[15] = step[15];
- output[16] = step[16] + step[19];
- output[17] = step[17] + step[18];
- output[18] = step[17] - step[18];
- output[19] = step[16] - step[19];
- output[20] = step[23] - step[20];
- output[21] = step[22] - step[21];
- output[22] = step[22] + step[21];
- output[23] = step[23] + step[20];
- output[24] = step[24] + step[27];
- output[25] = step[25] + step[26];
- output[26] = step[25] - step[26];
- output[27] = step[24] - step[27];
- output[28] = step[31] - step[28];
- output[29] = step[30] - step[29];
- output[30] = step[30] + step[29];
- output[31] = step[31] + step[28];
-
- range_check(output, 32, 18);
-
- // stage 6
- step[0] = output[0];
- step[1] = output[1];
- step[2] = output[2];
- step[3] = output[3];
- temp = output[4] * cospi_28_64 + output[7] * cospi_4_64;
- step[4] = (tran_low_t)fdct_round_shift(temp);
- temp = output[5] * cospi_12_64 + output[6] * cospi_20_64;
- step[5] = (tran_low_t)fdct_round_shift(temp);
- temp = output[6] * cospi_12_64 + output[5] * -cospi_20_64;
- step[6] = (tran_low_t)fdct_round_shift(temp);
- temp = output[7] * cospi_28_64 + output[4] * -cospi_4_64;
- step[7] = (tran_low_t)fdct_round_shift(temp);
- step[8] = output[8] + output[9];
- step[9] = output[8] - output[9];
- step[10] = output[11] - output[10];
- step[11] = output[11] + output[10];
- step[12] = output[12] + output[13];
- step[13] = output[12] - output[13];
- step[14] = output[15] - output[14];
- step[15] = output[15] + output[14];
- step[16] = output[16];
- temp = output[17] * -cospi_4_64 + output[30] * cospi_28_64;
- step[17] = (tran_low_t)fdct_round_shift(temp);
- temp = output[18] * -cospi_28_64 + output[29] * -cospi_4_64;
- step[18] = (tran_low_t)fdct_round_shift(temp);
- step[19] = output[19];
- step[20] = output[20];
- temp = output[21] * -cospi_20_64 + output[26] * cospi_12_64;
- step[21] = (tran_low_t)fdct_round_shift(temp);
- temp = output[22] * -cospi_12_64 + output[25] * -cospi_20_64;
- step[22] = (tran_low_t)fdct_round_shift(temp);
- step[23] = output[23];
- step[24] = output[24];
- temp = output[25] * cospi_12_64 + output[22] * -cospi_20_64;
- step[25] = (tran_low_t)fdct_round_shift(temp);
- temp = output[26] * cospi_20_64 + output[21] * cospi_12_64;
- step[26] = (tran_low_t)fdct_round_shift(temp);
- step[27] = output[27];
- step[28] = output[28];
- temp = output[29] * cospi_28_64 + output[18] * -cospi_4_64;
- step[29] = (tran_low_t)fdct_round_shift(temp);
- temp = output[30] * cospi_4_64 + output[17] * cospi_28_64;
- step[30] = (tran_low_t)fdct_round_shift(temp);
- step[31] = output[31];
-
- range_check(step, 32, 18);
-
- // stage 7
- output[0] = step[0];
- output[1] = step[1];
- output[2] = step[2];
- output[3] = step[3];
- output[4] = step[4];
- output[5] = step[5];
- output[6] = step[6];
- output[7] = step[7];
- temp = step[8] * cospi_30_64 + step[15] * cospi_2_64;
- output[8] = (tran_low_t)fdct_round_shift(temp);
- temp = step[9] * cospi_14_64 + step[14] * cospi_18_64;
- output[9] = (tran_low_t)fdct_round_shift(temp);
- temp = step[10] * cospi_22_64 + step[13] * cospi_10_64;
- output[10] = (tran_low_t)fdct_round_shift(temp);
- temp = step[11] * cospi_6_64 + step[12] * cospi_26_64;
- output[11] = (tran_low_t)fdct_round_shift(temp);
- temp = step[12] * cospi_6_64 + step[11] * -cospi_26_64;
- output[12] = (tran_low_t)fdct_round_shift(temp);
- temp = step[13] * cospi_22_64 + step[10] * -cospi_10_64;
- output[13] = (tran_low_t)fdct_round_shift(temp);
- temp = step[14] * cospi_14_64 + step[9] * -cospi_18_64;
- output[14] = (tran_low_t)fdct_round_shift(temp);
- temp = step[15] * cospi_30_64 + step[8] * -cospi_2_64;
- output[15] = (tran_low_t)fdct_round_shift(temp);
- output[16] = step[16] + step[17];
- output[17] = step[16] - step[17];
- output[18] = step[19] - step[18];
- output[19] = step[19] + step[18];
- output[20] = step[20] + step[21];
- output[21] = step[20] - step[21];
- output[22] = step[23] - step[22];
- output[23] = step[23] + step[22];
- output[24] = step[24] + step[25];
- output[25] = step[24] - step[25];
- output[26] = step[27] - step[26];
- output[27] = step[27] + step[26];
- output[28] = step[28] + step[29];
- output[29] = step[28] - step[29];
- output[30] = step[31] - step[30];
- output[31] = step[31] + step[30];
-
- range_check(output, 32, 18);
-
- // stage 8
- step[0] = output[0];
- step[1] = output[1];
- step[2] = output[2];
- step[3] = output[3];
- step[4] = output[4];
- step[5] = output[5];
- step[6] = output[6];
- step[7] = output[7];
- step[8] = output[8];
- step[9] = output[9];
- step[10] = output[10];
- step[11] = output[11];
- step[12] = output[12];
- step[13] = output[13];
- step[14] = output[14];
- step[15] = output[15];
- temp = output[16] * cospi_31_64 + output[31] * cospi_1_64;
- step[16] = (tran_low_t)fdct_round_shift(temp);
- temp = output[17] * cospi_15_64 + output[30] * cospi_17_64;
- step[17] = (tran_low_t)fdct_round_shift(temp);
- temp = output[18] * cospi_23_64 + output[29] * cospi_9_64;
- step[18] = (tran_low_t)fdct_round_shift(temp);
- temp = output[19] * cospi_7_64 + output[28] * cospi_25_64;
- step[19] = (tran_low_t)fdct_round_shift(temp);
- temp = output[20] * cospi_27_64 + output[27] * cospi_5_64;
- step[20] = (tran_low_t)fdct_round_shift(temp);
- temp = output[21] * cospi_11_64 + output[26] * cospi_21_64;
- step[21] = (tran_low_t)fdct_round_shift(temp);
- temp = output[22] * cospi_19_64 + output[25] * cospi_13_64;
- step[22] = (tran_low_t)fdct_round_shift(temp);
- temp = output[23] * cospi_3_64 + output[24] * cospi_29_64;
- step[23] = (tran_low_t)fdct_round_shift(temp);
- temp = output[24] * cospi_3_64 + output[23] * -cospi_29_64;
- step[24] = (tran_low_t)fdct_round_shift(temp);
- temp = output[25] * cospi_19_64 + output[22] * -cospi_13_64;
- step[25] = (tran_low_t)fdct_round_shift(temp);
- temp = output[26] * cospi_11_64 + output[21] * -cospi_21_64;
- step[26] = (tran_low_t)fdct_round_shift(temp);
- temp = output[27] * cospi_27_64 + output[20] * -cospi_5_64;
- step[27] = (tran_low_t)fdct_round_shift(temp);
- temp = output[28] * cospi_7_64 + output[19] * -cospi_25_64;
- step[28] = (tran_low_t)fdct_round_shift(temp);
- temp = output[29] * cospi_23_64 + output[18] * -cospi_9_64;
- step[29] = (tran_low_t)fdct_round_shift(temp);
- temp = output[30] * cospi_15_64 + output[17] * -cospi_17_64;
- step[30] = (tran_low_t)fdct_round_shift(temp);
- temp = output[31] * cospi_31_64 + output[16] * -cospi_1_64;
- step[31] = (tran_low_t)fdct_round_shift(temp);
-
- range_check(step, 32, 18);
-
- // stage 9
- output[0] = step[0];
- output[1] = step[16];
- output[2] = step[8];
- output[3] = step[24];
- output[4] = step[4];
- output[5] = step[20];
- output[6] = step[12];
- output[7] = step[28];
- output[8] = step[2];
- output[9] = step[18];
- output[10] = step[10];
- output[11] = step[26];
- output[12] = step[6];
- output[13] = step[22];
- output[14] = step[14];
- output[15] = step[30];
- output[16] = step[1];
- output[17] = step[17];
- output[18] = step[9];
- output[19] = step[25];
- output[20] = step[5];
- output[21] = step[21];
- output[22] = step[13];
- output[23] = step[29];
- output[24] = step[3];
- output[25] = step[19];
- output[26] = step[11];
- output[27] = step[27];
- output[28] = step[7];
- output[29] = step[23];
- output[30] = step[15];
- output[31] = step[31];
-
- range_check(output, 32, 18);
-}
-*/
-
-static void fadst4(const tran_low_t *input, tran_low_t *output) {
- tran_high_t x0, x1, x2, x3;
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7;
-
- x0 = input[0];
- x1 = input[1];
- x2 = input[2];
- x3 = input[3];
-
- if (!(x0 | x1 | x2 | x3)) {
- output[0] = output[1] = output[2] = output[3] = 0;
- return;
- }
-
- s0 = sinpi_1_9 * x0;
- s1 = sinpi_4_9 * x0;
- s2 = sinpi_2_9 * x1;
- s3 = sinpi_1_9 * x1;
- s4 = sinpi_3_9 * x2;
- s5 = sinpi_4_9 * x3;
- s6 = sinpi_2_9 * x3;
- s7 = x0 + x1 - x3;
-
- x0 = s0 + s2 + s5;
- x1 = sinpi_3_9 * s7;
- x2 = s1 - s3 + s6;
- x3 = s4;
-
- s0 = x0 + x3;
- s1 = x1;
- s2 = x2 - x3;
- s3 = x2 - x0 + x3;
-
- // 1-D transform scaling factor is sqrt(2).
- output[0] = (tran_low_t)fdct_round_shift(s0);
- output[1] = (tran_low_t)fdct_round_shift(s1);
- output[2] = (tran_low_t)fdct_round_shift(s2);
- output[3] = (tran_low_t)fdct_round_shift(s3);
-}
-
-static void fadst8(const tran_low_t *input, tran_low_t *output) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7;
-
- tran_high_t x0 = input[7];
- tran_high_t x1 = input[0];
- tran_high_t x2 = input[5];
- tran_high_t x3 = input[2];
- tran_high_t x4 = input[3];
- tran_high_t x5 = input[4];
- tran_high_t x6 = input[1];
- tran_high_t x7 = input[6];
-
- // stage 1
- s0 = cospi_2_64 * x0 + cospi_30_64 * x1;
- s1 = cospi_30_64 * x0 - cospi_2_64 * x1;
- s2 = cospi_10_64 * x2 + cospi_22_64 * x3;
- s3 = cospi_22_64 * x2 - cospi_10_64 * x3;
- s4 = cospi_18_64 * x4 + cospi_14_64 * x5;
- s5 = cospi_14_64 * x4 - cospi_18_64 * x5;
- s6 = cospi_26_64 * x6 + cospi_6_64 * x7;
- s7 = cospi_6_64 * x6 - cospi_26_64 * x7;
-
- x0 = fdct_round_shift(s0 + s4);
- x1 = fdct_round_shift(s1 + s5);
- x2 = fdct_round_shift(s2 + s6);
- x3 = fdct_round_shift(s3 + s7);
- x4 = fdct_round_shift(s0 - s4);
- x5 = fdct_round_shift(s1 - s5);
- x6 = fdct_round_shift(s2 - s6);
- x7 = fdct_round_shift(s3 - s7);
-
- // stage 2
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = cospi_8_64 * x4 + cospi_24_64 * x5;
- s5 = cospi_24_64 * x4 - cospi_8_64 * x5;
- s6 = - cospi_24_64 * x6 + cospi_8_64 * x7;
- s7 = cospi_8_64 * x6 + cospi_24_64 * x7;
-
- x0 = s0 + s2;
- x1 = s1 + s3;
- x2 = s0 - s2;
- x3 = s1 - s3;
- x4 = fdct_round_shift(s4 + s6);
- x5 = fdct_round_shift(s5 + s7);
- x6 = fdct_round_shift(s4 - s6);
- x7 = fdct_round_shift(s5 - s7);
-
- // stage 3
- s2 = cospi_16_64 * (x2 + x3);
- s3 = cospi_16_64 * (x2 - x3);
- s6 = cospi_16_64 * (x6 + x7);
- s7 = cospi_16_64 * (x6 - x7);
-
- x2 = fdct_round_shift(s2);
- x3 = fdct_round_shift(s3);
- x6 = fdct_round_shift(s6);
- x7 = fdct_round_shift(s7);
-
- output[0] = (tran_low_t)x0;
- output[1] = (tran_low_t)-x4;
- output[2] = (tran_low_t)x6;
- output[3] = (tran_low_t)-x2;
- output[4] = (tran_low_t)x3;
- output[5] = (tran_low_t)-x7;
- output[6] = (tran_low_t)x5;
- output[7] = (tran_low_t)-x1;
-}
-
-static void fadst16(const tran_low_t *input, tran_low_t *output) {
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7, s8;
- tran_high_t s9, s10, s11, s12, s13, s14, s15;
-
- tran_high_t x0 = input[15];
- tran_high_t x1 = input[0];
- tran_high_t x2 = input[13];
- tran_high_t x3 = input[2];
- tran_high_t x4 = input[11];
- tran_high_t x5 = input[4];
- tran_high_t x6 = input[9];
- tran_high_t x7 = input[6];
- tran_high_t x8 = input[7];
- tran_high_t x9 = input[8];
- tran_high_t x10 = input[5];
- tran_high_t x11 = input[10];
- tran_high_t x12 = input[3];
- tran_high_t x13 = input[12];
- tran_high_t x14 = input[1];
- tran_high_t x15 = input[14];
-
- // stage 1
- s0 = x0 * cospi_1_64 + x1 * cospi_31_64;
- s1 = x0 * cospi_31_64 - x1 * cospi_1_64;
- s2 = x2 * cospi_5_64 + x3 * cospi_27_64;
- s3 = x2 * cospi_27_64 - x3 * cospi_5_64;
- s4 = x4 * cospi_9_64 + x5 * cospi_23_64;
- s5 = x4 * cospi_23_64 - x5 * cospi_9_64;
- s6 = x6 * cospi_13_64 + x7 * cospi_19_64;
- s7 = x6 * cospi_19_64 - x7 * cospi_13_64;
- s8 = x8 * cospi_17_64 + x9 * cospi_15_64;
- s9 = x8 * cospi_15_64 - x9 * cospi_17_64;
- s10 = x10 * cospi_21_64 + x11 * cospi_11_64;
- s11 = x10 * cospi_11_64 - x11 * cospi_21_64;
- s12 = x12 * cospi_25_64 + x13 * cospi_7_64;
- s13 = x12 * cospi_7_64 - x13 * cospi_25_64;
- s14 = x14 * cospi_29_64 + x15 * cospi_3_64;
- s15 = x14 * cospi_3_64 - x15 * cospi_29_64;
-
- x0 = fdct_round_shift(s0 + s8);
- x1 = fdct_round_shift(s1 + s9);
- x2 = fdct_round_shift(s2 + s10);
- x3 = fdct_round_shift(s3 + s11);
- x4 = fdct_round_shift(s4 + s12);
- x5 = fdct_round_shift(s5 + s13);
- x6 = fdct_round_shift(s6 + s14);
- x7 = fdct_round_shift(s7 + s15);
- x8 = fdct_round_shift(s0 - s8);
- x9 = fdct_round_shift(s1 - s9);
- x10 = fdct_round_shift(s2 - s10);
- x11 = fdct_round_shift(s3 - s11);
- x12 = fdct_round_shift(s4 - s12);
- x13 = fdct_round_shift(s5 - s13);
- x14 = fdct_round_shift(s6 - s14);
- x15 = fdct_round_shift(s7 - s15);
-
- // stage 2
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = x4;
- s5 = x5;
- s6 = x6;
- s7 = x7;
- s8 = x8 * cospi_4_64 + x9 * cospi_28_64;
- s9 = x8 * cospi_28_64 - x9 * cospi_4_64;
- s10 = x10 * cospi_20_64 + x11 * cospi_12_64;
- s11 = x10 * cospi_12_64 - x11 * cospi_20_64;
- s12 = - x12 * cospi_28_64 + x13 * cospi_4_64;
- s13 = x12 * cospi_4_64 + x13 * cospi_28_64;
- s14 = - x14 * cospi_12_64 + x15 * cospi_20_64;
- s15 = x14 * cospi_20_64 + x15 * cospi_12_64;
-
- x0 = s0 + s4;
- x1 = s1 + s5;
- x2 = s2 + s6;
- x3 = s3 + s7;
- x4 = s0 - s4;
- x5 = s1 - s5;
- x6 = s2 - s6;
- x7 = s3 - s7;
- x8 = fdct_round_shift(s8 + s12);
- x9 = fdct_round_shift(s9 + s13);
- x10 = fdct_round_shift(s10 + s14);
- x11 = fdct_round_shift(s11 + s15);
- x12 = fdct_round_shift(s8 - s12);
- x13 = fdct_round_shift(s9 - s13);
- x14 = fdct_round_shift(s10 - s14);
- x15 = fdct_round_shift(s11 - s15);
-
- // stage 3
- s0 = x0;
- s1 = x1;
- s2 = x2;
- s3 = x3;
- s4 = x4 * cospi_8_64 + x5 * cospi_24_64;
- s5 = x4 * cospi_24_64 - x5 * cospi_8_64;
- s6 = - x6 * cospi_24_64 + x7 * cospi_8_64;
- s7 = x6 * cospi_8_64 + x7 * cospi_24_64;
- s8 = x8;
- s9 = x9;
- s10 = x10;
- s11 = x11;
- s12 = x12 * cospi_8_64 + x13 * cospi_24_64;
- s13 = x12 * cospi_24_64 - x13 * cospi_8_64;
- s14 = - x14 * cospi_24_64 + x15 * cospi_8_64;
- s15 = x14 * cospi_8_64 + x15 * cospi_24_64;
-
- x0 = s0 + s2;
- x1 = s1 + s3;
- x2 = s0 - s2;
- x3 = s1 - s3;
- x4 = fdct_round_shift(s4 + s6);
- x5 = fdct_round_shift(s5 + s7);
- x6 = fdct_round_shift(s4 - s6);
- x7 = fdct_round_shift(s5 - s7);
- x8 = s8 + s10;
- x9 = s9 + s11;
- x10 = s8 - s10;
- x11 = s9 - s11;
- x12 = fdct_round_shift(s12 + s14);
- x13 = fdct_round_shift(s13 + s15);
- x14 = fdct_round_shift(s12 - s14);
- x15 = fdct_round_shift(s13 - s15);
-
- // stage 4
- s2 = (- cospi_16_64) * (x2 + x3);
- s3 = cospi_16_64 * (x2 - x3);
- s6 = cospi_16_64 * (x6 + x7);
- s7 = cospi_16_64 * (- x6 + x7);
- s10 = cospi_16_64 * (x10 + x11);
- s11 = cospi_16_64 * (- x10 + x11);
- s14 = (- cospi_16_64) * (x14 + x15);
- s15 = cospi_16_64 * (x14 - x15);
-
- x2 = fdct_round_shift(s2);
- x3 = fdct_round_shift(s3);
- x6 = fdct_round_shift(s6);
- x7 = fdct_round_shift(s7);
- x10 = fdct_round_shift(s10);
- x11 = fdct_round_shift(s11);
- x14 = fdct_round_shift(s14);
- x15 = fdct_round_shift(s15);
-
- output[0] = (tran_low_t)x0;
- output[1] = (tran_low_t)-x8;
- output[2] = (tran_low_t)x12;
- output[3] = (tran_low_t)-x4;
- output[4] = (tran_low_t)x6;
- output[5] = (tran_low_t)x14;
- output[6] = (tran_low_t)x10;
- output[7] = (tran_low_t)x2;
- output[8] = (tran_low_t)x3;
- output[9] = (tran_low_t)x11;
- output[10] = (tran_low_t)x15;
- output[11] = (tran_low_t)x7;
- output[12] = (tran_low_t)x5;
- output[13] = (tran_low_t)-x13;
- output[14] = (tran_low_t)x9;
- output[15] = (tran_low_t)-x1;
-}
-
-static const transform_2d FHT_4[] = {
- { fdct4, fdct4 }, // DCT_DCT = 0
- { fadst4, fdct4 }, // ADST_DCT = 1
- { fdct4, fadst4 }, // DCT_ADST = 2
- { fadst4, fadst4 } // ADST_ADST = 3
-};
-
-static const transform_2d FHT_8[] = {
- { fdct8, fdct8 }, // DCT_DCT = 0
- { fadst8, fdct8 }, // ADST_DCT = 1
- { fdct8, fadst8 }, // DCT_ADST = 2
- { fadst8, fadst8 } // ADST_ADST = 3
-};
-
-static const transform_2d FHT_16[] = {
- { fdct16, fdct16 }, // DCT_DCT = 0
- { fadst16, fdct16 }, // ADST_DCT = 1
- { fdct16, fadst16 }, // DCT_ADST = 2
- { fadst16, fadst16 } // ADST_ADST = 3
-};
-
-void vp10_fht4x4_c(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- if (tx_type == DCT_DCT) {
- vpx_fdct4x4_c(input, output, stride);
- } else {
- tran_low_t out[4 * 4];
- int i, j;
- tran_low_t temp_in[4], temp_out[4];
- const transform_2d ht = FHT_4[tx_type];
-
- // Columns
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = input[j * stride + i] * 16;
- if (i == 0 && temp_in[0])
- temp_in[0] += 1;
- ht.cols(temp_in, temp_out);
- for (j = 0; j < 4; ++j)
- out[j * 4 + i] = temp_out[j];
- }
-
- // Rows
- for (i = 0; i < 4; ++i) {
- for (j = 0; j < 4; ++j)
- temp_in[j] = out[j + i * 4];
- ht.rows(temp_in, temp_out);
- for (j = 0; j < 4; ++j)
- output[j + i * 4] = (temp_out[j] + 1) >> 2;
- }
- }
-}
-
-void vp10_fdct8x8_quant_c(const int16_t *input, int stride,
- tran_low_t *coeff_ptr, intptr_t n_coeffs,
- int skip_block,
- const int16_t *zbin_ptr, const int16_t *round_ptr,
- const int16_t *quant_ptr,
- const int16_t *quant_shift_ptr,
- tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr,
- const int16_t *dequant_ptr,
- uint16_t *eob_ptr,
- const int16_t *scan, const int16_t *iscan) {
- int eob = -1;
-
- int i, j;
- tran_low_t intermediate[64];
-
- // Transform columns
- {
- tran_low_t *output = intermediate;
- tran_high_t s0, s1, s2, s3, s4, s5, s6, s7; // canbe16
- tran_high_t t0, t1, t2, t3; // needs32
- tran_high_t x0, x1, x2, x3; // canbe16
-
- int i;
- for (i = 0; i < 8; i++) {
- // stage 1
- s0 = (input[0 * stride] + input[7 * stride]) * 4;
- s1 = (input[1 * stride] + input[6 * stride]) * 4;
- s2 = (input[2 * stride] + input[5 * stride]) * 4;
- s3 = (input[3 * stride] + input[4 * stride]) * 4;
- s4 = (input[3 * stride] - input[4 * stride]) * 4;
- s5 = (input[2 * stride] - input[5 * stride]) * 4;
- s6 = (input[1 * stride] - input[6 * stride]) * 4;
- s7 = (input[0 * stride] - input[7 * stride]) * 4;
-
- // fdct4(step, step);
- x0 = s0 + s3;
- x1 = s1 + s2;
- x2 = s1 - s2;
- x3 = s0 - s3;
- t0 = (x0 + x1) * cospi_16_64;
- t1 = (x0 - x1) * cospi_16_64;
- t2 = x2 * cospi_24_64 + x3 * cospi_8_64;
- t3 = -x2 * cospi_8_64 + x3 * cospi_24_64;
- output[0 * 8] = (tran_low_t)fdct_round_shift(t0);
- output[2 * 8] = (tran_low_t)fdct_round_shift(t2);
- output[4 * 8] = (tran_low_t)fdct_round_shift(t1);
- output[6 * 8] = (tran_low_t)fdct_round_shift(t3);
-
- // stage 2
- t0 = (s6 - s5) * cospi_16_64;
- t1 = (s6 + s5) * cospi_16_64;
- t2 = fdct_round_shift(t0);
- t3 = fdct_round_shift(t1);
-
- // stage 3
- x0 = s4 + t2;
- x1 = s4 - t2;
- x2 = s7 - t3;
- x3 = s7 + t3;
-
- // stage 4
- t0 = x0 * cospi_28_64 + x3 * cospi_4_64;
- t1 = x1 * cospi_12_64 + x2 * cospi_20_64;
- t2 = x2 * cospi_12_64 + x1 * -cospi_20_64;
- t3 = x3 * cospi_28_64 + x0 * -cospi_4_64;
- output[1 * 8] = (tran_low_t)fdct_round_shift(t0);
- output[3 * 8] = (tran_low_t)fdct_round_shift(t2);
- output[5 * 8] = (tran_low_t)fdct_round_shift(t1);
- output[7 * 8] = (tran_low_t)fdct_round_shift(t3);
- input++;
- output++;
- }
- }
-
- // Rows
- for (i = 0; i < 8; ++i) {
- fdct8(&intermediate[i * 8], &coeff_ptr[i * 8]);
- for (j = 0; j < 8; ++j)
- coeff_ptr[j + i * 8] /= 2;
- }
-
- // TODO(jingning) Decide the need of these arguments after the
- // quantization process is completed.
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)iscan;
-
- memset(qcoeff_ptr, 0, n_coeffs * sizeof(*qcoeff_ptr));
- memset(dqcoeff_ptr, 0, n_coeffs * sizeof(*dqcoeff_ptr));
-
- if (!skip_block) {
- // Quantization pass: All coefficients with index >= zero_flag are
- // skippable. Note: zero_flag can be zero.
- for (i = 0; i < n_coeffs; i++) {
- const int rc = scan[i];
- const int coeff = coeff_ptr[rc];
- const int coeff_sign = (coeff >> 31);
- const int abs_coeff = (coeff ^ coeff_sign) - coeff_sign;
-
- int tmp = clamp(abs_coeff + round_ptr[rc != 0], INT16_MIN, INT16_MAX);
- tmp = (tmp * quant_ptr[rc != 0]) >> 16;
-
- qcoeff_ptr[rc] = (tmp ^ coeff_sign) - coeff_sign;
- dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0];
-
- if (tmp)
- eob = i;
- }
- }
- *eob_ptr = eob + 1;
-}
-
-void vp10_fht8x8_c(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- if (tx_type == DCT_DCT) {
- vpx_fdct8x8_c(input, output, stride);
- } else {
- tran_low_t out[64];
- int i, j;
- tran_low_t temp_in[8], temp_out[8];
- const transform_2d ht = FHT_8[tx_type];
-
- // Columns
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = input[j * stride + i] * 4;
- ht.cols(temp_in, temp_out);
- for (j = 0; j < 8; ++j)
- out[j * 8 + i] = temp_out[j];
- }
-
- // Rows
- for (i = 0; i < 8; ++i) {
- for (j = 0; j < 8; ++j)
- temp_in[j] = out[j + i * 8];
- ht.rows(temp_in, temp_out);
- for (j = 0; j < 8; ++j)
- output[j + i * 8] = (temp_out[j] + (temp_out[j] < 0)) >> 1;
- }
- }
-}
-
-/* 4-point reversible, orthonormal Walsh-Hadamard in 3.5 adds, 0.5 shifts per
- pixel. */
-void vp10_fwht4x4_c(const int16_t *input, tran_low_t *output, int stride) {
- int i;
- tran_high_t a1, b1, c1, d1, e1;
- const int16_t *ip_pass0 = input;
- const tran_low_t *ip = NULL;
- tran_low_t *op = output;
-
- for (i = 0; i < 4; i++) {
- a1 = ip_pass0[0 * stride];
- b1 = ip_pass0[1 * stride];
- c1 = ip_pass0[2 * stride];
- d1 = ip_pass0[3 * stride];
-
- a1 += b1;
- d1 = d1 - c1;
- e1 = (a1 - d1) >> 1;
- b1 = e1 - b1;
- c1 = e1 - c1;
- a1 -= c1;
- d1 += b1;
- op[0] = (tran_low_t)a1;
- op[4] = (tran_low_t)c1;
- op[8] = (tran_low_t)d1;
- op[12] = (tran_low_t)b1;
-
- ip_pass0++;
- op++;
- }
- ip = output;
- op = output;
-
- for (i = 0; i < 4; i++) {
- a1 = ip[0];
- b1 = ip[1];
- c1 = ip[2];
- d1 = ip[3];
-
- a1 += b1;
- d1 -= c1;
- e1 = (a1 - d1) >> 1;
- b1 = e1 - b1;
- c1 = e1 - c1;
- a1 -= c1;
- d1 += b1;
- op[0] = (tran_low_t)(a1 * UNIT_QUANT_FACTOR);
- op[1] = (tran_low_t)(c1 * UNIT_QUANT_FACTOR);
- op[2] = (tran_low_t)(d1 * UNIT_QUANT_FACTOR);
- op[3] = (tran_low_t)(b1 * UNIT_QUANT_FACTOR);
-
- ip += 4;
- op += 4;
- }
-}
-
-void vp10_fht16x16_c(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- if (tx_type == DCT_DCT) {
- vpx_fdct16x16_c(input, output, stride);
- } else {
- tran_low_t out[256];
- int i, j;
- tran_low_t temp_in[16], temp_out[16];
- const transform_2d ht = FHT_16[tx_type];
-
- // Columns
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = input[j * stride + i] * 4;
- ht.cols(temp_in, temp_out);
- for (j = 0; j < 16; ++j)
- out[j * 16 + i] = (temp_out[j] + 1 + (temp_out[j] < 0)) >> 2;
- }
-
- // Rows
- for (i = 0; i < 16; ++i) {
- for (j = 0; j < 16; ++j)
- temp_in[j] = out[j + i * 16];
- ht.rows(temp_in, temp_out);
- for (j = 0; j < 16; ++j)
- output[j + i * 16] = temp_out[j];
- }
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_fht4x4_c(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- vp10_fht4x4_c(input, output, stride, tx_type);
-}
-
-void vp10_highbd_fht8x8_c(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- vp10_fht8x8_c(input, output, stride, tx_type);
-}
-
-void vp10_highbd_fwht4x4_c(const int16_t *input, tran_low_t *output,
- int stride) {
- vp10_fwht4x4_c(input, output, stride);
-}
-
-void vp10_highbd_fht16x16_c(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- vp10_fht16x16_c(input, output, stride, tx_type);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <limits.h>
-#include "./vpx_dsp_rtcd.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_scale/yv12config.h"
-#include "vpx/vpx_integer.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/encoder/context_tree.h"
-#include "vp10/encoder/denoiser.h"
-
-/* The VP9 denoiser is a work-in-progress. It currently is only designed to work
- * with speed 6, though it (inexplicably) seems to also work with speed 5 (one
- * would need to modify the source code in vp10_pickmode.c and vp10_encoder.c to
- * make the calls to the vp10_denoiser_* functions when in speed 5).
- *
- * The implementation is very similar to that of the VP8 denoiser. While
- * choosing the motion vectors / reference frames, the denoiser is run, and if
- * it did not modify the signal to much, the denoised block is copied to the
- * signal.
- */
-
-#ifdef OUTPUT_YUV_DENOISED
-static void make_grayscale(YV12_BUFFER_CONFIG *yuv);
-#endif
-
-static int absdiff_thresh(BLOCK_SIZE bs, int increase_denoising) {
- (void)bs;
- return 3 + (increase_denoising ? 1 : 0);
-}
-
-static int delta_thresh(BLOCK_SIZE bs, int increase_denoising) {
- (void)bs;
- (void)increase_denoising;
- return 4;
-}
-
-static int noise_motion_thresh(BLOCK_SIZE bs, int increase_denoising) {
- (void)bs;
- (void)increase_denoising;
- return 625;
-}
-
-static unsigned int sse_thresh(BLOCK_SIZE bs, int increase_denoising) {
- return (1 << num_pels_log2_lookup[bs]) * (increase_denoising ? 60 : 40);
-}
-
-static int sse_diff_thresh(BLOCK_SIZE bs, int increase_denoising,
- int motion_magnitude) {
- if (motion_magnitude >
- noise_motion_thresh(bs, increase_denoising)) {
- return 0;
- } else {
- return (1 << num_pels_log2_lookup[bs]) * 20;
- }
-}
-
-int total_adj_strong_thresh(BLOCK_SIZE bs, int increase_denoising) {
- return (1 << num_pels_log2_lookup[bs]) * (increase_denoising ? 3 : 2);
-}
-
-static int total_adj_weak_thresh(BLOCK_SIZE bs, int increase_denoising) {
- return (1 << num_pels_log2_lookup[bs]) * (increase_denoising ? 3 : 2);
-}
-
-// TODO(jackychen): If increase_denoising is enabled in the future,
-// we might need to update the code for calculating 'total_adj' in
-// case the C code is not bit-exact with corresponding sse2 code.
-int vp10_denoiser_filter_c(const uint8_t *sig, int sig_stride,
- const uint8_t *mc_avg,
- int mc_avg_stride,
- uint8_t *avg, int avg_stride,
- int increase_denoising,
- BLOCK_SIZE bs,
- int motion_magnitude) {
- int r, c;
- const uint8_t *sig_start = sig;
- const uint8_t *mc_avg_start = mc_avg;
- uint8_t *avg_start = avg;
- int diff, adj, absdiff, delta;
- int adj_val[] = {3, 4, 6};
- int total_adj = 0;
- int shift_inc = 1;
-
- // If motion_magnitude is small, making the denoiser more aggressive by
- // increasing the adjustment for each level. Add another increment for
- // blocks that are labeled for increase denoising.
- if (motion_magnitude <= MOTION_MAGNITUDE_THRESHOLD) {
- if (increase_denoising) {
- shift_inc = 2;
- }
- adj_val[0] += shift_inc;
- adj_val[1] += shift_inc;
- adj_val[2] += shift_inc;
- }
-
- // First attempt to apply a strong temporal denoising filter.
- for (r = 0; r < (4 << b_height_log2_lookup[bs]); ++r) {
- for (c = 0; c < (4 << b_width_log2_lookup[bs]); ++c) {
- diff = mc_avg[c] - sig[c];
- absdiff = abs(diff);
-
- if (absdiff <= absdiff_thresh(bs, increase_denoising)) {
- avg[c] = mc_avg[c];
- total_adj += diff;
- } else {
- switch (absdiff) {
- case 4: case 5: case 6: case 7:
- adj = adj_val[0];
- break;
- case 8: case 9: case 10: case 11:
- case 12: case 13: case 14: case 15:
- adj = adj_val[1];
- break;
- default:
- adj = adj_val[2];
- }
- if (diff > 0) {
- avg[c] = VPXMIN(UINT8_MAX, sig[c] + adj);
- total_adj += adj;
- } else {
- avg[c] = VPXMAX(0, sig[c] - adj);
- total_adj -= adj;
- }
- }
- }
- sig += sig_stride;
- avg += avg_stride;
- mc_avg += mc_avg_stride;
- }
-
- // If the strong filter did not modify the signal too much, we're all set.
- if (abs(total_adj) <= total_adj_strong_thresh(bs, increase_denoising)) {
- return FILTER_BLOCK;
- }
-
- // Otherwise, we try to dampen the filter if the delta is not too high.
- delta = ((abs(total_adj) - total_adj_strong_thresh(bs, increase_denoising))
- >> num_pels_log2_lookup[bs]) + 1;
-
- if (delta >= delta_thresh(bs, increase_denoising)) {
- return COPY_BLOCK;
- }
-
- mc_avg = mc_avg_start;
- avg = avg_start;
- sig = sig_start;
- for (r = 0; r < (4 << b_height_log2_lookup[bs]); ++r) {
- for (c = 0; c < (4 << b_width_log2_lookup[bs]); ++c) {
- diff = mc_avg[c] - sig[c];
- adj = abs(diff);
- if (adj > delta) {
- adj = delta;
- }
- if (diff > 0) {
- // Diff positive means we made positive adjustment above
- // (in first try/attempt), so now make negative adjustment to bring
- // denoised signal down.
- avg[c] = VPXMAX(0, avg[c] - adj);
- total_adj -= adj;
- } else {
- // Diff negative means we made negative adjustment above
- // (in first try/attempt), so now make positive adjustment to bring
- // denoised signal up.
- avg[c] = VPXMIN(UINT8_MAX, avg[c] + adj);
- total_adj += adj;
- }
- }
- sig += sig_stride;
- avg += avg_stride;
- mc_avg += mc_avg_stride;
- }
-
- // We can use the filter if it has been sufficiently dampened
- if (abs(total_adj) <= total_adj_weak_thresh(bs, increase_denoising)) {
- return FILTER_BLOCK;
- }
- return COPY_BLOCK;
-}
-
-static uint8_t *block_start(uint8_t *framebuf, int stride,
- int mi_row, int mi_col) {
- return framebuf + (stride * mi_row * 8) + (mi_col * 8);
-}
-
-static VP9_DENOISER_DECISION perform_motion_compensation(VP9_DENOISER *denoiser,
- MACROBLOCK *mb,
- BLOCK_SIZE bs,
- int increase_denoising,
- int mi_row,
- int mi_col,
- PICK_MODE_CONTEXT *ctx,
- int *motion_magnitude
- ) {
- int mv_col, mv_row;
- int sse_diff = ctx->zeromv_sse - ctx->newmv_sse;
- MV_REFERENCE_FRAME frame;
- MACROBLOCKD *filter_mbd = &mb->e_mbd;
- MB_MODE_INFO *mbmi = &filter_mbd->mi[0]->mbmi;
- MB_MODE_INFO saved_mbmi;
- int i, j;
- struct buf_2d saved_dst[MAX_MB_PLANE];
- struct buf_2d saved_pre[MAX_MB_PLANE][2]; // 2 pre buffers
-
- mv_col = ctx->best_sse_mv.as_mv.col;
- mv_row = ctx->best_sse_mv.as_mv.row;
- *motion_magnitude = mv_row * mv_row + mv_col * mv_col;
- frame = ctx->best_reference_frame;
-
- saved_mbmi = *mbmi;
-
- // If the best reference frame uses inter-prediction and there is enough of a
- // difference in sum-squared-error, use it.
- if (frame != INTRA_FRAME &&
- sse_diff > sse_diff_thresh(bs, increase_denoising, *motion_magnitude)) {
- mbmi->ref_frame[0] = ctx->best_reference_frame;
- mbmi->mode = ctx->best_sse_inter_mode;
- mbmi->mv[0] = ctx->best_sse_mv;
- } else {
- // Otherwise, use the zero reference frame.
- frame = ctx->best_zeromv_reference_frame;
-
- mbmi->ref_frame[0] = ctx->best_zeromv_reference_frame;
- mbmi->mode = ZEROMV;
- mbmi->mv[0].as_int = 0;
-
- ctx->best_sse_inter_mode = ZEROMV;
- ctx->best_sse_mv.as_int = 0;
- ctx->newmv_sse = ctx->zeromv_sse;
- }
-
- if (ctx->newmv_sse > sse_thresh(bs, increase_denoising)) {
- // Restore everything to its original state
- *mbmi = saved_mbmi;
- return COPY_BLOCK;
- }
- if (*motion_magnitude >
- (noise_motion_thresh(bs, increase_denoising) << 3)) {
- // Restore everything to its original state
- *mbmi = saved_mbmi;
- return COPY_BLOCK;
- }
-
- // We will restore these after motion compensation.
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- for (j = 0; j < 2; ++j) {
- saved_pre[i][j] = filter_mbd->plane[i].pre[j];
- }
- saved_dst[i] = filter_mbd->plane[i].dst;
- }
-
- // Set the pointers in the MACROBLOCKD to point to the buffers in the denoiser
- // struct.
- for (j = 0; j < 2; ++j) {
- filter_mbd->plane[0].pre[j].buf =
- block_start(denoiser->running_avg_y[frame].y_buffer,
- denoiser->running_avg_y[frame].y_stride,
- mi_row, mi_col);
- filter_mbd->plane[0].pre[j].stride =
- denoiser->running_avg_y[frame].y_stride;
- filter_mbd->plane[1].pre[j].buf =
- block_start(denoiser->running_avg_y[frame].u_buffer,
- denoiser->running_avg_y[frame].uv_stride,
- mi_row, mi_col);
- filter_mbd->plane[1].pre[j].stride =
- denoiser->running_avg_y[frame].uv_stride;
- filter_mbd->plane[2].pre[j].buf =
- block_start(denoiser->running_avg_y[frame].v_buffer,
- denoiser->running_avg_y[frame].uv_stride,
- mi_row, mi_col);
- filter_mbd->plane[2].pre[j].stride =
- denoiser->running_avg_y[frame].uv_stride;
- }
- filter_mbd->plane[0].dst.buf =
- block_start(denoiser->mc_running_avg_y.y_buffer,
- denoiser->mc_running_avg_y.y_stride,
- mi_row, mi_col);
- filter_mbd->plane[0].dst.stride = denoiser->mc_running_avg_y.y_stride;
- filter_mbd->plane[1].dst.buf =
- block_start(denoiser->mc_running_avg_y.u_buffer,
- denoiser->mc_running_avg_y.uv_stride,
- mi_row, mi_col);
- filter_mbd->plane[1].dst.stride = denoiser->mc_running_avg_y.uv_stride;
- filter_mbd->plane[2].dst.buf =
- block_start(denoiser->mc_running_avg_y.v_buffer,
- denoiser->mc_running_avg_y.uv_stride,
- mi_row, mi_col);
- filter_mbd->plane[2].dst.stride = denoiser->mc_running_avg_y.uv_stride;
-
- vp10_build_inter_predictors_sby(filter_mbd, mv_row, mv_col, bs);
-
- // Restore everything to its original state
- *mbmi = saved_mbmi;
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- for (j = 0; j < 2; ++j) {
- filter_mbd->plane[i].pre[j] = saved_pre[i][j];
- }
- filter_mbd->plane[i].dst = saved_dst[i];
- }
-
- mv_row = ctx->best_sse_mv.as_mv.row;
- mv_col = ctx->best_sse_mv.as_mv.col;
-
- return FILTER_BLOCK;
-}
-
-void vp10_denoiser_denoise(VP9_DENOISER *denoiser, MACROBLOCK *mb,
- int mi_row, int mi_col, BLOCK_SIZE bs,
- PICK_MODE_CONTEXT *ctx) {
- int motion_magnitude = 0;
- VP9_DENOISER_DECISION decision = FILTER_BLOCK;
- YV12_BUFFER_CONFIG avg = denoiser->running_avg_y[INTRA_FRAME];
- YV12_BUFFER_CONFIG mc_avg = denoiser->mc_running_avg_y;
- uint8_t *avg_start = block_start(avg.y_buffer, avg.y_stride, mi_row, mi_col);
- uint8_t *mc_avg_start = block_start(mc_avg.y_buffer, mc_avg.y_stride,
- mi_row, mi_col);
- struct buf_2d src = mb->plane[0].src;
-
- decision = perform_motion_compensation(denoiser, mb, bs,
- denoiser->increase_denoising,
- mi_row, mi_col, ctx,
- &motion_magnitude);
-
- if (decision == FILTER_BLOCK) {
- decision = vp10_denoiser_filter(src.buf, src.stride,
- mc_avg_start, mc_avg.y_stride,
- avg_start, avg.y_stride,
- 0, bs, motion_magnitude);
- }
-
- if (decision == FILTER_BLOCK) {
- vpx_convolve_copy(avg_start, avg.y_stride, src.buf, src.stride,
- NULL, 0, NULL, 0,
- num_4x4_blocks_wide_lookup[bs] << 2,
- num_4x4_blocks_high_lookup[bs] << 2);
- } else { // COPY_BLOCK
- vpx_convolve_copy(src.buf, src.stride, avg_start, avg.y_stride,
- NULL, 0, NULL, 0,
- num_4x4_blocks_wide_lookup[bs] << 2,
- num_4x4_blocks_high_lookup[bs] << 2);
- }
-}
-
-static void copy_frame(YV12_BUFFER_CONFIG dest, const YV12_BUFFER_CONFIG src) {
- int r;
- const uint8_t *srcbuf = src.y_buffer;
- uint8_t *destbuf = dest.y_buffer;
-
- assert(dest.y_width == src.y_width);
- assert(dest.y_height == src.y_height);
-
- for (r = 0; r < dest.y_height; ++r) {
- memcpy(destbuf, srcbuf, dest.y_width);
- destbuf += dest.y_stride;
- srcbuf += src.y_stride;
- }
-}
-
-static void swap_frame_buffer(YV12_BUFFER_CONFIG *dest,
- YV12_BUFFER_CONFIG *src) {
- uint8_t *tmp_buf = dest->y_buffer;
- assert(dest->y_width == src->y_width);
- assert(dest->y_height == src->y_height);
- dest->y_buffer = src->y_buffer;
- src->y_buffer = tmp_buf;
-}
-
-void vp10_denoiser_update_frame_info(VP9_DENOISER *denoiser,
- YV12_BUFFER_CONFIG src,
- FRAME_TYPE frame_type,
- int refresh_alt_ref_frame,
- int refresh_golden_frame,
- int refresh_last_frame) {
- if (frame_type == KEY_FRAME) {
- int i;
- // Start at 1 so as not to overwrite the INTRA_FRAME
- for (i = 1; i < MAX_REF_FRAMES; ++i)
- copy_frame(denoiser->running_avg_y[i], src);
- return;
- }
-
- /* For non key frames */
- if (refresh_alt_ref_frame) {
- swap_frame_buffer(&denoiser->running_avg_y[ALTREF_FRAME],
- &denoiser->running_avg_y[INTRA_FRAME]);
- }
- if (refresh_golden_frame) {
- swap_frame_buffer(&denoiser->running_avg_y[GOLDEN_FRAME],
- &denoiser->running_avg_y[INTRA_FRAME]);
- }
- if (refresh_last_frame) {
- swap_frame_buffer(&denoiser->running_avg_y[LAST_FRAME],
- &denoiser->running_avg_y[INTRA_FRAME]);
- }
-}
-
-void vp10_denoiser_reset_frame_stats(PICK_MODE_CONTEXT *ctx) {
- ctx->zeromv_sse = UINT_MAX;
- ctx->newmv_sse = UINT_MAX;
-}
-
-void vp10_denoiser_update_frame_stats(MB_MODE_INFO *mbmi, unsigned int sse,
- PREDICTION_MODE mode,
- PICK_MODE_CONTEXT *ctx) {
- // TODO(tkopp): Use both MVs if possible
- if (mbmi->mv[0].as_int == 0 && sse < ctx->zeromv_sse) {
- ctx->zeromv_sse = sse;
- ctx->best_zeromv_reference_frame = mbmi->ref_frame[0];
- }
-
- if (mbmi->mv[0].as_int != 0 && sse < ctx->newmv_sse) {
- ctx->newmv_sse = sse;
- ctx->best_sse_inter_mode = mode;
- ctx->best_sse_mv = mbmi->mv[0];
- ctx->best_reference_frame = mbmi->ref_frame[0];
- }
-}
-
-int vp10_denoiser_alloc(VP9_DENOISER *denoiser, int width, int height,
- int ssx, int ssy,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- int border) {
- int i, fail;
- const int legacy_byte_alignment = 0;
- assert(denoiser != NULL);
-
- for (i = 0; i < MAX_REF_FRAMES; ++i) {
- fail = vpx_alloc_frame_buffer(&denoiser->running_avg_y[i], width, height,
- ssx, ssy,
-#if CONFIG_VP9_HIGHBITDEPTH
- use_highbitdepth,
-#endif
- border, legacy_byte_alignment);
- if (fail) {
- vp10_denoiser_free(denoiser);
- return 1;
- }
-#ifdef OUTPUT_YUV_DENOISED
- make_grayscale(&denoiser->running_avg_y[i]);
-#endif
- }
-
- fail = vpx_alloc_frame_buffer(&denoiser->mc_running_avg_y, width, height,
- ssx, ssy,
-#if CONFIG_VP9_HIGHBITDEPTH
- use_highbitdepth,
-#endif
- border, legacy_byte_alignment);
- if (fail) {
- vp10_denoiser_free(denoiser);
- return 1;
- }
-#ifdef OUTPUT_YUV_DENOISED
- make_grayscale(&denoiser->running_avg_y[i]);
-#endif
- denoiser->increase_denoising = 0;
- denoiser->frame_buffer_initialized = 1;
-
- return 0;
-}
-
-void vp10_denoiser_free(VP9_DENOISER *denoiser) {
- int i;
- denoiser->frame_buffer_initialized = 0;
- if (denoiser == NULL) {
- return;
- }
- for (i = 0; i < MAX_REF_FRAMES; ++i) {
- vpx_free_frame_buffer(&denoiser->running_avg_y[i]);
- }
- vpx_free_frame_buffer(&denoiser->mc_running_avg_y);
-}
-
-#ifdef OUTPUT_YUV_DENOISED
-static void make_grayscale(YV12_BUFFER_CONFIG *yuv) {
- int r, c;
- uint8_t *u = yuv->u_buffer;
- uint8_t *v = yuv->v_buffer;
-
- for (r = 0; r < yuv->uv_height; ++r) {
- for (c = 0; c < yuv->uv_width; ++c) {
- u[c] = UINT8_MAX / 2;
- v[c] = UINT8_MAX / 2;
- }
- u += yuv->uv_stride;
- v += yuv->uv_stride;
- }
-}
-#endif
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP9_ENCODER_DENOISER_H_
-#define VP9_ENCODER_DENOISER_H_
-
-#include "vp10/encoder/block.h"
-#include "vpx_scale/yv12config.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MOTION_MAGNITUDE_THRESHOLD (8 * 3)
-
-typedef enum vp10_denoiser_decision {
- COPY_BLOCK,
- FILTER_BLOCK
-} VP9_DENOISER_DECISION;
-
-typedef struct vp10_denoiser {
- YV12_BUFFER_CONFIG running_avg_y[MAX_REF_FRAMES];
- YV12_BUFFER_CONFIG mc_running_avg_y;
- int increase_denoising;
- int frame_buffer_initialized;
-} VP9_DENOISER;
-
-void vp10_denoiser_update_frame_info(VP9_DENOISER *denoiser,
- YV12_BUFFER_CONFIG src,
- FRAME_TYPE frame_type,
- int refresh_alt_ref_frame,
- int refresh_golden_frame,
- int refresh_last_frame);
-
-void vp10_denoiser_denoise(VP9_DENOISER *denoiser, MACROBLOCK *mb,
- int mi_row, int mi_col, BLOCK_SIZE bs,
- PICK_MODE_CONTEXT *ctx);
-
-void vp10_denoiser_reset_frame_stats(PICK_MODE_CONTEXT *ctx);
-
-void vp10_denoiser_update_frame_stats(MB_MODE_INFO *mbmi,
- unsigned int sse, PREDICTION_MODE mode,
- PICK_MODE_CONTEXT *ctx);
-
-int vp10_denoiser_alloc(VP9_DENOISER *denoiser, int width, int height,
- int ssx, int ssy,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- int border);
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
-int total_adj_strong_thresh(BLOCK_SIZE bs, int increase_denoising);
-#endif
-
-void vp10_denoiser_free(VP9_DENOISER *denoiser);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP9_ENCODER_DENOISER_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-#include <math.h>
-#include <stdio.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "./vpx_config.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/vpx_timer.h"
-#include "vpx_ports/system_state.h"
-
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/idct.h"
-#include "vp10/common/mvref_common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconintra.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/seg_common.h"
-#include "vp10/common/tile_common.h"
-
-#include "vp10/encoder/aq_complexity.h"
-#include "vp10/encoder/aq_cyclicrefresh.h"
-#include "vp10/encoder/aq_variance.h"
-#include "vp10/encoder/encodeframe.h"
-#include "vp10/encoder/encodemb.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/ethread.h"
-#include "vp10/encoder/extend.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/rdopt.h"
-#include "vp10/encoder/segmentation.h"
-#include "vp10/encoder/tokenize.h"
-
-static void encode_superblock(VP10_COMP *cpi, ThreadData * td,
- TOKENEXTRA **t, int output_enabled,
- int mi_row, int mi_col, BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx);
-
-// This is used as a reference when computing the source variance for the
-// purposes of activity masking.
-// Eventually this should be replaced by custom no-reference routines,
-// which will be faster.
-static const uint8_t VP9_VAR_OFFS[64] = {
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128
-};
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static const uint16_t VP9_HIGH_VAR_OFFS_8[64] = {
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128,
- 128, 128, 128, 128, 128, 128, 128, 128
-};
-
-static const uint16_t VP9_HIGH_VAR_OFFS_10[64] = {
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4,
- 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4, 128*4
-};
-
-static const uint16_t VP9_HIGH_VAR_OFFS_12[64] = {
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16,
- 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16, 128*16
-};
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-unsigned int vp10_get_sby_perpixel_variance(VP10_COMP *cpi,
- const struct buf_2d *ref,
- BLOCK_SIZE bs) {
- unsigned int sse;
- const unsigned int var = cpi->fn_ptr[bs].vf(ref->buf, ref->stride,
- VP9_VAR_OFFS, 0, &sse);
- return ROUND_POWER_OF_TWO(var, num_pels_log2_lookup[bs]);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-unsigned int vp10_high_get_sby_perpixel_variance(
- VP10_COMP *cpi, const struct buf_2d *ref, BLOCK_SIZE bs, int bd) {
- unsigned int var, sse;
- switch (bd) {
- case 10:
- var = cpi->fn_ptr[bs].vf(ref->buf, ref->stride,
- CONVERT_TO_BYTEPTR(VP9_HIGH_VAR_OFFS_10),
- 0, &sse);
- break;
- case 12:
- var = cpi->fn_ptr[bs].vf(ref->buf, ref->stride,
- CONVERT_TO_BYTEPTR(VP9_HIGH_VAR_OFFS_12),
- 0, &sse);
- break;
- case 8:
- default:
- var = cpi->fn_ptr[bs].vf(ref->buf, ref->stride,
- CONVERT_TO_BYTEPTR(VP9_HIGH_VAR_OFFS_8),
- 0, &sse);
- break;
- }
- return ROUND_POWER_OF_TWO(var, num_pels_log2_lookup[bs]);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static unsigned int get_sby_perpixel_diff_variance(VP10_COMP *cpi,
- const struct buf_2d *ref,
- int mi_row, int mi_col,
- BLOCK_SIZE bs) {
- unsigned int sse, var;
- uint8_t *last_y;
- const YV12_BUFFER_CONFIG *last = get_ref_frame_buffer(cpi, LAST_FRAME);
-
- assert(last != NULL);
- last_y =
- &last->y_buffer[mi_row * MI_SIZE * last->y_stride + mi_col * MI_SIZE];
- var = cpi->fn_ptr[bs].vf(ref->buf, ref->stride, last_y, last->y_stride, &sse);
- return ROUND_POWER_OF_TWO(var, num_pels_log2_lookup[bs]);
-}
-
-static BLOCK_SIZE get_rd_var_based_fixed_partition(VP10_COMP *cpi,
- MACROBLOCK *x,
- int mi_row,
- int mi_col) {
- unsigned int var = get_sby_perpixel_diff_variance(cpi, &x->plane[0].src,
- mi_row, mi_col,
- BLOCK_64X64);
- if (var < 8)
- return BLOCK_64X64;
- else if (var < 128)
- return BLOCK_32X32;
- else if (var < 2048)
- return BLOCK_16X16;
- else
- return BLOCK_8X8;
-}
-
-// Lighter version of set_offsets that only sets the mode info
-// pointers.
-static INLINE void set_mode_info_offsets(VP10_COMP *const cpi,
- MACROBLOCK *const x,
- MACROBLOCKD *const xd,
- int mi_row,
- int mi_col) {
- VP10_COMMON *const cm = &cpi->common;
- const int idx_str = xd->mi_stride * mi_row + mi_col;
- xd->mi = cm->mi_grid_visible + idx_str;
- xd->mi[0] = cm->mi + idx_str;
- x->mbmi_ext = cpi->mbmi_ext_base + (mi_row * cm->mi_cols + mi_col);
-}
-
-static void set_offsets(VP10_COMP *cpi, const TileInfo *const tile,
- MACROBLOCK *const x, int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *mbmi;
- const int mi_width = num_8x8_blocks_wide_lookup[bsize];
- const int mi_height = num_8x8_blocks_high_lookup[bsize];
- const struct segmentation *const seg = &cm->seg;
-
- set_skip_context(xd, mi_row, mi_col);
-
- set_mode_info_offsets(cpi, x, xd, mi_row, mi_col);
-
- mbmi = &xd->mi[0]->mbmi;
-
- // Set up destination pointers.
- vp10_setup_dst_planes(xd->plane, get_frame_new_buffer(cm), mi_row, mi_col);
-
- // Set up limit values for MV components.
- // Mv beyond the range do not produce new/different prediction block.
- x->mv_row_min = -(((mi_row + mi_height) * MI_SIZE) + VP9_INTERP_EXTEND);
- x->mv_col_min = -(((mi_col + mi_width) * MI_SIZE) + VP9_INTERP_EXTEND);
- x->mv_row_max = (cm->mi_rows - mi_row) * MI_SIZE + VP9_INTERP_EXTEND;
- x->mv_col_max = (cm->mi_cols - mi_col) * MI_SIZE + VP9_INTERP_EXTEND;
-
- // Set up distance of MB to edge of frame in 1/8th pel units.
- assert(!(mi_col & (mi_width - 1)) && !(mi_row & (mi_height - 1)));
- set_mi_row_col(xd, tile, mi_row, mi_height, mi_col, mi_width,
- cm->mi_rows, cm->mi_cols);
-
- // Set up source buffers.
- vp10_setup_src_planes(x, cpi->Source, mi_row, mi_col);
-
- // R/D setup.
- x->rddiv = cpi->rd.RDDIV;
- x->rdmult = cpi->rd.RDMULT;
-
- // Setup segment ID.
- if (seg->enabled) {
- if (cpi->oxcf.aq_mode != VARIANCE_AQ) {
- const uint8_t *const map = seg->update_map ? cpi->segmentation_map
- : cm->last_frame_seg_map;
- mbmi->segment_id = get_segment_id(cm, map, bsize, mi_row, mi_col);
- }
- vp10_init_plane_quantizers(cpi, x);
-
- x->encode_breakout = cpi->segment_encode_breakout[mbmi->segment_id];
- } else {
- mbmi->segment_id = 0;
- x->encode_breakout = cpi->encode_breakout;
- }
-
- // required by vp10_append_sub8x8_mvs_for_idx() and vp10_find_best_ref_mvs()
- xd->tile = *tile;
-}
-
-static void set_block_size(VP10_COMP * const cpi,
- MACROBLOCK *const x,
- MACROBLOCKD *const xd,
- int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- if (cpi->common.mi_cols > mi_col && cpi->common.mi_rows > mi_row) {
- set_mode_info_offsets(cpi, x, xd, mi_row, mi_col);
- xd->mi[0]->mbmi.sb_type = bsize;
- }
-}
-
-typedef struct {
- int64_t sum_square_error;
- int64_t sum_error;
- int log2_count;
- int variance;
-} var;
-
-typedef struct {
- var none;
- var horz[2];
- var vert[2];
-} partition_variance;
-
-typedef struct {
- partition_variance part_variances;
- var split[4];
-} v4x4;
-
-typedef struct {
- partition_variance part_variances;
- v4x4 split[4];
-} v8x8;
-
-typedef struct {
- partition_variance part_variances;
- v8x8 split[4];
-} v16x16;
-
-typedef struct {
- partition_variance part_variances;
- v16x16 split[4];
-} v32x32;
-
-typedef struct {
- partition_variance part_variances;
- v32x32 split[4];
-} v64x64;
-
-typedef struct {
- partition_variance *part_variances;
- var *split[4];
-} variance_node;
-
-typedef enum {
- V16X16,
- V32X32,
- V64X64,
-} TREE_LEVEL;
-
-static void tree_to_node(void *data, BLOCK_SIZE bsize, variance_node *node) {
- int i;
- node->part_variances = NULL;
- switch (bsize) {
- case BLOCK_64X64: {
- v64x64 *vt = (v64x64 *) data;
- node->part_variances = &vt->part_variances;
- for (i = 0; i < 4; i++)
- node->split[i] = &vt->split[i].part_variances.none;
- break;
- }
- case BLOCK_32X32: {
- v32x32 *vt = (v32x32 *) data;
- node->part_variances = &vt->part_variances;
- for (i = 0; i < 4; i++)
- node->split[i] = &vt->split[i].part_variances.none;
- break;
- }
- case BLOCK_16X16: {
- v16x16 *vt = (v16x16 *) data;
- node->part_variances = &vt->part_variances;
- for (i = 0; i < 4; i++)
- node->split[i] = &vt->split[i].part_variances.none;
- break;
- }
- case BLOCK_8X8: {
- v8x8 *vt = (v8x8 *) data;
- node->part_variances = &vt->part_variances;
- for (i = 0; i < 4; i++)
- node->split[i] = &vt->split[i].part_variances.none;
- break;
- }
- case BLOCK_4X4: {
- v4x4 *vt = (v4x4 *) data;
- node->part_variances = &vt->part_variances;
- for (i = 0; i < 4; i++)
- node->split[i] = &vt->split[i];
- break;
- }
- default: {
- assert(0);
- break;
- }
- }
-}
-
-// Set variance values given sum square error, sum error, count.
-static void fill_variance(int64_t s2, int64_t s, int c, var *v) {
- v->sum_square_error = s2;
- v->sum_error = s;
- v->log2_count = c;
-}
-
-static void get_variance(var *v) {
- v->variance = (int)(256 * (v->sum_square_error -
- ((v->sum_error * v->sum_error) >> v->log2_count)) >> v->log2_count);
-}
-
-static void sum_2_variances(const var *a, const var *b, var *r) {
- assert(a->log2_count == b->log2_count);
- fill_variance(a->sum_square_error + b->sum_square_error,
- a->sum_error + b->sum_error, a->log2_count + 1, r);
-}
-
-static void fill_variance_tree(void *data, BLOCK_SIZE bsize) {
- variance_node node;
- memset(&node, 0, sizeof(node));
- tree_to_node(data, bsize, &node);
- sum_2_variances(node.split[0], node.split[1], &node.part_variances->horz[0]);
- sum_2_variances(node.split[2], node.split[3], &node.part_variances->horz[1]);
- sum_2_variances(node.split[0], node.split[2], &node.part_variances->vert[0]);
- sum_2_variances(node.split[1], node.split[3], &node.part_variances->vert[1]);
- sum_2_variances(&node.part_variances->vert[0], &node.part_variances->vert[1],
- &node.part_variances->none);
-}
-
-static int set_vt_partitioning(VP10_COMP *cpi,
- MACROBLOCK *const x,
- MACROBLOCKD *const xd,
- void *data,
- BLOCK_SIZE bsize,
- int mi_row,
- int mi_col,
- int64_t threshold,
- BLOCK_SIZE bsize_min,
- int force_split) {
- VP10_COMMON * const cm = &cpi->common;
- variance_node vt;
- const int block_width = num_8x8_blocks_wide_lookup[bsize];
- const int block_height = num_8x8_blocks_high_lookup[bsize];
- const int low_res = (cm->width <= 352 && cm->height <= 288);
-
- assert(block_height == block_width);
- tree_to_node(data, bsize, &vt);
-
- if (force_split == 1)
- return 0;
-
- // For bsize=bsize_min (16x16/8x8 for 8x8/4x4 downsampling), select if
- // variance is below threshold, otherwise split will be selected.
- // No check for vert/horiz split as too few samples for variance.
- if (bsize == bsize_min) {
- // Variance already computed to set the force_split.
- if (low_res || cm->frame_type == KEY_FRAME)
- get_variance(&vt.part_variances->none);
- if (mi_col + block_width / 2 < cm->mi_cols &&
- mi_row + block_height / 2 < cm->mi_rows &&
- vt.part_variances->none.variance < threshold) {
- set_block_size(cpi, x, xd, mi_row, mi_col, bsize);
- return 1;
- }
- return 0;
- } else if (bsize > bsize_min) {
- // Variance already computed to set the force_split.
- if (low_res || cm->frame_type == KEY_FRAME)
- get_variance(&vt.part_variances->none);
- // For key frame: take split for bsize above 32X32 or very high variance.
- if (cm->frame_type == KEY_FRAME &&
- (bsize > BLOCK_32X32 ||
- vt.part_variances->none.variance > (threshold << 4))) {
- return 0;
- }
- // If variance is low, take the bsize (no split).
- if (mi_col + block_width / 2 < cm->mi_cols &&
- mi_row + block_height / 2 < cm->mi_rows &&
- vt.part_variances->none.variance < threshold) {
- set_block_size(cpi, x, xd, mi_row, mi_col, bsize);
- return 1;
- }
-
- // Check vertical split.
- if (mi_row + block_height / 2 < cm->mi_rows) {
- BLOCK_SIZE subsize = get_subsize(bsize, PARTITION_VERT);
- get_variance(&vt.part_variances->vert[0]);
- get_variance(&vt.part_variances->vert[1]);
- if (vt.part_variances->vert[0].variance < threshold &&
- vt.part_variances->vert[1].variance < threshold &&
- get_plane_block_size(subsize, &xd->plane[1]) < BLOCK_INVALID) {
- set_block_size(cpi, x, xd, mi_row, mi_col, subsize);
- set_block_size(cpi, x, xd, mi_row, mi_col + block_width / 2, subsize);
- return 1;
- }
- }
- // Check horizontal split.
- if (mi_col + block_width / 2 < cm->mi_cols) {
- BLOCK_SIZE subsize = get_subsize(bsize, PARTITION_HORZ);
- get_variance(&vt.part_variances->horz[0]);
- get_variance(&vt.part_variances->horz[1]);
- if (vt.part_variances->horz[0].variance < threshold &&
- vt.part_variances->horz[1].variance < threshold &&
- get_plane_block_size(subsize, &xd->plane[1]) < BLOCK_INVALID) {
- set_block_size(cpi, x, xd, mi_row, mi_col, subsize);
- set_block_size(cpi, x, xd, mi_row + block_height / 2, mi_col, subsize);
- return 1;
- }
- }
-
- return 0;
- }
- return 0;
-}
-
-// Set the variance split thresholds for following the block sizes:
-// 0 - threshold_64x64, 1 - threshold_32x32, 2 - threshold_16x16,
-// 3 - vbp_threshold_8x8. vbp_threshold_8x8 (to split to 4x4 partition) is
-// currently only used on key frame.
-static void set_vbp_thresholds(VP10_COMP *cpi, int64_t thresholds[], int q) {
- VP10_COMMON *const cm = &cpi->common;
- const int is_key_frame = (cm->frame_type == KEY_FRAME);
- const int threshold_multiplier = is_key_frame ? 20 : 1;
- const int64_t threshold_base = (int64_t)(threshold_multiplier *
- cpi->y_dequant[q][1]);
- if (is_key_frame) {
- thresholds[0] = threshold_base;
- thresholds[1] = threshold_base >> 2;
- thresholds[2] = threshold_base >> 2;
- thresholds[3] = threshold_base << 2;
- } else {
- thresholds[1] = threshold_base;
- if (cm->width <= 352 && cm->height <= 288) {
- thresholds[0] = threshold_base >> 2;
- thresholds[2] = threshold_base << 3;
- } else {
- thresholds[0] = threshold_base;
- thresholds[1] = (5 * threshold_base) >> 2;
- if (cm->width >= 1920 && cm->height >= 1080)
- thresholds[1] = (7 * threshold_base) >> 2;
- thresholds[2] = threshold_base << cpi->oxcf.speed;
- }
- }
-}
-
-void vp10_set_variance_partition_thresholds(VP10_COMP *cpi, int q) {
- VP10_COMMON *const cm = &cpi->common;
- SPEED_FEATURES *const sf = &cpi->sf;
- const int is_key_frame = (cm->frame_type == KEY_FRAME);
- if (sf->partition_search_type != VAR_BASED_PARTITION &&
- sf->partition_search_type != REFERENCE_PARTITION) {
- return;
- } else {
- set_vbp_thresholds(cpi, cpi->vbp_thresholds, q);
- // The thresholds below are not changed locally.
- if (is_key_frame) {
- cpi->vbp_threshold_sad = 0;
- cpi->vbp_bsize_min = BLOCK_8X8;
- } else {
- if (cm->width <= 352 && cm->height <= 288)
- cpi->vbp_threshold_sad = 100;
- else
- cpi->vbp_threshold_sad = (cpi->y_dequant[q][1] << 1) > 1000 ?
- (cpi->y_dequant[q][1] << 1) : 1000;
- cpi->vbp_bsize_min = BLOCK_16X16;
- }
- cpi->vbp_threshold_minmax = 15 + (q >> 3);
- }
-}
-
-// Compute the minmax over the 8x8 subblocks.
-static int compute_minmax_8x8(const uint8_t *s, int sp, const uint8_t *d,
- int dp, int x16_idx, int y16_idx,
-#if CONFIG_VP9_HIGHBITDEPTH
- int highbd_flag,
-#endif
- int pixels_wide,
- int pixels_high) {
- int k;
- int minmax_max = 0;
- int minmax_min = 255;
- // Loop over the 4 8x8 subblocks.
- for (k = 0; k < 4; k++) {
- int x8_idx = x16_idx + ((k & 1) << 3);
- int y8_idx = y16_idx + ((k >> 1) << 3);
- int min = 0;
- int max = 0;
- if (x8_idx < pixels_wide && y8_idx < pixels_high) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (highbd_flag & YV12_FLAG_HIGHBITDEPTH) {
- vp10_highbd_minmax_8x8(s + y8_idx * sp + x8_idx, sp,
- d + y8_idx * dp + x8_idx, dp,
- &min, &max);
- } else {
- vp10_minmax_8x8(s + y8_idx * sp + x8_idx, sp,
- d + y8_idx * dp + x8_idx, dp,
- &min, &max);
- }
-#else
- vp10_minmax_8x8(s + y8_idx * sp + x8_idx, sp,
- d + y8_idx * dp + x8_idx, dp,
- &min, &max);
-#endif
- if ((max - min) > minmax_max)
- minmax_max = (max - min);
- if ((max - min) < minmax_min)
- minmax_min = (max - min);
- }
- }
- return (minmax_max - minmax_min);
-}
-
-static void fill_variance_4x4avg(const uint8_t *s, int sp, const uint8_t *d,
- int dp, int x8_idx, int y8_idx, v8x8 *vst,
-#if CONFIG_VP9_HIGHBITDEPTH
- int highbd_flag,
-#endif
- int pixels_wide,
- int pixels_high,
- int is_key_frame) {
- int k;
- for (k = 0; k < 4; k++) {
- int x4_idx = x8_idx + ((k & 1) << 2);
- int y4_idx = y8_idx + ((k >> 1) << 2);
- unsigned int sse = 0;
- int sum = 0;
- if (x4_idx < pixels_wide && y4_idx < pixels_high) {
- int s_avg;
- int d_avg = 128;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (highbd_flag & YV12_FLAG_HIGHBITDEPTH) {
- s_avg = vp10_highbd_avg_4x4(s + y4_idx * sp + x4_idx, sp);
- if (!is_key_frame)
- d_avg = vp10_highbd_avg_4x4(d + y4_idx * dp + x4_idx, dp);
- } else {
- s_avg = vp10_avg_4x4(s + y4_idx * sp + x4_idx, sp);
- if (!is_key_frame)
- d_avg = vp10_avg_4x4(d + y4_idx * dp + x4_idx, dp);
- }
-#else
- s_avg = vp10_avg_4x4(s + y4_idx * sp + x4_idx, sp);
- if (!is_key_frame)
- d_avg = vp10_avg_4x4(d + y4_idx * dp + x4_idx, dp);
-#endif
- sum = s_avg - d_avg;
- sse = sum * sum;
- }
- fill_variance(sse, sum, 0, &vst->split[k].part_variances.none);
- }
-}
-
-static void fill_variance_8x8avg(const uint8_t *s, int sp, const uint8_t *d,
- int dp, int x16_idx, int y16_idx, v16x16 *vst,
-#if CONFIG_VP9_HIGHBITDEPTH
- int highbd_flag,
-#endif
- int pixels_wide,
- int pixels_high,
- int is_key_frame) {
- int k;
- for (k = 0; k < 4; k++) {
- int x8_idx = x16_idx + ((k & 1) << 3);
- int y8_idx = y16_idx + ((k >> 1) << 3);
- unsigned int sse = 0;
- int sum = 0;
- if (x8_idx < pixels_wide && y8_idx < pixels_high) {
- int s_avg;
- int d_avg = 128;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (highbd_flag & YV12_FLAG_HIGHBITDEPTH) {
- s_avg = vp10_highbd_avg_8x8(s + y8_idx * sp + x8_idx, sp);
- if (!is_key_frame)
- d_avg = vp10_highbd_avg_8x8(d + y8_idx * dp + x8_idx, dp);
- } else {
- s_avg = vp10_avg_8x8(s + y8_idx * sp + x8_idx, sp);
- if (!is_key_frame)
- d_avg = vp10_avg_8x8(d + y8_idx * dp + x8_idx, dp);
- }
-#else
- s_avg = vp10_avg_8x8(s + y8_idx * sp + x8_idx, sp);
- if (!is_key_frame)
- d_avg = vp10_avg_8x8(d + y8_idx * dp + x8_idx, dp);
-#endif
- sum = s_avg - d_avg;
- sse = sum * sum;
- }
- fill_variance(sse, sum, 0, &vst->split[k].part_variances.none);
- }
-}
-
-// This function chooses partitioning based on the variance between source and
-// reconstructed last, where variance is computed for down-sampled inputs.
-static int choose_partitioning(VP10_COMP *cpi,
- const TileInfo *const tile,
- MACROBLOCK *x,
- int mi_row, int mi_col) {
- VP10_COMMON * const cm = &cpi->common;
- MACROBLOCKD *xd = &x->e_mbd;
- int i, j, k, m;
- v64x64 vt;
- v16x16 vt2[16];
- int force_split[21];
- uint8_t *s;
- const uint8_t *d;
- int sp;
- int dp;
- int pixels_wide = 64, pixels_high = 64;
- int64_t thresholds[4] = {cpi->vbp_thresholds[0], cpi->vbp_thresholds[1],
- cpi->vbp_thresholds[2], cpi->vbp_thresholds[3]};
-
- // Always use 4x4 partition for key frame.
- const int is_key_frame = (cm->frame_type == KEY_FRAME);
- const int use_4x4_partition = is_key_frame;
- const int low_res = (cm->width <= 352 && cm->height <= 288);
- int variance4x4downsample[16];
-
- int segment_id = CR_SEGMENT_ID_BASE;
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ && cm->seg.enabled) {
- const uint8_t *const map = cm->seg.update_map ? cpi->segmentation_map :
- cm->last_frame_seg_map;
- segment_id = get_segment_id(cm, map, BLOCK_64X64, mi_row, mi_col);
-
- if (cyclic_refresh_segment_id_boosted(segment_id)) {
- int q = vp10_get_qindex(&cm->seg, segment_id, cm->base_qindex);
- set_vbp_thresholds(cpi, thresholds, q);
- }
- }
-
- set_offsets(cpi, tile, x, mi_row, mi_col, BLOCK_64X64);
-
- if (xd->mb_to_right_edge < 0)
- pixels_wide += (xd->mb_to_right_edge >> 3);
- if (xd->mb_to_bottom_edge < 0)
- pixels_high += (xd->mb_to_bottom_edge >> 3);
-
- s = x->plane[0].src.buf;
- sp = x->plane[0].src.stride;
-
- if (!is_key_frame) {
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- unsigned int uv_sad;
- const YV12_BUFFER_CONFIG *yv12 = get_ref_frame_buffer(cpi, LAST_FRAME);
-
- const YV12_BUFFER_CONFIG *yv12_g = NULL;
- unsigned int y_sad, y_sad_g;
- const BLOCK_SIZE bsize = BLOCK_32X32
- + (mi_col + 4 < cm->mi_cols) * 2 + (mi_row + 4 < cm->mi_rows);
-
- assert(yv12 != NULL);
- yv12_g = get_ref_frame_buffer(cpi, GOLDEN_FRAME);
-
- if (yv12_g && yv12_g != yv12) {
- vp10_setup_pre_planes(xd, 0, yv12_g, mi_row, mi_col,
- &cm->frame_refs[GOLDEN_FRAME - 1].sf);
- y_sad_g = cpi->fn_ptr[bsize].sdf(x->plane[0].src.buf,
- x->plane[0].src.stride,
- xd->plane[0].pre[0].buf,
- xd->plane[0].pre[0].stride);
- } else {
- y_sad_g = UINT_MAX;
- }
-
- vp10_setup_pre_planes(xd, 0, yv12, mi_row, mi_col,
- &cm->frame_refs[LAST_FRAME - 1].sf);
- mbmi->ref_frame[0] = LAST_FRAME;
- mbmi->ref_frame[1] = NONE;
- mbmi->sb_type = BLOCK_64X64;
- mbmi->mv[0].as_int = 0;
- mbmi->interp_filter = BILINEAR;
-
- y_sad = vp10_int_pro_motion_estimation(cpi, x, bsize, mi_row, mi_col);
- if (y_sad_g < y_sad) {
- vp10_setup_pre_planes(xd, 0, yv12_g, mi_row, mi_col,
- &cm->frame_refs[GOLDEN_FRAME - 1].sf);
- mbmi->ref_frame[0] = GOLDEN_FRAME;
- mbmi->mv[0].as_int = 0;
- y_sad = y_sad_g;
- } else {
- x->pred_mv[LAST_FRAME] = mbmi->mv[0].as_mv;
- }
-
- vp10_build_inter_predictors_sb(xd, mi_row, mi_col, BLOCK_64X64);
-
- for (i = 1; i <= 2; ++i) {
- struct macroblock_plane *p = &x->plane[i];
- struct macroblockd_plane *pd = &xd->plane[i];
- const BLOCK_SIZE bs = get_plane_block_size(bsize, pd);
-
- if (bs == BLOCK_INVALID)
- uv_sad = UINT_MAX;
- else
- uv_sad = cpi->fn_ptr[bs].sdf(p->src.buf, p->src.stride,
- pd->dst.buf, pd->dst.stride);
-
- x->color_sensitivity[i - 1] = uv_sad > (y_sad >> 2);
- }
-
- d = xd->plane[0].dst.buf;
- dp = xd->plane[0].dst.stride;
-
- // If the y_sad is very small, take 64x64 as partition and exit.
- // Don't check on boosted segment for now, as 64x64 is suppressed there.
- if (segment_id == CR_SEGMENT_ID_BASE &&
- y_sad < cpi->vbp_threshold_sad) {
- const int block_width = num_8x8_blocks_wide_lookup[BLOCK_64X64];
- const int block_height = num_8x8_blocks_high_lookup[BLOCK_64X64];
- if (mi_col + block_width / 2 < cm->mi_cols &&
- mi_row + block_height / 2 < cm->mi_rows) {
- set_block_size(cpi, x, xd, mi_row, mi_col, BLOCK_64X64);
- return 0;
- }
- }
- } else {
- d = VP9_VAR_OFFS;
- dp = 0;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (xd->bd) {
- case 10:
- d = CONVERT_TO_BYTEPTR(VP9_HIGH_VAR_OFFS_10);
- break;
- case 12:
- d = CONVERT_TO_BYTEPTR(VP9_HIGH_VAR_OFFS_12);
- break;
- case 8:
- default:
- d = CONVERT_TO_BYTEPTR(VP9_HIGH_VAR_OFFS_8);
- break;
- }
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
-
- // Index for force_split: 0 for 64x64, 1-4 for 32x32 blocks,
- // 5-20 for the 16x16 blocks.
- force_split[0] = 0;
- // Fill in the entire tree of 8x8 (or 4x4 under some conditions) variances
- // for splits.
- for (i = 0; i < 4; i++) {
- const int x32_idx = ((i & 1) << 5);
- const int y32_idx = ((i >> 1) << 5);
- const int i2 = i << 2;
- force_split[i + 1] = 0;
- for (j = 0; j < 4; j++) {
- const int x16_idx = x32_idx + ((j & 1) << 4);
- const int y16_idx = y32_idx + ((j >> 1) << 4);
- const int split_index = 5 + i2 + j;
- v16x16 *vst = &vt.split[i].split[j];
- force_split[split_index] = 0;
- variance4x4downsample[i2 + j] = 0;
- if (!is_key_frame) {
- fill_variance_8x8avg(s, sp, d, dp, x16_idx, y16_idx, vst,
-#if CONFIG_VP9_HIGHBITDEPTH
- xd->cur_buf->flags,
-#endif
- pixels_wide,
- pixels_high,
- is_key_frame);
- fill_variance_tree(&vt.split[i].split[j], BLOCK_16X16);
- get_variance(&vt.split[i].split[j].part_variances.none);
- if (vt.split[i].split[j].part_variances.none.variance >
- thresholds[2]) {
- // 16X16 variance is above threshold for split, so force split to 8x8
- // for this 16x16 block (this also forces splits for upper levels).
- force_split[split_index] = 1;
- force_split[i + 1] = 1;
- force_split[0] = 1;
- } else if (vt.split[i].split[j].part_variances.none.variance >
- thresholds[1] &&
- !cyclic_refresh_segment_id_boosted(segment_id)) {
- // We have some nominal amount of 16x16 variance (based on average),
- // compute the minmax over the 8x8 sub-blocks, and if above threshold,
- // force split to 8x8 block for this 16x16 block.
- int minmax = compute_minmax_8x8(s, sp, d, dp, x16_idx, y16_idx,
-#if CONFIG_VP9_HIGHBITDEPTH
- xd->cur_buf->flags,
-#endif
- pixels_wide, pixels_high);
- if (minmax > cpi->vbp_threshold_minmax) {
- force_split[split_index] = 1;
- force_split[i + 1] = 1;
- force_split[0] = 1;
- }
- }
- }
- if (is_key_frame || (low_res &&
- vt.split[i].split[j].part_variances.none.variance >
- (thresholds[1] << 1))) {
- force_split[split_index] = 0;
- // Go down to 4x4 down-sampling for variance.
- variance4x4downsample[i2 + j] = 1;
- for (k = 0; k < 4; k++) {
- int x8_idx = x16_idx + ((k & 1) << 3);
- int y8_idx = y16_idx + ((k >> 1) << 3);
- v8x8 *vst2 = is_key_frame ? &vst->split[k] :
- &vt2[i2 + j].split[k];
- fill_variance_4x4avg(s, sp, d, dp, x8_idx, y8_idx, vst2,
-#if CONFIG_VP9_HIGHBITDEPTH
- xd->cur_buf->flags,
-#endif
- pixels_wide,
- pixels_high,
- is_key_frame);
- }
- }
- }
- }
-
- // Fill the rest of the variance tree by summing split partition values.
- for (i = 0; i < 4; i++) {
- const int i2 = i << 2;
- for (j = 0; j < 4; j++) {
- if (variance4x4downsample[i2 + j] == 1) {
- v16x16 *vtemp = (!is_key_frame) ? &vt2[i2 + j] :
- &vt.split[i].split[j];
- for (m = 0; m < 4; m++)
- fill_variance_tree(&vtemp->split[m], BLOCK_8X8);
- fill_variance_tree(vtemp, BLOCK_16X16);
- }
- }
- fill_variance_tree(&vt.split[i], BLOCK_32X32);
- // If variance of this 32x32 block is above the threshold, force the block
- // to split. This also forces a split on the upper (64x64) level.
- if (!force_split[i + 1]) {
- get_variance(&vt.split[i].part_variances.none);
- if (vt.split[i].part_variances.none.variance > thresholds[1]) {
- force_split[i + 1] = 1;
- force_split[0] = 1;
- }
- }
- }
- if (!force_split[0]) {
- fill_variance_tree(&vt, BLOCK_64X64);
- get_variance(&vt.part_variances.none);
- }
-
- // Now go through the entire structure, splitting every block size until
- // we get to one that's got a variance lower than our threshold.
- if ( mi_col + 8 > cm->mi_cols || mi_row + 8 > cm->mi_rows ||
- !set_vt_partitioning(cpi, x, xd, &vt, BLOCK_64X64, mi_row, mi_col,
- thresholds[0], BLOCK_16X16, force_split[0])) {
- for (i = 0; i < 4; ++i) {
- const int x32_idx = ((i & 1) << 2);
- const int y32_idx = ((i >> 1) << 2);
- const int i2 = i << 2;
- if (!set_vt_partitioning(cpi, x, xd, &vt.split[i], BLOCK_32X32,
- (mi_row + y32_idx), (mi_col + x32_idx),
- thresholds[1], BLOCK_16X16,
- force_split[i + 1])) {
- for (j = 0; j < 4; ++j) {
- const int x16_idx = ((j & 1) << 1);
- const int y16_idx = ((j >> 1) << 1);
- // For inter frames: if variance4x4downsample[] == 1 for this 16x16
- // block, then the variance is based on 4x4 down-sampling, so use vt2
- // in set_vt_partioning(), otherwise use vt.
- v16x16 *vtemp = (!is_key_frame &&
- variance4x4downsample[i2 + j] == 1) ?
- &vt2[i2 + j] : &vt.split[i].split[j];
- if (!set_vt_partitioning(cpi, x, xd, vtemp, BLOCK_16X16,
- mi_row + y32_idx + y16_idx,
- mi_col + x32_idx + x16_idx,
- thresholds[2],
- cpi->vbp_bsize_min,
- force_split[5 + i2 + j])) {
- for (k = 0; k < 4; ++k) {
- const int x8_idx = (k & 1);
- const int y8_idx = (k >> 1);
- if (use_4x4_partition) {
- if (!set_vt_partitioning(cpi, x, xd, &vtemp->split[k],
- BLOCK_8X8,
- mi_row + y32_idx + y16_idx + y8_idx,
- mi_col + x32_idx + x16_idx + x8_idx,
- thresholds[3], BLOCK_8X8, 0)) {
- set_block_size(cpi, x, xd,
- (mi_row + y32_idx + y16_idx + y8_idx),
- (mi_col + x32_idx + x16_idx + x8_idx),
- BLOCK_4X4);
- }
- } else {
- set_block_size(cpi, x, xd,
- (mi_row + y32_idx + y16_idx + y8_idx),
- (mi_col + x32_idx + x16_idx + x8_idx),
- BLOCK_8X8);
- }
- }
- }
- }
- }
- }
- }
- return 0;
-}
-
-static void update_state(VP10_COMP *cpi, ThreadData *td,
- PICK_MODE_CONTEXT *ctx,
- int mi_row, int mi_col, BLOCK_SIZE bsize,
- int output_enabled) {
- int i, x_idx, y;
- VP10_COMMON *const cm = &cpi->common;
- RD_COUNTS *const rdc = &td->rd_counts;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- struct macroblock_plane *const p = x->plane;
- struct macroblockd_plane *const pd = xd->plane;
- MODE_INFO *mi = &ctx->mic;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- MODE_INFO *mi_addr = xd->mi[0];
- const struct segmentation *const seg = &cm->seg;
- const int bw = num_8x8_blocks_wide_lookup[mi->mbmi.sb_type];
- const int bh = num_8x8_blocks_high_lookup[mi->mbmi.sb_type];
- const int x_mis = VPXMIN(bw, cm->mi_cols - mi_col);
- const int y_mis = VPXMIN(bh, cm->mi_rows - mi_row);
- MV_REF *const frame_mvs =
- cm->cur_frame->mvs + mi_row * cm->mi_cols + mi_col;
- int w, h;
-
- const int mis = cm->mi_stride;
- const int mi_width = num_8x8_blocks_wide_lookup[bsize];
- const int mi_height = num_8x8_blocks_high_lookup[bsize];
- int max_plane;
-
- assert(mi->mbmi.sb_type == bsize);
-
- *mi_addr = *mi;
- *x->mbmi_ext = ctx->mbmi_ext;
-
- // If segmentation in use
- if (seg->enabled) {
- // For in frame complexity AQ copy the segment id from the segment map.
- if (cpi->oxcf.aq_mode == COMPLEXITY_AQ) {
- const uint8_t *const map = seg->update_map ? cpi->segmentation_map
- : cm->last_frame_seg_map;
- mi_addr->mbmi.segment_id =
- get_segment_id(cm, map, bsize, mi_row, mi_col);
- }
- // Else for cyclic refresh mode update the segment map, set the segment id
- // and then update the quantizer.
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ) {
- vp10_cyclic_refresh_update_segment(cpi, &xd->mi[0]->mbmi, mi_row,
- mi_col, bsize, ctx->rate, ctx->dist,
- x->skip);
- }
- }
-
- max_plane = is_inter_block(mbmi) ? MAX_MB_PLANE : 1;
- for (i = 0; i < max_plane; ++i) {
- p[i].coeff = ctx->coeff_pbuf[i][1];
- p[i].qcoeff = ctx->qcoeff_pbuf[i][1];
- pd[i].dqcoeff = ctx->dqcoeff_pbuf[i][1];
- p[i].eobs = ctx->eobs_pbuf[i][1];
- }
-
- for (i = max_plane; i < MAX_MB_PLANE; ++i) {
- p[i].coeff = ctx->coeff_pbuf[i][2];
- p[i].qcoeff = ctx->qcoeff_pbuf[i][2];
- pd[i].dqcoeff = ctx->dqcoeff_pbuf[i][2];
- p[i].eobs = ctx->eobs_pbuf[i][2];
- }
-
- for (i = 0; i < 2; ++i)
- pd[i].color_index_map = ctx->color_index_map[i];
-
- // Restore the coding context of the MB to that that was in place
- // when the mode was picked for it
- for (y = 0; y < mi_height; y++)
- for (x_idx = 0; x_idx < mi_width; x_idx++)
- if ((xd->mb_to_right_edge >> (3 + MI_SIZE_LOG2)) + mi_width > x_idx
- && (xd->mb_to_bottom_edge >> (3 + MI_SIZE_LOG2)) + mi_height > y) {
- xd->mi[x_idx + y * mis] = mi_addr;
- }
-
- if (cpi->oxcf.aq_mode)
- vp10_init_plane_quantizers(cpi, x);
-
- if (is_inter_block(mbmi) && mbmi->sb_type < BLOCK_8X8) {
- mbmi->mv[0].as_int = mi->bmi[3].as_mv[0].as_int;
- mbmi->mv[1].as_int = mi->bmi[3].as_mv[1].as_int;
- }
-
- x->skip = ctx->skip;
- memcpy(x->zcoeff_blk[mbmi->tx_size], ctx->zcoeff_blk,
- sizeof(ctx->zcoeff_blk[0]) * ctx->num_4x4_blk);
-
- if (!output_enabled)
- return;
-
-#if CONFIG_INTERNAL_STATS
- if (frame_is_intra_only(cm)) {
- static const int kf_mode_index[] = {
- THR_DC /*DC_PRED*/,
- THR_V_PRED /*V_PRED*/,
- THR_H_PRED /*H_PRED*/,
- THR_D45_PRED /*D45_PRED*/,
- THR_D135_PRED /*D135_PRED*/,
- THR_D117_PRED /*D117_PRED*/,
- THR_D153_PRED /*D153_PRED*/,
- THR_D207_PRED /*D207_PRED*/,
- THR_D63_PRED /*D63_PRED*/,
- THR_TM /*TM_PRED*/,
- };
- ++cpi->mode_chosen_counts[kf_mode_index[mbmi->mode]];
- } else {
- // Note how often each mode chosen as best
- ++cpi->mode_chosen_counts[ctx->best_mode_index];
- }
-#endif
- if (!frame_is_intra_only(cm)) {
- if (is_inter_block(mbmi)) {
- vp10_update_mv_count(td);
-
- if (cm->interp_filter == SWITCHABLE) {
- const int ctx = vp10_get_pred_context_switchable_interp(xd);
- ++td->counts->switchable_interp[ctx][mbmi->interp_filter];
- }
- }
-
- rdc->comp_pred_diff[SINGLE_REFERENCE] += ctx->single_pred_diff;
- rdc->comp_pred_diff[COMPOUND_REFERENCE] += ctx->comp_pred_diff;
- rdc->comp_pred_diff[REFERENCE_MODE_SELECT] += ctx->hybrid_pred_diff;
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- rdc->filter_diff[i] += ctx->best_filter_diff[i];
- }
-
- for (h = 0; h < y_mis; ++h) {
- MV_REF *const frame_mv = frame_mvs + h * cm->mi_cols;
- for (w = 0; w < x_mis; ++w) {
- MV_REF *const mv = frame_mv + w;
- mv->ref_frame[0] = mi->mbmi.ref_frame[0];
- mv->ref_frame[1] = mi->mbmi.ref_frame[1];
- mv->mv[0].as_int = mi->mbmi.mv[0].as_int;
- mv->mv[1].as_int = mi->mbmi.mv[1].as_int;
- }
- }
-}
-
-void vp10_setup_src_planes(MACROBLOCK *x, const YV12_BUFFER_CONFIG *src,
- int mi_row, int mi_col) {
- uint8_t *const buffers[3] = {src->y_buffer, src->u_buffer, src->v_buffer };
- const int strides[3] = {src->y_stride, src->uv_stride, src->uv_stride };
- int i;
-
- // Set current frame pointer.
- x->e_mbd.cur_buf = src;
-
- for (i = 0; i < MAX_MB_PLANE; i++)
- setup_pred_plane(&x->plane[i].src, buffers[i], strides[i], mi_row, mi_col,
- NULL, x->e_mbd.plane[i].subsampling_x,
- x->e_mbd.plane[i].subsampling_y);
-}
-
-static int set_segment_rdmult(VP10_COMP *const cpi,
- MACROBLOCK *const x,
- int8_t segment_id) {
- int segment_qindex;
- VP10_COMMON *const cm = &cpi->common;
- vp10_init_plane_quantizers(cpi, x);
- vpx_clear_system_state();
- segment_qindex = vp10_get_qindex(&cm->seg, segment_id,
- cm->base_qindex);
- return vp10_compute_rd_mult(cpi, segment_qindex + cm->y_dc_delta_q);
-}
-
-static void rd_pick_sb_modes(VP10_COMP *cpi,
- TileDataEnc *tile_data,
- MACROBLOCK *const x,
- int mi_row, int mi_col, RD_COST *rd_cost,
- BLOCK_SIZE bsize, PICK_MODE_CONTEXT *ctx,
- int64_t best_rd) {
- VP10_COMMON *const cm = &cpi->common;
- TileInfo *const tile_info = &tile_data->tile_info;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *mbmi;
- struct macroblock_plane *const p = x->plane;
- struct macroblockd_plane *const pd = xd->plane;
- const AQ_MODE aq_mode = cpi->oxcf.aq_mode;
- int i, orig_rdmult;
-
- vpx_clear_system_state();
-
- // Use the lower precision, but faster, 32x32 fdct for mode selection.
- x->use_lp32x32fdct = 1;
-
- set_offsets(cpi, tile_info, x, mi_row, mi_col, bsize);
- mbmi = &xd->mi[0]->mbmi;
- mbmi->sb_type = bsize;
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- p[i].coeff = ctx->coeff_pbuf[i][0];
- p[i].qcoeff = ctx->qcoeff_pbuf[i][0];
- pd[i].dqcoeff = ctx->dqcoeff_pbuf[i][0];
- p[i].eobs = ctx->eobs_pbuf[i][0];
- }
-
- if (cm->current_video_frame == 0 && cm->allow_screen_content_tools) {
- for (i = 0; i < 2; ++i) {
- if (ctx->color_index_map[i] == 0) {
- CHECK_MEM_ERROR(cm, ctx->color_index_map[i],
- vpx_memalign(16, (ctx->num_4x4_blk << 4) *
- sizeof(*ctx->color_index_map[i])));
- }
- }
- }
- for (i = 0; i < 2; ++i)
- pd[i].color_index_map = ctx->color_index_map[i];
-
- ctx->is_coded = 0;
- ctx->skippable = 0;
- ctx->pred_pixel_ready = 0;
- x->skip_recode = 0;
-
- // Set to zero to make sure we do not use the previous encoded frame stats
- mbmi->skip = 0;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- x->source_variance =
- vp10_high_get_sby_perpixel_variance(cpi, &x->plane[0].src,
- bsize, xd->bd);
- } else {
- x->source_variance =
- vp10_get_sby_perpixel_variance(cpi, &x->plane[0].src, bsize);
- }
-#else
- x->source_variance =
- vp10_get_sby_perpixel_variance(cpi, &x->plane[0].src, bsize);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Save rdmult before it might be changed, so it can be restored later.
- orig_rdmult = x->rdmult;
-
- if (aq_mode == VARIANCE_AQ) {
- const int energy = bsize <= BLOCK_16X16 ? x->mb_energy
- : vp10_block_energy(cpi, x, bsize);
- if (cm->frame_type == KEY_FRAME ||
- cpi->refresh_alt_ref_frame ||
- (cpi->refresh_golden_frame && !cpi->rc.is_src_frame_alt_ref)) {
- mbmi->segment_id = vp10_vaq_segment_id(energy);
- } else {
- const uint8_t *const map = cm->seg.update_map ? cpi->segmentation_map
- : cm->last_frame_seg_map;
- mbmi->segment_id = get_segment_id(cm, map, bsize, mi_row, mi_col);
- }
- x->rdmult = set_segment_rdmult(cpi, x, mbmi->segment_id);
- } else if (aq_mode == COMPLEXITY_AQ) {
- x->rdmult = set_segment_rdmult(cpi, x, mbmi->segment_id);
- } else if (aq_mode == CYCLIC_REFRESH_AQ) {
- const uint8_t *const map = cm->seg.update_map ? cpi->segmentation_map
- : cm->last_frame_seg_map;
- // If segment is boosted, use rdmult for that segment.
- if (cyclic_refresh_segment_id_boosted(
- get_segment_id(cm, map, bsize, mi_row, mi_col)))
- x->rdmult = vp10_cyclic_refresh_get_rdmult(cpi->cyclic_refresh);
- }
-
- // Find best coding mode & reconstruct the MB so it is available
- // as a predictor for MBs that follow in the SB
- if (frame_is_intra_only(cm)) {
- vp10_rd_pick_intra_mode_sb(cpi, x, rd_cost, bsize, ctx, best_rd);
- } else {
- if (bsize >= BLOCK_8X8) {
- if (segfeature_active(&cm->seg, mbmi->segment_id, SEG_LVL_SKIP))
- vp10_rd_pick_inter_mode_sb_seg_skip(cpi, tile_data, x, rd_cost, bsize,
- ctx, best_rd);
- else
- vp10_rd_pick_inter_mode_sb(cpi, tile_data, x, mi_row, mi_col,
- rd_cost, bsize, ctx, best_rd);
- } else {
- vp10_rd_pick_inter_mode_sub8x8(cpi, tile_data, x, mi_row, mi_col,
- rd_cost, bsize, ctx, best_rd);
- }
- }
-
-
- // Examine the resulting rate and for AQ mode 2 make a segment choice.
- if ((rd_cost->rate != INT_MAX) &&
- (aq_mode == COMPLEXITY_AQ) && (bsize >= BLOCK_16X16) &&
- (cm->frame_type == KEY_FRAME ||
- cpi->refresh_alt_ref_frame ||
- (cpi->refresh_golden_frame && !cpi->rc.is_src_frame_alt_ref))) {
- vp10_caq_select_segment(cpi, x, bsize, mi_row, mi_col, rd_cost->rate);
- }
-
- x->rdmult = orig_rdmult;
-
- // TODO(jingning) The rate-distortion optimization flow needs to be
- // refactored to provide proper exit/return handle.
- if (rd_cost->rate == INT_MAX)
- rd_cost->rdcost = INT64_MAX;
-
- ctx->rate = rd_cost->rate;
- ctx->dist = rd_cost->dist;
-}
-
-static void update_stats(VP10_COMMON *cm, ThreadData *td) {
- const MACROBLOCK *x = &td->mb;
- const MACROBLOCKD *const xd = &x->e_mbd;
- const MODE_INFO *const mi = xd->mi[0];
- const MB_MODE_INFO *const mbmi = &mi->mbmi;
- const MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
- const BLOCK_SIZE bsize = mbmi->sb_type;
-
- if (!frame_is_intra_only(cm)) {
- FRAME_COUNTS *const counts = td->counts;
- const int inter_block = is_inter_block(mbmi);
- const int seg_ref_active = segfeature_active(&cm->seg, mbmi->segment_id,
- SEG_LVL_REF_FRAME);
- if (!seg_ref_active) {
- counts->intra_inter[vp10_get_intra_inter_context(xd)][inter_block]++;
- // If the segment reference feature is enabled we have only a single
- // reference frame allowed for the segment so exclude it from
- // the reference frame counts used to work out probabilities.
- if (inter_block) {
- const MV_REFERENCE_FRAME ref0 = mbmi->ref_frame[0];
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- counts->comp_inter[vp10_get_reference_mode_context(cm, xd)]
- [has_second_ref(mbmi)]++;
-
- if (has_second_ref(mbmi)) {
- counts->comp_ref[vp10_get_pred_context_comp_ref_p(cm, xd)]
- [ref0 == GOLDEN_FRAME]++;
- } else {
- counts->single_ref[vp10_get_pred_context_single_ref_p1(xd)][0]
- [ref0 != LAST_FRAME]++;
- if (ref0 != LAST_FRAME)
- counts->single_ref[vp10_get_pred_context_single_ref_p2(xd)][1]
- [ref0 != GOLDEN_FRAME]++;
- }
- }
- }
- if (inter_block &&
- !segfeature_active(&cm->seg, mbmi->segment_id, SEG_LVL_SKIP)) {
- const int mode_ctx = mbmi_ext->mode_context[mbmi->ref_frame[0]];
- if (bsize >= BLOCK_8X8) {
- const PREDICTION_MODE mode = mbmi->mode;
- ++counts->inter_mode[mode_ctx][INTER_OFFSET(mode)];
- } else {
- const int num_4x4_w = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[bsize];
- int idx, idy;
- for (idy = 0; idy < 2; idy += num_4x4_h) {
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- const int j = idy * 2 + idx;
- const PREDICTION_MODE b_mode = mi->bmi[j].as_mode;
- ++counts->inter_mode[mode_ctx][INTER_OFFSET(b_mode)];
- }
- }
- }
- }
- }
-}
-
-static void restore_context(MACROBLOCK *const x, int mi_row, int mi_col,
- ENTROPY_CONTEXT a[16 * MAX_MB_PLANE],
- ENTROPY_CONTEXT l[16 * MAX_MB_PLANE],
- PARTITION_CONTEXT sa[8], PARTITION_CONTEXT sl[8],
- BLOCK_SIZE bsize) {
- MACROBLOCKD *const xd = &x->e_mbd;
- int p;
- const int num_4x4_blocks_wide = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_blocks_high = num_4x4_blocks_high_lookup[bsize];
- int mi_width = num_8x8_blocks_wide_lookup[bsize];
- int mi_height = num_8x8_blocks_high_lookup[bsize];
- for (p = 0; p < MAX_MB_PLANE; p++) {
- memcpy(
- xd->above_context[p] + ((mi_col * 2) >> xd->plane[p].subsampling_x),
- a + num_4x4_blocks_wide * p,
- (sizeof(ENTROPY_CONTEXT) * num_4x4_blocks_wide) >>
- xd->plane[p].subsampling_x);
- memcpy(
- xd->left_context[p]
- + ((mi_row & MI_MASK) * 2 >> xd->plane[p].subsampling_y),
- l + num_4x4_blocks_high * p,
- (sizeof(ENTROPY_CONTEXT) * num_4x4_blocks_high) >>
- xd->plane[p].subsampling_y);
- }
- memcpy(xd->above_seg_context + mi_col, sa,
- sizeof(*xd->above_seg_context) * mi_width);
- memcpy(xd->left_seg_context + (mi_row & MI_MASK), sl,
- sizeof(xd->left_seg_context[0]) * mi_height);
-}
-
-static void save_context(MACROBLOCK *const x, int mi_row, int mi_col,
- ENTROPY_CONTEXT a[16 * MAX_MB_PLANE],
- ENTROPY_CONTEXT l[16 * MAX_MB_PLANE],
- PARTITION_CONTEXT sa[8], PARTITION_CONTEXT sl[8],
- BLOCK_SIZE bsize) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- int p;
- const int num_4x4_blocks_wide = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_blocks_high = num_4x4_blocks_high_lookup[bsize];
- int mi_width = num_8x8_blocks_wide_lookup[bsize];
- int mi_height = num_8x8_blocks_high_lookup[bsize];
-
- // buffer the above/left context information of the block in search.
- for (p = 0; p < MAX_MB_PLANE; ++p) {
- memcpy(
- a + num_4x4_blocks_wide * p,
- xd->above_context[p] + (mi_col * 2 >> xd->plane[p].subsampling_x),
- (sizeof(ENTROPY_CONTEXT) * num_4x4_blocks_wide) >>
- xd->plane[p].subsampling_x);
- memcpy(
- l + num_4x4_blocks_high * p,
- xd->left_context[p]
- + ((mi_row & MI_MASK) * 2 >> xd->plane[p].subsampling_y),
- (sizeof(ENTROPY_CONTEXT) * num_4x4_blocks_high) >>
- xd->plane[p].subsampling_y);
- }
- memcpy(sa, xd->above_seg_context + mi_col,
- sizeof(*xd->above_seg_context) * mi_width);
- memcpy(sl, xd->left_seg_context + (mi_row & MI_MASK),
- sizeof(xd->left_seg_context[0]) * mi_height);
-}
-
-static void encode_b(VP10_COMP *cpi, const TileInfo *const tile,
- ThreadData *td,
- TOKENEXTRA **tp, int mi_row, int mi_col,
- int output_enabled, BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx) {
- MACROBLOCK *const x = &td->mb;
- set_offsets(cpi, tile, x, mi_row, mi_col, bsize);
- update_state(cpi, td, ctx, mi_row, mi_col, bsize, output_enabled);
- encode_superblock(cpi, td, tp, output_enabled, mi_row, mi_col, bsize, ctx);
-
- if (output_enabled) {
- update_stats(&cpi->common, td);
- }
-}
-
-static void encode_sb(VP10_COMP *cpi, ThreadData *td,
- const TileInfo *const tile,
- TOKENEXTRA **tp, int mi_row, int mi_col,
- int output_enabled, BLOCK_SIZE bsize,
- PC_TREE *pc_tree) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
-
- const int bsl = b_width_log2_lookup[bsize], hbs = (1 << bsl) / 4;
- int ctx;
- PARTITION_TYPE partition;
- BLOCK_SIZE subsize = bsize;
-
- if (mi_row >= cm->mi_rows || mi_col >= cm->mi_cols)
- return;
-
- if (bsize >= BLOCK_8X8) {
- ctx = partition_plane_context(xd, mi_row, mi_col, bsize);
- subsize = get_subsize(bsize, pc_tree->partitioning);
- } else {
- ctx = 0;
- subsize = BLOCK_4X4;
- }
-
- partition = partition_lookup[bsl][subsize];
- if (output_enabled && bsize != BLOCK_4X4)
- td->counts->partition[ctx][partition]++;
-
- switch (partition) {
- case PARTITION_NONE:
- encode_b(cpi, tile, td, tp, mi_row, mi_col, output_enabled, subsize,
- &pc_tree->none);
- break;
- case PARTITION_VERT:
- encode_b(cpi, tile, td, tp, mi_row, mi_col, output_enabled, subsize,
- &pc_tree->vertical[0]);
- if (mi_col + hbs < cm->mi_cols && bsize > BLOCK_8X8) {
- encode_b(cpi, tile, td, tp, mi_row, mi_col + hbs, output_enabled,
- subsize, &pc_tree->vertical[1]);
- }
- break;
- case PARTITION_HORZ:
- encode_b(cpi, tile, td, tp, mi_row, mi_col, output_enabled, subsize,
- &pc_tree->horizontal[0]);
- if (mi_row + hbs < cm->mi_rows && bsize > BLOCK_8X8) {
- encode_b(cpi, tile, td, tp, mi_row + hbs, mi_col, output_enabled,
- subsize, &pc_tree->horizontal[1]);
- }
- break;
- case PARTITION_SPLIT:
- if (bsize == BLOCK_8X8) {
- encode_b(cpi, tile, td, tp, mi_row, mi_col, output_enabled, subsize,
- pc_tree->leaf_split[0]);
- } else {
- encode_sb(cpi, td, tile, tp, mi_row, mi_col, output_enabled, subsize,
- pc_tree->split[0]);
- encode_sb(cpi, td, tile, tp, mi_row, mi_col + hbs, output_enabled,
- subsize, pc_tree->split[1]);
- encode_sb(cpi, td, tile, tp, mi_row + hbs, mi_col, output_enabled,
- subsize, pc_tree->split[2]);
- encode_sb(cpi, td, tile, tp, mi_row + hbs, mi_col + hbs, output_enabled,
- subsize, pc_tree->split[3]);
- }
- break;
- default:
- assert(0 && "Invalid partition type.");
- break;
- }
-
- if (partition != PARTITION_SPLIT || bsize == BLOCK_8X8)
- update_partition_context(xd, mi_row, mi_col, subsize, bsize);
-}
-
-// Check to see if the given partition size is allowed for a specified number
-// of 8x8 block rows and columns remaining in the image.
-// If not then return the largest allowed partition size
-static BLOCK_SIZE find_partition_size(BLOCK_SIZE bsize,
- int rows_left, int cols_left,
- int *bh, int *bw) {
- if (rows_left <= 0 || cols_left <= 0) {
- return VPXMIN(bsize, BLOCK_8X8);
- } else {
- for (; bsize > 0; bsize -= 3) {
- *bh = num_8x8_blocks_high_lookup[bsize];
- *bw = num_8x8_blocks_wide_lookup[bsize];
- if ((*bh <= rows_left) && (*bw <= cols_left)) {
- break;
- }
- }
- }
- return bsize;
-}
-
-static void set_partial_b64x64_partition(MODE_INFO *mi, int mis,
- int bh_in, int bw_in, int row8x8_remaining, int col8x8_remaining,
- BLOCK_SIZE bsize, MODE_INFO **mi_8x8) {
- int bh = bh_in;
- int r, c;
- for (r = 0; r < MI_BLOCK_SIZE; r += bh) {
- int bw = bw_in;
- for (c = 0; c < MI_BLOCK_SIZE; c += bw) {
- const int index = r * mis + c;
- mi_8x8[index] = mi + index;
- mi_8x8[index]->mbmi.sb_type = find_partition_size(bsize,
- row8x8_remaining - r, col8x8_remaining - c, &bh, &bw);
- }
- }
-}
-
-// This function attempts to set all mode info entries in a given SB64
-// to the same block partition size.
-// However, at the bottom and right borders of the image the requested size
-// may not be allowed in which case this code attempts to choose the largest
-// allowable partition.
-static void set_fixed_partitioning(VP10_COMP *cpi, const TileInfo *const tile,
- MODE_INFO **mi_8x8, int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- VP10_COMMON *const cm = &cpi->common;
- const int mis = cm->mi_stride;
- const int row8x8_remaining = tile->mi_row_end - mi_row;
- const int col8x8_remaining = tile->mi_col_end - mi_col;
- int block_row, block_col;
- MODE_INFO *mi_upper_left = cm->mi + mi_row * mis + mi_col;
- int bh = num_8x8_blocks_high_lookup[bsize];
- int bw = num_8x8_blocks_wide_lookup[bsize];
-
- assert((row8x8_remaining > 0) && (col8x8_remaining > 0));
-
- // Apply the requested partition size to the SB64 if it is all "in image"
- if ((col8x8_remaining >= MI_BLOCK_SIZE) &&
- (row8x8_remaining >= MI_BLOCK_SIZE)) {
- for (block_row = 0; block_row < MI_BLOCK_SIZE; block_row += bh) {
- for (block_col = 0; block_col < MI_BLOCK_SIZE; block_col += bw) {
- int index = block_row * mis + block_col;
- mi_8x8[index] = mi_upper_left + index;
- mi_8x8[index]->mbmi.sb_type = bsize;
- }
- }
- } else {
- // Else this is a partial SB64.
- set_partial_b64x64_partition(mi_upper_left, mis, bh, bw, row8x8_remaining,
- col8x8_remaining, bsize, mi_8x8);
- }
-}
-
-static void rd_use_partition(VP10_COMP *cpi,
- ThreadData *td,
- TileDataEnc *tile_data,
- MODE_INFO **mi_8x8, TOKENEXTRA **tp,
- int mi_row, int mi_col,
- BLOCK_SIZE bsize,
- int *rate, int64_t *dist,
- int do_recon, PC_TREE *pc_tree) {
- VP10_COMMON *const cm = &cpi->common;
- TileInfo *const tile_info = &tile_data->tile_info;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- const int mis = cm->mi_stride;
- const int bsl = b_width_log2_lookup[bsize];
- const int mi_step = num_4x4_blocks_wide_lookup[bsize] / 2;
- const int bss = (1 << bsl) / 4;
- int i, pl;
- PARTITION_TYPE partition = PARTITION_NONE;
- BLOCK_SIZE subsize;
- ENTROPY_CONTEXT l[16 * MAX_MB_PLANE], a[16 * MAX_MB_PLANE];
- PARTITION_CONTEXT sl[8], sa[8];
- RD_COST last_part_rdc, none_rdc, chosen_rdc;
- BLOCK_SIZE sub_subsize = BLOCK_4X4;
- int splits_below = 0;
- BLOCK_SIZE bs_type = mi_8x8[0]->mbmi.sb_type;
- int do_partition_search = 1;
- PICK_MODE_CONTEXT *ctx = &pc_tree->none;
-
- if (mi_row >= cm->mi_rows || mi_col >= cm->mi_cols)
- return;
-
- assert(num_4x4_blocks_wide_lookup[bsize] ==
- num_4x4_blocks_high_lookup[bsize]);
-
- vp10_rd_cost_reset(&last_part_rdc);
- vp10_rd_cost_reset(&none_rdc);
- vp10_rd_cost_reset(&chosen_rdc);
-
- partition = partition_lookup[bsl][bs_type];
- subsize = get_subsize(bsize, partition);
-
- pc_tree->partitioning = partition;
- save_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
-
- if (bsize == BLOCK_16X16 && cpi->oxcf.aq_mode) {
- set_offsets(cpi, tile_info, x, mi_row, mi_col, bsize);
- x->mb_energy = vp10_block_energy(cpi, x, bsize);
- }
-
- if (do_partition_search &&
- cpi->sf.partition_search_type == SEARCH_PARTITION &&
- cpi->sf.adjust_partitioning_from_last_frame) {
- // Check if any of the sub blocks are further split.
- if (partition == PARTITION_SPLIT && subsize > BLOCK_8X8) {
- sub_subsize = get_subsize(subsize, PARTITION_SPLIT);
- splits_below = 1;
- for (i = 0; i < 4; i++) {
- int jj = i >> 1, ii = i & 0x01;
- MODE_INFO *this_mi = mi_8x8[jj * bss * mis + ii * bss];
- if (this_mi && this_mi->mbmi.sb_type >= sub_subsize) {
- splits_below = 0;
- }
- }
- }
-
- // If partition is not none try none unless each of the 4 splits are split
- // even further..
- if (partition != PARTITION_NONE && !splits_below &&
- mi_row + (mi_step >> 1) < cm->mi_rows &&
- mi_col + (mi_step >> 1) < cm->mi_cols) {
- pc_tree->partitioning = PARTITION_NONE;
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &none_rdc, bsize,
- ctx, INT64_MAX);
-
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
-
- if (none_rdc.rate < INT_MAX) {
- none_rdc.rate += cpi->partition_cost[pl][PARTITION_NONE];
- none_rdc.rdcost = RDCOST(x->rdmult, x->rddiv, none_rdc.rate,
- none_rdc.dist);
- }
-
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- mi_8x8[0]->mbmi.sb_type = bs_type;
- pc_tree->partitioning = partition;
- }
- }
-
- switch (partition) {
- case PARTITION_NONE:
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &last_part_rdc,
- bsize, ctx, INT64_MAX);
- break;
- case PARTITION_HORZ:
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &last_part_rdc,
- subsize, &pc_tree->horizontal[0],
- INT64_MAX);
- if (last_part_rdc.rate != INT_MAX &&
- bsize >= BLOCK_8X8 && mi_row + (mi_step >> 1) < cm->mi_rows) {
- RD_COST tmp_rdc;
- PICK_MODE_CONTEXT *ctx = &pc_tree->horizontal[0];
- vp10_rd_cost_init(&tmp_rdc);
- update_state(cpi, td, ctx, mi_row, mi_col, subsize, 0);
- encode_superblock(cpi, td, tp, 0, mi_row, mi_col, subsize, ctx);
- rd_pick_sb_modes(cpi, tile_data, x,
- mi_row + (mi_step >> 1), mi_col, &tmp_rdc,
- subsize, &pc_tree->horizontal[1], INT64_MAX);
- if (tmp_rdc.rate == INT_MAX || tmp_rdc.dist == INT64_MAX) {
- vp10_rd_cost_reset(&last_part_rdc);
- break;
- }
- last_part_rdc.rate += tmp_rdc.rate;
- last_part_rdc.dist += tmp_rdc.dist;
- last_part_rdc.rdcost += tmp_rdc.rdcost;
- }
- break;
- case PARTITION_VERT:
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &last_part_rdc,
- subsize, &pc_tree->vertical[0], INT64_MAX);
- if (last_part_rdc.rate != INT_MAX &&
- bsize >= BLOCK_8X8 && mi_col + (mi_step >> 1) < cm->mi_cols) {
- RD_COST tmp_rdc;
- PICK_MODE_CONTEXT *ctx = &pc_tree->vertical[0];
- vp10_rd_cost_init(&tmp_rdc);
- update_state(cpi, td, ctx, mi_row, mi_col, subsize, 0);
- encode_superblock(cpi, td, tp, 0, mi_row, mi_col, subsize, ctx);
- rd_pick_sb_modes(cpi, tile_data, x,
- mi_row, mi_col + (mi_step >> 1), &tmp_rdc,
- subsize, &pc_tree->vertical[bsize > BLOCK_8X8],
- INT64_MAX);
- if (tmp_rdc.rate == INT_MAX || tmp_rdc.dist == INT64_MAX) {
- vp10_rd_cost_reset(&last_part_rdc);
- break;
- }
- last_part_rdc.rate += tmp_rdc.rate;
- last_part_rdc.dist += tmp_rdc.dist;
- last_part_rdc.rdcost += tmp_rdc.rdcost;
- }
- break;
- case PARTITION_SPLIT:
- if (bsize == BLOCK_8X8) {
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &last_part_rdc,
- subsize, pc_tree->leaf_split[0], INT64_MAX);
- break;
- }
- last_part_rdc.rate = 0;
- last_part_rdc.dist = 0;
- last_part_rdc.rdcost = 0;
- for (i = 0; i < 4; i++) {
- int x_idx = (i & 1) * (mi_step >> 1);
- int y_idx = (i >> 1) * (mi_step >> 1);
- int jj = i >> 1, ii = i & 0x01;
- RD_COST tmp_rdc;
- if ((mi_row + y_idx >= cm->mi_rows) || (mi_col + x_idx >= cm->mi_cols))
- continue;
-
- vp10_rd_cost_init(&tmp_rdc);
- rd_use_partition(cpi, td, tile_data,
- mi_8x8 + jj * bss * mis + ii * bss, tp,
- mi_row + y_idx, mi_col + x_idx, subsize,
- &tmp_rdc.rate, &tmp_rdc.dist,
- i != 3, pc_tree->split[i]);
- if (tmp_rdc.rate == INT_MAX || tmp_rdc.dist == INT64_MAX) {
- vp10_rd_cost_reset(&last_part_rdc);
- break;
- }
- last_part_rdc.rate += tmp_rdc.rate;
- last_part_rdc.dist += tmp_rdc.dist;
- }
- break;
- default:
- assert(0);
- break;
- }
-
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
- if (last_part_rdc.rate < INT_MAX) {
- last_part_rdc.rate += cpi->partition_cost[pl][partition];
- last_part_rdc.rdcost = RDCOST(x->rdmult, x->rddiv,
- last_part_rdc.rate, last_part_rdc.dist);
- }
-
- if (do_partition_search
- && cpi->sf.adjust_partitioning_from_last_frame
- && cpi->sf.partition_search_type == SEARCH_PARTITION
- && partition != PARTITION_SPLIT && bsize > BLOCK_8X8
- && (mi_row + mi_step < cm->mi_rows ||
- mi_row + (mi_step >> 1) == cm->mi_rows)
- && (mi_col + mi_step < cm->mi_cols ||
- mi_col + (mi_step >> 1) == cm->mi_cols)) {
- BLOCK_SIZE split_subsize = get_subsize(bsize, PARTITION_SPLIT);
- chosen_rdc.rate = 0;
- chosen_rdc.dist = 0;
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- pc_tree->partitioning = PARTITION_SPLIT;
-
- // Split partition.
- for (i = 0; i < 4; i++) {
- int x_idx = (i & 1) * (mi_step >> 1);
- int y_idx = (i >> 1) * (mi_step >> 1);
- RD_COST tmp_rdc;
- ENTROPY_CONTEXT l[16 * MAX_MB_PLANE], a[16 * MAX_MB_PLANE];
- PARTITION_CONTEXT sl[8], sa[8];
-
- if ((mi_row + y_idx >= cm->mi_rows) || (mi_col + x_idx >= cm->mi_cols))
- continue;
-
- save_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- pc_tree->split[i]->partitioning = PARTITION_NONE;
- rd_pick_sb_modes(cpi, tile_data, x,
- mi_row + y_idx, mi_col + x_idx, &tmp_rdc,
- split_subsize, &pc_tree->split[i]->none, INT64_MAX);
-
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
-
- if (tmp_rdc.rate == INT_MAX || tmp_rdc.dist == INT64_MAX) {
- vp10_rd_cost_reset(&chosen_rdc);
- break;
- }
-
- chosen_rdc.rate += tmp_rdc.rate;
- chosen_rdc.dist += tmp_rdc.dist;
-
- if (i != 3)
- encode_sb(cpi, td, tile_info, tp, mi_row + y_idx, mi_col + x_idx, 0,
- split_subsize, pc_tree->split[i]);
-
- pl = partition_plane_context(xd, mi_row + y_idx, mi_col + x_idx,
- split_subsize);
- chosen_rdc.rate += cpi->partition_cost[pl][PARTITION_NONE];
- }
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
- if (chosen_rdc.rate < INT_MAX) {
- chosen_rdc.rate += cpi->partition_cost[pl][PARTITION_SPLIT];
- chosen_rdc.rdcost = RDCOST(x->rdmult, x->rddiv,
- chosen_rdc.rate, chosen_rdc.dist);
- }
- }
-
- // If last_part is better set the partitioning to that.
- if (last_part_rdc.rdcost < chosen_rdc.rdcost) {
- mi_8x8[0]->mbmi.sb_type = bsize;
- if (bsize >= BLOCK_8X8)
- pc_tree->partitioning = partition;
- chosen_rdc = last_part_rdc;
- }
- // If none was better set the partitioning to that.
- if (none_rdc.rdcost < chosen_rdc.rdcost) {
- if (bsize >= BLOCK_8X8)
- pc_tree->partitioning = PARTITION_NONE;
- chosen_rdc = none_rdc;
- }
-
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
-
- // We must have chosen a partitioning and encoding or we'll fail later on.
- // No other opportunities for success.
- if (bsize == BLOCK_64X64)
- assert(chosen_rdc.rate < INT_MAX && chosen_rdc.dist < INT64_MAX);
-
- if (do_recon) {
- int output_enabled = (bsize == BLOCK_64X64);
- encode_sb(cpi, td, tile_info, tp, mi_row, mi_col, output_enabled, bsize,
- pc_tree);
- }
-
- *rate = chosen_rdc.rate;
- *dist = chosen_rdc.dist;
-}
-
-static const BLOCK_SIZE min_partition_size[BLOCK_SIZES] = {
- BLOCK_4X4, BLOCK_4X4, BLOCK_4X4,
- BLOCK_4X4, BLOCK_4X4, BLOCK_4X4,
- BLOCK_8X8, BLOCK_8X8, BLOCK_8X8,
- BLOCK_16X16, BLOCK_16X16, BLOCK_16X16,
- BLOCK_16X16
-};
-
-static const BLOCK_SIZE max_partition_size[BLOCK_SIZES] = {
- BLOCK_8X8, BLOCK_16X16, BLOCK_16X16,
- BLOCK_16X16, BLOCK_32X32, BLOCK_32X32,
- BLOCK_32X32, BLOCK_64X64, BLOCK_64X64,
- BLOCK_64X64, BLOCK_64X64, BLOCK_64X64,
- BLOCK_64X64
-};
-
-
-// Look at all the mode_info entries for blocks that are part of this
-// partition and find the min and max values for sb_type.
-// At the moment this is designed to work on a 64x64 SB but could be
-// adjusted to use a size parameter.
-//
-// The min and max are assumed to have been initialized prior to calling this
-// function so repeat calls can accumulate a min and max of more than one sb64.
-static void get_sb_partition_size_range(MACROBLOCKD *xd, MODE_INFO **mi_8x8,
- BLOCK_SIZE *min_block_size,
- BLOCK_SIZE *max_block_size,
- int bs_hist[BLOCK_SIZES]) {
- int sb_width_in_blocks = MI_BLOCK_SIZE;
- int sb_height_in_blocks = MI_BLOCK_SIZE;
- int i, j;
- int index = 0;
-
- // Check the sb_type for each block that belongs to this region.
- for (i = 0; i < sb_height_in_blocks; ++i) {
- for (j = 0; j < sb_width_in_blocks; ++j) {
- MODE_INFO *mi = mi_8x8[index+j];
- BLOCK_SIZE sb_type = mi ? mi->mbmi.sb_type : 0;
- bs_hist[sb_type]++;
- *min_block_size = VPXMIN(*min_block_size, sb_type);
- *max_block_size = VPXMAX(*max_block_size, sb_type);
- }
- index += xd->mi_stride;
- }
-}
-
-// Next square block size less or equal than current block size.
-static const BLOCK_SIZE next_square_size[BLOCK_SIZES] = {
- BLOCK_4X4, BLOCK_4X4, BLOCK_4X4,
- BLOCK_8X8, BLOCK_8X8, BLOCK_8X8,
- BLOCK_16X16, BLOCK_16X16, BLOCK_16X16,
- BLOCK_32X32, BLOCK_32X32, BLOCK_32X32,
- BLOCK_64X64
-};
-
-// Look at neighboring blocks and set a min and max partition size based on
-// what they chose.
-static void rd_auto_partition_range(VP10_COMP *cpi, const TileInfo *const tile,
- MACROBLOCKD *const xd,
- int mi_row, int mi_col,
- BLOCK_SIZE *min_block_size,
- BLOCK_SIZE *max_block_size) {
- VP10_COMMON *const cm = &cpi->common;
- MODE_INFO **mi = xd->mi;
- const int left_in_image = xd->left_available && mi[-1];
- const int above_in_image = xd->up_available && mi[-xd->mi_stride];
- const int row8x8_remaining = tile->mi_row_end - mi_row;
- const int col8x8_remaining = tile->mi_col_end - mi_col;
- int bh, bw;
- BLOCK_SIZE min_size = BLOCK_4X4;
- BLOCK_SIZE max_size = BLOCK_64X64;
- int bs_hist[BLOCK_SIZES] = {0};
-
- // Trap case where we do not have a prediction.
- if (left_in_image || above_in_image || cm->frame_type != KEY_FRAME) {
- // Default "min to max" and "max to min"
- min_size = BLOCK_64X64;
- max_size = BLOCK_4X4;
-
- // NOTE: each call to get_sb_partition_size_range() uses the previous
- // passed in values for min and max as a starting point.
- // Find the min and max partition used in previous frame at this location
- if (cm->frame_type != KEY_FRAME) {
- MODE_INFO **prev_mi =
- &cm->prev_mi_grid_visible[mi_row * xd->mi_stride + mi_col];
- get_sb_partition_size_range(xd, prev_mi, &min_size, &max_size, bs_hist);
- }
- // Find the min and max partition sizes used in the left SB64
- if (left_in_image) {
- MODE_INFO **left_sb64_mi = &mi[-MI_BLOCK_SIZE];
- get_sb_partition_size_range(xd, left_sb64_mi, &min_size, &max_size,
- bs_hist);
- }
- // Find the min and max partition sizes used in the above SB64.
- if (above_in_image) {
- MODE_INFO **above_sb64_mi = &mi[-xd->mi_stride * MI_BLOCK_SIZE];
- get_sb_partition_size_range(xd, above_sb64_mi, &min_size, &max_size,
- bs_hist);
- }
-
- // Adjust observed min and max for "relaxed" auto partition case.
- if (cpi->sf.auto_min_max_partition_size == RELAXED_NEIGHBORING_MIN_MAX) {
- min_size = min_partition_size[min_size];
- max_size = max_partition_size[max_size];
- }
- }
-
- // Check border cases where max and min from neighbors may not be legal.
- max_size = find_partition_size(max_size,
- row8x8_remaining, col8x8_remaining,
- &bh, &bw);
- // Test for blocks at the edge of the active image.
- // This may be the actual edge of the image or where there are formatting
- // bars.
- if (vp10_active_edge_sb(cpi, mi_row, mi_col)) {
- min_size = BLOCK_4X4;
- } else {
- min_size =
- VPXMIN(cpi->sf.rd_auto_partition_min_limit, VPXMIN(min_size, max_size));
- }
-
- // When use_square_partition_only is true, make sure at least one square
- // partition is allowed by selecting the next smaller square size as
- // *min_block_size.
- if (cpi->sf.use_square_partition_only &&
- next_square_size[max_size] < min_size) {
- min_size = next_square_size[max_size];
- }
-
- *min_block_size = min_size;
- *max_block_size = max_size;
-}
-
-// TODO(jingning) refactor functions setting partition search range
-static void set_partition_range(VP10_COMMON *cm, MACROBLOCKD *xd,
- int mi_row, int mi_col, BLOCK_SIZE bsize,
- BLOCK_SIZE *min_bs, BLOCK_SIZE *max_bs) {
- int mi_width = num_8x8_blocks_wide_lookup[bsize];
- int mi_height = num_8x8_blocks_high_lookup[bsize];
- int idx, idy;
-
- MODE_INFO *mi;
- const int idx_str = cm->mi_stride * mi_row + mi_col;
- MODE_INFO **prev_mi = &cm->prev_mi_grid_visible[idx_str];
- BLOCK_SIZE bs, min_size, max_size;
-
- min_size = BLOCK_64X64;
- max_size = BLOCK_4X4;
-
- if (prev_mi) {
- for (idy = 0; idy < mi_height; ++idy) {
- for (idx = 0; idx < mi_width; ++idx) {
- mi = prev_mi[idy * cm->mi_stride + idx];
- bs = mi ? mi->mbmi.sb_type : bsize;
- min_size = VPXMIN(min_size, bs);
- max_size = VPXMAX(max_size, bs);
- }
- }
- }
-
- if (xd->left_available) {
- for (idy = 0; idy < mi_height; ++idy) {
- mi = xd->mi[idy * cm->mi_stride - 1];
- bs = mi ? mi->mbmi.sb_type : bsize;
- min_size = VPXMIN(min_size, bs);
- max_size = VPXMAX(max_size, bs);
- }
- }
-
- if (xd->up_available) {
- for (idx = 0; idx < mi_width; ++idx) {
- mi = xd->mi[idx - cm->mi_stride];
- bs = mi ? mi->mbmi.sb_type : bsize;
- min_size = VPXMIN(min_size, bs);
- max_size = VPXMAX(max_size, bs);
- }
- }
-
- if (min_size == max_size) {
- min_size = min_partition_size[min_size];
- max_size = max_partition_size[max_size];
- }
-
- *min_bs = min_size;
- *max_bs = max_size;
-}
-
-static INLINE void store_pred_mv(MACROBLOCK *x, PICK_MODE_CONTEXT *ctx) {
- memcpy(ctx->pred_mv, x->pred_mv, sizeof(x->pred_mv));
-}
-
-static INLINE void load_pred_mv(MACROBLOCK *x, PICK_MODE_CONTEXT *ctx) {
- memcpy(x->pred_mv, ctx->pred_mv, sizeof(x->pred_mv));
-}
-
-#if CONFIG_FP_MB_STATS
-const int num_16x16_blocks_wide_lookup[BLOCK_SIZES] =
- {1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 4, 4};
-const int num_16x16_blocks_high_lookup[BLOCK_SIZES] =
- {1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 4, 2, 4};
-const int qindex_skip_threshold_lookup[BLOCK_SIZES] =
- {0, 10, 10, 30, 40, 40, 60, 80, 80, 90, 100, 100, 120};
-const int qindex_split_threshold_lookup[BLOCK_SIZES] =
- {0, 3, 3, 7, 15, 15, 30, 40, 40, 60, 80, 80, 120};
-const int complexity_16x16_blocks_threshold[BLOCK_SIZES] =
- {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 4, 6};
-
-typedef enum {
- MV_ZERO = 0,
- MV_LEFT = 1,
- MV_UP = 2,
- MV_RIGHT = 3,
- MV_DOWN = 4,
- MV_INVALID
-} MOTION_DIRECTION;
-
-static INLINE MOTION_DIRECTION get_motion_direction_fp(uint8_t fp_byte) {
- if (fp_byte & FPMB_MOTION_ZERO_MASK) {
- return MV_ZERO;
- } else if (fp_byte & FPMB_MOTION_LEFT_MASK) {
- return MV_LEFT;
- } else if (fp_byte & FPMB_MOTION_RIGHT_MASK) {
- return MV_RIGHT;
- } else if (fp_byte & FPMB_MOTION_UP_MASK) {
- return MV_UP;
- } else {
- return MV_DOWN;
- }
-}
-
-static INLINE int get_motion_inconsistency(MOTION_DIRECTION this_mv,
- MOTION_DIRECTION that_mv) {
- if (this_mv == that_mv) {
- return 0;
- } else {
- return abs(this_mv - that_mv) == 2 ? 2 : 1;
- }
-}
-#endif
-
-// TODO(jingning,jimbankoski,rbultje): properly skip partition types that are
-// unlikely to be selected depending on previous rate-distortion optimization
-// results, for encoding speed-up.
-static void rd_pick_partition(VP10_COMP *cpi, ThreadData *td,
- TileDataEnc *tile_data,
- TOKENEXTRA **tp, int mi_row, int mi_col,
- BLOCK_SIZE bsize, RD_COST *rd_cost,
- int64_t best_rd, PC_TREE *pc_tree) {
- VP10_COMMON *const cm = &cpi->common;
- TileInfo *const tile_info = &tile_data->tile_info;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- const int mi_step = num_8x8_blocks_wide_lookup[bsize] / 2;
- ENTROPY_CONTEXT l[16 * MAX_MB_PLANE], a[16 * MAX_MB_PLANE];
- PARTITION_CONTEXT sl[8], sa[8];
- TOKENEXTRA *tp_orig = *tp;
- PICK_MODE_CONTEXT *ctx = &pc_tree->none;
- int i, pl;
- BLOCK_SIZE subsize;
- RD_COST this_rdc, sum_rdc, best_rdc;
- int do_split = bsize >= BLOCK_8X8;
- int do_rect = 1;
-
- // Override skipping rectangular partition operations for edge blocks
- const int force_horz_split = (mi_row + mi_step >= cm->mi_rows);
- const int force_vert_split = (mi_col + mi_step >= cm->mi_cols);
- const int xss = x->e_mbd.plane[1].subsampling_x;
- const int yss = x->e_mbd.plane[1].subsampling_y;
-
- BLOCK_SIZE min_size = x->min_partition_size;
- BLOCK_SIZE max_size = x->max_partition_size;
-
-#if CONFIG_FP_MB_STATS
- unsigned int src_diff_var = UINT_MAX;
- int none_complexity = 0;
-#endif
-
- int partition_none_allowed = !force_horz_split && !force_vert_split;
- int partition_horz_allowed = !force_vert_split && yss <= xss &&
- bsize >= BLOCK_8X8;
- int partition_vert_allowed = !force_horz_split && xss <= yss &&
- bsize >= BLOCK_8X8;
- (void) *tp_orig;
-
- assert(num_8x8_blocks_wide_lookup[bsize] ==
- num_8x8_blocks_high_lookup[bsize]);
-
- vp10_rd_cost_init(&this_rdc);
- vp10_rd_cost_init(&sum_rdc);
- vp10_rd_cost_reset(&best_rdc);
- best_rdc.rdcost = best_rd;
-
- set_offsets(cpi, tile_info, x, mi_row, mi_col, bsize);
-
- if (bsize == BLOCK_16X16 && cpi->oxcf.aq_mode)
- x->mb_energy = vp10_block_energy(cpi, x, bsize);
-
- if (cpi->sf.cb_partition_search && bsize == BLOCK_16X16) {
- int cb_partition_search_ctrl = ((pc_tree->index == 0 || pc_tree->index == 3)
- + get_chessboard_index(cm->current_video_frame)) & 0x1;
-
- if (cb_partition_search_ctrl && bsize > min_size && bsize < max_size)
- set_partition_range(cm, xd, mi_row, mi_col, bsize, &min_size, &max_size);
- }
-
- // Determine partition types in search according to the speed features.
- // The threshold set here has to be of square block size.
- if (cpi->sf.auto_min_max_partition_size) {
- partition_none_allowed &= (bsize <= max_size && bsize >= min_size);
- partition_horz_allowed &= ((bsize <= max_size && bsize > min_size) ||
- force_horz_split);
- partition_vert_allowed &= ((bsize <= max_size && bsize > min_size) ||
- force_vert_split);
- do_split &= bsize > min_size;
- }
- if (cpi->sf.use_square_partition_only) {
- partition_horz_allowed &= force_horz_split;
- partition_vert_allowed &= force_vert_split;
- }
-
- save_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- set_offsets(cpi, tile_info, x, mi_row, mi_col, bsize);
- src_diff_var = get_sby_perpixel_diff_variance(cpi, &x->plane[0].src,
- mi_row, mi_col, bsize);
- }
-#endif
-
-#if CONFIG_FP_MB_STATS
- // Decide whether we shall split directly and skip searching NONE by using
- // the first pass block statistics
- if (cpi->use_fp_mb_stats && bsize >= BLOCK_32X32 && do_split &&
- partition_none_allowed && src_diff_var > 4 &&
- cm->base_qindex < qindex_split_threshold_lookup[bsize]) {
- int mb_row = mi_row >> 1;
- int mb_col = mi_col >> 1;
- int mb_row_end =
- VPXMIN(mb_row + num_16x16_blocks_high_lookup[bsize], cm->mb_rows);
- int mb_col_end =
- VPXMIN(mb_col + num_16x16_blocks_wide_lookup[bsize], cm->mb_cols);
- int r, c;
-
- // compute a complexity measure, basically measure inconsistency of motion
- // vectors obtained from the first pass in the current block
- for (r = mb_row; r < mb_row_end ; r++) {
- for (c = mb_col; c < mb_col_end; c++) {
- const int mb_index = r * cm->mb_cols + c;
-
- MOTION_DIRECTION this_mv;
- MOTION_DIRECTION right_mv;
- MOTION_DIRECTION bottom_mv;
-
- this_mv =
- get_motion_direction_fp(cpi->twopass.this_frame_mb_stats[mb_index]);
-
- // to its right
- if (c != mb_col_end - 1) {
- right_mv = get_motion_direction_fp(
- cpi->twopass.this_frame_mb_stats[mb_index + 1]);
- none_complexity += get_motion_inconsistency(this_mv, right_mv);
- }
-
- // to its bottom
- if (r != mb_row_end - 1) {
- bottom_mv = get_motion_direction_fp(
- cpi->twopass.this_frame_mb_stats[mb_index + cm->mb_cols]);
- none_complexity += get_motion_inconsistency(this_mv, bottom_mv);
- }
-
- // do not count its left and top neighbors to avoid double counting
- }
- }
-
- if (none_complexity > complexity_16x16_blocks_threshold[bsize]) {
- partition_none_allowed = 0;
- }
- }
-#endif
-
- // PARTITION_NONE
- if (partition_none_allowed) {
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col,
- &this_rdc, bsize, ctx, best_rdc.rdcost);
- if (this_rdc.rate != INT_MAX) {
- if (bsize >= BLOCK_8X8) {
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
- this_rdc.rate += cpi->partition_cost[pl][PARTITION_NONE];
- this_rdc.rdcost = RDCOST(x->rdmult, x->rddiv,
- this_rdc.rate, this_rdc.dist);
- }
-
- if (this_rdc.rdcost < best_rdc.rdcost) {
- int64_t dist_breakout_thr = cpi->sf.partition_search_breakout_dist_thr;
- int rate_breakout_thr = cpi->sf.partition_search_breakout_rate_thr;
-
- best_rdc = this_rdc;
- if (bsize >= BLOCK_8X8)
- pc_tree->partitioning = PARTITION_NONE;
-
- // Adjust dist breakout threshold according to the partition size.
- dist_breakout_thr >>= 8 - (b_width_log2_lookup[bsize] +
- b_height_log2_lookup[bsize]);
-
- rate_breakout_thr *= num_pels_log2_lookup[bsize];
-
- // If all y, u, v transform blocks in this partition are skippable, and
- // the dist & rate are within the thresholds, the partition search is
- // terminated for current branch of the partition search tree.
- // The dist & rate thresholds are set to 0 at speed 0 to disable the
- // early termination at that speed.
- if (!x->e_mbd.lossless[xd->mi[0]->mbmi.segment_id] &&
- (ctx->skippable && best_rdc.dist < dist_breakout_thr &&
- best_rdc.rate < rate_breakout_thr)) {
- do_split = 0;
- do_rect = 0;
- }
-
-#if CONFIG_FP_MB_STATS
- // Check if every 16x16 first pass block statistics has zero
- // motion and the corresponding first pass residue is small enough.
- // If that is the case, check the difference variance between the
- // current frame and the last frame. If the variance is small enough,
- // stop further splitting in RD optimization
- if (cpi->use_fp_mb_stats && do_split != 0 &&
- cm->base_qindex > qindex_skip_threshold_lookup[bsize]) {
- int mb_row = mi_row >> 1;
- int mb_col = mi_col >> 1;
- int mb_row_end =
- VPXMIN(mb_row + num_16x16_blocks_high_lookup[bsize], cm->mb_rows);
- int mb_col_end =
- VPXMIN(mb_col + num_16x16_blocks_wide_lookup[bsize], cm->mb_cols);
- int r, c;
-
- int skip = 1;
- for (r = mb_row; r < mb_row_end; r++) {
- for (c = mb_col; c < mb_col_end; c++) {
- const int mb_index = r * cm->mb_cols + c;
- if (!(cpi->twopass.this_frame_mb_stats[mb_index] &
- FPMB_MOTION_ZERO_MASK) ||
- !(cpi->twopass.this_frame_mb_stats[mb_index] &
- FPMB_ERROR_SMALL_MASK)) {
- skip = 0;
- break;
- }
- }
- if (skip == 0) {
- break;
- }
- }
- if (skip) {
- if (src_diff_var == UINT_MAX) {
- set_offsets(cpi, tile_info, x, mi_row, mi_col, bsize);
- src_diff_var = get_sby_perpixel_diff_variance(
- cpi, &x->plane[0].src, mi_row, mi_col, bsize);
- }
- if (src_diff_var < 8) {
- do_split = 0;
- do_rect = 0;
- }
- }
- }
-#endif
- }
- }
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- }
-
- // store estimated motion vector
- if (cpi->sf.adaptive_motion_search)
- store_pred_mv(x, ctx);
-
- // PARTITION_SPLIT
- // TODO(jingning): use the motion vectors given by the above search as
- // the starting point of motion search in the following partition type check.
- if (do_split) {
- subsize = get_subsize(bsize, PARTITION_SPLIT);
- if (bsize == BLOCK_8X8) {
- i = 4;
- if (cpi->sf.adaptive_pred_interp_filter && partition_none_allowed)
- pc_tree->leaf_split[0]->pred_interp_filter =
- ctx->mic.mbmi.interp_filter;
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &sum_rdc, subsize,
- pc_tree->leaf_split[0], best_rdc.rdcost);
- if (sum_rdc.rate == INT_MAX)
- sum_rdc.rdcost = INT64_MAX;
- } else {
- for (i = 0; i < 4 && sum_rdc.rdcost < best_rdc.rdcost; ++i) {
- const int x_idx = (i & 1) * mi_step;
- const int y_idx = (i >> 1) * mi_step;
-
- if (mi_row + y_idx >= cm->mi_rows || mi_col + x_idx >= cm->mi_cols)
- continue;
-
- if (cpi->sf.adaptive_motion_search)
- load_pred_mv(x, ctx);
-
- pc_tree->split[i]->index = i;
- rd_pick_partition(cpi, td, tile_data, tp,
- mi_row + y_idx, mi_col + x_idx,
- subsize, &this_rdc,
- best_rdc.rdcost - sum_rdc.rdcost, pc_tree->split[i]);
-
- if (this_rdc.rate == INT_MAX) {
- sum_rdc.rdcost = INT64_MAX;
- break;
- } else {
- sum_rdc.rate += this_rdc.rate;
- sum_rdc.dist += this_rdc.dist;
- sum_rdc.rdcost += this_rdc.rdcost;
- }
- }
- }
-
- if (sum_rdc.rdcost < best_rdc.rdcost && i == 4) {
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
- sum_rdc.rate += cpi->partition_cost[pl][PARTITION_SPLIT];
- sum_rdc.rdcost = RDCOST(x->rdmult, x->rddiv,
- sum_rdc.rate, sum_rdc.dist);
-
- if (sum_rdc.rdcost < best_rdc.rdcost) {
- best_rdc = sum_rdc;
- pc_tree->partitioning = PARTITION_SPLIT;
- }
- } else {
- // skip rectangular partition test when larger block size
- // gives better rd cost
- if (cpi->sf.less_rectangular_check)
- do_rect &= !partition_none_allowed;
- }
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- }
-
- // PARTITION_HORZ
- if (partition_horz_allowed &&
- (do_rect || vp10_active_h_edge(cpi, mi_row, mi_step))) {
- subsize = get_subsize(bsize, PARTITION_HORZ);
- if (cpi->sf.adaptive_motion_search)
- load_pred_mv(x, ctx);
- if (cpi->sf.adaptive_pred_interp_filter && bsize == BLOCK_8X8 &&
- partition_none_allowed)
- pc_tree->horizontal[0].pred_interp_filter =
- ctx->mic.mbmi.interp_filter;
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &sum_rdc, subsize,
- &pc_tree->horizontal[0], best_rdc.rdcost);
-
- if (sum_rdc.rdcost < best_rdc.rdcost && mi_row + mi_step < cm->mi_rows &&
- bsize > BLOCK_8X8) {
- PICK_MODE_CONTEXT *ctx = &pc_tree->horizontal[0];
- update_state(cpi, td, ctx, mi_row, mi_col, subsize, 0);
- encode_superblock(cpi, td, tp, 0, mi_row, mi_col, subsize, ctx);
-
- if (cpi->sf.adaptive_motion_search)
- load_pred_mv(x, ctx);
- if (cpi->sf.adaptive_pred_interp_filter && bsize == BLOCK_8X8 &&
- partition_none_allowed)
- pc_tree->horizontal[1].pred_interp_filter =
- ctx->mic.mbmi.interp_filter;
- rd_pick_sb_modes(cpi, tile_data, x, mi_row + mi_step, mi_col,
- &this_rdc, subsize, &pc_tree->horizontal[1],
- best_rdc.rdcost - sum_rdc.rdcost);
- if (this_rdc.rate == INT_MAX) {
- sum_rdc.rdcost = INT64_MAX;
- } else {
- sum_rdc.rate += this_rdc.rate;
- sum_rdc.dist += this_rdc.dist;
- sum_rdc.rdcost += this_rdc.rdcost;
- }
- }
-
- if (sum_rdc.rdcost < best_rdc.rdcost) {
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
- sum_rdc.rate += cpi->partition_cost[pl][PARTITION_HORZ];
- sum_rdc.rdcost = RDCOST(x->rdmult, x->rddiv, sum_rdc.rate, sum_rdc.dist);
- if (sum_rdc.rdcost < best_rdc.rdcost) {
- best_rdc = sum_rdc;
- pc_tree->partitioning = PARTITION_HORZ;
- }
- }
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- }
- // PARTITION_VERT
- if (partition_vert_allowed &&
- (do_rect || vp10_active_v_edge(cpi, mi_col, mi_step))) {
- subsize = get_subsize(bsize, PARTITION_VERT);
-
- if (cpi->sf.adaptive_motion_search)
- load_pred_mv(x, ctx);
- if (cpi->sf.adaptive_pred_interp_filter && bsize == BLOCK_8X8 &&
- partition_none_allowed)
- pc_tree->vertical[0].pred_interp_filter =
- ctx->mic.mbmi.interp_filter;
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col, &sum_rdc, subsize,
- &pc_tree->vertical[0], best_rdc.rdcost);
- if (sum_rdc.rdcost < best_rdc.rdcost && mi_col + mi_step < cm->mi_cols &&
- bsize > BLOCK_8X8) {
- update_state(cpi, td, &pc_tree->vertical[0], mi_row, mi_col, subsize, 0);
- encode_superblock(cpi, td, tp, 0, mi_row, mi_col, subsize,
- &pc_tree->vertical[0]);
-
- if (cpi->sf.adaptive_motion_search)
- load_pred_mv(x, ctx);
- if (cpi->sf.adaptive_pred_interp_filter && bsize == BLOCK_8X8 &&
- partition_none_allowed)
- pc_tree->vertical[1].pred_interp_filter =
- ctx->mic.mbmi.interp_filter;
- rd_pick_sb_modes(cpi, tile_data, x, mi_row, mi_col + mi_step,
- &this_rdc, subsize,
- &pc_tree->vertical[1], best_rdc.rdcost - sum_rdc.rdcost);
- if (this_rdc.rate == INT_MAX) {
- sum_rdc.rdcost = INT64_MAX;
- } else {
- sum_rdc.rate += this_rdc.rate;
- sum_rdc.dist += this_rdc.dist;
- sum_rdc.rdcost += this_rdc.rdcost;
- }
- }
-
- if (sum_rdc.rdcost < best_rdc.rdcost) {
- pl = partition_plane_context(xd, mi_row, mi_col, bsize);
- sum_rdc.rate += cpi->partition_cost[pl][PARTITION_VERT];
- sum_rdc.rdcost = RDCOST(x->rdmult, x->rddiv,
- sum_rdc.rate, sum_rdc.dist);
- if (sum_rdc.rdcost < best_rdc.rdcost) {
- best_rdc = sum_rdc;
- pc_tree->partitioning = PARTITION_VERT;
- }
- }
- restore_context(x, mi_row, mi_col, a, l, sa, sl, bsize);
- }
-
- // TODO(jbb): This code added so that we avoid static analysis
- // warning related to the fact that best_rd isn't used after this
- // point. This code should be refactored so that the duplicate
- // checks occur in some sub function and thus are used...
- (void) best_rd;
- *rd_cost = best_rdc;
-
-
- if (best_rdc.rate < INT_MAX && best_rdc.dist < INT64_MAX &&
- pc_tree->index != 3) {
- int output_enabled = (bsize == BLOCK_64X64);
- encode_sb(cpi, td, tile_info, tp, mi_row, mi_col, output_enabled,
- bsize, pc_tree);
- }
-
- if (bsize == BLOCK_64X64) {
- assert(tp_orig < *tp || (tp_orig == *tp && xd->mi[0]->mbmi.skip));
- assert(best_rdc.rate < INT_MAX);
- assert(best_rdc.dist < INT64_MAX);
- } else {
- assert(tp_orig == *tp);
- }
-}
-
-static void encode_rd_sb_row(VP10_COMP *cpi,
- ThreadData *td,
- TileDataEnc *tile_data,
- int mi_row,
- TOKENEXTRA **tp) {
- VP10_COMMON *const cm = &cpi->common;
- TileInfo *const tile_info = &tile_data->tile_info;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- SPEED_FEATURES *const sf = &cpi->sf;
- int mi_col;
-
- // Initialize the left context for the new SB row
- memset(&xd->left_context, 0, sizeof(xd->left_context));
- memset(xd->left_seg_context, 0, sizeof(xd->left_seg_context));
-
- // Code each SB in the row
- for (mi_col = tile_info->mi_col_start; mi_col < tile_info->mi_col_end;
- mi_col += MI_BLOCK_SIZE) {
- const struct segmentation *const seg = &cm->seg;
- int dummy_rate;
- int64_t dummy_dist;
- RD_COST dummy_rdc;
- int i;
- int seg_skip = 0;
-
- const int idx_str = cm->mi_stride * mi_row + mi_col;
- MODE_INFO **mi = cm->mi_grid_visible + idx_str;
-
- if (sf->adaptive_pred_interp_filter) {
- for (i = 0; i < 64; ++i)
- td->leaf_tree[i].pred_interp_filter = SWITCHABLE;
-
- for (i = 0; i < 64; ++i) {
- td->pc_tree[i].vertical[0].pred_interp_filter = SWITCHABLE;
- td->pc_tree[i].vertical[1].pred_interp_filter = SWITCHABLE;
- td->pc_tree[i].horizontal[0].pred_interp_filter = SWITCHABLE;
- td->pc_tree[i].horizontal[1].pred_interp_filter = SWITCHABLE;
- }
- }
-
- vp10_zero(x->pred_mv);
- td->pc_root->index = 0;
-
- if (seg->enabled) {
- const uint8_t *const map = seg->update_map ? cpi->segmentation_map
- : cm->last_frame_seg_map;
- int segment_id = get_segment_id(cm, map, BLOCK_64X64, mi_row, mi_col);
- seg_skip = segfeature_active(seg, segment_id, SEG_LVL_SKIP);
- }
-
- x->source_variance = UINT_MAX;
- if (sf->partition_search_type == FIXED_PARTITION || seg_skip) {
- const BLOCK_SIZE bsize =
- seg_skip ? BLOCK_64X64 : sf->always_this_block_size;
- set_offsets(cpi, tile_info, x, mi_row, mi_col, BLOCK_64X64);
- set_fixed_partitioning(cpi, tile_info, mi, mi_row, mi_col, bsize);
- rd_use_partition(cpi, td, tile_data, mi, tp, mi_row, mi_col,
- BLOCK_64X64, &dummy_rate, &dummy_dist, 1, td->pc_root);
- } else if (cpi->partition_search_skippable_frame) {
- BLOCK_SIZE bsize;
- set_offsets(cpi, tile_info, x, mi_row, mi_col, BLOCK_64X64);
- bsize = get_rd_var_based_fixed_partition(cpi, x, mi_row, mi_col);
- set_fixed_partitioning(cpi, tile_info, mi, mi_row, mi_col, bsize);
- rd_use_partition(cpi, td, tile_data, mi, tp, mi_row, mi_col,
- BLOCK_64X64, &dummy_rate, &dummy_dist, 1, td->pc_root);
- } else if (sf->partition_search_type == VAR_BASED_PARTITION &&
- cm->frame_type != KEY_FRAME) {
- choose_partitioning(cpi, tile_info, x, mi_row, mi_col);
- rd_use_partition(cpi, td, tile_data, mi, tp, mi_row, mi_col,
- BLOCK_64X64, &dummy_rate, &dummy_dist, 1, td->pc_root);
- } else {
- // If required set upper and lower partition size limits
- if (sf->auto_min_max_partition_size) {
- set_offsets(cpi, tile_info, x, mi_row, mi_col, BLOCK_64X64);
- rd_auto_partition_range(cpi, tile_info, xd, mi_row, mi_col,
- &x->min_partition_size,
- &x->max_partition_size);
- }
- rd_pick_partition(cpi, td, tile_data, tp, mi_row, mi_col, BLOCK_64X64,
- &dummy_rdc, INT64_MAX, td->pc_root);
- }
- }
-}
-
-static void init_encode_frame_mb_context(VP10_COMP *cpi) {
- MACROBLOCK *const x = &cpi->td.mb;
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- const int aligned_mi_cols = mi_cols_aligned_to_sb(cm->mi_cols);
-
- // Copy data over into macro block data structures.
- vp10_setup_src_planes(x, cpi->Source, 0, 0);
-
- vp10_setup_block_planes(&x->e_mbd, cm->subsampling_x, cm->subsampling_y);
-
- // Note: this memset assumes above_context[0], [1] and [2]
- // are allocated as part of the same buffer.
- memset(xd->above_context[0], 0,
- sizeof(*xd->above_context[0]) *
- 2 * aligned_mi_cols * MAX_MB_PLANE);
- memset(xd->above_seg_context, 0,
- sizeof(*xd->above_seg_context) * aligned_mi_cols);
-}
-
-static int check_dual_ref_flags(VP10_COMP *cpi) {
- const int ref_flags = cpi->ref_frame_flags;
-
- if (segfeature_active(&cpi->common.seg, 1, SEG_LVL_REF_FRAME)) {
- return 0;
- } else {
- return (!!(ref_flags & VP9_GOLD_FLAG) + !!(ref_flags & VP9_LAST_FLAG)
- + !!(ref_flags & VP9_ALT_FLAG)) >= 2;
- }
-}
-
-static void reset_skip_tx_size(VP10_COMMON *cm, TX_SIZE max_tx_size) {
- int mi_row, mi_col;
- const int mis = cm->mi_stride;
- MODE_INFO **mi_ptr = cm->mi_grid_visible;
-
- for (mi_row = 0; mi_row < cm->mi_rows; ++mi_row, mi_ptr += mis) {
- for (mi_col = 0; mi_col < cm->mi_cols; ++mi_col) {
- if (mi_ptr[mi_col]->mbmi.tx_size > max_tx_size)
- mi_ptr[mi_col]->mbmi.tx_size = max_tx_size;
- }
- }
-}
-
-static MV_REFERENCE_FRAME get_frame_type(const VP10_COMP *cpi) {
- if (frame_is_intra_only(&cpi->common))
- return INTRA_FRAME;
- else if (cpi->rc.is_src_frame_alt_ref && cpi->refresh_golden_frame)
- return ALTREF_FRAME;
- else if (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)
- return GOLDEN_FRAME;
- else
- return LAST_FRAME;
-}
-
-static TX_MODE select_tx_mode(const VP10_COMP *cpi, MACROBLOCKD *const xd) {
- if (!cpi->common.seg.enabled && xd->lossless[0])
- return ONLY_4X4;
- if (cpi->sf.tx_size_search_method == USE_LARGESTALL)
- return ALLOW_32X32;
- else if (cpi->sf.tx_size_search_method == USE_FULL_RD||
- cpi->sf.tx_size_search_method == USE_TX_8X8)
- return TX_MODE_SELECT;
- else
- return cpi->common.tx_mode;
-}
-
-void vp10_init_tile_data(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int tile_rows = 1 << cm->log2_tile_rows;
- int tile_col, tile_row;
- TOKENEXTRA *pre_tok = cpi->tile_tok[0][0];
- int tile_tok = 0;
-
- if (cpi->tile_data == NULL || cpi->allocated_tiles < tile_cols * tile_rows) {
- if (cpi->tile_data != NULL)
- vpx_free(cpi->tile_data);
- CHECK_MEM_ERROR(cm, cpi->tile_data,
- vpx_malloc(tile_cols * tile_rows * sizeof(*cpi->tile_data)));
- cpi->allocated_tiles = tile_cols * tile_rows;
-
- for (tile_row = 0; tile_row < tile_rows; ++tile_row)
- for (tile_col = 0; tile_col < tile_cols; ++tile_col) {
- TileDataEnc *tile_data =
- &cpi->tile_data[tile_row * tile_cols + tile_col];
- int i, j;
- for (i = 0; i < BLOCK_SIZES; ++i) {
- for (j = 0; j < MAX_MODES; ++j) {
- tile_data->thresh_freq_fact[i][j] = 32;
- tile_data->mode_map[i][j] = j;
- }
- }
- }
- }
-
- for (tile_row = 0; tile_row < tile_rows; ++tile_row) {
- for (tile_col = 0; tile_col < tile_cols; ++tile_col) {
- TileInfo *tile_info =
- &cpi->tile_data[tile_row * tile_cols + tile_col].tile_info;
- vp10_tile_init(tile_info, cm, tile_row, tile_col);
-
- cpi->tile_tok[tile_row][tile_col] = pre_tok + tile_tok;
- pre_tok = cpi->tile_tok[tile_row][tile_col];
- tile_tok = allocated_tokens(*tile_info);
- }
- }
-}
-
-void vp10_encode_tile(VP10_COMP *cpi, ThreadData *td,
- int tile_row, int tile_col) {
- VP10_COMMON *const cm = &cpi->common;
- const int tile_cols = 1 << cm->log2_tile_cols;
- TileDataEnc *this_tile =
- &cpi->tile_data[tile_row * tile_cols + tile_col];
- const TileInfo * const tile_info = &this_tile->tile_info;
- TOKENEXTRA *tok = cpi->tile_tok[tile_row][tile_col];
- int mi_row;
-
- for (mi_row = tile_info->mi_row_start; mi_row < tile_info->mi_row_end;
- mi_row += MI_BLOCK_SIZE) {
- encode_rd_sb_row(cpi, td, this_tile, mi_row, &tok);
- }
- cpi->tok_count[tile_row][tile_col] =
- (unsigned int)(tok - cpi->tile_tok[tile_row][tile_col]);
- assert(tok - cpi->tile_tok[tile_row][tile_col] <=
- allocated_tokens(*tile_info));
-}
-
-static void encode_tiles(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int tile_rows = 1 << cm->log2_tile_rows;
- int tile_col, tile_row;
-
- vp10_init_tile_data(cpi);
-
- for (tile_row = 0; tile_row < tile_rows; ++tile_row)
- for (tile_col = 0; tile_col < tile_cols; ++tile_col)
- vp10_encode_tile(cpi, &cpi->td, tile_row, tile_col);
-}
-
-#if CONFIG_FP_MB_STATS
-static int input_fpmb_stats(FIRSTPASS_MB_STATS *firstpass_mb_stats,
- VP10_COMMON *cm, uint8_t **this_frame_mb_stats) {
- uint8_t *mb_stats_in = firstpass_mb_stats->mb_stats_start +
- cm->current_video_frame * cm->MBs * sizeof(uint8_t);
-
- if (mb_stats_in > firstpass_mb_stats->mb_stats_end)
- return EOF;
-
- *this_frame_mb_stats = mb_stats_in;
-
- return 1;
-}
-#endif
-
-static void encode_frame_internal(VP10_COMP *cpi) {
- ThreadData *const td = &cpi->td;
- MACROBLOCK *const x = &td->mb;
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- RD_COUNTS *const rdc = &cpi->td.rd_counts;
- int i;
-
- xd->mi = cm->mi_grid_visible;
- xd->mi[0] = cm->mi;
-
- vp10_zero(*td->counts);
- vp10_zero(rdc->coef_counts);
- vp10_zero(rdc->comp_pred_diff);
- vp10_zero(rdc->filter_diff);
-
- for (i = 0; i < (cm->seg.enabled ? MAX_SEGMENTS : 1); ++i) {
-#if CONFIG_MISC_FIXES
- const int qindex = vp10_get_qindex(&cm->seg, i, cm->base_qindex);
-#endif
- xd->lossless[i] = cm->y_dc_delta_q == 0 &&
-#if CONFIG_MISC_FIXES
- qindex == 0 &&
-#else
- cm->base_qindex == 0 &&
-#endif
- cm->uv_dc_delta_q == 0 &&
- cm->uv_ac_delta_q == 0;
- }
-
- if (!cm->seg.enabled && xd->lossless[0])
- x->optimize = 0;
-
- cm->tx_mode = select_tx_mode(cpi, xd);
-
- vp10_frame_init_quantizer(cpi);
-
- vp10_initialize_rd_consts(cpi);
- vp10_initialize_me_consts(cpi, x, cm->base_qindex);
- init_encode_frame_mb_context(cpi);
- cm->use_prev_frame_mvs = !cm->error_resilient_mode &&
- cm->width == cm->last_width &&
- cm->height == cm->last_height &&
- !cm->intra_only &&
- cm->last_show_frame;
- // Special case: set prev_mi to NULL when the previous mode info
- // context cannot be used.
- cm->prev_mi = cm->use_prev_frame_mvs ?
- cm->prev_mip + cm->mi_stride + 1 : NULL;
-
- x->quant_fp = cpi->sf.use_quant_fp;
- vp10_zero(x->skip_txfm);
-
- {
- struct vpx_usec_timer emr_timer;
- vpx_usec_timer_start(&emr_timer);
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- input_fpmb_stats(&cpi->twopass.firstpass_mb_stats, cm,
- &cpi->twopass.this_frame_mb_stats);
- }
-#endif
-
- // If allowed, encoding tiles in parallel with one thread handling one tile.
- if (VPXMIN(cpi->oxcf.max_threads, 1 << cm->log2_tile_cols) > 1)
- vp10_encode_tiles_mt(cpi);
- else
- encode_tiles(cpi);
-
- vpx_usec_timer_mark(&emr_timer);
- cpi->time_encode_sb_row += vpx_usec_timer_elapsed(&emr_timer);
- }
-
-#if 0
- // Keep record of the total distortion this time around for future use
- cpi->last_frame_distortion = cpi->frame_distortion;
-#endif
-}
-
-static INTERP_FILTER get_interp_filter(
- const int64_t threshes[SWITCHABLE_FILTER_CONTEXTS], int is_alt_ref) {
- if (!is_alt_ref &&
- threshes[EIGHTTAP_SMOOTH] > threshes[EIGHTTAP] &&
- threshes[EIGHTTAP_SMOOTH] > threshes[EIGHTTAP_SHARP] &&
- threshes[EIGHTTAP_SMOOTH] > threshes[SWITCHABLE - 1]) {
- return EIGHTTAP_SMOOTH;
- } else if (threshes[EIGHTTAP_SHARP] > threshes[EIGHTTAP] &&
- threshes[EIGHTTAP_SHARP] > threshes[SWITCHABLE - 1]) {
- return EIGHTTAP_SHARP;
- } else if (threshes[EIGHTTAP] > threshes[SWITCHABLE - 1]) {
- return EIGHTTAP;
- } else {
- return SWITCHABLE;
- }
-}
-
-void vp10_encode_frame(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
-
- // In the longer term the encoder should be generalized to match the
- // decoder such that we allow compound where one of the 3 buffers has a
- // different sign bias and that buffer is then the fixed ref. However, this
- // requires further work in the rd loop. For now the only supported encoder
- // side behavior is where the ALT ref buffer has opposite sign bias to
- // the other two.
- if (!frame_is_intra_only(cm)) {
- if ((cm->ref_frame_sign_bias[ALTREF_FRAME] ==
- cm->ref_frame_sign_bias[GOLDEN_FRAME]) ||
- (cm->ref_frame_sign_bias[ALTREF_FRAME] ==
- cm->ref_frame_sign_bias[LAST_FRAME])) {
- cpi->allow_comp_inter_inter = 0;
- } else {
- cpi->allow_comp_inter_inter = 1;
- cm->comp_fixed_ref = ALTREF_FRAME;
- cm->comp_var_ref[0] = LAST_FRAME;
- cm->comp_var_ref[1] = GOLDEN_FRAME;
- }
- } else {
- cpi->allow_comp_inter_inter = 0;
- }
-
- if (cpi->sf.frame_parameter_update) {
- int i;
- RD_OPT *const rd_opt = &cpi->rd;
- FRAME_COUNTS *counts = cpi->td.counts;
- RD_COUNTS *const rdc = &cpi->td.rd_counts;
-
- // This code does a single RD pass over the whole frame assuming
- // either compound, single or hybrid prediction as per whatever has
- // worked best for that type of frame in the past.
- // It also predicts whether another coding mode would have worked
- // better that this coding mode. If that is the case, it remembers
- // that for subsequent frames.
- // It does the same analysis for transform size selection also.
- const MV_REFERENCE_FRAME frame_type = get_frame_type(cpi);
- int64_t *const mode_thrs = rd_opt->prediction_type_threshes[frame_type];
- int64_t *const filter_thrs = rd_opt->filter_threshes[frame_type];
- const int is_alt_ref = frame_type == ALTREF_FRAME;
-
- /* prediction (compound, single or hybrid) mode selection */
- if (is_alt_ref || !cpi->allow_comp_inter_inter)
- cm->reference_mode = SINGLE_REFERENCE;
- else if (mode_thrs[COMPOUND_REFERENCE] > mode_thrs[SINGLE_REFERENCE] &&
- mode_thrs[COMPOUND_REFERENCE] >
- mode_thrs[REFERENCE_MODE_SELECT] &&
- check_dual_ref_flags(cpi) &&
- cpi->static_mb_pct == 100)
- cm->reference_mode = COMPOUND_REFERENCE;
- else if (mode_thrs[SINGLE_REFERENCE] > mode_thrs[REFERENCE_MODE_SELECT])
- cm->reference_mode = SINGLE_REFERENCE;
- else
- cm->reference_mode = REFERENCE_MODE_SELECT;
-
- if (cm->interp_filter == SWITCHABLE)
- cm->interp_filter = get_interp_filter(filter_thrs, is_alt_ref);
-
- encode_frame_internal(cpi);
-
- for (i = 0; i < REFERENCE_MODES; ++i)
- mode_thrs[i] = (mode_thrs[i] + rdc->comp_pred_diff[i] / cm->MBs) / 2;
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- filter_thrs[i] = (filter_thrs[i] + rdc->filter_diff[i] / cm->MBs) / 2;
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT) {
- int single_count_zero = 0;
- int comp_count_zero = 0;
-
- for (i = 0; i < COMP_INTER_CONTEXTS; i++) {
- single_count_zero += counts->comp_inter[i][0];
- comp_count_zero += counts->comp_inter[i][1];
- }
-
- if (comp_count_zero == 0) {
- cm->reference_mode = SINGLE_REFERENCE;
- vp10_zero(counts->comp_inter);
- } else if (single_count_zero == 0) {
- cm->reference_mode = COMPOUND_REFERENCE;
- vp10_zero(counts->comp_inter);
- }
- }
-
- if (cm->tx_mode == TX_MODE_SELECT) {
- int count4x4 = 0;
- int count8x8_lp = 0, count8x8_8x8p = 0;
- int count16x16_16x16p = 0, count16x16_lp = 0;
- int count32x32 = 0;
-
- for (i = 0; i < TX_SIZE_CONTEXTS; ++i) {
- count4x4 += counts->tx.p32x32[i][TX_4X4];
- count4x4 += counts->tx.p16x16[i][TX_4X4];
- count4x4 += counts->tx.p8x8[i][TX_4X4];
-
- count8x8_lp += counts->tx.p32x32[i][TX_8X8];
- count8x8_lp += counts->tx.p16x16[i][TX_8X8];
- count8x8_8x8p += counts->tx.p8x8[i][TX_8X8];
-
- count16x16_16x16p += counts->tx.p16x16[i][TX_16X16];
- count16x16_lp += counts->tx.p32x32[i][TX_16X16];
- count32x32 += counts->tx.p32x32[i][TX_32X32];
- }
- if (count4x4 == 0 && count16x16_lp == 0 && count16x16_16x16p == 0 &&
- count32x32 == 0) {
- cm->tx_mode = ALLOW_8X8;
- reset_skip_tx_size(cm, TX_8X8);
- } else if (count8x8_8x8p == 0 && count16x16_16x16p == 0 &&
- count8x8_lp == 0 && count16x16_lp == 0 && count32x32 == 0) {
- cm->tx_mode = ONLY_4X4;
- reset_skip_tx_size(cm, TX_4X4);
- } else if (count8x8_lp == 0 && count16x16_lp == 0 && count4x4 == 0) {
- cm->tx_mode = ALLOW_32X32;
- } else if (count32x32 == 0 && count8x8_lp == 0 && count4x4 == 0) {
- cm->tx_mode = ALLOW_16X16;
- reset_skip_tx_size(cm, TX_16X16);
- }
- }
- } else {
- cm->reference_mode = SINGLE_REFERENCE;
- encode_frame_internal(cpi);
- }
-}
-
-static void sum_intra_stats(FRAME_COUNTS *counts, const MODE_INFO *mi,
- const MODE_INFO *above_mi, const MODE_INFO *left_mi,
- const int intraonly) {
- const PREDICTION_MODE y_mode = mi->mbmi.mode;
- const PREDICTION_MODE uv_mode = mi->mbmi.uv_mode;
- const BLOCK_SIZE bsize = mi->mbmi.sb_type;
-
- if (bsize < BLOCK_8X8) {
- int idx, idy;
- const int num_4x4_w = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[bsize];
- for (idy = 0; idy < 2; idy += num_4x4_h)
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- const int bidx = idy * 2 + idx;
- const PREDICTION_MODE bmode = mi->bmi[bidx].as_mode;
- if (intraonly) {
- const PREDICTION_MODE a = vp10_above_block_mode(mi, above_mi, bidx);
- const PREDICTION_MODE l = vp10_left_block_mode(mi, left_mi, bidx);
- ++counts->kf_y_mode[a][l][bmode];
- } else {
- ++counts->y_mode[0][bmode];
- }
- }
- } else {
- if (intraonly) {
- const PREDICTION_MODE above = vp10_above_block_mode(mi, above_mi, 0);
- const PREDICTION_MODE left = vp10_left_block_mode(mi, left_mi, 0);
- ++counts->kf_y_mode[above][left][y_mode];
- } else {
- ++counts->y_mode[size_group_lookup[bsize]][y_mode];
- }
- }
-
- ++counts->uv_mode[y_mode][uv_mode];
-}
-
-static void encode_superblock(VP10_COMP *cpi, ThreadData *td,
- TOKENEXTRA **t, int output_enabled,
- int mi_row, int mi_col, BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- MODE_INFO **mi_8x8 = xd->mi;
- MODE_INFO *mi = mi_8x8[0];
- MB_MODE_INFO *mbmi = &mi->mbmi;
- const int seg_skip = segfeature_active(&cm->seg, mbmi->segment_id,
- SEG_LVL_SKIP);
- const int mis = cm->mi_stride;
- const int mi_width = num_8x8_blocks_wide_lookup[bsize];
- const int mi_height = num_8x8_blocks_high_lookup[bsize];
-
- x->skip_recode = !x->select_tx_size && mbmi->sb_type >= BLOCK_8X8 &&
- cpi->oxcf.aq_mode != COMPLEXITY_AQ &&
- cpi->oxcf.aq_mode != CYCLIC_REFRESH_AQ &&
- cpi->sf.allow_skip_recode;
-
- if (!x->skip_recode)
- memset(x->skip_txfm, 0, sizeof(x->skip_txfm));
-
- x->skip_optimize = ctx->is_coded;
- ctx->is_coded = 1;
- x->use_lp32x32fdct = cpi->sf.use_lp32x32fdct;
-
- if (!is_inter_block(mbmi)) {
- int plane;
- mbmi->skip = 1;
- for (plane = 0; plane < MAX_MB_PLANE; ++plane)
- vp10_encode_intra_block_plane(x, VPXMAX(bsize, BLOCK_8X8), plane);
- if (output_enabled)
- sum_intra_stats(td->counts, mi, xd->above_mi, xd->left_mi,
- frame_is_intra_only(cm));
-
- if (bsize >= BLOCK_8X8 && output_enabled) {
- if (mbmi->palette_mode_info.palette_size[0] > 0) {
- mbmi->palette_mode_info.palette_first_color_idx[0] =
- xd->plane[0].color_index_map[0];
- // TODO(huisu): this increases the use of token buffer. Needs stretch
- // test to verify.
- vp10_tokenize_palette_sb(td, bsize, 0, t);
- }
- }
- vp10_tokenize_sb(cpi, td, t, !output_enabled, VPXMAX(bsize, BLOCK_8X8));
- } else {
- int ref;
- const int is_compound = has_second_ref(mbmi);
- set_ref_ptrs(cm, xd, mbmi->ref_frame[0], mbmi->ref_frame[1]);
- for (ref = 0; ref < 1 + is_compound; ++ref) {
- YV12_BUFFER_CONFIG *cfg = get_ref_frame_buffer(cpi,
- mbmi->ref_frame[ref]);
- assert(cfg != NULL);
- vp10_setup_pre_planes(xd, ref, cfg, mi_row, mi_col,
- &xd->block_refs[ref]->sf);
- }
- if (!(cpi->sf.reuse_inter_pred_sby && ctx->pred_pixel_ready) || seg_skip)
- vp10_build_inter_predictors_sby(xd, mi_row, mi_col,
- VPXMAX(bsize, BLOCK_8X8));
-
- vp10_build_inter_predictors_sbuv(xd, mi_row, mi_col,
- VPXMAX(bsize, BLOCK_8X8));
-
- vp10_encode_sb(x, VPXMAX(bsize, BLOCK_8X8));
- vp10_tokenize_sb(cpi, td, t, !output_enabled, VPXMAX(bsize, BLOCK_8X8));
- }
-
- if (output_enabled) {
- if (cm->tx_mode == TX_MODE_SELECT &&
- mbmi->sb_type >= BLOCK_8X8 &&
- !(is_inter_block(mbmi) && (mbmi->skip || seg_skip))) {
- ++get_tx_counts(max_txsize_lookup[bsize], get_tx_size_context(xd),
- &td->counts->tx)[mbmi->tx_size];
- } else {
- int x, y;
- TX_SIZE tx_size;
- // The new intra coding scheme requires no change of transform size
- if (is_inter_block(&mi->mbmi)) {
- tx_size = VPXMIN(tx_mode_to_biggest_tx_size[cm->tx_mode],
- max_txsize_lookup[bsize]);
- } else {
- tx_size = (bsize >= BLOCK_8X8) ? mbmi->tx_size : TX_4X4;
- }
-
- for (y = 0; y < mi_height; y++)
- for (x = 0; x < mi_width; x++)
- if (mi_col + x < cm->mi_cols && mi_row + y < cm->mi_rows)
- mi_8x8[mis * y + x]->mbmi.tx_size = tx_size;
- }
- ++td->counts->tx.tx_totals[mbmi->tx_size];
- ++td->counts->tx.tx_totals[get_uv_tx_size(mbmi, &xd->plane[1])];
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_ENCODEFRAME_H_
-#define VP10_ENCODER_ENCODEFRAME_H_
-
-#include "vpx/vpx_integer.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct macroblock;
-struct yv12_buffer_config;
-struct VP10_COMP;
-struct ThreadData;
-
-// Constants used in SOURCE_VAR_BASED_PARTITION
-#define VAR_HIST_MAX_BG_VAR 1000
-#define VAR_HIST_FACTOR 10
-#define VAR_HIST_BINS (VAR_HIST_MAX_BG_VAR / VAR_HIST_FACTOR + 1)
-#define VAR_HIST_LARGE_CUT_OFF 75
-#define VAR_HIST_SMALL_CUT_OFF 45
-
-void vp10_setup_src_planes(struct macroblock *x,
- const struct yv12_buffer_config *src,
- int mi_row, int mi_col);
-
-void vp10_encode_frame(struct VP10_COMP *cpi);
-
-void vp10_init_tile_data(struct VP10_COMP *cpi);
-void vp10_encode_tile(struct VP10_COMP *cpi, struct ThreadData *td,
- int tile_row, int tile_col);
-
-void vp10_set_variance_partition_thresholds(struct VP10_COMP *cpi, int q);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_ENCODEFRAME_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#include "./vp10_rtcd.h"
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-
-#include "vpx_dsp/quantize.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/idct.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/reconintra.h"
-#include "vp10/common/scan.h"
-
-#include "vp10/encoder/encodemb.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/tokenize.h"
-
-struct optimize_ctx {
- ENTROPY_CONTEXT ta[MAX_MB_PLANE][16];
- ENTROPY_CONTEXT tl[MAX_MB_PLANE][16];
-};
-
-void vp10_subtract_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane) {
- struct macroblock_plane *const p = &x->plane[plane];
- const struct macroblockd_plane *const pd = &x->e_mbd.plane[plane];
- const BLOCK_SIZE plane_bsize = get_plane_block_size(bsize, pd);
- const int bw = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- const int bh = 4 * num_4x4_blocks_high_lookup[plane_bsize];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (x->e_mbd.cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- vpx_highbd_subtract_block(bh, bw, p->src_diff, bw, p->src.buf,
- p->src.stride, pd->dst.buf, pd->dst.stride,
- x->e_mbd.bd);
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- vpx_subtract_block(bh, bw, p->src_diff, bw, p->src.buf, p->src.stride,
- pd->dst.buf, pd->dst.stride);
-}
-
-#define RDTRUNC(RM, DM, R, D) ((128 + (R) * (RM)) & 0xFF)
-
-typedef struct vp10_token_state {
- int rate;
- int error;
- int next;
- int16_t token;
- short qc;
-} vp10_token_state;
-
-// TODO(jimbankoski): experiment to find optimal RD numbers.
-static const int plane_rd_mult[PLANE_TYPES] = { 4, 2 };
-
-#define UPDATE_RD_COST()\
-{\
- rd_cost0 = RDCOST(rdmult, rddiv, rate0, error0);\
- rd_cost1 = RDCOST(rdmult, rddiv, rate1, error1);\
- if (rd_cost0 == rd_cost1) {\
- rd_cost0 = RDTRUNC(rdmult, rddiv, rate0, error0);\
- rd_cost1 = RDTRUNC(rdmult, rddiv, rate1, error1);\
- }\
-}
-
-// This function is a place holder for now but may ultimately need
-// to scan previous tokens to work out the correct context.
-static int trellis_get_coeff_context(const int16_t *scan,
- const int16_t *nb,
- int idx, int token,
- uint8_t *token_cache) {
- int bak = token_cache[scan[idx]], pt;
- token_cache[scan[idx]] = vp10_pt_energy_class[token];
- pt = get_coef_context(nb, token_cache, idx + 1);
- token_cache[scan[idx]] = bak;
- return pt;
-}
-
-static int optimize_b(MACROBLOCK *mb, int plane, int block,
- TX_SIZE tx_size, int ctx) {
- MACROBLOCKD *const xd = &mb->e_mbd;
- struct macroblock_plane *const p = &mb->plane[plane];
- struct macroblockd_plane *const pd = &xd->plane[plane];
- const int ref = is_inter_block(&xd->mi[0]->mbmi);
- vp10_token_state tokens[1025][2];
- unsigned best_index[1025][2];
- uint8_t token_cache[1024];
- const tran_low_t *const coeff = BLOCK_OFFSET(mb->plane[plane].coeff, block);
- tran_low_t *const qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- const int eob = p->eobs[block];
- const PLANE_TYPE type = pd->plane_type;
- const int default_eob = 16 << (tx_size << 1);
- const int mul = 1 + (tx_size == TX_32X32);
- const int16_t *dequant_ptr = pd->dequant;
- const uint8_t *const band_translate = get_band_translate(tx_size);
- TX_TYPE tx_type = get_tx_type(type, xd, block);
- const scan_order *const so = get_scan(tx_size, tx_type);
- const int16_t *const scan = so->scan;
- const int16_t *const nb = so->neighbors;
- int next = eob, sz = 0;
- int64_t rdmult = mb->rdmult * plane_rd_mult[type], rddiv = mb->rddiv;
- int64_t rd_cost0, rd_cost1;
- int rate0, rate1, error0, error1;
- int16_t t0, t1;
- EXTRABIT e0;
- int best, band, pt, i, final_eob;
-#if CONFIG_VP9_HIGHBITDEPTH
- const int16_t *cat6_high_cost = vp10_get_high_cost_table(xd->bd);
-#else
- const int16_t *cat6_high_cost = vp10_get_high_cost_table(8);
-#endif
-
- assert((!type && !plane) || (type && plane));
- assert(eob <= default_eob);
-
- /* Now set up a Viterbi trellis to evaluate alternative roundings. */
- if (!ref)
- rdmult = (rdmult * 9) >> 4;
-
- /* Initialize the sentinel node of the trellis. */
- tokens[eob][0].rate = 0;
- tokens[eob][0].error = 0;
- tokens[eob][0].next = default_eob;
- tokens[eob][0].token = EOB_TOKEN;
- tokens[eob][0].qc = 0;
- tokens[eob][1] = tokens[eob][0];
-
- for (i = 0; i < eob; i++)
- token_cache[scan[i]] =
- vp10_pt_energy_class[vp10_get_token(qcoeff[scan[i]])];
-
- for (i = eob; i-- > 0;) {
- int base_bits, d2, dx;
- const int rc = scan[i];
- int x = qcoeff[rc];
- /* Only add a trellis state for non-zero coefficients. */
- if (x) {
- int shortcut = 0;
- error0 = tokens[next][0].error;
- error1 = tokens[next][1].error;
- /* Evaluate the first possibility for this state. */
- rate0 = tokens[next][0].rate;
- rate1 = tokens[next][1].rate;
- vp10_get_token_extra(x, &t0, &e0);
- /* Consider both possible successor states. */
- if (next < default_eob) {
- band = band_translate[i + 1];
- pt = trellis_get_coeff_context(scan, nb, i, t0, token_cache);
- rate0 += mb->token_costs[tx_size][type][ref][band][0][pt]
- [tokens[next][0].token];
- rate1 += mb->token_costs[tx_size][type][ref][band][0][pt]
- [tokens[next][1].token];
- }
- UPDATE_RD_COST();
- /* And pick the best. */
- best = rd_cost1 < rd_cost0;
- base_bits = vp10_get_cost(t0, e0, cat6_high_cost);
- dx = mul * (dqcoeff[rc] - coeff[rc]);
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- dx >>= xd->bd - 8;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- d2 = dx * dx;
- tokens[i][0].rate = base_bits + (best ? rate1 : rate0);
- tokens[i][0].error = d2 + (best ? error1 : error0);
- tokens[i][0].next = next;
- tokens[i][0].token = t0;
- tokens[i][0].qc = x;
- best_index[i][0] = best;
-
- /* Evaluate the second possibility for this state. */
- rate0 = tokens[next][0].rate;
- rate1 = tokens[next][1].rate;
-
- if ((abs(x) * dequant_ptr[rc != 0] > abs(coeff[rc]) * mul) &&
- (abs(x) * dequant_ptr[rc != 0] < abs(coeff[rc]) * mul +
- dequant_ptr[rc != 0]))
- shortcut = 1;
- else
- shortcut = 0;
-
- if (shortcut) {
- sz = -(x < 0);
- x -= 2 * sz + 1;
- }
-
- /* Consider both possible successor states. */
- if (!x) {
- /* If we reduced this coefficient to zero, check to see if
- * we need to move the EOB back here.
- */
- t0 = tokens[next][0].token == EOB_TOKEN ? EOB_TOKEN : ZERO_TOKEN;
- t1 = tokens[next][1].token == EOB_TOKEN ? EOB_TOKEN : ZERO_TOKEN;
- e0 = 0;
- } else {
- vp10_get_token_extra(x, &t0, &e0);
- t1 = t0;
- }
- if (next < default_eob) {
- band = band_translate[i + 1];
- if (t0 != EOB_TOKEN) {
- pt = trellis_get_coeff_context(scan, nb, i, t0, token_cache);
- rate0 += mb->token_costs[tx_size][type][ref][band][!x][pt]
- [tokens[next][0].token];
- }
- if (t1 != EOB_TOKEN) {
- pt = trellis_get_coeff_context(scan, nb, i, t1, token_cache);
- rate1 += mb->token_costs[tx_size][type][ref][band][!x][pt]
- [tokens[next][1].token];
- }
- }
-
- UPDATE_RD_COST();
- /* And pick the best. */
- best = rd_cost1 < rd_cost0;
- base_bits = vp10_get_cost(t0, e0, cat6_high_cost);
-
- if (shortcut) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- dx -= ((dequant_ptr[rc != 0] >> (xd->bd - 8)) + sz) ^ sz;
- } else {
- dx -= (dequant_ptr[rc != 0] + sz) ^ sz;
- }
-#else
- dx -= (dequant_ptr[rc != 0] + sz) ^ sz;
-#endif // CONFIG_VP9_HIGHBITDEPTH
- d2 = dx * dx;
- }
- tokens[i][1].rate = base_bits + (best ? rate1 : rate0);
- tokens[i][1].error = d2 + (best ? error1 : error0);
- tokens[i][1].next = next;
- tokens[i][1].token = best ? t1 : t0;
- tokens[i][1].qc = x;
- best_index[i][1] = best;
- /* Finally, make this the new head of the trellis. */
- next = i;
- } else {
- /* There's no choice to make for a zero coefficient, so we don't
- * add a new trellis node, but we do need to update the costs.
- */
- band = band_translate[i + 1];
- t0 = tokens[next][0].token;
- t1 = tokens[next][1].token;
- /* Update the cost of each path if we're past the EOB token. */
- if (t0 != EOB_TOKEN) {
- tokens[next][0].rate +=
- mb->token_costs[tx_size][type][ref][band][1][0][t0];
- tokens[next][0].token = ZERO_TOKEN;
- }
- if (t1 != EOB_TOKEN) {
- tokens[next][1].rate +=
- mb->token_costs[tx_size][type][ref][band][1][0][t1];
- tokens[next][1].token = ZERO_TOKEN;
- }
- best_index[i][0] = best_index[i][1] = 0;
- /* Don't update next, because we didn't add a new node. */
- }
- }
-
- /* Now pick the best path through the whole trellis. */
- band = band_translate[i + 1];
- rate0 = tokens[next][0].rate;
- rate1 = tokens[next][1].rate;
- error0 = tokens[next][0].error;
- error1 = tokens[next][1].error;
- t0 = tokens[next][0].token;
- t1 = tokens[next][1].token;
- rate0 += mb->token_costs[tx_size][type][ref][band][0][ctx][t0];
- rate1 += mb->token_costs[tx_size][type][ref][band][0][ctx][t1];
- UPDATE_RD_COST();
- best = rd_cost1 < rd_cost0;
- final_eob = -1;
- memset(qcoeff, 0, sizeof(*qcoeff) * (16 << (tx_size * 2)));
- memset(dqcoeff, 0, sizeof(*dqcoeff) * (16 << (tx_size * 2)));
- for (i = next; i < eob; i = next) {
- const int x = tokens[i][best].qc;
- const int rc = scan[i];
- if (x) {
- final_eob = i;
- }
-
- qcoeff[rc] = x;
- dqcoeff[rc] = (x * dequant_ptr[rc != 0]) / mul;
-
- next = tokens[i][best].next;
- best = best_index[i][best];
- }
- final_eob++;
-
- mb->plane[plane].eobs[block] = final_eob;
- return final_eob;
-}
-
-static INLINE void fdct32x32(int rd_transform,
- const int16_t *src, tran_low_t *dst,
- int src_stride) {
- if (rd_transform)
- vpx_fdct32x32_rd(src, dst, src_stride);
- else
- vpx_fdct32x32(src, dst, src_stride);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static INLINE void highbd_fdct32x32(int rd_transform, const int16_t *src,
- tran_low_t *dst, int src_stride) {
- if (rd_transform)
- vpx_highbd_fdct32x32_rd(src, dst, src_stride);
- else
- vpx_highbd_fdct32x32(src, dst, src_stride);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_xform_quant_fp(MACROBLOCK *x, int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size) {
- MACROBLOCKD *const xd = &x->e_mbd;
- const struct macroblock_plane *const p = &x->plane[plane];
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- PLANE_TYPE plane_type = (plane == 0) ? PLANE_TYPE_Y : PLANE_TYPE_UV;
- TX_TYPE tx_type = get_tx_type(plane_type, xd, block);
- const scan_order *const scan_order = get_scan(tx_size, tx_type);
- tran_low_t *const coeff = BLOCK_OFFSET(p->coeff, block);
- tran_low_t *const qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- uint16_t *const eob = &p->eobs[block];
- const int diff_stride = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- const int16_t *src_diff;
- src_diff = &p->src_diff[4 * (blk_row * diff_stride + blk_col)];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_32X32:
- highbd_fdct32x32(x->use_lp32x32fdct, src_diff, coeff, diff_stride);
- vp10_highbd_quantize_fp_32x32(coeff, 1024, x->skip_block, p->zbin,
- p->round_fp, p->quant_fp, p->quant_shift,
- qcoeff, dqcoeff, pd->dequant,
- eob, scan_order->scan,
- scan_order->iscan);
- break;
- case TX_16X16:
- vpx_highbd_fdct16x16(src_diff, coeff, diff_stride);
- vp10_highbd_quantize_fp(coeff, 256, x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_8X8:
- vpx_highbd_fdct8x8(src_diff, coeff, diff_stride);
- vp10_highbd_quantize_fp(coeff, 64, x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_4X4:
- if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- vp10_highbd_fwht4x4(src_diff, coeff, diff_stride);
- } else {
- vpx_highbd_fdct4x4(src_diff, coeff, diff_stride);
- }
- vp10_highbd_quantize_fp(coeff, 16, x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- default:
- assert(0);
- }
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- switch (tx_size) {
- case TX_32X32:
- fdct32x32(x->use_lp32x32fdct, src_diff, coeff, diff_stride);
- vp10_quantize_fp_32x32(coeff, 1024, x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob, scan_order->scan,
- scan_order->iscan);
- break;
- case TX_16X16:
- vpx_fdct16x16(src_diff, coeff, diff_stride);
- vp10_quantize_fp(coeff, 256, x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_8X8:
- vp10_fdct8x8_quant(src_diff, diff_stride, coeff, 64,
- x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_4X4:
- if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- vp10_fwht4x4(src_diff, coeff, diff_stride);
- } else {
- vpx_fdct4x4(src_diff, coeff, diff_stride);
- }
- vp10_quantize_fp(coeff, 16, x->skip_block, p->zbin, p->round_fp,
- p->quant_fp, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_xform_quant_dc(MACROBLOCK *x, int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size) {
- MACROBLOCKD *const xd = &x->e_mbd;
- const struct macroblock_plane *const p = &x->plane[plane];
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- tran_low_t *const coeff = BLOCK_OFFSET(p->coeff, block);
- tran_low_t *const qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- uint16_t *const eob = &p->eobs[block];
- const int diff_stride = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- const int16_t *src_diff;
- src_diff = &p->src_diff[4 * (blk_row * diff_stride + blk_col)];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_32X32:
- vpx_highbd_fdct32x32_1(src_diff, coeff, diff_stride);
- vpx_highbd_quantize_dc_32x32(coeff, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- case TX_16X16:
- vpx_highbd_fdct16x16_1(src_diff, coeff, diff_stride);
- vpx_highbd_quantize_dc(coeff, 256, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- case TX_8X8:
- vpx_highbd_fdct8x8_1(src_diff, coeff, diff_stride);
- vpx_highbd_quantize_dc(coeff, 64, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- case TX_4X4:
- if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- vp10_highbd_fwht4x4(src_diff, coeff, diff_stride);
- } else {
- vpx_highbd_fdct4x4(src_diff, coeff, diff_stride);
- }
- vpx_highbd_quantize_dc(coeff, 16, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- default:
- assert(0);
- }
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- switch (tx_size) {
- case TX_32X32:
- vpx_fdct32x32_1(src_diff, coeff, diff_stride);
- vpx_quantize_dc_32x32(coeff, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- case TX_16X16:
- vpx_fdct16x16_1(src_diff, coeff, diff_stride);
- vpx_quantize_dc(coeff, 256, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- case TX_8X8:
- vpx_fdct8x8_1(src_diff, coeff, diff_stride);
- vpx_quantize_dc(coeff, 64, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- case TX_4X4:
- if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- vp10_fwht4x4(src_diff, coeff, diff_stride);
- } else {
- vpx_fdct4x4(src_diff, coeff, diff_stride);
- }
- vpx_quantize_dc(coeff, 16, x->skip_block, p->round,
- p->quant_fp[0], qcoeff, dqcoeff,
- pd->dequant[0], eob);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_fwd_txfm_4x4(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type, int lossless) {
- if (lossless) {
- vp10_fwht4x4(src_diff, coeff, diff_stride);
- } else {
- switch (tx_type) {
- case DCT_DCT:
- vpx_fdct4x4(src_diff, coeff, diff_stride);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_fht4x4(src_diff, coeff, diff_stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
- }
-}
-
-static void fwd_txfm_8x8(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_fht8x8(src_diff, coeff, diff_stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static void fwd_txfm_16x16(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_fht16x16(src_diff, coeff, diff_stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static void fwd_txfm_32x32(int rd_transform, const int16_t *src_diff,
- tran_low_t *coeff, int diff_stride,
- TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- fdct32x32(rd_transform, src_diff, coeff, diff_stride);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- assert(0);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_fwd_txfm_4x4(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type, int lossless) {
- if (lossless) {
- assert(tx_type == DCT_DCT);
- vp10_highbd_fwht4x4(src_diff, coeff, diff_stride);
- } else {
- switch (tx_type) {
- case DCT_DCT:
- vpx_highbd_fdct4x4(src_diff, coeff, diff_stride);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_highbd_fht4x4(src_diff, coeff, diff_stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
- }
-}
-
-static void highbd_fwd_txfm_8x8(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vpx_highbd_fdct8x8(src_diff, coeff, diff_stride);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_highbd_fht8x8(src_diff, coeff, diff_stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static void highbd_fwd_txfm_16x16(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- vpx_highbd_fdct16x16(src_diff, coeff, diff_stride);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- vp10_highbd_fht16x16(src_diff, coeff, diff_stride, tx_type);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static void highbd_fwd_txfm_32x32(int rd_transform, const int16_t *src_diff,
- tran_low_t *coeff, int diff_stride,
- TX_TYPE tx_type) {
- switch (tx_type) {
- case DCT_DCT:
- highbd_fdct32x32(rd_transform, src_diff, coeff, diff_stride);
- break;
- case ADST_DCT:
- case DCT_ADST:
- case ADST_ADST:
- assert(0);
- break;
- default:
- assert(0);
- break;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_xform_quant(MACROBLOCK *x, int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size) {
- MACROBLOCKD *const xd = &x->e_mbd;
- const struct macroblock_plane *const p = &x->plane[plane];
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- PLANE_TYPE plane_type = (plane == 0) ? PLANE_TYPE_Y : PLANE_TYPE_UV;
- TX_TYPE tx_type = get_tx_type(plane_type, xd, block);
- const scan_order *const scan_order = get_scan(tx_size, tx_type);
- tran_low_t *const coeff = BLOCK_OFFSET(p->coeff, block);
- tran_low_t *const qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- uint16_t *const eob = &p->eobs[block];
- const int diff_stride = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- const int16_t *src_diff;
- src_diff = &p->src_diff[4 * (blk_row * diff_stride + blk_col)];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_32X32:
- highbd_fwd_txfm_32x32(x->use_lp32x32fdct, src_diff, coeff, diff_stride,
- tx_type);
- vpx_highbd_quantize_b_32x32(coeff, 1024, x->skip_block, p->zbin,
- p->round, p->quant, p->quant_shift, qcoeff,
- dqcoeff, pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_16X16:
- highbd_fwd_txfm_16x16(src_diff, coeff, diff_stride, tx_type);
- vpx_highbd_quantize_b(coeff, 256, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_8X8:
- highbd_fwd_txfm_8x8(src_diff, coeff, diff_stride, tx_type);
- vpx_highbd_quantize_b(coeff, 64, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_4X4:
- vp10_highbd_fwd_txfm_4x4(src_diff, coeff, diff_stride, tx_type,
- xd->lossless[xd->mi[0]->mbmi.segment_id]);
- vpx_highbd_quantize_b(coeff, 16, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- default:
- assert(0);
- }
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- switch (tx_size) {
- case TX_32X32:
- fwd_txfm_32x32(x->use_lp32x32fdct, src_diff, coeff, diff_stride, tx_type);
- vpx_quantize_b_32x32(coeff, 1024, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob, scan_order->scan,
- scan_order->iscan);
- break;
- case TX_16X16:
- fwd_txfm_16x16(src_diff, coeff, diff_stride, tx_type);
- vpx_quantize_b(coeff, 256, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_8X8:
- fwd_txfm_8x8(src_diff, coeff, diff_stride, tx_type);
- vpx_quantize_b(coeff, 64, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- case TX_4X4:
- vp10_fwd_txfm_4x4(src_diff, coeff, diff_stride, tx_type,
- xd->lossless[xd->mi[0]->mbmi.segment_id]);
- vpx_quantize_b(coeff, 16, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static void encode_block(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg) {
- struct encode_b_args *const args = arg;
- MACROBLOCK *const x = args->x;
- MACROBLOCKD *const xd = &x->e_mbd;
- struct optimize_ctx *const ctx = args->ctx;
- struct macroblock_plane *const p = &x->plane[plane];
- struct macroblockd_plane *const pd = &xd->plane[plane];
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- uint8_t *dst;
- ENTROPY_CONTEXT *a, *l;
- TX_TYPE tx_type = get_tx_type(pd->plane_type, xd, block);
- dst = &pd->dst.buf[4 * blk_row * pd->dst.stride + 4 * blk_col];
- a = &ctx->ta[plane][blk_col];
- l = &ctx->tl[plane][blk_row];
-
- // TODO(jingning): per transformed block zero forcing only enabled for
- // luma component. will integrate chroma components as well.
- if (x->zcoeff_blk[tx_size][block] && plane == 0) {
- p->eobs[block] = 0;
- *a = *l = 0;
- return;
- }
-
- if (!x->skip_recode) {
- if (x->quant_fp) {
- // Encoding process for rtc mode
- if (x->skip_txfm[0] == SKIP_TXFM_AC_DC && plane == 0) {
- // skip forward transform
- p->eobs[block] = 0;
- *a = *l = 0;
- return;
- } else {
- vp10_xform_quant_fp(x, plane, block, blk_row, blk_col,
- plane_bsize, tx_size);
- }
- } else {
- if (max_txsize_lookup[plane_bsize] == tx_size) {
- int txfm_blk_index = (plane << 2) + (block >> (tx_size << 1));
- if (x->skip_txfm[txfm_blk_index] == SKIP_TXFM_NONE) {
- // full forward transform and quantization
- vp10_xform_quant(x, plane, block, blk_row, blk_col,
- plane_bsize, tx_size);
- } else if (x->skip_txfm[txfm_blk_index] == SKIP_TXFM_AC_ONLY) {
- // fast path forward transform and quantization
- vp10_xform_quant_dc(x, plane, block, blk_row, blk_col,
- plane_bsize, tx_size);
- } else {
- // skip forward transform
- p->eobs[block] = 0;
- *a = *l = 0;
- return;
- }
- } else {
- vp10_xform_quant(x, plane, block, blk_row, blk_col,
- plane_bsize, tx_size);
- }
- }
- }
-
- if (x->optimize && (!x->skip_recode || !x->skip_optimize)) {
- const int ctx = combine_entropy_contexts(*a, *l);
- *a = *l = optimize_b(x, plane, block, tx_size, ctx) > 0;
- } else {
- *a = *l = p->eobs[block] > 0;
- }
-
- if (p->eobs[block])
- *(args->skip) = 0;
-
- if (p->eobs[block] == 0)
- return;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_32X32:
- vp10_highbd_inv_txfm_add_32x32(dqcoeff, dst, pd->dst.stride,
- p->eobs[block], xd->bd, tx_type);
- break;
- case TX_16X16:
- vp10_highbd_inv_txfm_add_16x16(dqcoeff, dst, pd->dst.stride,
- p->eobs[block], xd->bd, tx_type);
- break;
- case TX_8X8:
- vp10_highbd_inv_txfm_add_8x8(dqcoeff, dst, pd->dst.stride,
- p->eobs[block], xd->bd, tx_type);
- break;
- case TX_4X4:
- // this is like vp10_short_idct4x4 but has a special case around eob<=1
- // which is significant (not just an optimization) for the lossless
- // case.
- vp10_highbd_inv_txfm_add_4x4(dqcoeff, dst, pd->dst.stride,
- p->eobs[block], xd->bd, tx_type,
- xd->lossless[xd->mi[0]->mbmi.segment_id]);
- break;
- default:
- assert(0 && "Invalid transform size");
- break;
- }
-
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- switch (tx_size) {
- case TX_32X32:
- vp10_inv_txfm_add_32x32(dqcoeff, dst, pd->dst.stride, p->eobs[block],
- tx_type);
- break;
- case TX_16X16:
- vp10_inv_txfm_add_16x16(dqcoeff, dst, pd->dst.stride, p->eobs[block],
- tx_type);
- break;
- case TX_8X8:
- vp10_inv_txfm_add_8x8(dqcoeff, dst, pd->dst.stride, p->eobs[block],
- tx_type);
- break;
- case TX_4X4:
- // this is like vp10_short_idct4x4 but has a special case around eob<=1
- // which is significant (not just an optimization) for the lossless
- // case.
- vp10_inv_txfm_add_4x4(dqcoeff, dst, pd->dst.stride, p->eobs[block],
- tx_type, xd->lossless[xd->mi[0]->mbmi.segment_id]);
- break;
- default:
- assert(0 && "Invalid transform size");
- break;
- }
-}
-
-static void encode_block_pass1(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg) {
- MACROBLOCK *const x = (MACROBLOCK *)arg;
- MACROBLOCKD *const xd = &x->e_mbd;
- struct macroblock_plane *const p = &x->plane[plane];
- struct macroblockd_plane *const pd = &xd->plane[plane];
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- uint8_t *dst;
- dst = &pd->dst.buf[4 * blk_row * pd->dst.stride + 4 * blk_col];
-
- vp10_xform_quant(x, plane, block, blk_row, blk_col, plane_bsize, tx_size);
-
- if (p->eobs[block] > 0) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- if (xd->lossless[0]) {
- vp10_highbd_iwht4x4_add(dqcoeff, dst, pd->dst.stride,
- p->eobs[block], xd->bd);
- } else {
- vp10_highbd_idct4x4_add(dqcoeff, dst, pd->dst.stride,
- p->eobs[block], xd->bd);
- }
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- if (xd->lossless[0]) {
- vp10_iwht4x4_add(dqcoeff, dst, pd->dst.stride, p->eobs[block]);
- } else {
- vp10_idct4x4_add(dqcoeff, dst, pd->dst.stride, p->eobs[block]);
- }
- }
-}
-
-void vp10_encode_sby_pass1(MACROBLOCK *x, BLOCK_SIZE bsize) {
- vp10_subtract_plane(x, bsize, 0);
- vp10_foreach_transformed_block_in_plane(&x->e_mbd, bsize, 0,
- encode_block_pass1, x);
-}
-
-void vp10_encode_sb(MACROBLOCK *x, BLOCK_SIZE bsize) {
- MACROBLOCKD *const xd = &x->e_mbd;
- struct optimize_ctx ctx;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- struct encode_b_args arg = {x, &ctx, &mbmi->skip};
- int plane;
-
- mbmi->skip = 1;
-
- if (x->skip)
- return;
-
- for (plane = 0; plane < MAX_MB_PLANE; ++plane) {
- if (!x->skip_recode)
- vp10_subtract_plane(x, bsize, plane);
-
- if (x->optimize && (!x->skip_recode || !x->skip_optimize)) {
- const struct macroblockd_plane* const pd = &xd->plane[plane];
- const TX_SIZE tx_size = plane ? get_uv_tx_size(mbmi, pd) : mbmi->tx_size;
- vp10_get_entropy_contexts(bsize, tx_size, pd,
- ctx.ta[plane], ctx.tl[plane]);
- }
-
- vp10_foreach_transformed_block_in_plane(xd, bsize, plane, encode_block,
- &arg);
- }
-}
-
-void vp10_encode_block_intra(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg) {
- struct encode_b_args* const args = arg;
- MACROBLOCK *const x = args->x;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- struct macroblock_plane *const p = &x->plane[plane];
- struct macroblockd_plane *const pd = &xd->plane[plane];
- tran_low_t *coeff = BLOCK_OFFSET(p->coeff, block);
- tran_low_t *qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- tran_low_t *dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
- PLANE_TYPE plane_type = (plane == 0) ? PLANE_TYPE_Y : PLANE_TYPE_UV;
- TX_TYPE tx_type = get_tx_type(plane_type, xd, block);
- const scan_order *const scan_order = get_scan(tx_size, tx_type);
- PREDICTION_MODE mode;
- const int bwl = b_width_log2_lookup[plane_bsize];
- const int bhl = b_height_log2_lookup[plane_bsize];
- const int diff_stride = 4 * (1 << bwl);
- uint8_t *src, *dst;
- int16_t *src_diff;
- uint16_t *eob = &p->eobs[block];
- const int src_stride = p->src.stride;
- const int dst_stride = pd->dst.stride;
- dst = &pd->dst.buf[4 * (blk_row * dst_stride + blk_col)];
- src = &p->src.buf[4 * (blk_row * src_stride + blk_col)];
- src_diff = &p->src_diff[4 * (blk_row * diff_stride + blk_col)];
-
- mode = plane == 0 ? get_y_mode(xd->mi[0], block) : mbmi->uv_mode;
- vp10_predict_intra_block(xd, bwl, bhl, tx_size, mode, dst, dst_stride,
- dst, dst_stride, blk_col, blk_row, plane);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- switch (tx_size) {
- case TX_32X32:
- if (!x->skip_recode) {
- vpx_highbd_subtract_block(32, 32, src_diff, diff_stride,
- src, src_stride, dst, dst_stride, xd->bd);
- highbd_fwd_txfm_32x32(x->use_lp32x32fdct, src_diff, coeff,
- diff_stride, tx_type);
- vpx_highbd_quantize_b_32x32(coeff, 1024, x->skip_block, p->zbin,
- p->round, p->quant, p->quant_shift,
- qcoeff, dqcoeff, pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- }
- if (*eob)
- vp10_highbd_inv_txfm_add_32x32(dqcoeff, dst, dst_stride, *eob, xd->bd,
- tx_type);
- break;
- case TX_16X16:
- if (!x->skip_recode) {
- vpx_highbd_subtract_block(16, 16, src_diff, diff_stride,
- src, src_stride, dst, dst_stride, xd->bd);
- highbd_fwd_txfm_16x16(src_diff, coeff, diff_stride, tx_type);
- vpx_highbd_quantize_b(coeff, 256, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- }
- if (*eob)
- vp10_highbd_inv_txfm_add_16x16(dqcoeff, dst, dst_stride, *eob, xd->bd,
- tx_type);
- break;
- case TX_8X8:
- if (!x->skip_recode) {
- vpx_highbd_subtract_block(8, 8, src_diff, diff_stride,
- src, src_stride, dst, dst_stride, xd->bd);
- highbd_fwd_txfm_8x8(src_diff, coeff, diff_stride, tx_type);
- vpx_highbd_quantize_b(coeff, 64, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- }
- if (*eob)
- vp10_highbd_inv_txfm_add_8x8(dqcoeff, dst, dst_stride, *eob, xd->bd,
- tx_type);
- break;
- case TX_4X4:
- if (!x->skip_recode) {
- vpx_highbd_subtract_block(4, 4, src_diff, diff_stride,
- src, src_stride, dst, dst_stride, xd->bd);
- vp10_highbd_fwd_txfm_4x4(src_diff, coeff, diff_stride, tx_type,
- xd->lossless[mbmi->segment_id]);
- vpx_highbd_quantize_b(coeff, 16, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob,
- scan_order->scan, scan_order->iscan);
- }
-
- if (*eob)
- // this is like vp10_short_idct4x4 but has a special case around
- // eob<=1 which is significant (not just an optimization) for the
- // lossless case.
- vp10_highbd_inv_txfm_add_4x4(dqcoeff, dst, dst_stride, *eob, xd->bd,
- tx_type, xd->lossless[mbmi->segment_id]);
- break;
- default:
- assert(0);
- return;
- }
- if (*eob)
- *(args->skip) = 0;
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- switch (tx_size) {
- case TX_32X32:
- if (!x->skip_recode) {
- vpx_subtract_block(32, 32, src_diff, diff_stride,
- src, src_stride, dst, dst_stride);
- fwd_txfm_32x32(x->use_lp32x32fdct, src_diff, coeff, diff_stride,
- tx_type);
- vpx_quantize_b_32x32(coeff, 1024, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob, scan_order->scan,
- scan_order->iscan);
- }
- if (*eob)
- vp10_inv_txfm_add_32x32(dqcoeff, dst, dst_stride, *eob, tx_type);
- break;
- case TX_16X16:
- if (!x->skip_recode) {
- vpx_subtract_block(16, 16, src_diff, diff_stride,
- src, src_stride, dst, dst_stride);
- fwd_txfm_16x16(src_diff, coeff, diff_stride, tx_type);
- vpx_quantize_b(coeff, 256, x->skip_block, p->zbin, p->round,
- p->quant, p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob, scan_order->scan,
- scan_order->iscan);
- }
- if (*eob)
- vp10_inv_txfm_add_16x16(dqcoeff, dst, dst_stride, *eob, tx_type);
- break;
- case TX_8X8:
- if (!x->skip_recode) {
- vpx_subtract_block(8, 8, src_diff, diff_stride,
- src, src_stride, dst, dst_stride);
- fwd_txfm_8x8(src_diff, coeff, diff_stride, tx_type);
- vpx_quantize_b(coeff, 64, x->skip_block, p->zbin, p->round, p->quant,
- p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob, scan_order->scan,
- scan_order->iscan);
- }
- if (*eob)
- vp10_inv_txfm_add_8x8(dqcoeff, dst, dst_stride, *eob, tx_type);
- break;
- case TX_4X4:
- if (!x->skip_recode) {
- vpx_subtract_block(4, 4, src_diff, diff_stride,
- src, src_stride, dst, dst_stride);
- vp10_fwd_txfm_4x4(src_diff, coeff, diff_stride, tx_type,
- xd->lossless[mbmi->segment_id]);
- vpx_quantize_b(coeff, 16, x->skip_block, p->zbin, p->round, p->quant,
- p->quant_shift, qcoeff, dqcoeff,
- pd->dequant, eob, scan_order->scan,
- scan_order->iscan);
- }
-
- if (*eob) {
- // this is like vp10_short_idct4x4 but has a special case around eob<=1
- // which is significant (not just an optimization) for the lossless
- // case.
- vp10_inv_txfm_add_4x4(dqcoeff, dst, dst_stride, *eob, tx_type,
- xd->lossless[mbmi->segment_id]);
- }
- break;
- default:
- assert(0);
- break;
- }
- if (*eob)
- *(args->skip) = 0;
-}
-
-void vp10_encode_intra_block_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- struct encode_b_args arg = {x, NULL, &xd->mi[0]->mbmi.skip};
-
- vp10_foreach_transformed_block_in_plane(xd, bsize, plane,
- vp10_encode_block_intra, &arg);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_ENCODEMB_H_
-#define VP10_ENCODER_ENCODEMB_H_
-
-#include "./vpx_config.h"
-#include "vp10/encoder/block.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct encode_b_args {
- MACROBLOCK *x;
- struct optimize_ctx *ctx;
- int8_t *skip;
-};
-void vp10_encode_sb(MACROBLOCK *x, BLOCK_SIZE bsize);
-void vp10_encode_sby_pass1(MACROBLOCK *x, BLOCK_SIZE bsize);
-void vp10_xform_quant_fp(MACROBLOCK *x, int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size);
-void vp10_xform_quant_dc(MACROBLOCK *x, int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size);
-void vp10_xform_quant(MACROBLOCK *x, int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size);
-
-void vp10_subtract_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane);
-
-void vp10_encode_block_intra(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg);
-
-void vp10_encode_intra_block_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane);
-
-void vp10_fwd_txfm_4x4(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type, int lossless);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_fwd_txfm_4x4(const int16_t *src_diff, tran_low_t *coeff,
- int diff_stride, TX_TYPE tx_type, int lossless);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_ENCODEMB_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-
-#include "vp10/common/common.h"
-#include "vp10/common/entropymode.h"
-
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/subexp.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-
-static struct vp10_token mv_joint_encodings[MV_JOINTS];
-static struct vp10_token mv_class_encodings[MV_CLASSES];
-static struct vp10_token mv_fp_encodings[MV_FP_SIZE];
-static struct vp10_token mv_class0_encodings[CLASS0_SIZE];
-
-void vp10_entropy_mv_init(void) {
- vp10_tokens_from_tree(mv_joint_encodings, vp10_mv_joint_tree);
- vp10_tokens_from_tree(mv_class_encodings, vp10_mv_class_tree);
- vp10_tokens_from_tree(mv_class0_encodings, vp10_mv_class0_tree);
- vp10_tokens_from_tree(mv_fp_encodings, vp10_mv_fp_tree);
-}
-
-static void encode_mv_component(vpx_writer* w, int comp,
- const nmv_component* mvcomp, int usehp) {
- int offset;
- const int sign = comp < 0;
- const int mag = sign ? -comp : comp;
- const int mv_class = vp10_get_mv_class(mag - 1, &offset);
- const int d = offset >> 3; // int mv data
- const int fr = (offset >> 1) & 3; // fractional mv data
- const int hp = offset & 1; // high precision mv data
-
- assert(comp != 0);
-
- // Sign
- vpx_write(w, sign, mvcomp->sign);
-
- // Class
- vp10_write_token(w, vp10_mv_class_tree, mvcomp->classes,
- &mv_class_encodings[mv_class]);
-
- // Integer bits
- if (mv_class == MV_CLASS_0) {
- vp10_write_token(w, vp10_mv_class0_tree, mvcomp->class0,
- &mv_class0_encodings[d]);
- } else {
- int i;
- const int n = mv_class + CLASS0_BITS - 1; // number of bits
- for (i = 0; i < n; ++i)
- vpx_write(w, (d >> i) & 1, mvcomp->bits[i]);
- }
-
- // Fractional bits
- vp10_write_token(w, vp10_mv_fp_tree,
- mv_class == MV_CLASS_0 ? mvcomp->class0_fp[d] : mvcomp->fp,
- &mv_fp_encodings[fr]);
-
- // High precision bit
- if (usehp)
- vpx_write(w, hp,
- mv_class == MV_CLASS_0 ? mvcomp->class0_hp : mvcomp->hp);
-}
-
-
-static void build_nmv_component_cost_table(int *mvcost,
- const nmv_component* const mvcomp,
- int usehp) {
- int i, v;
- int sign_cost[2], class_cost[MV_CLASSES], class0_cost[CLASS0_SIZE];
- int bits_cost[MV_OFFSET_BITS][2];
- int class0_fp_cost[CLASS0_SIZE][MV_FP_SIZE], fp_cost[MV_FP_SIZE];
- int class0_hp_cost[2], hp_cost[2];
-
- sign_cost[0] = vp10_cost_zero(mvcomp->sign);
- sign_cost[1] = vp10_cost_one(mvcomp->sign);
- vp10_cost_tokens(class_cost, mvcomp->classes, vp10_mv_class_tree);
- vp10_cost_tokens(class0_cost, mvcomp->class0, vp10_mv_class0_tree);
- for (i = 0; i < MV_OFFSET_BITS; ++i) {
- bits_cost[i][0] = vp10_cost_zero(mvcomp->bits[i]);
- bits_cost[i][1] = vp10_cost_one(mvcomp->bits[i]);
- }
-
- for (i = 0; i < CLASS0_SIZE; ++i)
- vp10_cost_tokens(class0_fp_cost[i], mvcomp->class0_fp[i], vp10_mv_fp_tree);
- vp10_cost_tokens(fp_cost, mvcomp->fp, vp10_mv_fp_tree);
-
- if (usehp) {
- class0_hp_cost[0] = vp10_cost_zero(mvcomp->class0_hp);
- class0_hp_cost[1] = vp10_cost_one(mvcomp->class0_hp);
- hp_cost[0] = vp10_cost_zero(mvcomp->hp);
- hp_cost[1] = vp10_cost_one(mvcomp->hp);
- }
- mvcost[0] = 0;
- for (v = 1; v <= MV_MAX; ++v) {
- int z, c, o, d, e, f, cost = 0;
- z = v - 1;
- c = vp10_get_mv_class(z, &o);
- cost += class_cost[c];
- d = (o >> 3); /* int mv data */
- f = (o >> 1) & 3; /* fractional pel mv data */
- e = (o & 1); /* high precision mv data */
- if (c == MV_CLASS_0) {
- cost += class0_cost[d];
- } else {
- int i, b;
- b = c + CLASS0_BITS - 1; /* number of bits */
- for (i = 0; i < b; ++i)
- cost += bits_cost[i][((d >> i) & 1)];
- }
- if (c == MV_CLASS_0) {
- cost += class0_fp_cost[d][f];
- } else {
- cost += fp_cost[f];
- }
- if (usehp) {
- if (c == MV_CLASS_0) {
- cost += class0_hp_cost[e];
- } else {
- cost += hp_cost[e];
- }
- }
- mvcost[v] = cost + sign_cost[0];
- mvcost[-v] = cost + sign_cost[1];
- }
-}
-
-static void update_mv(vpx_writer *w, const unsigned int ct[2], vpx_prob *cur_p,
- vpx_prob upd_p) {
-#if CONFIG_MISC_FIXES
- (void) upd_p;
- vp10_cond_prob_diff_update(w, cur_p, ct);
-#else
- const vpx_prob new_p = get_binary_prob(ct[0], ct[1]) | 1;
- const int update = cost_branch256(ct, *cur_p) + vp10_cost_zero(upd_p) >
- cost_branch256(ct, new_p) + vp10_cost_one(upd_p) + 7 * 256;
- vpx_write(w, update, upd_p);
- if (update) {
- *cur_p = new_p;
- vpx_write_literal(w, new_p >> 1, 7);
- }
-#endif
-}
-
-static void write_mv_update(const vpx_tree_index *tree,
- vpx_prob probs[/*n - 1*/],
- const unsigned int counts[/*n - 1*/],
- int n, vpx_writer *w) {
- int i;
- unsigned int branch_ct[32][2];
-
- // Assuming max number of probabilities <= 32
- assert(n <= 32);
-
- vp10_tree_probs_from_distribution(tree, branch_ct, counts);
- for (i = 0; i < n - 1; ++i)
- update_mv(w, branch_ct[i], &probs[i], MV_UPDATE_PROB);
-}
-
-void vp10_write_nmv_probs(VP10_COMMON *cm, int usehp, vpx_writer *w,
- nmv_context_counts *const counts) {
- int i, j;
- nmv_context *const mvc = &cm->fc->nmvc;
-
- write_mv_update(vp10_mv_joint_tree, mvc->joints, counts->joints, MV_JOINTS, w);
-
- for (i = 0; i < 2; ++i) {
- nmv_component *comp = &mvc->comps[i];
- nmv_component_counts *comp_counts = &counts->comps[i];
-
- update_mv(w, comp_counts->sign, &comp->sign, MV_UPDATE_PROB);
- write_mv_update(vp10_mv_class_tree, comp->classes, comp_counts->classes,
- MV_CLASSES, w);
- write_mv_update(vp10_mv_class0_tree, comp->class0, comp_counts->class0,
- CLASS0_SIZE, w);
- for (j = 0; j < MV_OFFSET_BITS; ++j)
- update_mv(w, comp_counts->bits[j], &comp->bits[j], MV_UPDATE_PROB);
- }
-
- for (i = 0; i < 2; ++i) {
- for (j = 0; j < CLASS0_SIZE; ++j)
- write_mv_update(vp10_mv_fp_tree, mvc->comps[i].class0_fp[j],
- counts->comps[i].class0_fp[j], MV_FP_SIZE, w);
-
- write_mv_update(vp10_mv_fp_tree, mvc->comps[i].fp, counts->comps[i].fp,
- MV_FP_SIZE, w);
- }
-
- if (usehp) {
- for (i = 0; i < 2; ++i) {
- update_mv(w, counts->comps[i].class0_hp, &mvc->comps[i].class0_hp,
- MV_UPDATE_PROB);
- update_mv(w, counts->comps[i].hp, &mvc->comps[i].hp, MV_UPDATE_PROB);
- }
- }
-}
-
-void vp10_encode_mv(VP10_COMP* cpi, vpx_writer* w,
- const MV* mv, const MV* ref,
- const nmv_context* mvctx, int usehp) {
- const MV diff = {mv->row - ref->row,
- mv->col - ref->col};
- const MV_JOINT_TYPE j = vp10_get_mv_joint(&diff);
- usehp = usehp && vp10_use_mv_hp(ref);
-
- vp10_write_token(w, vp10_mv_joint_tree, mvctx->joints, &mv_joint_encodings[j]);
- if (mv_joint_vertical(j))
- encode_mv_component(w, diff.row, &mvctx->comps[0], usehp);
-
- if (mv_joint_horizontal(j))
- encode_mv_component(w, diff.col, &mvctx->comps[1], usehp);
-
- // If auto_mv_step_size is enabled then keep track of the largest
- // motion vector component used.
- if (cpi->sf.mv.auto_mv_step_size) {
- unsigned int maxv = VPXMAX(abs(mv->row), abs(mv->col)) >> 3;
- cpi->max_mv_magnitude = VPXMAX(maxv, cpi->max_mv_magnitude);
- }
-}
-
-void vp10_build_nmv_cost_table(int *mvjoint, int *mvcost[2],
- const nmv_context* ctx, int usehp) {
- vp10_cost_tokens(mvjoint, ctx->joints, vp10_mv_joint_tree);
- build_nmv_component_cost_table(mvcost[0], &ctx->comps[0], usehp);
- build_nmv_component_cost_table(mvcost[1], &ctx->comps[1], usehp);
-}
-
-static void inc_mvs(const MB_MODE_INFO *mbmi, const MB_MODE_INFO_EXT *mbmi_ext,
- const int_mv mvs[2],
- nmv_context_counts *counts) {
- int i;
-
- for (i = 0; i < 1 + has_second_ref(mbmi); ++i) {
- const MV *ref = &mbmi_ext->ref_mvs[mbmi->ref_frame[i]][0].as_mv;
- const MV diff = {mvs[i].as_mv.row - ref->row,
- mvs[i].as_mv.col - ref->col};
- vp10_inc_mv(&diff, counts, vp10_use_mv_hp(ref));
- }
-}
-
-void vp10_update_mv_count(ThreadData *td) {
- const MACROBLOCKD *xd = &td->mb.e_mbd;
- const MODE_INFO *mi = xd->mi[0];
- const MB_MODE_INFO *const mbmi = &mi->mbmi;
- const MB_MODE_INFO_EXT *mbmi_ext = td->mb.mbmi_ext;
-
- if (mbmi->sb_type < BLOCK_8X8) {
- const int num_4x4_w = num_4x4_blocks_wide_lookup[mbmi->sb_type];
- const int num_4x4_h = num_4x4_blocks_high_lookup[mbmi->sb_type];
- int idx, idy;
-
- for (idy = 0; idy < 2; idy += num_4x4_h) {
- for (idx = 0; idx < 2; idx += num_4x4_w) {
- const int i = idy * 2 + idx;
- if (mi->bmi[i].as_mode == NEWMV)
- inc_mvs(mbmi, mbmi_ext, mi->bmi[i].as_mv, &td->counts->mv);
- }
- }
- } else {
- if (mbmi->mode == NEWMV)
- inc_mvs(mbmi, mbmi_ext, mbmi->mv, &td->counts->mv);
- }
-}
-
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_ENCODEMV_H_
-#define VP10_ENCODER_ENCODEMV_H_
-
-#include "vp10/encoder/encoder.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_entropy_mv_init(void);
-
-void vp10_write_nmv_probs(VP10_COMMON *cm, int usehp, vpx_writer *w,
- nmv_context_counts *const counts);
-
-void vp10_encode_mv(VP10_COMP *cpi, vpx_writer* w, const MV* mv, const MV* ref,
- const nmv_context* mvctx, int usehp);
-
-void vp10_build_nmv_cost_table(int *mvjoint, int *mvcost[2],
- const nmv_context* mvctx, int usehp);
-
-void vp10_update_mv_count(ThreadData *td);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_ENCODEMV_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-#include <stdio.h>
-#include <limits.h>
-
-#include "./vpx_config.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/filter.h"
-#include "vp10/common/idct.h"
-#if CONFIG_VP9_POSTPROC
-#include "vp10/common/postproc.h"
-#endif
-#include "vp10/common/reconinter.h"
-#include "vp10/common/reconintra.h"
-#include "vp10/common/tile_common.h"
-
-#include "vp10/encoder/aq_complexity.h"
-#include "vp10/encoder/aq_cyclicrefresh.h"
-#include "vp10/encoder/aq_variance.h"
-#include "vp10/encoder/bitstream.h"
-#include "vp10/encoder/context_tree.h"
-#include "vp10/encoder/encodeframe.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/ethread.h"
-#include "vp10/encoder/firstpass.h"
-#include "vp10/encoder/mbgraph.h"
-#include "vp10/encoder/picklpf.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/resize.h"
-#include "vp10/encoder/segmentation.h"
-#include "vp10/encoder/skin_detection.h"
-#include "vp10/encoder/speed_features.h"
-#include "vp10/encoder/temporal_filter.h"
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "./vpx_scale_rtcd.h"
-#include "vpx/internal/vpx_psnr.h"
-#if CONFIG_INTERNAL_STATS
-#include "vpx_dsp/ssim.h"
-#endif
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_dsp/vpx_filter.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-#include "vpx_ports/vpx_timer.h"
-#include "vpx_scale/vpx_scale.h"
-
-#define AM_SEGMENT_ID_INACTIVE 7
-#define AM_SEGMENT_ID_ACTIVE 0
-
-#define SHARP_FILTER_QTHRESH 0 /* Q threshold for 8-tap sharp filter */
-
-#define ALTREF_HIGH_PRECISION_MV 1 // Whether to use high precision mv
- // for altref computation.
-#define HIGH_PRECISION_MV_QTHRESH 200 // Q threshold for high precision
- // mv. Choose a very high value for
- // now so that HIGH_PRECISION is always
- // chosen.
-// #define OUTPUT_YUV_REC
-
-#ifdef OUTPUT_YUV_DENOISED
-FILE *yuv_denoised_file = NULL;
-#endif
-#ifdef OUTPUT_YUV_SKINMAP
-FILE *yuv_skinmap_file = NULL;
-#endif
-#ifdef OUTPUT_YUV_REC
-FILE *yuv_rec_file;
-#endif
-
-#if 0
-FILE *framepsnr;
-FILE *kf_list;
-FILE *keyfile;
-#endif
-
-static INLINE void Scale2Ratio(VPX_SCALING mode, int *hr, int *hs) {
- switch (mode) {
- case NORMAL:
- *hr = 1;
- *hs = 1;
- break;
- case FOURFIVE:
- *hr = 4;
- *hs = 5;
- break;
- case THREEFIVE:
- *hr = 3;
- *hs = 5;
- break;
- case ONETWO:
- *hr = 1;
- *hs = 2;
- break;
- default:
- *hr = 1;
- *hs = 1;
- assert(0);
- break;
- }
-}
-
-// Mark all inactive blocks as active. Other segmentation features may be set
-// so memset cannot be used, instead only inactive blocks should be reset.
-static void suppress_active_map(VP10_COMP *cpi) {
- unsigned char *const seg_map = cpi->segmentation_map;
- int i;
- if (cpi->active_map.enabled || cpi->active_map.update)
- for (i = 0; i < cpi->common.mi_rows * cpi->common.mi_cols; ++i)
- if (seg_map[i] == AM_SEGMENT_ID_INACTIVE)
- seg_map[i] = AM_SEGMENT_ID_ACTIVE;
-}
-
-static void apply_active_map(VP10_COMP *cpi) {
- struct segmentation *const seg = &cpi->common.seg;
- unsigned char *const seg_map = cpi->segmentation_map;
- const unsigned char *const active_map = cpi->active_map.map;
- int i;
-
- assert(AM_SEGMENT_ID_ACTIVE == CR_SEGMENT_ID_BASE);
-
- if (frame_is_intra_only(&cpi->common)) {
- cpi->active_map.enabled = 0;
- cpi->active_map.update = 1;
- }
-
- if (cpi->active_map.update) {
- if (cpi->active_map.enabled) {
- for (i = 0; i < cpi->common.mi_rows * cpi->common.mi_cols; ++i)
- if (seg_map[i] == AM_SEGMENT_ID_ACTIVE) seg_map[i] = active_map[i];
- vp10_enable_segmentation(seg);
- vp10_enable_segfeature(seg, AM_SEGMENT_ID_INACTIVE, SEG_LVL_SKIP);
- vp10_enable_segfeature(seg, AM_SEGMENT_ID_INACTIVE, SEG_LVL_ALT_LF);
- // Setting the data to -MAX_LOOP_FILTER will result in the computed loop
- // filter level being zero regardless of the value of seg->abs_delta.
- vp10_set_segdata(seg, AM_SEGMENT_ID_INACTIVE,
- SEG_LVL_ALT_LF, -MAX_LOOP_FILTER);
- } else {
- vp10_disable_segfeature(seg, AM_SEGMENT_ID_INACTIVE, SEG_LVL_SKIP);
- vp10_disable_segfeature(seg, AM_SEGMENT_ID_INACTIVE, SEG_LVL_ALT_LF);
- if (seg->enabled) {
- seg->update_data = 1;
- seg->update_map = 1;
- }
- }
- cpi->active_map.update = 0;
- }
-}
-
-int vp10_set_active_map(VP10_COMP* cpi,
- unsigned char* new_map_16x16,
- int rows,
- int cols) {
- if (rows == cpi->common.mb_rows && cols == cpi->common.mb_cols) {
- unsigned char *const active_map_8x8 = cpi->active_map.map;
- const int mi_rows = cpi->common.mi_rows;
- const int mi_cols = cpi->common.mi_cols;
- cpi->active_map.update = 1;
- if (new_map_16x16) {
- int r, c;
- for (r = 0; r < mi_rows; ++r) {
- for (c = 0; c < mi_cols; ++c) {
- active_map_8x8[r * mi_cols + c] =
- new_map_16x16[(r >> 1) * cols + (c >> 1)]
- ? AM_SEGMENT_ID_ACTIVE
- : AM_SEGMENT_ID_INACTIVE;
- }
- }
- cpi->active_map.enabled = 1;
- } else {
- cpi->active_map.enabled = 0;
- }
- return 0;
- } else {
- return -1;
- }
-}
-
-int vp10_get_active_map(VP10_COMP* cpi,
- unsigned char* new_map_16x16,
- int rows,
- int cols) {
- if (rows == cpi->common.mb_rows && cols == cpi->common.mb_cols &&
- new_map_16x16) {
- unsigned char* const seg_map_8x8 = cpi->segmentation_map;
- const int mi_rows = cpi->common.mi_rows;
- const int mi_cols = cpi->common.mi_cols;
- memset(new_map_16x16, !cpi->active_map.enabled, rows * cols);
- if (cpi->active_map.enabled) {
- int r, c;
- for (r = 0; r < mi_rows; ++r) {
- for (c = 0; c < mi_cols; ++c) {
- // Cyclic refresh segments are considered active despite not having
- // AM_SEGMENT_ID_ACTIVE
- new_map_16x16[(r >> 1) * cols + (c >> 1)] |=
- seg_map_8x8[r * mi_cols + c] != AM_SEGMENT_ID_INACTIVE;
- }
- }
- }
- return 0;
- } else {
- return -1;
- }
-}
-
-void vp10_set_high_precision_mv(VP10_COMP *cpi, int allow_high_precision_mv) {
- MACROBLOCK *const mb = &cpi->td.mb;
- cpi->common.allow_high_precision_mv = allow_high_precision_mv;
- if (cpi->common.allow_high_precision_mv) {
- mb->mvcost = mb->nmvcost_hp;
- mb->mvsadcost = mb->nmvsadcost_hp;
- } else {
- mb->mvcost = mb->nmvcost;
- mb->mvsadcost = mb->nmvsadcost;
- }
-}
-
-static void setup_frame(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- // Set up entropy context depending on frame type. The decoder mandates
- // the use of the default context, index 0, for keyframes and inter
- // frames where the error_resilient_mode or intra_only flag is set. For
- // other inter-frames the encoder currently uses only two contexts;
- // context 1 for ALTREF frames and context 0 for the others.
- if (frame_is_intra_only(cm) || cm->error_resilient_mode) {
- vp10_setup_past_independence(cm);
- } else {
- cm->frame_context_idx = cpi->refresh_alt_ref_frame;
- }
-
- if (cm->frame_type == KEY_FRAME) {
- cpi->refresh_golden_frame = 1;
- cpi->refresh_alt_ref_frame = 1;
- vp10_zero(cpi->interp_filter_selected);
- } else {
- *cm->fc = cm->frame_contexts[cm->frame_context_idx];
- vp10_zero(cpi->interp_filter_selected[0]);
- }
-}
-
-static void vp10_enc_setup_mi(VP10_COMMON *cm) {
- int i;
- cm->mi = cm->mip + cm->mi_stride + 1;
- memset(cm->mip, 0, cm->mi_stride * (cm->mi_rows + 1) * sizeof(*cm->mip));
- cm->prev_mi = cm->prev_mip + cm->mi_stride + 1;
- // Clear top border row
- memset(cm->prev_mip, 0, sizeof(*cm->prev_mip) * cm->mi_stride);
- // Clear left border column
- for (i = 1; i < cm->mi_rows + 1; ++i)
- memset(&cm->prev_mip[i * cm->mi_stride], 0, sizeof(*cm->prev_mip));
-
- cm->mi_grid_visible = cm->mi_grid_base + cm->mi_stride + 1;
- cm->prev_mi_grid_visible = cm->prev_mi_grid_base + cm->mi_stride + 1;
-
- memset(cm->mi_grid_base, 0,
- cm->mi_stride * (cm->mi_rows + 1) * sizeof(*cm->mi_grid_base));
-}
-
-static int vp10_enc_alloc_mi(VP10_COMMON *cm, int mi_size) {
- cm->mip = vpx_calloc(mi_size, sizeof(*cm->mip));
- if (!cm->mip)
- return 1;
- cm->prev_mip = vpx_calloc(mi_size, sizeof(*cm->prev_mip));
- if (!cm->prev_mip)
- return 1;
- cm->mi_alloc_size = mi_size;
-
- cm->mi_grid_base = (MODE_INFO **)vpx_calloc(mi_size, sizeof(MODE_INFO*));
- if (!cm->mi_grid_base)
- return 1;
- cm->prev_mi_grid_base = (MODE_INFO **)vpx_calloc(mi_size, sizeof(MODE_INFO*));
- if (!cm->prev_mi_grid_base)
- return 1;
-
- return 0;
-}
-
-static void vp10_enc_free_mi(VP10_COMMON *cm) {
- vpx_free(cm->mip);
- cm->mip = NULL;
- vpx_free(cm->prev_mip);
- cm->prev_mip = NULL;
- vpx_free(cm->mi_grid_base);
- cm->mi_grid_base = NULL;
- vpx_free(cm->prev_mi_grid_base);
- cm->prev_mi_grid_base = NULL;
-}
-
-static void vp10_swap_mi_and_prev_mi(VP10_COMMON *cm) {
- // Current mip will be the prev_mip for the next frame.
- MODE_INFO **temp_base = cm->prev_mi_grid_base;
- MODE_INFO *temp = cm->prev_mip;
- cm->prev_mip = cm->mip;
- cm->mip = temp;
-
- // Update the upper left visible macroblock ptrs.
- cm->mi = cm->mip + cm->mi_stride + 1;
- cm->prev_mi = cm->prev_mip + cm->mi_stride + 1;
-
- cm->prev_mi_grid_base = cm->mi_grid_base;
- cm->mi_grid_base = temp_base;
- cm->mi_grid_visible = cm->mi_grid_base + cm->mi_stride + 1;
- cm->prev_mi_grid_visible = cm->prev_mi_grid_base + cm->mi_stride + 1;
-}
-
-void vp10_initialize_enc(void) {
- static volatile int init_done = 0;
-
- if (!init_done) {
- vp10_rtcd();
- vpx_dsp_rtcd();
- vpx_scale_rtcd();
- vp10_init_intra_predictors();
- vp10_init_me_luts();
- vp10_rc_init_minq_luts();
- vp10_entropy_mv_init();
- vp10_temporal_filter_init();
- init_done = 1;
- }
-}
-
-static void dealloc_compressor_data(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
-
- vpx_free(cpi->mbmi_ext_base);
- cpi->mbmi_ext_base = NULL;
-
- vpx_free(cpi->tile_data);
- cpi->tile_data = NULL;
-
- // Delete sementation map
- vpx_free(cpi->segmentation_map);
- cpi->segmentation_map = NULL;
- vpx_free(cpi->coding_context.last_frame_seg_map_copy);
- cpi->coding_context.last_frame_seg_map_copy = NULL;
-
- vpx_free(cpi->nmvcosts[0]);
- vpx_free(cpi->nmvcosts[1]);
- cpi->nmvcosts[0] = NULL;
- cpi->nmvcosts[1] = NULL;
-
- vpx_free(cpi->nmvcosts_hp[0]);
- vpx_free(cpi->nmvcosts_hp[1]);
- cpi->nmvcosts_hp[0] = NULL;
- cpi->nmvcosts_hp[1] = NULL;
-
- vpx_free(cpi->nmvsadcosts[0]);
- vpx_free(cpi->nmvsadcosts[1]);
- cpi->nmvsadcosts[0] = NULL;
- cpi->nmvsadcosts[1] = NULL;
-
- vpx_free(cpi->nmvsadcosts_hp[0]);
- vpx_free(cpi->nmvsadcosts_hp[1]);
- cpi->nmvsadcosts_hp[0] = NULL;
- cpi->nmvsadcosts_hp[1] = NULL;
-
- vp10_cyclic_refresh_free(cpi->cyclic_refresh);
- cpi->cyclic_refresh = NULL;
-
- vpx_free(cpi->active_map.map);
- cpi->active_map.map = NULL;
-
- vp10_free_ref_frame_buffers(cm->buffer_pool);
-#if CONFIG_VP9_POSTPROC
- vp10_free_postproc_buffers(cm);
-#endif
- vp10_free_context_buffers(cm);
-
- vpx_free_frame_buffer(&cpi->last_frame_uf);
- vpx_free_frame_buffer(&cpi->scaled_source);
- vpx_free_frame_buffer(&cpi->scaled_last_source);
- vpx_free_frame_buffer(&cpi->alt_ref_buffer);
- vp10_lookahead_destroy(cpi->lookahead);
-
- vpx_free(cpi->tile_tok[0][0]);
- cpi->tile_tok[0][0] = 0;
-
- vp10_free_pc_tree(&cpi->td);
-
- if (cpi->common.allow_screen_content_tools)
- vpx_free(cpi->td.mb.palette_buffer);
-
- if (cpi->source_diff_var != NULL) {
- vpx_free(cpi->source_diff_var);
- cpi->source_diff_var = NULL;
- }
-}
-
-static void save_coding_context(VP10_COMP *cpi) {
- CODING_CONTEXT *const cc = &cpi->coding_context;
- VP10_COMMON *cm = &cpi->common;
-
- // Stores a snapshot of key state variables which can subsequently be
- // restored with a call to vp10_restore_coding_context. These functions are
- // intended for use in a re-code loop in vp10_compress_frame where the
- // quantizer value is adjusted between loop iterations.
- vp10_copy(cc->nmvjointcost, cpi->td.mb.nmvjointcost);
-
- memcpy(cc->nmvcosts[0], cpi->nmvcosts[0],
- MV_VALS * sizeof(*cpi->nmvcosts[0]));
- memcpy(cc->nmvcosts[1], cpi->nmvcosts[1],
- MV_VALS * sizeof(*cpi->nmvcosts[1]));
- memcpy(cc->nmvcosts_hp[0], cpi->nmvcosts_hp[0],
- MV_VALS * sizeof(*cpi->nmvcosts_hp[0]));
- memcpy(cc->nmvcosts_hp[1], cpi->nmvcosts_hp[1],
- MV_VALS * sizeof(*cpi->nmvcosts_hp[1]));
-
-#if !CONFIG_MISC_FIXES
- vp10_copy(cc->segment_pred_probs, cm->segp.pred_probs);
-#endif
-
- memcpy(cpi->coding_context.last_frame_seg_map_copy,
- cm->last_frame_seg_map, (cm->mi_rows * cm->mi_cols));
-
- vp10_copy(cc->last_ref_lf_deltas, cm->lf.last_ref_deltas);
- vp10_copy(cc->last_mode_lf_deltas, cm->lf.last_mode_deltas);
-
- cc->fc = *cm->fc;
-}
-
-static void restore_coding_context(VP10_COMP *cpi) {
- CODING_CONTEXT *const cc = &cpi->coding_context;
- VP10_COMMON *cm = &cpi->common;
-
- // Restore key state variables to the snapshot state stored in the
- // previous call to vp10_save_coding_context.
- vp10_copy(cpi->td.mb.nmvjointcost, cc->nmvjointcost);
-
- memcpy(cpi->nmvcosts[0], cc->nmvcosts[0], MV_VALS * sizeof(*cc->nmvcosts[0]));
- memcpy(cpi->nmvcosts[1], cc->nmvcosts[1], MV_VALS * sizeof(*cc->nmvcosts[1]));
- memcpy(cpi->nmvcosts_hp[0], cc->nmvcosts_hp[0],
- MV_VALS * sizeof(*cc->nmvcosts_hp[0]));
- memcpy(cpi->nmvcosts_hp[1], cc->nmvcosts_hp[1],
- MV_VALS * sizeof(*cc->nmvcosts_hp[1]));
-
-#if !CONFIG_MISC_FIXES
- vp10_copy(cm->segp.pred_probs, cc->segment_pred_probs);
-#endif
-
- memcpy(cm->last_frame_seg_map,
- cpi->coding_context.last_frame_seg_map_copy,
- (cm->mi_rows * cm->mi_cols));
-
- vp10_copy(cm->lf.last_ref_deltas, cc->last_ref_lf_deltas);
- vp10_copy(cm->lf.last_mode_deltas, cc->last_mode_lf_deltas);
-
- *cm->fc = cc->fc;
-}
-
-static void configure_static_seg_features(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- const RATE_CONTROL *const rc = &cpi->rc;
- struct segmentation *const seg = &cm->seg;
-
- int high_q = (int)(rc->avg_q > 48.0);
- int qi_delta;
-
- // Disable and clear down for KF
- if (cm->frame_type == KEY_FRAME) {
- // Clear down the global segmentation map
- memset(cpi->segmentation_map, 0, cm->mi_rows * cm->mi_cols);
- seg->update_map = 0;
- seg->update_data = 0;
- cpi->static_mb_pct = 0;
-
- // Disable segmentation
- vp10_disable_segmentation(seg);
-
- // Clear down the segment features.
- vp10_clearall_segfeatures(seg);
- } else if (cpi->refresh_alt_ref_frame) {
- // If this is an alt ref frame
- // Clear down the global segmentation map
- memset(cpi->segmentation_map, 0, cm->mi_rows * cm->mi_cols);
- seg->update_map = 0;
- seg->update_data = 0;
- cpi->static_mb_pct = 0;
-
- // Disable segmentation and individual segment features by default
- vp10_disable_segmentation(seg);
- vp10_clearall_segfeatures(seg);
-
- // Scan frames from current to arf frame.
- // This function re-enables segmentation if appropriate.
- vp10_update_mbgraph_stats(cpi);
-
- // If segmentation was enabled set those features needed for the
- // arf itself.
- if (seg->enabled) {
- seg->update_map = 1;
- seg->update_data = 1;
-
- qi_delta = vp10_compute_qdelta(rc, rc->avg_q, rc->avg_q * 0.875,
- cm->bit_depth);
- vp10_set_segdata(seg, 1, SEG_LVL_ALT_Q, qi_delta - 2);
- vp10_set_segdata(seg, 1, SEG_LVL_ALT_LF, -2);
-
- vp10_enable_segfeature(seg, 1, SEG_LVL_ALT_Q);
- vp10_enable_segfeature(seg, 1, SEG_LVL_ALT_LF);
-
- // Where relevant assume segment data is delta data
- seg->abs_delta = SEGMENT_DELTADATA;
- }
- } else if (seg->enabled) {
- // All other frames if segmentation has been enabled
-
- // First normal frame in a valid gf or alt ref group
- if (rc->frames_since_golden == 0) {
- // Set up segment features for normal frames in an arf group
- if (rc->source_alt_ref_active) {
- seg->update_map = 0;
- seg->update_data = 1;
- seg->abs_delta = SEGMENT_DELTADATA;
-
- qi_delta = vp10_compute_qdelta(rc, rc->avg_q, rc->avg_q * 1.125,
- cm->bit_depth);
- vp10_set_segdata(seg, 1, SEG_LVL_ALT_Q, qi_delta + 2);
- vp10_enable_segfeature(seg, 1, SEG_LVL_ALT_Q);
-
- vp10_set_segdata(seg, 1, SEG_LVL_ALT_LF, -2);
- vp10_enable_segfeature(seg, 1, SEG_LVL_ALT_LF);
-
- // Segment coding disabled for compred testing
- if (high_q || (cpi->static_mb_pct == 100)) {
- vp10_set_segdata(seg, 1, SEG_LVL_REF_FRAME, ALTREF_FRAME);
- vp10_enable_segfeature(seg, 1, SEG_LVL_REF_FRAME);
- vp10_enable_segfeature(seg, 1, SEG_LVL_SKIP);
- }
- } else {
- // Disable segmentation and clear down features if alt ref
- // is not active for this group
-
- vp10_disable_segmentation(seg);
-
- memset(cpi->segmentation_map, 0, cm->mi_rows * cm->mi_cols);
-
- seg->update_map = 0;
- seg->update_data = 0;
-
- vp10_clearall_segfeatures(seg);
- }
- } else if (rc->is_src_frame_alt_ref) {
- // Special case where we are coding over the top of a previous
- // alt ref frame.
- // Segment coding disabled for compred testing
-
- // Enable ref frame features for segment 0 as well
- vp10_enable_segfeature(seg, 0, SEG_LVL_REF_FRAME);
- vp10_enable_segfeature(seg, 1, SEG_LVL_REF_FRAME);
-
- // All mbs should use ALTREF_FRAME
- vp10_clear_segdata(seg, 0, SEG_LVL_REF_FRAME);
- vp10_set_segdata(seg, 0, SEG_LVL_REF_FRAME, ALTREF_FRAME);
- vp10_clear_segdata(seg, 1, SEG_LVL_REF_FRAME);
- vp10_set_segdata(seg, 1, SEG_LVL_REF_FRAME, ALTREF_FRAME);
-
- // Skip all MBs if high Q (0,0 mv and skip coeffs)
- if (high_q) {
- vp10_enable_segfeature(seg, 0, SEG_LVL_SKIP);
- vp10_enable_segfeature(seg, 1, SEG_LVL_SKIP);
- }
- // Enable data update
- seg->update_data = 1;
- } else {
- // All other frames.
-
- // No updates.. leave things as they are.
- seg->update_map = 0;
- seg->update_data = 0;
- }
- }
-}
-
-static void update_reference_segmentation_map(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- MODE_INFO **mi_8x8_ptr = cm->mi_grid_visible;
- uint8_t *cache_ptr = cm->last_frame_seg_map;
- int row, col;
-
- for (row = 0; row < cm->mi_rows; row++) {
- MODE_INFO **mi_8x8 = mi_8x8_ptr;
- uint8_t *cache = cache_ptr;
- for (col = 0; col < cm->mi_cols; col++, mi_8x8++, cache++)
- cache[0] = mi_8x8[0]->mbmi.segment_id;
- mi_8x8_ptr += cm->mi_stride;
- cache_ptr += cm->mi_cols;
- }
-}
-
-static void alloc_raw_frame_buffers(VP10_COMP *cpi) {
- VP10_COMMON *cm = &cpi->common;
- const VP10EncoderConfig *oxcf = &cpi->oxcf;
-
- if (!cpi->lookahead)
- cpi->lookahead = vp10_lookahead_init(oxcf->width, oxcf->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- oxcf->lag_in_frames);
- if (!cpi->lookahead)
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate lag buffers");
-
- // TODO(agrange) Check if ARF is enabled and skip allocation if not.
- if (vpx_realloc_frame_buffer(&cpi->alt_ref_buffer,
- oxcf->width, oxcf->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate altref buffer");
-}
-
-static void alloc_util_frame_buffers(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- if (vpx_realloc_frame_buffer(&cpi->last_frame_uf,
- cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate last frame buffer");
-
- if (vpx_realloc_frame_buffer(&cpi->scaled_source,
- cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate scaled source buffer");
-
- if (vpx_realloc_frame_buffer(&cpi->scaled_last_source,
- cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate scaled last source buffer");
-}
-
-
-static int alloc_context_buffers_ext(VP10_COMP *cpi) {
- VP10_COMMON *cm = &cpi->common;
- int mi_size = cm->mi_cols * cm->mi_rows;
-
- cpi->mbmi_ext_base = vpx_calloc(mi_size, sizeof(*cpi->mbmi_ext_base));
- if (!cpi->mbmi_ext_base)
- return 1;
-
- return 0;
-}
-
-void vp10_alloc_compressor_data(VP10_COMP *cpi) {
- VP10_COMMON *cm = &cpi->common;
-
- vp10_alloc_context_buffers(cm, cm->width, cm->height);
-
- alloc_context_buffers_ext(cpi);
-
- vpx_free(cpi->tile_tok[0][0]);
-
- {
- unsigned int tokens = get_token_alloc(cm->mb_rows, cm->mb_cols);
- CHECK_MEM_ERROR(cm, cpi->tile_tok[0][0],
- vpx_calloc(tokens, sizeof(*cpi->tile_tok[0][0])));
- }
-
- vp10_setup_pc_tree(&cpi->common, &cpi->td);
-}
-
-void vp10_new_framerate(VP10_COMP *cpi, double framerate) {
- cpi->framerate = framerate < 0.1 ? 30 : framerate;
- vp10_rc_update_framerate(cpi);
-}
-
-static void set_tile_limits(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
-
- int min_log2_tile_cols, max_log2_tile_cols;
- vp10_get_tile_n_bits(cm->mi_cols, &min_log2_tile_cols, &max_log2_tile_cols);
-
- cm->log2_tile_cols = clamp(cpi->oxcf.tile_columns,
- min_log2_tile_cols, max_log2_tile_cols);
- cm->log2_tile_rows = cpi->oxcf.tile_rows;
-}
-
-static void update_frame_size(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &cpi->td.mb.e_mbd;
-
- vp10_set_mb_mi(cm, cm->width, cm->height);
- vp10_init_context_buffers(cm);
- vp10_init_macroblockd(cm, xd, NULL);
- memset(cpi->mbmi_ext_base, 0,
- cm->mi_rows * cm->mi_cols * sizeof(*cpi->mbmi_ext_base));
-
- set_tile_limits(cpi);
-}
-
-static void init_buffer_indices(VP10_COMP *cpi) {
- cpi->lst_fb_idx = 0;
- cpi->gld_fb_idx = 1;
- cpi->alt_fb_idx = 2;
-}
-
-static void init_config(struct VP10_COMP *cpi, VP10EncoderConfig *oxcf) {
- VP10_COMMON *const cm = &cpi->common;
-
- cpi->oxcf = *oxcf;
- cpi->framerate = oxcf->init_framerate;
-
- cm->profile = oxcf->profile;
- cm->bit_depth = oxcf->bit_depth;
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth = oxcf->use_highbitdepth;
-#endif
- cm->color_space = oxcf->color_space;
- cm->color_range = oxcf->color_range;
-
- cm->width = oxcf->width;
- cm->height = oxcf->height;
- vp10_alloc_compressor_data(cpi);
-
- // Single thread case: use counts in common.
- cpi->td.counts = &cm->counts;
-
- // change includes all joint functionality
- vp10_change_config(cpi, oxcf);
-
- cpi->static_mb_pct = 0;
- cpi->ref_frame_flags = 0;
-
- init_buffer_indices(cpi);
-}
-
-static void set_rc_buffer_sizes(RATE_CONTROL *rc,
- const VP10EncoderConfig *oxcf) {
- const int64_t bandwidth = oxcf->target_bandwidth;
- const int64_t starting = oxcf->starting_buffer_level_ms;
- const int64_t optimal = oxcf->optimal_buffer_level_ms;
- const int64_t maximum = oxcf->maximum_buffer_size_ms;
-
- rc->starting_buffer_level = starting * bandwidth / 1000;
- rc->optimal_buffer_level = (optimal == 0) ? bandwidth / 8
- : optimal * bandwidth / 1000;
- rc->maximum_buffer_size = (maximum == 0) ? bandwidth / 8
- : maximum * bandwidth / 1000;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-#define HIGHBD_BFP(BT, SDF, SDAF, VF, SVF, SVAF, SDX3F, SDX8F, SDX4DF) \
- cpi->fn_ptr[BT].sdf = SDF; \
- cpi->fn_ptr[BT].sdaf = SDAF; \
- cpi->fn_ptr[BT].vf = VF; \
- cpi->fn_ptr[BT].svf = SVF; \
- cpi->fn_ptr[BT].svaf = SVAF; \
- cpi->fn_ptr[BT].sdx3f = SDX3F; \
- cpi->fn_ptr[BT].sdx8f = SDX8F; \
- cpi->fn_ptr[BT].sdx4df = SDX4DF;
-
-#define MAKE_BFP_SAD_WRAPPER(fnname) \
-static unsigned int fnname##_bits8(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride) { \
- return fnname(src_ptr, source_stride, ref_ptr, ref_stride); \
-} \
-static unsigned int fnname##_bits10(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride) { \
- return fnname(src_ptr, source_stride, ref_ptr, ref_stride) >> 2; \
-} \
-static unsigned int fnname##_bits12(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride) { \
- return fnname(src_ptr, source_stride, ref_ptr, ref_stride) >> 4; \
-}
-
-#define MAKE_BFP_SADAVG_WRAPPER(fnname) static unsigned int \
-fnname##_bits8(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- const uint8_t *second_pred) { \
- return fnname(src_ptr, source_stride, ref_ptr, ref_stride, second_pred); \
-} \
-static unsigned int fnname##_bits10(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- const uint8_t *second_pred) { \
- return fnname(src_ptr, source_stride, ref_ptr, ref_stride, \
- second_pred) >> 2; \
-} \
-static unsigned int fnname##_bits12(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- const uint8_t *second_pred) { \
- return fnname(src_ptr, source_stride, ref_ptr, ref_stride, \
- second_pred) >> 4; \
-}
-
-#define MAKE_BFP_SAD3_WRAPPER(fnname) \
-static void fnname##_bits8(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- unsigned int *sad_array) { \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
-} \
-static void fnname##_bits10(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- unsigned int *sad_array) { \
- int i; \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
- for (i = 0; i < 3; i++) \
- sad_array[i] >>= 2; \
-} \
-static void fnname##_bits12(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- unsigned int *sad_array) { \
- int i; \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
- for (i = 0; i < 3; i++) \
- sad_array[i] >>= 4; \
-}
-
-#define MAKE_BFP_SAD8_WRAPPER(fnname) \
-static void fnname##_bits8(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- unsigned int *sad_array) { \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
-} \
-static void fnname##_bits10(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- unsigned int *sad_array) { \
- int i; \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
- for (i = 0; i < 8; i++) \
- sad_array[i] >>= 2; \
-} \
-static void fnname##_bits12(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t *ref_ptr, \
- int ref_stride, \
- unsigned int *sad_array) { \
- int i; \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
- for (i = 0; i < 8; i++) \
- sad_array[i] >>= 4; \
-}
-#define MAKE_BFP_SAD4D_WRAPPER(fnname) \
-static void fnname##_bits8(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t* const ref_ptr[], \
- int ref_stride, \
- unsigned int *sad_array) { \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
-} \
-static void fnname##_bits10(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t* const ref_ptr[], \
- int ref_stride, \
- unsigned int *sad_array) { \
- int i; \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
- for (i = 0; i < 4; i++) \
- sad_array[i] >>= 2; \
-} \
-static void fnname##_bits12(const uint8_t *src_ptr, \
- int source_stride, \
- const uint8_t* const ref_ptr[], \
- int ref_stride, \
- unsigned int *sad_array) { \
- int i; \
- fnname(src_ptr, source_stride, ref_ptr, ref_stride, sad_array); \
- for (i = 0; i < 4; i++) \
- sad_array[i] >>= 4; \
-}
-
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad32x16)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad32x16_avg)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad32x16x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad16x32)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad16x32_avg)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad16x32x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad64x32)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad64x32_avg)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad64x32x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad32x64)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad32x64_avg)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad32x64x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad32x32)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad32x32_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad32x32x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad32x32x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad32x32x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad64x64)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad64x64_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad64x64x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad64x64x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad64x64x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad16x16)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad16x16_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad16x16x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad16x16x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad16x16x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad16x8)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad16x8_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad16x8x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad16x8x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad16x8x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad8x16)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad8x16_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad8x16x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad8x16x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad8x16x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad8x8)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad8x8_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad8x8x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad8x8x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad8x8x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad8x4)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad8x4_avg)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad8x4x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad8x4x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad4x8)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad4x8_avg)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad4x8x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad4x8x4d)
-MAKE_BFP_SAD_WRAPPER(vpx_highbd_sad4x4)
-MAKE_BFP_SADAVG_WRAPPER(vpx_highbd_sad4x4_avg)
-MAKE_BFP_SAD3_WRAPPER(vpx_highbd_sad4x4x3)
-MAKE_BFP_SAD8_WRAPPER(vpx_highbd_sad4x4x8)
-MAKE_BFP_SAD4D_WRAPPER(vpx_highbd_sad4x4x4d)
-
-static void highbd_set_var_fns(VP10_COMP *const cpi) {
- VP10_COMMON *const cm = &cpi->common;
- if (cm->use_highbitdepth) {
- switch (cm->bit_depth) {
- case VPX_BITS_8:
- HIGHBD_BFP(BLOCK_32X16,
- vpx_highbd_sad32x16_bits8,
- vpx_highbd_sad32x16_avg_bits8,
- vpx_highbd_8_variance32x16,
- vpx_highbd_8_sub_pixel_variance32x16,
- vpx_highbd_8_sub_pixel_avg_variance32x16,
- NULL,
- NULL,
- vpx_highbd_sad32x16x4d_bits8)
-
- HIGHBD_BFP(BLOCK_16X32,
- vpx_highbd_sad16x32_bits8,
- vpx_highbd_sad16x32_avg_bits8,
- vpx_highbd_8_variance16x32,
- vpx_highbd_8_sub_pixel_variance16x32,
- vpx_highbd_8_sub_pixel_avg_variance16x32,
- NULL,
- NULL,
- vpx_highbd_sad16x32x4d_bits8)
-
- HIGHBD_BFP(BLOCK_64X32,
- vpx_highbd_sad64x32_bits8,
- vpx_highbd_sad64x32_avg_bits8,
- vpx_highbd_8_variance64x32,
- vpx_highbd_8_sub_pixel_variance64x32,
- vpx_highbd_8_sub_pixel_avg_variance64x32,
- NULL,
- NULL,
- vpx_highbd_sad64x32x4d_bits8)
-
- HIGHBD_BFP(BLOCK_32X64,
- vpx_highbd_sad32x64_bits8,
- vpx_highbd_sad32x64_avg_bits8,
- vpx_highbd_8_variance32x64,
- vpx_highbd_8_sub_pixel_variance32x64,
- vpx_highbd_8_sub_pixel_avg_variance32x64,
- NULL,
- NULL,
- vpx_highbd_sad32x64x4d_bits8)
-
- HIGHBD_BFP(BLOCK_32X32,
- vpx_highbd_sad32x32_bits8,
- vpx_highbd_sad32x32_avg_bits8,
- vpx_highbd_8_variance32x32,
- vpx_highbd_8_sub_pixel_variance32x32,
- vpx_highbd_8_sub_pixel_avg_variance32x32,
- vpx_highbd_sad32x32x3_bits8,
- vpx_highbd_sad32x32x8_bits8,
- vpx_highbd_sad32x32x4d_bits8)
-
- HIGHBD_BFP(BLOCK_64X64,
- vpx_highbd_sad64x64_bits8,
- vpx_highbd_sad64x64_avg_bits8,
- vpx_highbd_8_variance64x64,
- vpx_highbd_8_sub_pixel_variance64x64,
- vpx_highbd_8_sub_pixel_avg_variance64x64,
- vpx_highbd_sad64x64x3_bits8,
- vpx_highbd_sad64x64x8_bits8,
- vpx_highbd_sad64x64x4d_bits8)
-
- HIGHBD_BFP(BLOCK_16X16,
- vpx_highbd_sad16x16_bits8,
- vpx_highbd_sad16x16_avg_bits8,
- vpx_highbd_8_variance16x16,
- vpx_highbd_8_sub_pixel_variance16x16,
- vpx_highbd_8_sub_pixel_avg_variance16x16,
- vpx_highbd_sad16x16x3_bits8,
- vpx_highbd_sad16x16x8_bits8,
- vpx_highbd_sad16x16x4d_bits8)
-
- HIGHBD_BFP(BLOCK_16X8,
- vpx_highbd_sad16x8_bits8,
- vpx_highbd_sad16x8_avg_bits8,
- vpx_highbd_8_variance16x8,
- vpx_highbd_8_sub_pixel_variance16x8,
- vpx_highbd_8_sub_pixel_avg_variance16x8,
- vpx_highbd_sad16x8x3_bits8,
- vpx_highbd_sad16x8x8_bits8,
- vpx_highbd_sad16x8x4d_bits8)
-
- HIGHBD_BFP(BLOCK_8X16,
- vpx_highbd_sad8x16_bits8,
- vpx_highbd_sad8x16_avg_bits8,
- vpx_highbd_8_variance8x16,
- vpx_highbd_8_sub_pixel_variance8x16,
- vpx_highbd_8_sub_pixel_avg_variance8x16,
- vpx_highbd_sad8x16x3_bits8,
- vpx_highbd_sad8x16x8_bits8,
- vpx_highbd_sad8x16x4d_bits8)
-
- HIGHBD_BFP(BLOCK_8X8,
- vpx_highbd_sad8x8_bits8,
- vpx_highbd_sad8x8_avg_bits8,
- vpx_highbd_8_variance8x8,
- vpx_highbd_8_sub_pixel_variance8x8,
- vpx_highbd_8_sub_pixel_avg_variance8x8,
- vpx_highbd_sad8x8x3_bits8,
- vpx_highbd_sad8x8x8_bits8,
- vpx_highbd_sad8x8x4d_bits8)
-
- HIGHBD_BFP(BLOCK_8X4,
- vpx_highbd_sad8x4_bits8,
- vpx_highbd_sad8x4_avg_bits8,
- vpx_highbd_8_variance8x4,
- vpx_highbd_8_sub_pixel_variance8x4,
- vpx_highbd_8_sub_pixel_avg_variance8x4,
- NULL,
- vpx_highbd_sad8x4x8_bits8,
- vpx_highbd_sad8x4x4d_bits8)
-
- HIGHBD_BFP(BLOCK_4X8,
- vpx_highbd_sad4x8_bits8,
- vpx_highbd_sad4x8_avg_bits8,
- vpx_highbd_8_variance4x8,
- vpx_highbd_8_sub_pixel_variance4x8,
- vpx_highbd_8_sub_pixel_avg_variance4x8,
- NULL,
- vpx_highbd_sad4x8x8_bits8,
- vpx_highbd_sad4x8x4d_bits8)
-
- HIGHBD_BFP(BLOCK_4X4,
- vpx_highbd_sad4x4_bits8,
- vpx_highbd_sad4x4_avg_bits8,
- vpx_highbd_8_variance4x4,
- vpx_highbd_8_sub_pixel_variance4x4,
- vpx_highbd_8_sub_pixel_avg_variance4x4,
- vpx_highbd_sad4x4x3_bits8,
- vpx_highbd_sad4x4x8_bits8,
- vpx_highbd_sad4x4x4d_bits8)
- break;
-
- case VPX_BITS_10:
- HIGHBD_BFP(BLOCK_32X16,
- vpx_highbd_sad32x16_bits10,
- vpx_highbd_sad32x16_avg_bits10,
- vpx_highbd_10_variance32x16,
- vpx_highbd_10_sub_pixel_variance32x16,
- vpx_highbd_10_sub_pixel_avg_variance32x16,
- NULL,
- NULL,
- vpx_highbd_sad32x16x4d_bits10)
-
- HIGHBD_BFP(BLOCK_16X32,
- vpx_highbd_sad16x32_bits10,
- vpx_highbd_sad16x32_avg_bits10,
- vpx_highbd_10_variance16x32,
- vpx_highbd_10_sub_pixel_variance16x32,
- vpx_highbd_10_sub_pixel_avg_variance16x32,
- NULL,
- NULL,
- vpx_highbd_sad16x32x4d_bits10)
-
- HIGHBD_BFP(BLOCK_64X32,
- vpx_highbd_sad64x32_bits10,
- vpx_highbd_sad64x32_avg_bits10,
- vpx_highbd_10_variance64x32,
- vpx_highbd_10_sub_pixel_variance64x32,
- vpx_highbd_10_sub_pixel_avg_variance64x32,
- NULL,
- NULL,
- vpx_highbd_sad64x32x4d_bits10)
-
- HIGHBD_BFP(BLOCK_32X64,
- vpx_highbd_sad32x64_bits10,
- vpx_highbd_sad32x64_avg_bits10,
- vpx_highbd_10_variance32x64,
- vpx_highbd_10_sub_pixel_variance32x64,
- vpx_highbd_10_sub_pixel_avg_variance32x64,
- NULL,
- NULL,
- vpx_highbd_sad32x64x4d_bits10)
-
- HIGHBD_BFP(BLOCK_32X32,
- vpx_highbd_sad32x32_bits10,
- vpx_highbd_sad32x32_avg_bits10,
- vpx_highbd_10_variance32x32,
- vpx_highbd_10_sub_pixel_variance32x32,
- vpx_highbd_10_sub_pixel_avg_variance32x32,
- vpx_highbd_sad32x32x3_bits10,
- vpx_highbd_sad32x32x8_bits10,
- vpx_highbd_sad32x32x4d_bits10)
-
- HIGHBD_BFP(BLOCK_64X64,
- vpx_highbd_sad64x64_bits10,
- vpx_highbd_sad64x64_avg_bits10,
- vpx_highbd_10_variance64x64,
- vpx_highbd_10_sub_pixel_variance64x64,
- vpx_highbd_10_sub_pixel_avg_variance64x64,
- vpx_highbd_sad64x64x3_bits10,
- vpx_highbd_sad64x64x8_bits10,
- vpx_highbd_sad64x64x4d_bits10)
-
- HIGHBD_BFP(BLOCK_16X16,
- vpx_highbd_sad16x16_bits10,
- vpx_highbd_sad16x16_avg_bits10,
- vpx_highbd_10_variance16x16,
- vpx_highbd_10_sub_pixel_variance16x16,
- vpx_highbd_10_sub_pixel_avg_variance16x16,
- vpx_highbd_sad16x16x3_bits10,
- vpx_highbd_sad16x16x8_bits10,
- vpx_highbd_sad16x16x4d_bits10)
-
- HIGHBD_BFP(BLOCK_16X8,
- vpx_highbd_sad16x8_bits10,
- vpx_highbd_sad16x8_avg_bits10,
- vpx_highbd_10_variance16x8,
- vpx_highbd_10_sub_pixel_variance16x8,
- vpx_highbd_10_sub_pixel_avg_variance16x8,
- vpx_highbd_sad16x8x3_bits10,
- vpx_highbd_sad16x8x8_bits10,
- vpx_highbd_sad16x8x4d_bits10)
-
- HIGHBD_BFP(BLOCK_8X16,
- vpx_highbd_sad8x16_bits10,
- vpx_highbd_sad8x16_avg_bits10,
- vpx_highbd_10_variance8x16,
- vpx_highbd_10_sub_pixel_variance8x16,
- vpx_highbd_10_sub_pixel_avg_variance8x16,
- vpx_highbd_sad8x16x3_bits10,
- vpx_highbd_sad8x16x8_bits10,
- vpx_highbd_sad8x16x4d_bits10)
-
- HIGHBD_BFP(BLOCK_8X8,
- vpx_highbd_sad8x8_bits10,
- vpx_highbd_sad8x8_avg_bits10,
- vpx_highbd_10_variance8x8,
- vpx_highbd_10_sub_pixel_variance8x8,
- vpx_highbd_10_sub_pixel_avg_variance8x8,
- vpx_highbd_sad8x8x3_bits10,
- vpx_highbd_sad8x8x8_bits10,
- vpx_highbd_sad8x8x4d_bits10)
-
- HIGHBD_BFP(BLOCK_8X4,
- vpx_highbd_sad8x4_bits10,
- vpx_highbd_sad8x4_avg_bits10,
- vpx_highbd_10_variance8x4,
- vpx_highbd_10_sub_pixel_variance8x4,
- vpx_highbd_10_sub_pixel_avg_variance8x4,
- NULL,
- vpx_highbd_sad8x4x8_bits10,
- vpx_highbd_sad8x4x4d_bits10)
-
- HIGHBD_BFP(BLOCK_4X8,
- vpx_highbd_sad4x8_bits10,
- vpx_highbd_sad4x8_avg_bits10,
- vpx_highbd_10_variance4x8,
- vpx_highbd_10_sub_pixel_variance4x8,
- vpx_highbd_10_sub_pixel_avg_variance4x8,
- NULL,
- vpx_highbd_sad4x8x8_bits10,
- vpx_highbd_sad4x8x4d_bits10)
-
- HIGHBD_BFP(BLOCK_4X4,
- vpx_highbd_sad4x4_bits10,
- vpx_highbd_sad4x4_avg_bits10,
- vpx_highbd_10_variance4x4,
- vpx_highbd_10_sub_pixel_variance4x4,
- vpx_highbd_10_sub_pixel_avg_variance4x4,
- vpx_highbd_sad4x4x3_bits10,
- vpx_highbd_sad4x4x8_bits10,
- vpx_highbd_sad4x4x4d_bits10)
- break;
-
- case VPX_BITS_12:
- HIGHBD_BFP(BLOCK_32X16,
- vpx_highbd_sad32x16_bits12,
- vpx_highbd_sad32x16_avg_bits12,
- vpx_highbd_12_variance32x16,
- vpx_highbd_12_sub_pixel_variance32x16,
- vpx_highbd_12_sub_pixel_avg_variance32x16,
- NULL,
- NULL,
- vpx_highbd_sad32x16x4d_bits12)
-
- HIGHBD_BFP(BLOCK_16X32,
- vpx_highbd_sad16x32_bits12,
- vpx_highbd_sad16x32_avg_bits12,
- vpx_highbd_12_variance16x32,
- vpx_highbd_12_sub_pixel_variance16x32,
- vpx_highbd_12_sub_pixel_avg_variance16x32,
- NULL,
- NULL,
- vpx_highbd_sad16x32x4d_bits12)
-
- HIGHBD_BFP(BLOCK_64X32,
- vpx_highbd_sad64x32_bits12,
- vpx_highbd_sad64x32_avg_bits12,
- vpx_highbd_12_variance64x32,
- vpx_highbd_12_sub_pixel_variance64x32,
- vpx_highbd_12_sub_pixel_avg_variance64x32,
- NULL,
- NULL,
- vpx_highbd_sad64x32x4d_bits12)
-
- HIGHBD_BFP(BLOCK_32X64,
- vpx_highbd_sad32x64_bits12,
- vpx_highbd_sad32x64_avg_bits12,
- vpx_highbd_12_variance32x64,
- vpx_highbd_12_sub_pixel_variance32x64,
- vpx_highbd_12_sub_pixel_avg_variance32x64,
- NULL,
- NULL,
- vpx_highbd_sad32x64x4d_bits12)
-
- HIGHBD_BFP(BLOCK_32X32,
- vpx_highbd_sad32x32_bits12,
- vpx_highbd_sad32x32_avg_bits12,
- vpx_highbd_12_variance32x32,
- vpx_highbd_12_sub_pixel_variance32x32,
- vpx_highbd_12_sub_pixel_avg_variance32x32,
- vpx_highbd_sad32x32x3_bits12,
- vpx_highbd_sad32x32x8_bits12,
- vpx_highbd_sad32x32x4d_bits12)
-
- HIGHBD_BFP(BLOCK_64X64,
- vpx_highbd_sad64x64_bits12,
- vpx_highbd_sad64x64_avg_bits12,
- vpx_highbd_12_variance64x64,
- vpx_highbd_12_sub_pixel_variance64x64,
- vpx_highbd_12_sub_pixel_avg_variance64x64,
- vpx_highbd_sad64x64x3_bits12,
- vpx_highbd_sad64x64x8_bits12,
- vpx_highbd_sad64x64x4d_bits12)
-
- HIGHBD_BFP(BLOCK_16X16,
- vpx_highbd_sad16x16_bits12,
- vpx_highbd_sad16x16_avg_bits12,
- vpx_highbd_12_variance16x16,
- vpx_highbd_12_sub_pixel_variance16x16,
- vpx_highbd_12_sub_pixel_avg_variance16x16,
- vpx_highbd_sad16x16x3_bits12,
- vpx_highbd_sad16x16x8_bits12,
- vpx_highbd_sad16x16x4d_bits12)
-
- HIGHBD_BFP(BLOCK_16X8,
- vpx_highbd_sad16x8_bits12,
- vpx_highbd_sad16x8_avg_bits12,
- vpx_highbd_12_variance16x8,
- vpx_highbd_12_sub_pixel_variance16x8,
- vpx_highbd_12_sub_pixel_avg_variance16x8,
- vpx_highbd_sad16x8x3_bits12,
- vpx_highbd_sad16x8x8_bits12,
- vpx_highbd_sad16x8x4d_bits12)
-
- HIGHBD_BFP(BLOCK_8X16,
- vpx_highbd_sad8x16_bits12,
- vpx_highbd_sad8x16_avg_bits12,
- vpx_highbd_12_variance8x16,
- vpx_highbd_12_sub_pixel_variance8x16,
- vpx_highbd_12_sub_pixel_avg_variance8x16,
- vpx_highbd_sad8x16x3_bits12,
- vpx_highbd_sad8x16x8_bits12,
- vpx_highbd_sad8x16x4d_bits12)
-
- HIGHBD_BFP(BLOCK_8X8,
- vpx_highbd_sad8x8_bits12,
- vpx_highbd_sad8x8_avg_bits12,
- vpx_highbd_12_variance8x8,
- vpx_highbd_12_sub_pixel_variance8x8,
- vpx_highbd_12_sub_pixel_avg_variance8x8,
- vpx_highbd_sad8x8x3_bits12,
- vpx_highbd_sad8x8x8_bits12,
- vpx_highbd_sad8x8x4d_bits12)
-
- HIGHBD_BFP(BLOCK_8X4,
- vpx_highbd_sad8x4_bits12,
- vpx_highbd_sad8x4_avg_bits12,
- vpx_highbd_12_variance8x4,
- vpx_highbd_12_sub_pixel_variance8x4,
- vpx_highbd_12_sub_pixel_avg_variance8x4,
- NULL,
- vpx_highbd_sad8x4x8_bits12,
- vpx_highbd_sad8x4x4d_bits12)
-
- HIGHBD_BFP(BLOCK_4X8,
- vpx_highbd_sad4x8_bits12,
- vpx_highbd_sad4x8_avg_bits12,
- vpx_highbd_12_variance4x8,
- vpx_highbd_12_sub_pixel_variance4x8,
- vpx_highbd_12_sub_pixel_avg_variance4x8,
- NULL,
- vpx_highbd_sad4x8x8_bits12,
- vpx_highbd_sad4x8x4d_bits12)
-
- HIGHBD_BFP(BLOCK_4X4,
- vpx_highbd_sad4x4_bits12,
- vpx_highbd_sad4x4_avg_bits12,
- vpx_highbd_12_variance4x4,
- vpx_highbd_12_sub_pixel_variance4x4,
- vpx_highbd_12_sub_pixel_avg_variance4x4,
- vpx_highbd_sad4x4x3_bits12,
- vpx_highbd_sad4x4x8_bits12,
- vpx_highbd_sad4x4x4d_bits12)
- break;
-
- default:
- assert(0 && "cm->bit_depth should be VPX_BITS_8, "
- "VPX_BITS_10 or VPX_BITS_12");
- }
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void realloc_segmentation_maps(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
-
- // Create the encoder segmentation map and set all entries to 0
- vpx_free(cpi->segmentation_map);
- CHECK_MEM_ERROR(cm, cpi->segmentation_map,
- vpx_calloc(cm->mi_rows * cm->mi_cols, 1));
-
- // Create a map used for cyclic background refresh.
- if (cpi->cyclic_refresh)
- vp10_cyclic_refresh_free(cpi->cyclic_refresh);
- CHECK_MEM_ERROR(cm, cpi->cyclic_refresh,
- vp10_cyclic_refresh_alloc(cm->mi_rows, cm->mi_cols));
-
- // Create a map used to mark inactive areas.
- vpx_free(cpi->active_map.map);
- CHECK_MEM_ERROR(cm, cpi->active_map.map,
- vpx_calloc(cm->mi_rows * cm->mi_cols, 1));
-
- // And a place holder structure is the coding context
- // for use if we want to save and restore it
- vpx_free(cpi->coding_context.last_frame_seg_map_copy);
- CHECK_MEM_ERROR(cm, cpi->coding_context.last_frame_seg_map_copy,
- vpx_calloc(cm->mi_rows * cm->mi_cols, 1));
-}
-
-void vp10_change_config(struct VP10_COMP *cpi, const VP10EncoderConfig *oxcf) {
- VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
-
- if (cm->profile != oxcf->profile)
- cm->profile = oxcf->profile;
- cm->bit_depth = oxcf->bit_depth;
- cm->color_space = oxcf->color_space;
- cm->color_range = oxcf->color_range;
-
- if (cm->profile <= PROFILE_1)
- assert(cm->bit_depth == VPX_BITS_8);
- else
- assert(cm->bit_depth > VPX_BITS_8);
-
- cpi->oxcf = *oxcf;
-#if CONFIG_VP9_HIGHBITDEPTH
- cpi->td.mb.e_mbd.bd = (int)cm->bit_depth;
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- rc->baseline_gf_interval = (MIN_GF_INTERVAL + MAX_GF_INTERVAL) / 2;
-
- cpi->refresh_golden_frame = 0;
- cpi->refresh_last_frame = 1;
- cm->refresh_frame_context =
- oxcf->error_resilient_mode ? REFRESH_FRAME_CONTEXT_OFF :
- oxcf->frame_parallel_decoding_mode ? REFRESH_FRAME_CONTEXT_FORWARD
- : REFRESH_FRAME_CONTEXT_BACKWARD;
- cm->reset_frame_context = RESET_FRAME_CONTEXT_NONE;
-
- cm->allow_screen_content_tools = (cpi->oxcf.content == VP9E_CONTENT_SCREEN);
- if (cm->allow_screen_content_tools) {
- MACROBLOCK *x = &cpi->td.mb;
- if (x->palette_buffer == 0) {
- CHECK_MEM_ERROR(cm, x->palette_buffer,
- vpx_memalign(16, sizeof(*x->palette_buffer)));
- }
- }
-
- vp10_reset_segment_features(cm);
- vp10_set_high_precision_mv(cpi, 0);
-
- {
- int i;
-
- for (i = 0; i < MAX_SEGMENTS; i++)
- cpi->segment_encode_breakout[i] = cpi->oxcf.encode_breakout;
- }
- cpi->encode_breakout = cpi->oxcf.encode_breakout;
-
- set_rc_buffer_sizes(rc, &cpi->oxcf);
-
- // Under a configuration change, where maximum_buffer_size may change,
- // keep buffer level clipped to the maximum allowed buffer size.
- rc->bits_off_target = VPXMIN(rc->bits_off_target, rc->maximum_buffer_size);
- rc->buffer_level = VPXMIN(rc->buffer_level, rc->maximum_buffer_size);
-
- // Set up frame rate and related parameters rate control values.
- vp10_new_framerate(cpi, cpi->framerate);
-
- // Set absolute upper and lower quality limits
- rc->worst_quality = cpi->oxcf.worst_allowed_q;
- rc->best_quality = cpi->oxcf.best_allowed_q;
-
- cm->interp_filter = cpi->sf.default_interp_filter;
-
- if (cpi->oxcf.render_width > 0 && cpi->oxcf.render_height > 0) {
- cm->render_width = cpi->oxcf.render_width;
- cm->render_height = cpi->oxcf.render_height;
- } else {
- cm->render_width = cpi->oxcf.width;
- cm->render_height = cpi->oxcf.height;
- }
- cm->width = cpi->oxcf.width;
- cm->height = cpi->oxcf.height;
-
- if (cpi->initial_width) {
- if (cm->width > cpi->initial_width || cm->height > cpi->initial_height) {
- vp10_free_context_buffers(cm);
- vp10_alloc_compressor_data(cpi);
- realloc_segmentation_maps(cpi);
- cpi->initial_width = cpi->initial_height = 0;
- }
- }
- update_frame_size(cpi);
-
- cpi->alt_ref_source = NULL;
- rc->is_src_frame_alt_ref = 0;
-
-#if 0
- // Experimental RD Code
- cpi->frame_distortion = 0;
- cpi->last_frame_distortion = 0;
-#endif
-
- set_tile_limits(cpi);
-
- cpi->ext_refresh_frame_flags_pending = 0;
- cpi->ext_refresh_frame_context_pending = 0;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- highbd_set_var_fns(cpi);
-#endif
-}
-
-#ifndef M_LOG2_E
-#define M_LOG2_E 0.693147180559945309417
-#endif
-#define log2f(x) (log (x) / (float) M_LOG2_E)
-
-static void cal_nmvjointsadcost(int *mvjointsadcost) {
- mvjointsadcost[0] = 600;
- mvjointsadcost[1] = 300;
- mvjointsadcost[2] = 300;
- mvjointsadcost[3] = 300;
-}
-
-static void cal_nmvsadcosts(int *mvsadcost[2]) {
- int i = 1;
-
- mvsadcost[0][0] = 0;
- mvsadcost[1][0] = 0;
-
- do {
- double z = 256 * (2 * (log2f(8 * i) + .6));
- mvsadcost[0][i] = (int)z;
- mvsadcost[1][i] = (int)z;
- mvsadcost[0][-i] = (int)z;
- mvsadcost[1][-i] = (int)z;
- } while (++i <= MV_MAX);
-}
-
-static void cal_nmvsadcosts_hp(int *mvsadcost[2]) {
- int i = 1;
-
- mvsadcost[0][0] = 0;
- mvsadcost[1][0] = 0;
-
- do {
- double z = 256 * (2 * (log2f(8 * i) + .6));
- mvsadcost[0][i] = (int)z;
- mvsadcost[1][i] = (int)z;
- mvsadcost[0][-i] = (int)z;
- mvsadcost[1][-i] = (int)z;
- } while (++i <= MV_MAX);
-}
-
-
-VP10_COMP *vp10_create_compressor(VP10EncoderConfig *oxcf,
- BufferPool *const pool) {
- unsigned int i;
- VP10_COMP *volatile const cpi = vpx_memalign(32, sizeof(VP10_COMP));
- VP10_COMMON *volatile const cm = cpi != NULL ? &cpi->common : NULL;
-
- if (!cm)
- return NULL;
-
- vp10_zero(*cpi);
-
- if (setjmp(cm->error.jmp)) {
- cm->error.setjmp = 0;
- vp10_remove_compressor(cpi);
- return 0;
- }
-
- cm->error.setjmp = 1;
- cm->alloc_mi = vp10_enc_alloc_mi;
- cm->free_mi = vp10_enc_free_mi;
- cm->setup_mi = vp10_enc_setup_mi;
-
- CHECK_MEM_ERROR(cm, cm->fc,
- (FRAME_CONTEXT *)vpx_calloc(1, sizeof(*cm->fc)));
- CHECK_MEM_ERROR(cm, cm->frame_contexts,
- (FRAME_CONTEXT *)vpx_calloc(FRAME_CONTEXTS,
- sizeof(*cm->frame_contexts)));
-
- cpi->resize_state = 0;
- cpi->resize_avg_qp = 0;
- cpi->resize_buffer_underflow = 0;
- cpi->common.buffer_pool = pool;
-
- init_config(cpi, oxcf);
- vp10_rc_init(&cpi->oxcf, oxcf->pass, &cpi->rc);
-
- cm->current_video_frame = 0;
- cpi->partition_search_skippable_frame = 0;
- cpi->tile_data = NULL;
-
- realloc_segmentation_maps(cpi);
-
- CHECK_MEM_ERROR(cm, cpi->nmvcosts[0],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvcosts[0])));
- CHECK_MEM_ERROR(cm, cpi->nmvcosts[1],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvcosts[1])));
- CHECK_MEM_ERROR(cm, cpi->nmvcosts_hp[0],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvcosts_hp[0])));
- CHECK_MEM_ERROR(cm, cpi->nmvcosts_hp[1],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvcosts_hp[1])));
- CHECK_MEM_ERROR(cm, cpi->nmvsadcosts[0],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvsadcosts[0])));
- CHECK_MEM_ERROR(cm, cpi->nmvsadcosts[1],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvsadcosts[1])));
- CHECK_MEM_ERROR(cm, cpi->nmvsadcosts_hp[0],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvsadcosts_hp[0])));
- CHECK_MEM_ERROR(cm, cpi->nmvsadcosts_hp[1],
- vpx_calloc(MV_VALS, sizeof(*cpi->nmvsadcosts_hp[1])));
-
- for (i = 0; i < (sizeof(cpi->mbgraph_stats) /
- sizeof(cpi->mbgraph_stats[0])); i++) {
- CHECK_MEM_ERROR(cm, cpi->mbgraph_stats[i].mb_stats,
- vpx_calloc(cm->MBs *
- sizeof(*cpi->mbgraph_stats[i].mb_stats), 1));
- }
-
-#if CONFIG_FP_MB_STATS
- cpi->use_fp_mb_stats = 0;
- if (cpi->use_fp_mb_stats) {
- // a place holder used to store the first pass mb stats in the first pass
- CHECK_MEM_ERROR(cm, cpi->twopass.frame_mb_stats_buf,
- vpx_calloc(cm->MBs * sizeof(uint8_t), 1));
- } else {
- cpi->twopass.frame_mb_stats_buf = NULL;
- }
-#endif
-
- cpi->refresh_alt_ref_frame = 0;
- cpi->multi_arf_last_grp_enabled = 0;
-
- cpi->b_calculate_psnr = CONFIG_INTERNAL_STATS;
-#if CONFIG_INTERNAL_STATS
- cpi->b_calculate_ssimg = 0;
- cpi->b_calculate_blockiness = 1;
- cpi->b_calculate_consistency = 1;
- cpi->total_inconsistency = 0;
- cpi->psnr.worst = 100.0;
- cpi->worst_ssim = 100.0;
-
- cpi->count = 0;
- cpi->bytes = 0;
-
- if (cpi->b_calculate_psnr) {
- cpi->total_sq_error = 0;
- cpi->total_samples = 0;
-
- cpi->totalp_sq_error = 0;
- cpi->totalp_samples = 0;
-
- cpi->tot_recode_hits = 0;
- cpi->summed_quality = 0;
- cpi->summed_weights = 0;
- cpi->summedp_quality = 0;
- cpi->summedp_weights = 0;
- }
-
- if (cpi->b_calculate_ssimg) {
- cpi->ssimg.worst= 100.0;
- }
- cpi->fastssim.worst = 100.0;
-
- cpi->psnrhvs.worst = 100.0;
-
- if (cpi->b_calculate_blockiness) {
- cpi->total_blockiness = 0;
- cpi->worst_blockiness = 0.0;
- }
-
- if (cpi->b_calculate_consistency) {
- cpi->ssim_vars = vpx_malloc(sizeof(*cpi->ssim_vars) *
- 4 * cpi->common.mi_rows * cpi->common.mi_cols);
- cpi->worst_consistency = 100.0;
- }
-
-#endif
-
- cpi->first_time_stamp_ever = INT64_MAX;
-
- cal_nmvjointsadcost(cpi->td.mb.nmvjointsadcost);
- cpi->td.mb.nmvcost[0] = &cpi->nmvcosts[0][MV_MAX];
- cpi->td.mb.nmvcost[1] = &cpi->nmvcosts[1][MV_MAX];
- cpi->td.mb.nmvsadcost[0] = &cpi->nmvsadcosts[0][MV_MAX];
- cpi->td.mb.nmvsadcost[1] = &cpi->nmvsadcosts[1][MV_MAX];
- cal_nmvsadcosts(cpi->td.mb.nmvsadcost);
-
- cpi->td.mb.nmvcost_hp[0] = &cpi->nmvcosts_hp[0][MV_MAX];
- cpi->td.mb.nmvcost_hp[1] = &cpi->nmvcosts_hp[1][MV_MAX];
- cpi->td.mb.nmvsadcost_hp[0] = &cpi->nmvsadcosts_hp[0][MV_MAX];
- cpi->td.mb.nmvsadcost_hp[1] = &cpi->nmvsadcosts_hp[1][MV_MAX];
- cal_nmvsadcosts_hp(cpi->td.mb.nmvsadcost_hp);
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
-#ifdef OUTPUT_YUV_DENOISED
- yuv_denoised_file = fopen("denoised.yuv", "ab");
-#endif
-#endif
-#ifdef OUTPUT_YUV_SKINMAP
- yuv_skinmap_file = fopen("skinmap.yuv", "ab");
-#endif
-#ifdef OUTPUT_YUV_REC
- yuv_rec_file = fopen("rec.yuv", "wb");
-#endif
-
-#if 0
- framepsnr = fopen("framepsnr.stt", "a");
- kf_list = fopen("kf_list.stt", "w");
-#endif
-
- cpi->allow_encode_breakout = ENCODE_BREAKOUT_ENABLED;
-
- if (oxcf->pass == 1) {
- vp10_init_first_pass(cpi);
- } else if (oxcf->pass == 2) {
- const size_t packet_sz = sizeof(FIRSTPASS_STATS);
- const int packets = (int)(oxcf->two_pass_stats_in.sz / packet_sz);
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- const size_t psz = cpi->common.MBs * sizeof(uint8_t);
- const int ps = (int)(oxcf->firstpass_mb_stats_in.sz / psz);
-
- cpi->twopass.firstpass_mb_stats.mb_stats_start =
- oxcf->firstpass_mb_stats_in.buf;
- cpi->twopass.firstpass_mb_stats.mb_stats_end =
- cpi->twopass.firstpass_mb_stats.mb_stats_start +
- (ps - 1) * cpi->common.MBs * sizeof(uint8_t);
- }
-#endif
-
- cpi->twopass.stats_in_start = oxcf->two_pass_stats_in.buf;
- cpi->twopass.stats_in = cpi->twopass.stats_in_start;
- cpi->twopass.stats_in_end = &cpi->twopass.stats_in[packets - 1];
-
- vp10_init_second_pass(cpi);
- }
-
- vp10_set_speed_features_framesize_independent(cpi);
- vp10_set_speed_features_framesize_dependent(cpi);
-
- // Allocate memory to store variances for a frame.
- CHECK_MEM_ERROR(cm, cpi->source_diff_var,
- vpx_calloc(cm->MBs, sizeof(diff)));
- cpi->source_var_thresh = 0;
- cpi->frames_till_next_var_check = 0;
-
-#define BFP(BT, SDF, SDAF, VF, SVF, SVAF, SDX3F, SDX8F, SDX4DF)\
- cpi->fn_ptr[BT].sdf = SDF; \
- cpi->fn_ptr[BT].sdaf = SDAF; \
- cpi->fn_ptr[BT].vf = VF; \
- cpi->fn_ptr[BT].svf = SVF; \
- cpi->fn_ptr[BT].svaf = SVAF; \
- cpi->fn_ptr[BT].sdx3f = SDX3F; \
- cpi->fn_ptr[BT].sdx8f = SDX8F; \
- cpi->fn_ptr[BT].sdx4df = SDX4DF;
-
- BFP(BLOCK_32X16, vpx_sad32x16, vpx_sad32x16_avg,
- vpx_variance32x16, vpx_sub_pixel_variance32x16,
- vpx_sub_pixel_avg_variance32x16, NULL, NULL, vpx_sad32x16x4d)
-
- BFP(BLOCK_16X32, vpx_sad16x32, vpx_sad16x32_avg,
- vpx_variance16x32, vpx_sub_pixel_variance16x32,
- vpx_sub_pixel_avg_variance16x32, NULL, NULL, vpx_sad16x32x4d)
-
- BFP(BLOCK_64X32, vpx_sad64x32, vpx_sad64x32_avg,
- vpx_variance64x32, vpx_sub_pixel_variance64x32,
- vpx_sub_pixel_avg_variance64x32, NULL, NULL, vpx_sad64x32x4d)
-
- BFP(BLOCK_32X64, vpx_sad32x64, vpx_sad32x64_avg,
- vpx_variance32x64, vpx_sub_pixel_variance32x64,
- vpx_sub_pixel_avg_variance32x64, NULL, NULL, vpx_sad32x64x4d)
-
- BFP(BLOCK_32X32, vpx_sad32x32, vpx_sad32x32_avg,
- vpx_variance32x32, vpx_sub_pixel_variance32x32,
- vpx_sub_pixel_avg_variance32x32, vpx_sad32x32x3, vpx_sad32x32x8,
- vpx_sad32x32x4d)
-
- BFP(BLOCK_64X64, vpx_sad64x64, vpx_sad64x64_avg,
- vpx_variance64x64, vpx_sub_pixel_variance64x64,
- vpx_sub_pixel_avg_variance64x64, vpx_sad64x64x3, vpx_sad64x64x8,
- vpx_sad64x64x4d)
-
- BFP(BLOCK_16X16, vpx_sad16x16, vpx_sad16x16_avg,
- vpx_variance16x16, vpx_sub_pixel_variance16x16,
- vpx_sub_pixel_avg_variance16x16, vpx_sad16x16x3, vpx_sad16x16x8,
- vpx_sad16x16x4d)
-
- BFP(BLOCK_16X8, vpx_sad16x8, vpx_sad16x8_avg,
- vpx_variance16x8, vpx_sub_pixel_variance16x8,
- vpx_sub_pixel_avg_variance16x8,
- vpx_sad16x8x3, vpx_sad16x8x8, vpx_sad16x8x4d)
-
- BFP(BLOCK_8X16, vpx_sad8x16, vpx_sad8x16_avg,
- vpx_variance8x16, vpx_sub_pixel_variance8x16,
- vpx_sub_pixel_avg_variance8x16,
- vpx_sad8x16x3, vpx_sad8x16x8, vpx_sad8x16x4d)
-
- BFP(BLOCK_8X8, vpx_sad8x8, vpx_sad8x8_avg,
- vpx_variance8x8, vpx_sub_pixel_variance8x8,
- vpx_sub_pixel_avg_variance8x8,
- vpx_sad8x8x3, vpx_sad8x8x8, vpx_sad8x8x4d)
-
- BFP(BLOCK_8X4, vpx_sad8x4, vpx_sad8x4_avg,
- vpx_variance8x4, vpx_sub_pixel_variance8x4,
- vpx_sub_pixel_avg_variance8x4, NULL, vpx_sad8x4x8, vpx_sad8x4x4d)
-
- BFP(BLOCK_4X8, vpx_sad4x8, vpx_sad4x8_avg,
- vpx_variance4x8, vpx_sub_pixel_variance4x8,
- vpx_sub_pixel_avg_variance4x8, NULL, vpx_sad4x8x8, vpx_sad4x8x4d)
-
- BFP(BLOCK_4X4, vpx_sad4x4, vpx_sad4x4_avg,
- vpx_variance4x4, vpx_sub_pixel_variance4x4,
- vpx_sub_pixel_avg_variance4x4,
- vpx_sad4x4x3, vpx_sad4x4x8, vpx_sad4x4x4d)
-
-#if CONFIG_VP9_HIGHBITDEPTH
- highbd_set_var_fns(cpi);
-#endif
-
- /* vp10_init_quantizer() is first called here. Add check in
- * vp10_frame_init_quantizer() so that vp10_init_quantizer is only
- * called later when needed. This will avoid unnecessary calls of
- * vp10_init_quantizer() for every frame.
- */
- vp10_init_quantizer(cpi);
-
- vp10_loop_filter_init(cm);
-
- cm->error.setjmp = 0;
-
- return cpi;
-}
-#define SNPRINT(H, T) \
- snprintf((H) + strlen(H), sizeof(H) - strlen(H), (T))
-
-#define SNPRINT2(H, T, V) \
- snprintf((H) + strlen(H), sizeof(H) - strlen(H), (T), (V))
-
-void vp10_remove_compressor(VP10_COMP *cpi) {
- VP10_COMMON *cm;
- unsigned int i;
- int t;
-
- if (!cpi)
- return;
-
- cm = &cpi->common;
- if (cm->current_video_frame > 0) {
-#if CONFIG_INTERNAL_STATS
- vpx_clear_system_state();
-
- if (cpi->oxcf.pass != 1) {
- char headings[512] = {0};
- char results[512] = {0};
- FILE *f = fopen("opsnr.stt", "a");
- double time_encoded = (cpi->last_end_time_stamp_seen
- - cpi->first_time_stamp_ever) / 10000000.000;
- double total_encode_time = (cpi->time_receive_data +
- cpi->time_compress_data) / 1000.000;
- const double dr =
- (double)cpi->bytes * (double) 8 / (double)1000 / time_encoded;
- const double peak = (double)((1 << cpi->oxcf.input_bit_depth) - 1);
-
- if (cpi->b_calculate_psnr) {
- const double total_psnr =
- vpx_sse_to_psnr((double)cpi->total_samples, peak,
- (double)cpi->total_sq_error);
- const double totalp_psnr =
- vpx_sse_to_psnr((double)cpi->totalp_samples, peak,
- (double)cpi->totalp_sq_error);
- const double total_ssim = 100 * pow(cpi->summed_quality /
- cpi->summed_weights, 8.0);
- const double totalp_ssim = 100 * pow(cpi->summedp_quality /
- cpi->summedp_weights, 8.0);
-
- snprintf(headings, sizeof(headings),
- "Bitrate\tAVGPsnr\tGLBPsnr\tAVPsnrP\tGLPsnrP\t"
- "VPXSSIM\tVPSSIMP\tFASTSIM\tPSNRHVS\t"
- "WstPsnr\tWstSsim\tWstFast\tWstHVS");
- snprintf(results, sizeof(results),
- "%7.2f\t%7.3f\t%7.3f\t%7.3f\t%7.3f\t"
- "%7.3f\t%7.3f\t%7.3f\t%7.3f\t"
- "%7.3f\t%7.3f\t%7.3f\t%7.3f",
- dr, cpi->psnr.stat[ALL] / cpi->count, total_psnr,
- cpi->psnrp.stat[ALL] / cpi->count, totalp_psnr,
- total_ssim, totalp_ssim,
- cpi->fastssim.stat[ALL] / cpi->count,
- cpi->psnrhvs.stat[ALL] / cpi->count,
- cpi->psnr.worst, cpi->worst_ssim, cpi->fastssim.worst,
- cpi->psnrhvs.worst);
-
- if (cpi->b_calculate_blockiness) {
- SNPRINT(headings, "\t Block\tWstBlck");
- SNPRINT2(results, "\t%7.3f", cpi->total_blockiness / cpi->count);
- SNPRINT2(results, "\t%7.3f", cpi->worst_blockiness);
- }
-
- if (cpi->b_calculate_consistency) {
- double consistency =
- vpx_sse_to_psnr((double)cpi->totalp_samples, peak,
- (double)cpi->total_inconsistency);
-
- SNPRINT(headings, "\tConsist\tWstCons");
- SNPRINT2(results, "\t%7.3f", consistency);
- SNPRINT2(results, "\t%7.3f", cpi->worst_consistency);
- }
-
- if (cpi->b_calculate_ssimg) {
- SNPRINT(headings, "\t SSIMG\tWtSSIMG");
- SNPRINT2(results, "\t%7.3f", cpi->ssimg.stat[ALL] / cpi->count);
- SNPRINT2(results, "\t%7.3f", cpi->ssimg.worst);
- }
-
- fprintf(f, "%s\t Time\n", headings);
- fprintf(f, "%s\t%8.0f\n", results, total_encode_time);
- }
-
- fclose(f);
- }
-
-#endif
-
-#if 0
- {
- printf("\n_pick_loop_filter_level:%d\n", cpi->time_pick_lpf / 1000);
- printf("\n_frames recive_data encod_mb_row compress_frame Total\n");
- printf("%6d %10ld %10ld %10ld %10ld\n", cpi->common.current_video_frame,
- cpi->time_receive_data / 1000, cpi->time_encode_sb_row / 1000,
- cpi->time_compress_data / 1000,
- (cpi->time_receive_data + cpi->time_compress_data) / 1000);
- }
-#endif
- }
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
- vp10_denoiser_free(&(cpi->denoiser));
-#endif
-
- for (t = 0; t < cpi->num_workers; ++t) {
- VPxWorker *const worker = &cpi->workers[t];
- EncWorkerData *const thread_data = &cpi->tile_thr_data[t];
-
- // Deallocate allocated threads.
- vpx_get_worker_interface()->end(worker);
-
- // Deallocate allocated thread data.
- if (t < cpi->num_workers - 1) {
- if (cpi->common.allow_screen_content_tools)
- vpx_free(thread_data->td->mb.palette_buffer);
- vpx_free(thread_data->td->counts);
- vp10_free_pc_tree(thread_data->td);
- vpx_free(thread_data->td);
- }
- }
- vpx_free(cpi->tile_thr_data);
- vpx_free(cpi->workers);
-
- if (cpi->num_workers > 1)
- vp10_loop_filter_dealloc(&cpi->lf_row_sync);
-
- dealloc_compressor_data(cpi);
-
- for (i = 0; i < sizeof(cpi->mbgraph_stats) /
- sizeof(cpi->mbgraph_stats[0]); ++i) {
- vpx_free(cpi->mbgraph_stats[i].mb_stats);
- }
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- vpx_free(cpi->twopass.frame_mb_stats_buf);
- cpi->twopass.frame_mb_stats_buf = NULL;
- }
-#endif
-
- vp10_remove_common(cm);
- vp10_free_ref_frame_buffers(cm->buffer_pool);
-#if CONFIG_VP9_POSTPROC
- vp10_free_postproc_buffers(cm);
-#endif
- vpx_free(cpi);
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
-#ifdef OUTPUT_YUV_DENOISED
- fclose(yuv_denoised_file);
-#endif
-#endif
-#ifdef OUTPUT_YUV_SKINMAP
- fclose(yuv_skinmap_file);
-#endif
-#ifdef OUTPUT_YUV_REC
- fclose(yuv_rec_file);
-#endif
-
-#if 0
-
- if (keyfile)
- fclose(keyfile);
-
- if (framepsnr)
- fclose(framepsnr);
-
- if (kf_list)
- fclose(kf_list);
-
-#endif
-}
-
-/* TODO(yaowu): The block_variance calls the unoptimized versions of variance()
- * and highbd_8_variance(). It should not.
- */
-static void encoder_variance(const uint8_t *a, int a_stride,
- const uint8_t *b, int b_stride,
- int w, int h, unsigned int *sse, int *sum) {
- int i, j;
-
- *sum = 0;
- *sse = 0;
-
- for (i = 0; i < h; i++) {
- for (j = 0; j < w; j++) {
- const int diff = a[j] - b[j];
- *sum += diff;
- *sse += diff * diff;
- }
-
- a += a_stride;
- b += b_stride;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void encoder_highbd_variance64(const uint8_t *a8, int a_stride,
- const uint8_t *b8, int b_stride,
- int w, int h, uint64_t *sse,
- uint64_t *sum) {
- int i, j;
-
- uint16_t *a = CONVERT_TO_SHORTPTR(a8);
- uint16_t *b = CONVERT_TO_SHORTPTR(b8);
- *sum = 0;
- *sse = 0;
-
- for (i = 0; i < h; i++) {
- for (j = 0; j < w; j++) {
- const int diff = a[j] - b[j];
- *sum += diff;
- *sse += diff * diff;
- }
- a += a_stride;
- b += b_stride;
- }
-}
-
-static void encoder_highbd_8_variance(const uint8_t *a8, int a_stride,
- const uint8_t *b8, int b_stride,
- int w, int h,
- unsigned int *sse, int *sum) {
- uint64_t sse_long = 0;
- uint64_t sum_long = 0;
- encoder_highbd_variance64(a8, a_stride, b8, b_stride, w, h,
- &sse_long, &sum_long);
- *sse = (unsigned int)sse_long;
- *sum = (int)sum_long;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static int64_t get_sse(const uint8_t *a, int a_stride,
- const uint8_t *b, int b_stride,
- int width, int height) {
- const int dw = width % 16;
- const int dh = height % 16;
- int64_t total_sse = 0;
- unsigned int sse = 0;
- int sum = 0;
- int x, y;
-
- if (dw > 0) {
- encoder_variance(&a[width - dw], a_stride, &b[width - dw], b_stride,
- dw, height, &sse, &sum);
- total_sse += sse;
- }
-
- if (dh > 0) {
- encoder_variance(&a[(height - dh) * a_stride], a_stride,
- &b[(height - dh) * b_stride], b_stride,
- width - dw, dh, &sse, &sum);
- total_sse += sse;
- }
-
- for (y = 0; y < height / 16; ++y) {
- const uint8_t *pa = a;
- const uint8_t *pb = b;
- for (x = 0; x < width / 16; ++x) {
- vpx_mse16x16(pa, a_stride, pb, b_stride, &sse);
- total_sse += sse;
-
- pa += 16;
- pb += 16;
- }
-
- a += 16 * a_stride;
- b += 16 * b_stride;
- }
-
- return total_sse;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static int64_t highbd_get_sse_shift(const uint8_t *a8, int a_stride,
- const uint8_t *b8, int b_stride,
- int width, int height,
- unsigned int input_shift) {
- const uint16_t *a = CONVERT_TO_SHORTPTR(a8);
- const uint16_t *b = CONVERT_TO_SHORTPTR(b8);
- int64_t total_sse = 0;
- int x, y;
- for (y = 0; y < height; ++y) {
- for (x = 0; x < width; ++x) {
- int64_t diff;
- diff = (a[x] >> input_shift) - (b[x] >> input_shift);
- total_sse += diff * diff;
- }
- a += a_stride;
- b += b_stride;
- }
- return total_sse;
-}
-
-static int64_t highbd_get_sse(const uint8_t *a, int a_stride,
- const uint8_t *b, int b_stride,
- int width, int height) {
- int64_t total_sse = 0;
- int x, y;
- const int dw = width % 16;
- const int dh = height % 16;
- unsigned int sse = 0;
- int sum = 0;
- if (dw > 0) {
- encoder_highbd_8_variance(&a[width - dw], a_stride,
- &b[width - dw], b_stride,
- dw, height, &sse, &sum);
- total_sse += sse;
- }
- if (dh > 0) {
- encoder_highbd_8_variance(&a[(height - dh) * a_stride], a_stride,
- &b[(height - dh) * b_stride], b_stride,
- width - dw, dh, &sse, &sum);
- total_sse += sse;
- }
- for (y = 0; y < height / 16; ++y) {
- const uint8_t *pa = a;
- const uint8_t *pb = b;
- for (x = 0; x < width / 16; ++x) {
- vpx_highbd_8_mse16x16(pa, a_stride, pb, b_stride, &sse);
- total_sse += sse;
- pa += 16;
- pb += 16;
- }
- a += 16 * a_stride;
- b += 16 * b_stride;
- }
- return total_sse;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-typedef struct {
- double psnr[4]; // total/y/u/v
- uint64_t sse[4]; // total/y/u/v
- uint32_t samples[4]; // total/y/u/v
-} PSNR_STATS;
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void calc_highbd_psnr(const YV12_BUFFER_CONFIG *a,
- const YV12_BUFFER_CONFIG *b,
- PSNR_STATS *psnr,
- unsigned int bit_depth,
- unsigned int in_bit_depth) {
- const int widths[3] =
- {a->y_crop_width, a->uv_crop_width, a->uv_crop_width };
- const int heights[3] =
- {a->y_crop_height, a->uv_crop_height, a->uv_crop_height};
- const uint8_t *a_planes[3] = {a->y_buffer, a->u_buffer, a->v_buffer };
- const int a_strides[3] = {a->y_stride, a->uv_stride, a->uv_stride};
- const uint8_t *b_planes[3] = {b->y_buffer, b->u_buffer, b->v_buffer };
- const int b_strides[3] = {b->y_stride, b->uv_stride, b->uv_stride};
- int i;
- uint64_t total_sse = 0;
- uint32_t total_samples = 0;
- const double peak = (double)((1 << in_bit_depth) - 1);
- const unsigned int input_shift = bit_depth - in_bit_depth;
-
- for (i = 0; i < 3; ++i) {
- const int w = widths[i];
- const int h = heights[i];
- const uint32_t samples = w * h;
- uint64_t sse;
- if (a->flags & YV12_FLAG_HIGHBITDEPTH) {
- if (input_shift) {
- sse = highbd_get_sse_shift(a_planes[i], a_strides[i],
- b_planes[i], b_strides[i], w, h,
- input_shift);
- } else {
- sse = highbd_get_sse(a_planes[i], a_strides[i],
- b_planes[i], b_strides[i], w, h);
- }
- } else {
- sse = get_sse(a_planes[i], a_strides[i],
- b_planes[i], b_strides[i],
- w, h);
- }
- psnr->sse[1 + i] = sse;
- psnr->samples[1 + i] = samples;
- psnr->psnr[1 + i] = vpx_sse_to_psnr(samples, peak, (double)sse);
-
- total_sse += sse;
- total_samples += samples;
- }
-
- psnr->sse[0] = total_sse;
- psnr->samples[0] = total_samples;
- psnr->psnr[0] = vpx_sse_to_psnr((double)total_samples, peak,
- (double)total_sse);
-}
-
-#else // !CONFIG_VP9_HIGHBITDEPTH
-
-static void calc_psnr(const YV12_BUFFER_CONFIG *a, const YV12_BUFFER_CONFIG *b,
- PSNR_STATS *psnr) {
- static const double peak = 255.0;
- const int widths[3] = {
- a->y_crop_width, a->uv_crop_width, a->uv_crop_width};
- const int heights[3] = {
- a->y_crop_height, a->uv_crop_height, a->uv_crop_height};
- const uint8_t *a_planes[3] = {a->y_buffer, a->u_buffer, a->v_buffer};
- const int a_strides[3] = {a->y_stride, a->uv_stride, a->uv_stride};
- const uint8_t *b_planes[3] = {b->y_buffer, b->u_buffer, b->v_buffer};
- const int b_strides[3] = {b->y_stride, b->uv_stride, b->uv_stride};
- int i;
- uint64_t total_sse = 0;
- uint32_t total_samples = 0;
-
- for (i = 0; i < 3; ++i) {
- const int w = widths[i];
- const int h = heights[i];
- const uint32_t samples = w * h;
- const uint64_t sse = get_sse(a_planes[i], a_strides[i],
- b_planes[i], b_strides[i],
- w, h);
- psnr->sse[1 + i] = sse;
- psnr->samples[1 + i] = samples;
- psnr->psnr[1 + i] = vpx_sse_to_psnr(samples, peak, (double)sse);
-
- total_sse += sse;
- total_samples += samples;
- }
-
- psnr->sse[0] = total_sse;
- psnr->samples[0] = total_samples;
- psnr->psnr[0] = vpx_sse_to_psnr((double)total_samples, peak,
- (double)total_sse);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static void generate_psnr_packet(VP10_COMP *cpi) {
- struct vpx_codec_cx_pkt pkt;
- int i;
- PSNR_STATS psnr;
-#if CONFIG_VP9_HIGHBITDEPTH
- calc_highbd_psnr(cpi->Source, cpi->common.frame_to_show, &psnr,
- cpi->td.mb.e_mbd.bd, cpi->oxcf.input_bit_depth);
-#else
- calc_psnr(cpi->Source, cpi->common.frame_to_show, &psnr);
-#endif
-
- for (i = 0; i < 4; ++i) {
- pkt.data.psnr.samples[i] = psnr.samples[i];
- pkt.data.psnr.sse[i] = psnr.sse[i];
- pkt.data.psnr.psnr[i] = psnr.psnr[i];
- }
- pkt.kind = VPX_CODEC_PSNR_PKT;
- vpx_codec_pkt_list_add(cpi->output_pkt_list, &pkt);
-}
-
-int vp10_use_as_reference(VP10_COMP *cpi, int ref_frame_flags) {
- if (ref_frame_flags > 7)
- return -1;
-
- cpi->ref_frame_flags = ref_frame_flags;
- return 0;
-}
-
-void vp10_update_reference(VP10_COMP *cpi, int ref_frame_flags) {
- cpi->ext_refresh_golden_frame = (ref_frame_flags & VP9_GOLD_FLAG) != 0;
- cpi->ext_refresh_alt_ref_frame = (ref_frame_flags & VP9_ALT_FLAG) != 0;
- cpi->ext_refresh_last_frame = (ref_frame_flags & VP9_LAST_FLAG) != 0;
- cpi->ext_refresh_frame_flags_pending = 1;
-}
-
-static YV12_BUFFER_CONFIG *get_vp10_ref_frame_buffer(VP10_COMP *cpi,
- VP9_REFFRAME ref_frame_flag) {
- MV_REFERENCE_FRAME ref_frame = NONE;
- if (ref_frame_flag == VP9_LAST_FLAG)
- ref_frame = LAST_FRAME;
- else if (ref_frame_flag == VP9_GOLD_FLAG)
- ref_frame = GOLDEN_FRAME;
- else if (ref_frame_flag == VP9_ALT_FLAG)
- ref_frame = ALTREF_FRAME;
-
- return ref_frame == NONE ? NULL : get_ref_frame_buffer(cpi, ref_frame);
-}
-
-int vp10_copy_reference_enc(VP10_COMP *cpi, VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd) {
- YV12_BUFFER_CONFIG *cfg = get_vp10_ref_frame_buffer(cpi, ref_frame_flag);
- if (cfg) {
- vp8_yv12_copy_frame(cfg, sd);
- return 0;
- } else {
- return -1;
- }
-}
-
-int vp10_set_reference_enc(VP10_COMP *cpi, VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd) {
- YV12_BUFFER_CONFIG *cfg = get_vp10_ref_frame_buffer(cpi, ref_frame_flag);
- if (cfg) {
- vp8_yv12_copy_frame(sd, cfg);
- return 0;
- } else {
- return -1;
- }
-}
-
-int vp10_update_entropy(VP10_COMP * cpi, int update) {
- cpi->ext_refresh_frame_context = update;
- cpi->ext_refresh_frame_context_pending = 1;
- return 0;
-}
-
-#if defined(OUTPUT_YUV_DENOISED) || defined(OUTPUT_YUV_SKINMAP)
-// The denoiser buffer is allocated as a YUV 440 buffer. This function writes it
-// as YUV 420. We simply use the top-left pixels of the UV buffers, since we do
-// not denoise the UV channels at this time. If ever we implement UV channel
-// denoising we will have to modify this.
-void vp10_write_yuv_frame_420(YV12_BUFFER_CONFIG *s, FILE *f) {
- uint8_t *src = s->y_buffer;
- int h = s->y_height;
-
- do {
- fwrite(src, s->y_width, 1, f);
- src += s->y_stride;
- } while (--h);
-
- src = s->u_buffer;
- h = s->uv_height;
-
- do {
- fwrite(src, s->uv_width, 1, f);
- src += s->uv_stride;
- } while (--h);
-
- src = s->v_buffer;
- h = s->uv_height;
-
- do {
- fwrite(src, s->uv_width, 1, f);
- src += s->uv_stride;
- } while (--h);
-}
-#endif
-
-#ifdef OUTPUT_YUV_REC
-void vp10_write_yuv_rec_frame(VP10_COMMON *cm) {
- YV12_BUFFER_CONFIG *s = cm->frame_to_show;
- uint8_t *src = s->y_buffer;
- int h = cm->height;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (s->flags & YV12_FLAG_HIGHBITDEPTH) {
- uint16_t *src16 = CONVERT_TO_SHORTPTR(s->y_buffer);
-
- do {
- fwrite(src16, s->y_width, 2, yuv_rec_file);
- src16 += s->y_stride;
- } while (--h);
-
- src16 = CONVERT_TO_SHORTPTR(s->u_buffer);
- h = s->uv_height;
-
- do {
- fwrite(src16, s->uv_width, 2, yuv_rec_file);
- src16 += s->uv_stride;
- } while (--h);
-
- src16 = CONVERT_TO_SHORTPTR(s->v_buffer);
- h = s->uv_height;
-
- do {
- fwrite(src16, s->uv_width, 2, yuv_rec_file);
- src16 += s->uv_stride;
- } while (--h);
-
- fflush(yuv_rec_file);
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- do {
- fwrite(src, s->y_width, 1, yuv_rec_file);
- src += s->y_stride;
- } while (--h);
-
- src = s->u_buffer;
- h = s->uv_height;
-
- do {
- fwrite(src, s->uv_width, 1, yuv_rec_file);
- src += s->uv_stride;
- } while (--h);
-
- src = s->v_buffer;
- h = s->uv_height;
-
- do {
- fwrite(src, s->uv_width, 1, yuv_rec_file);
- src += s->uv_stride;
- } while (--h);
-
- fflush(yuv_rec_file);
-}
-#endif
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void scale_and_extend_frame_nonnormative(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst,
- int bd) {
-#else
-static void scale_and_extend_frame_nonnormative(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst) {
-#endif // CONFIG_VP9_HIGHBITDEPTH
- // TODO(dkovalev): replace YV12_BUFFER_CONFIG with vpx_image_t
- int i;
- const uint8_t *const srcs[3] = {src->y_buffer, src->u_buffer, src->v_buffer};
- const int src_strides[3] = {src->y_stride, src->uv_stride, src->uv_stride};
- const int src_widths[3] = {src->y_crop_width, src->uv_crop_width,
- src->uv_crop_width };
- const int src_heights[3] = {src->y_crop_height, src->uv_crop_height,
- src->uv_crop_height};
- uint8_t *const dsts[3] = {dst->y_buffer, dst->u_buffer, dst->v_buffer};
- const int dst_strides[3] = {dst->y_stride, dst->uv_stride, dst->uv_stride};
- const int dst_widths[3] = {dst->y_crop_width, dst->uv_crop_width,
- dst->uv_crop_width};
- const int dst_heights[3] = {dst->y_crop_height, dst->uv_crop_height,
- dst->uv_crop_height};
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (src->flags & YV12_FLAG_HIGHBITDEPTH) {
- vp10_highbd_resize_plane(srcs[i], src_heights[i], src_widths[i],
- src_strides[i], dsts[i], dst_heights[i],
- dst_widths[i], dst_strides[i], bd);
- } else {
- vp10_resize_plane(srcs[i], src_heights[i], src_widths[i], src_strides[i],
- dsts[i], dst_heights[i], dst_widths[i], dst_strides[i]);
- }
-#else
- vp10_resize_plane(srcs[i], src_heights[i], src_widths[i], src_strides[i],
- dsts[i], dst_heights[i], dst_widths[i], dst_strides[i]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
- vpx_extend_frame_borders(dst);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void scale_and_extend_frame(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst, int bd) {
-#else
-static void scale_and_extend_frame(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst) {
-#endif // CONFIG_VP9_HIGHBITDEPTH
- const int src_w = src->y_crop_width;
- const int src_h = src->y_crop_height;
- const int dst_w = dst->y_crop_width;
- const int dst_h = dst->y_crop_height;
- const uint8_t *const srcs[3] = {src->y_buffer, src->u_buffer, src->v_buffer};
- const int src_strides[3] = {src->y_stride, src->uv_stride, src->uv_stride};
- uint8_t *const dsts[3] = {dst->y_buffer, dst->u_buffer, dst->v_buffer};
- const int dst_strides[3] = {dst->y_stride, dst->uv_stride, dst->uv_stride};
- const InterpKernel *const kernel = vp10_filter_kernels[EIGHTTAP];
- int x, y, i;
-
- for (y = 0; y < dst_h; y += 16) {
- for (x = 0; x < dst_w; x += 16) {
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- const int factor = (i == 0 || i == 3 ? 1 : 2);
- const int x_q4 = x * (16 / factor) * src_w / dst_w;
- const int y_q4 = y * (16 / factor) * src_h / dst_h;
- const int src_stride = src_strides[i];
- const int dst_stride = dst_strides[i];
- const uint8_t *src_ptr = srcs[i] + (y / factor) * src_h / dst_h *
- src_stride + (x / factor) * src_w / dst_w;
- uint8_t *dst_ptr = dsts[i] + (y / factor) * dst_stride + (x / factor);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (src->flags & YV12_FLAG_HIGHBITDEPTH) {
- vpx_highbd_convolve8(src_ptr, src_stride, dst_ptr, dst_stride,
- kernel[x_q4 & 0xf], 16 * src_w / dst_w,
- kernel[y_q4 & 0xf], 16 * src_h / dst_h,
- 16 / factor, 16 / factor, bd);
- } else {
- vpx_convolve8(src_ptr, src_stride, dst_ptr, dst_stride,
- kernel[x_q4 & 0xf], 16 * src_w / dst_w,
- kernel[y_q4 & 0xf], 16 * src_h / dst_h,
- 16 / factor, 16 / factor);
- }
-#else
- vpx_convolve8(src_ptr, src_stride, dst_ptr, dst_stride,
- kernel[x_q4 & 0xf], 16 * src_w / dst_w,
- kernel[y_q4 & 0xf], 16 * src_h / dst_h,
- 16 / factor, 16 / factor);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
- }
- }
-
- vpx_extend_frame_borders(dst);
-}
-
-static int scale_down(VP10_COMP *cpi, int q) {
- RATE_CONTROL *const rc = &cpi->rc;
- GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- int scale = 0;
- assert(frame_is_kf_gf_arf(cpi));
-
- if (rc->frame_size_selector == UNSCALED &&
- q >= rc->rf_level_maxq[gf_group->rf_level[gf_group->index]]) {
- const int max_size_thresh = (int)(rate_thresh_mult[SCALE_STEP1]
- * VPXMAX(rc->this_frame_target, rc->avg_frame_bandwidth));
- scale = rc->projected_frame_size > max_size_thresh ? 1 : 0;
- }
- return scale;
-}
-
-// Function to test for conditions that indicate we should loop
-// back and recode a frame.
-static int recode_loop_test(VP10_COMP *cpi,
- int high_limit, int low_limit,
- int q, int maxq, int minq) {
- const RATE_CONTROL *const rc = &cpi->rc;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- const int frame_is_kfgfarf = frame_is_kf_gf_arf(cpi);
- int force_recode = 0;
-
- if ((rc->projected_frame_size >= rc->max_frame_bandwidth) ||
- (cpi->sf.recode_loop == ALLOW_RECODE) ||
- (frame_is_kfgfarf &&
- (cpi->sf.recode_loop == ALLOW_RECODE_KFARFGF))) {
- if (frame_is_kfgfarf &&
- (oxcf->resize_mode == RESIZE_DYNAMIC) &&
- scale_down(cpi, q)) {
- // Code this group at a lower resolution.
- cpi->resize_pending = 1;
- return 1;
- }
-
- // TODO(agrange) high_limit could be greater than the scale-down threshold.
- if ((rc->projected_frame_size > high_limit && q < maxq) ||
- (rc->projected_frame_size < low_limit && q > minq)) {
- force_recode = 1;
- } else if (cpi->oxcf.rc_mode == VPX_CQ) {
- // Deal with frame undershoot and whether or not we are
- // below the automatically set cq level.
- if (q > oxcf->cq_level &&
- rc->projected_frame_size < ((rc->this_frame_target * 7) >> 3)) {
- force_recode = 1;
- }
- }
- }
- return force_recode;
-}
-
-void vp10_update_reference_frames(VP10_COMP *cpi) {
- VP10_COMMON * const cm = &cpi->common;
- BufferPool *const pool = cm->buffer_pool;
-
- // At this point the new frame has been encoded.
- // If any buffer copy / swapping is signaled it should be done here.
- if (cm->frame_type == KEY_FRAME) {
- ref_cnt_fb(pool->frame_bufs,
- &cm->ref_frame_map[cpi->gld_fb_idx], cm->new_fb_idx);
- ref_cnt_fb(pool->frame_bufs,
- &cm->ref_frame_map[cpi->alt_fb_idx], cm->new_fb_idx);
- } else if (vp10_preserve_existing_gf(cpi)) {
- // We have decided to preserve the previously existing golden frame as our
- // new ARF frame. However, in the short term in function
- // vp10_bitstream.c::get_refresh_mask() we left it in the GF slot and, if
- // we're updating the GF with the current decoded frame, we save it to the
- // ARF slot instead.
- // We now have to update the ARF with the current frame and swap gld_fb_idx
- // and alt_fb_idx so that, overall, we've stored the old GF in the new ARF
- // slot and, if we're updating the GF, the current frame becomes the new GF.
- int tmp;
-
- ref_cnt_fb(pool->frame_bufs,
- &cm->ref_frame_map[cpi->alt_fb_idx], cm->new_fb_idx);
-
- tmp = cpi->alt_fb_idx;
- cpi->alt_fb_idx = cpi->gld_fb_idx;
- cpi->gld_fb_idx = tmp;
- } else { /* For non key/golden frames */
- if (cpi->refresh_alt_ref_frame) {
- int arf_idx = cpi->alt_fb_idx;
- if ((cpi->oxcf.pass == 2) && cpi->multi_arf_allowed) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- arf_idx = gf_group->arf_update_idx[gf_group->index];
- }
-
- ref_cnt_fb(pool->frame_bufs,
- &cm->ref_frame_map[arf_idx], cm->new_fb_idx);
- memcpy(cpi->interp_filter_selected[ALTREF_FRAME],
- cpi->interp_filter_selected[0],
- sizeof(cpi->interp_filter_selected[0]));
- }
-
- if (cpi->refresh_golden_frame) {
- ref_cnt_fb(pool->frame_bufs,
- &cm->ref_frame_map[cpi->gld_fb_idx], cm->new_fb_idx);
- if (!cpi->rc.is_src_frame_alt_ref)
- memcpy(cpi->interp_filter_selected[GOLDEN_FRAME],
- cpi->interp_filter_selected[0],
- sizeof(cpi->interp_filter_selected[0]));
- else
- memcpy(cpi->interp_filter_selected[GOLDEN_FRAME],
- cpi->interp_filter_selected[ALTREF_FRAME],
- sizeof(cpi->interp_filter_selected[ALTREF_FRAME]));
- }
- }
-
- if (cpi->refresh_last_frame) {
- ref_cnt_fb(pool->frame_bufs,
- &cm->ref_frame_map[cpi->lst_fb_idx], cm->new_fb_idx);
- if (!cpi->rc.is_src_frame_alt_ref)
- memcpy(cpi->interp_filter_selected[LAST_FRAME],
- cpi->interp_filter_selected[0],
- sizeof(cpi->interp_filter_selected[0]));
- }
-#if CONFIG_VP9_TEMPORAL_DENOISING
- if (cpi->oxcf.noise_sensitivity > 0) {
- vp10_denoiser_update_frame_info(&cpi->denoiser,
- *cpi->Source,
- cpi->common.frame_type,
- cpi->refresh_alt_ref_frame,
- cpi->refresh_golden_frame,
- cpi->refresh_last_frame);
- }
-#endif
-}
-
-static void loopfilter_frame(VP10_COMP *cpi, VP10_COMMON *cm) {
- MACROBLOCKD *xd = &cpi->td.mb.e_mbd;
- struct loopfilter *lf = &cm->lf;
- if (is_lossless_requested(&cpi->oxcf)) {
- lf->filter_level = 0;
- } else {
- struct vpx_usec_timer timer;
-
- vpx_clear_system_state();
-
- vpx_usec_timer_start(&timer);
-
- vp10_pick_filter_level(cpi->Source, cpi, cpi->sf.lpf_pick);
-
- vpx_usec_timer_mark(&timer);
- cpi->time_pick_lpf += vpx_usec_timer_elapsed(&timer);
- }
-
- if (lf->filter_level > 0) {
- if (cpi->num_workers > 1)
- vp10_loop_filter_frame_mt(cm->frame_to_show, cm, xd->plane,
- lf->filter_level, 0, 0,
- cpi->workers, cpi->num_workers,
- &cpi->lf_row_sync);
- else
- vp10_loop_filter_frame(cm->frame_to_show, cm, xd, lf->filter_level, 0, 0);
- }
-
- vpx_extend_frame_inner_borders(cm->frame_to_show);
-}
-
-static INLINE void alloc_frame_mvs(const VP10_COMMON *cm,
- int buffer_idx) {
- RefCntBuffer *const new_fb_ptr = &cm->buffer_pool->frame_bufs[buffer_idx];
- if (new_fb_ptr->mvs == NULL ||
- new_fb_ptr->mi_rows < cm->mi_rows ||
- new_fb_ptr->mi_cols < cm->mi_cols) {
- vpx_free(new_fb_ptr->mvs);
- new_fb_ptr->mvs =
- (MV_REF *)vpx_calloc(cm->mi_rows * cm->mi_cols,
- sizeof(*new_fb_ptr->mvs));
- new_fb_ptr->mi_rows = cm->mi_rows;
- new_fb_ptr->mi_cols = cm->mi_cols;
- }
-}
-
-void vp10_scale_references(VP10_COMP *cpi) {
- VP10_COMMON *cm = &cpi->common;
- MV_REFERENCE_FRAME ref_frame;
- const VP9_REFFRAME ref_mask[3] = {VP9_LAST_FLAG, VP9_GOLD_FLAG, VP9_ALT_FLAG};
-
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ++ref_frame) {
- // Need to convert from VP9_REFFRAME to index into ref_mask (subtract 1).
- if (cpi->ref_frame_flags & ref_mask[ref_frame - 1]) {
- BufferPool *const pool = cm->buffer_pool;
- const YV12_BUFFER_CONFIG *const ref = get_ref_frame_buffer(cpi,
- ref_frame);
-
- if (ref == NULL) {
- cpi->scaled_ref_idx[ref_frame - 1] = INVALID_IDX;
- continue;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (ref->y_crop_width != cm->width || ref->y_crop_height != cm->height) {
- RefCntBuffer *new_fb_ptr = NULL;
- int force_scaling = 0;
- int new_fb = cpi->scaled_ref_idx[ref_frame - 1];
- if (new_fb == INVALID_IDX) {
- new_fb = get_free_fb(cm);
- force_scaling = 1;
- }
- if (new_fb == INVALID_IDX)
- return;
- new_fb_ptr = &pool->frame_bufs[new_fb];
- if (force_scaling ||
- new_fb_ptr->buf.y_crop_width != cm->width ||
- new_fb_ptr->buf.y_crop_height != cm->height) {
- vpx_realloc_frame_buffer(&new_fb_ptr->buf,
- cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
- cm->use_highbitdepth,
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL);
- scale_and_extend_frame(ref, &new_fb_ptr->buf, (int)cm->bit_depth);
- cpi->scaled_ref_idx[ref_frame - 1] = new_fb;
- alloc_frame_mvs(cm, new_fb);
- }
-#else
- if (ref->y_crop_width != cm->width || ref->y_crop_height != cm->height) {
- RefCntBuffer *new_fb_ptr = NULL;
- int force_scaling = 0;
- int new_fb = cpi->scaled_ref_idx[ref_frame - 1];
- if (new_fb == INVALID_IDX) {
- new_fb = get_free_fb(cm);
- force_scaling = 1;
- }
- if (new_fb == INVALID_IDX)
- return;
- new_fb_ptr = &pool->frame_bufs[new_fb];
- if (force_scaling ||
- new_fb_ptr->buf.y_crop_width != cm->width ||
- new_fb_ptr->buf.y_crop_height != cm->height) {
- vpx_realloc_frame_buffer(&new_fb_ptr->buf,
- cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL);
- scale_and_extend_frame(ref, &new_fb_ptr->buf);
- cpi->scaled_ref_idx[ref_frame - 1] = new_fb;
- alloc_frame_mvs(cm, new_fb);
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- } else {
- const int buf_idx = get_ref_frame_buf_idx(cpi, ref_frame);
- RefCntBuffer *const buf = &pool->frame_bufs[buf_idx];
- buf->buf.y_crop_width = ref->y_crop_width;
- buf->buf.y_crop_height = ref->y_crop_height;
- cpi->scaled_ref_idx[ref_frame - 1] = buf_idx;
- ++buf->ref_count;
- }
- } else {
- if (cpi->oxcf.pass != 0)
- cpi->scaled_ref_idx[ref_frame - 1] = INVALID_IDX;
- }
- }
-}
-
-static void release_scaled_references(VP10_COMP *cpi) {
- VP10_COMMON *cm = &cpi->common;
- int i;
- if (cpi->oxcf.pass == 0) {
- // Only release scaled references under certain conditions:
- // if reference will be updated, or if scaled reference has same resolution.
- int refresh[3];
- refresh[0] = (cpi->refresh_last_frame) ? 1 : 0;
- refresh[1] = (cpi->refresh_golden_frame) ? 1 : 0;
- refresh[2] = (cpi->refresh_alt_ref_frame) ? 1 : 0;
- for (i = LAST_FRAME; i <= ALTREF_FRAME; ++i) {
- const int idx = cpi->scaled_ref_idx[i - 1];
- RefCntBuffer *const buf = idx != INVALID_IDX ?
- &cm->buffer_pool->frame_bufs[idx] : NULL;
- const YV12_BUFFER_CONFIG *const ref = get_ref_frame_buffer(cpi, i);
- if (buf != NULL &&
- (refresh[i - 1] ||
- (buf->buf.y_crop_width == ref->y_crop_width &&
- buf->buf.y_crop_height == ref->y_crop_height))) {
- --buf->ref_count;
- cpi->scaled_ref_idx[i -1] = INVALID_IDX;
- }
- }
- } else {
- for (i = 0; i < MAX_REF_FRAMES; ++i) {
- const int idx = cpi->scaled_ref_idx[i];
- RefCntBuffer *const buf = idx != INVALID_IDX ?
- &cm->buffer_pool->frame_bufs[idx] : NULL;
- if (buf != NULL) {
- --buf->ref_count;
- cpi->scaled_ref_idx[i] = INVALID_IDX;
- }
- }
- }
-}
-
-static void full_to_model_count(unsigned int *model_count,
- unsigned int *full_count) {
- int n;
- model_count[ZERO_TOKEN] = full_count[ZERO_TOKEN];
- model_count[ONE_TOKEN] = full_count[ONE_TOKEN];
- model_count[TWO_TOKEN] = full_count[TWO_TOKEN];
- for (n = THREE_TOKEN; n < EOB_TOKEN; ++n)
- model_count[TWO_TOKEN] += full_count[n];
- model_count[EOB_MODEL_TOKEN] = full_count[EOB_TOKEN];
-}
-
-static void full_to_model_counts(vp10_coeff_count_model *model_count,
- vp10_coeff_count *full_count) {
- int i, j, k, l;
-
- for (i = 0; i < PLANE_TYPES; ++i)
- for (j = 0; j < REF_TYPES; ++j)
- for (k = 0; k < COEF_BANDS; ++k)
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l)
- full_to_model_count(model_count[i][j][k][l], full_count[i][j][k][l]);
-}
-
-#if 0 && CONFIG_INTERNAL_STATS
-static void output_frame_level_debug_stats(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- FILE *const f = fopen("tmp.stt", cm->current_video_frame ? "a" : "w");
- int64_t recon_err;
-
- vpx_clear_system_state();
-
- recon_err = vp10_get_y_sse(cpi->Source, get_frame_new_buffer(cm));
-
- if (cpi->twopass.total_left_stats.coded_error != 0.0)
- fprintf(f, "%10u %dx%d %d %d %10d %10d %10d %10d"
- "%10"PRId64" %10"PRId64" %5d %5d %10"PRId64" "
- "%10"PRId64" %10"PRId64" %10d "
- "%7.2lf %7.2lf %7.2lf %7.2lf %7.2lf"
- "%6d %6d %5d %5d %5d "
- "%10"PRId64" %10.3lf"
- "%10lf %8u %10"PRId64" %10d %10d %10d\n",
- cpi->common.current_video_frame,
- cm->width, cm->height,
- cpi->rc.source_alt_ref_pending,
- cpi->rc.source_alt_ref_active,
- cpi->rc.this_frame_target,
- cpi->rc.projected_frame_size,
- cpi->rc.projected_frame_size / cpi->common.MBs,
- (cpi->rc.projected_frame_size - cpi->rc.this_frame_target),
- cpi->rc.vbr_bits_off_target,
- cpi->rc.vbr_bits_off_target_fast,
- cpi->twopass.extend_minq,
- cpi->twopass.extend_minq_fast,
- cpi->rc.total_target_vs_actual,
- (cpi->rc.starting_buffer_level - cpi->rc.bits_off_target),
- cpi->rc.total_actual_bits, cm->base_qindex,
- vp10_convert_qindex_to_q(cm->base_qindex, cm->bit_depth),
- (double)vp10_dc_quant(cm->base_qindex, 0, cm->bit_depth) / 4.0,
- vp10_convert_qindex_to_q(cpi->twopass.active_worst_quality,
- cm->bit_depth),
- cpi->rc.avg_q,
- vp10_convert_qindex_to_q(cpi->oxcf.cq_level, cm->bit_depth),
- cpi->refresh_last_frame, cpi->refresh_golden_frame,
- cpi->refresh_alt_ref_frame, cm->frame_type, cpi->rc.gfu_boost,
- cpi->twopass.bits_left,
- cpi->twopass.total_left_stats.coded_error,
- cpi->twopass.bits_left /
- (1 + cpi->twopass.total_left_stats.coded_error),
- cpi->tot_recode_hits, recon_err, cpi->rc.kf_boost,
- cpi->twopass.kf_zeromotion_pct,
- cpi->twopass.fr_content_type);
-
- fclose(f);
-
- if (0) {
- FILE *const fmodes = fopen("Modes.stt", "a");
- int i;
-
- fprintf(fmodes, "%6d:%1d:%1d:%1d ", cpi->common.current_video_frame,
- cm->frame_type, cpi->refresh_golden_frame,
- cpi->refresh_alt_ref_frame);
-
- for (i = 0; i < MAX_MODES; ++i)
- fprintf(fmodes, "%5d ", cpi->mode_chosen_counts[i]);
-
- fprintf(fmodes, "\n");
-
- fclose(fmodes);
- }
-}
-#endif
-
-static void set_mv_search_params(VP10_COMP *cpi) {
- const VP10_COMMON *const cm = &cpi->common;
- const unsigned int max_mv_def = VPXMIN(cm->width, cm->height);
-
- // Default based on max resolution.
- cpi->mv_step_param = vp10_init_search_range(max_mv_def);
-
- if (cpi->sf.mv.auto_mv_step_size) {
- if (frame_is_intra_only(cm)) {
- // Initialize max_mv_magnitude for use in the first INTER frame
- // after a key/intra-only frame.
- cpi->max_mv_magnitude = max_mv_def;
- } else {
- if (cm->show_frame) {
- // Allow mv_steps to correspond to twice the max mv magnitude found
- // in the previous frame, capped by the default max_mv_magnitude based
- // on resolution.
- cpi->mv_step_param = vp10_init_search_range(
- VPXMIN(max_mv_def, 2 * cpi->max_mv_magnitude));
- }
- cpi->max_mv_magnitude = 0;
- }
- }
-}
-
-static void set_size_independent_vars(VP10_COMP *cpi) {
- vp10_set_speed_features_framesize_independent(cpi);
- vp10_set_rd_speed_thresholds(cpi);
- vp10_set_rd_speed_thresholds_sub8x8(cpi);
- cpi->common.interp_filter = cpi->sf.default_interp_filter;
-}
-
-static void set_size_dependent_vars(VP10_COMP *cpi, int *q,
- int *bottom_index, int *top_index) {
- VP10_COMMON *const cm = &cpi->common;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
-
- // Setup variables that depend on the dimensions of the frame.
- vp10_set_speed_features_framesize_dependent(cpi);
-
- // Decide q and q bounds.
- *q = vp10_rc_pick_q_and_bounds(cpi, bottom_index, top_index);
-
- if (!frame_is_intra_only(cm)) {
- vp10_set_high_precision_mv(cpi, (*q) < HIGH_PRECISION_MV_QTHRESH);
- }
-
- // Configure experimental use of segmentation for enhanced coding of
- // static regions if indicated.
- // Only allowed in the second pass of a two pass encode, as it requires
- // lagged coding, and if the relevant speed feature flag is set.
- if (oxcf->pass == 2 && cpi->sf.static_segmentation)
- configure_static_seg_features(cpi);
-
-#if CONFIG_VP9_POSTPROC
- if (oxcf->noise_sensitivity > 0) {
- int l = 0;
- switch (oxcf->noise_sensitivity) {
- case 1:
- l = 20;
- break;
- case 2:
- l = 40;
- break;
- case 3:
- l = 60;
- break;
- case 4:
- case 5:
- l = 100;
- break;
- case 6:
- l = 150;
- break;
- }
- vp10_denoise(cpi->Source, cpi->Source, l);
- }
-#endif // CONFIG_VP9_POSTPROC
-}
-
-static void init_motion_estimation(VP10_COMP *cpi) {
- int y_stride = cpi->scaled_source.y_stride;
-
- if (cpi->sf.mv.search_method == NSTEP) {
- vp10_init3smotion_compensation(&cpi->ss_cfg, y_stride);
- } else if (cpi->sf.mv.search_method == DIAMOND) {
- vp10_init_dsmotion_compensation(&cpi->ss_cfg, y_stride);
- }
-}
-
-static void set_frame_size(VP10_COMP *cpi) {
- int ref_frame;
- VP10_COMMON *const cm = &cpi->common;
- VP10EncoderConfig *const oxcf = &cpi->oxcf;
- MACROBLOCKD *const xd = &cpi->td.mb.e_mbd;
-
- if (oxcf->pass == 2 &&
- oxcf->rc_mode == VPX_VBR &&
- ((oxcf->resize_mode == RESIZE_FIXED && cm->current_video_frame == 0) ||
- (oxcf->resize_mode == RESIZE_DYNAMIC && cpi->resize_pending))) {
- vp10_calculate_coded_size(
- cpi, &oxcf->scaled_frame_width, &oxcf->scaled_frame_height);
-
- // There has been a change in frame size.
- vp10_set_size_literal(cpi, oxcf->scaled_frame_width,
- oxcf->scaled_frame_height);
- }
-
- if (oxcf->pass == 0 &&
- oxcf->rc_mode == VPX_CBR &&
- oxcf->resize_mode == RESIZE_DYNAMIC) {
- if (cpi->resize_pending == 1) {
- oxcf->scaled_frame_width =
- (cm->width * cpi->resize_scale_num) / cpi->resize_scale_den;
- oxcf->scaled_frame_height =
- (cm->height * cpi->resize_scale_num) /cpi->resize_scale_den;
- } else if (cpi->resize_pending == -1) {
- // Go back up to original size.
- oxcf->scaled_frame_width = oxcf->width;
- oxcf->scaled_frame_height = oxcf->height;
- }
- if (cpi->resize_pending != 0) {
- // There has been a change in frame size.
- vp10_set_size_literal(cpi,
- oxcf->scaled_frame_width,
- oxcf->scaled_frame_height);
-
- // TODO(agrange) Scale cpi->max_mv_magnitude if frame-size has changed.
- set_mv_search_params(cpi);
- }
- }
-
- if (oxcf->pass == 2) {
- vp10_set_target_rate(cpi);
- }
-
- alloc_frame_mvs(cm, cm->new_fb_idx);
-
- // Reset the frame pointers to the current frame size.
- vpx_realloc_frame_buffer(get_frame_new_buffer(cm),
- cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment,
- NULL, NULL, NULL);
-
- alloc_util_frame_buffers(cpi);
- init_motion_estimation(cpi);
-
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ++ref_frame) {
- RefBuffer *const ref_buf = &cm->frame_refs[ref_frame - 1];
- const int buf_idx = get_ref_frame_buf_idx(cpi, ref_frame);
-
- ref_buf->idx = buf_idx;
-
- if (buf_idx != INVALID_IDX) {
- YV12_BUFFER_CONFIG *const buf = &cm->buffer_pool->frame_bufs[buf_idx].buf;
- ref_buf->buf = buf;
-#if CONFIG_VP9_HIGHBITDEPTH
- vp10_setup_scale_factors_for_frame(&ref_buf->sf,
- buf->y_crop_width, buf->y_crop_height,
- cm->width, cm->height,
- (buf->flags & YV12_FLAG_HIGHBITDEPTH) ?
- 1 : 0);
-#else
- vp10_setup_scale_factors_for_frame(&ref_buf->sf,
- buf->y_crop_width, buf->y_crop_height,
- cm->width, cm->height);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- if (vp10_is_scaled(&ref_buf->sf))
- vpx_extend_frame_borders(buf);
- } else {
- ref_buf->buf = NULL;
- }
- }
-
- set_ref_ptrs(cm, xd, LAST_FRAME, LAST_FRAME);
-}
-
-static void encode_without_recode_loop(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- int q = 0, bottom_index = 0, top_index = 0; // Dummy variables.
-
- vpx_clear_system_state();
-
- set_frame_size(cpi);
-
- // For 1 pass CBR under dynamic resize mode: use faster scaling for source.
- // Only for 2x2 scaling for now.
- if (cpi->oxcf.pass == 0 &&
- cpi->oxcf.rc_mode == VPX_CBR &&
- cpi->oxcf.resize_mode == RESIZE_DYNAMIC &&
- cpi->un_scaled_source->y_width == (cm->width << 1) &&
- cpi->un_scaled_source->y_height == (cm->height << 1)) {
- cpi->Source = vp10_scale_if_required_fast(cm,
- cpi->un_scaled_source,
- &cpi->scaled_source);
- if (cpi->unscaled_last_source != NULL)
- cpi->Last_Source = vp10_scale_if_required_fast(cm,
- cpi->unscaled_last_source,
- &cpi->scaled_last_source);
- } else {
- cpi->Source = vp10_scale_if_required(cm, cpi->un_scaled_source,
- &cpi->scaled_source);
- if (cpi->unscaled_last_source != NULL)
- cpi->Last_Source = vp10_scale_if_required(cm, cpi->unscaled_last_source,
- &cpi->scaled_last_source);
- }
-
- if (frame_is_intra_only(cm) == 0) {
- vp10_scale_references(cpi);
- }
-
- set_size_independent_vars(cpi);
- set_size_dependent_vars(cpi, &q, &bottom_index, &top_index);
-
- vp10_set_quantizer(cm, q);
- vp10_set_variance_partition_thresholds(cpi, q);
-
- setup_frame(cpi);
-
- suppress_active_map(cpi);
- // Variance adaptive and in frame q adjustment experiments are mutually
- // exclusive.
- if (cpi->oxcf.aq_mode == VARIANCE_AQ) {
- vp10_vaq_frame_setup(cpi);
- } else if (cpi->oxcf.aq_mode == COMPLEXITY_AQ) {
- vp10_setup_in_frame_q_adj(cpi);
- } else if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ) {
- vp10_cyclic_refresh_setup(cpi);
- }
- apply_active_map(cpi);
-
- // transform / motion compensation build reconstruction frame
- vp10_encode_frame(cpi);
-
- // Update some stats from cyclic refresh, and check if we should not update
- // golden reference, for 1 pass CBR.
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ &&
- cm->frame_type != KEY_FRAME &&
- (cpi->oxcf.pass == 0 && cpi->oxcf.rc_mode == VPX_CBR))
- vp10_cyclic_refresh_check_golden_update(cpi);
-
- // Update the skip mb flag probabilities based on the distribution
- // seen in the last encoder iteration.
- // update_base_skip_probs(cpi);
- vpx_clear_system_state();
-}
-
-static void encode_with_recode_loop(VP10_COMP *cpi,
- size_t *size,
- uint8_t *dest) {
- VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- int bottom_index, top_index;
- int loop_count = 0;
- int loop_at_this_size = 0;
- int loop = 0;
- int overshoot_seen = 0;
- int undershoot_seen = 0;
- int frame_over_shoot_limit;
- int frame_under_shoot_limit;
- int q = 0, q_low = 0, q_high = 0;
-
- set_size_independent_vars(cpi);
-
- do {
- vpx_clear_system_state();
-
- set_frame_size(cpi);
-
- if (loop_count == 0 || cpi->resize_pending != 0) {
- set_size_dependent_vars(cpi, &q, &bottom_index, &top_index);
-
- // TODO(agrange) Scale cpi->max_mv_magnitude if frame-size has changed.
- set_mv_search_params(cpi);
-
- // Reset the loop state for new frame size.
- overshoot_seen = 0;
- undershoot_seen = 0;
-
- // Reconfiguration for change in frame size has concluded.
- cpi->resize_pending = 0;
-
- q_low = bottom_index;
- q_high = top_index;
-
- loop_at_this_size = 0;
- }
-
- // Decide frame size bounds first time through.
- if (loop_count == 0) {
- vp10_rc_compute_frame_size_bounds(cpi, rc->this_frame_target,
- &frame_under_shoot_limit,
- &frame_over_shoot_limit);
- }
-
- cpi->Source = vp10_scale_if_required(cm, cpi->un_scaled_source,
- &cpi->scaled_source);
-
- if (cpi->unscaled_last_source != NULL)
- cpi->Last_Source = vp10_scale_if_required(cm, cpi->unscaled_last_source,
- &cpi->scaled_last_source);
-
- if (frame_is_intra_only(cm) == 0) {
- if (loop_count > 0) {
- release_scaled_references(cpi);
- }
- vp10_scale_references(cpi);
- }
-
- vp10_set_quantizer(cm, q);
-
- if (loop_count == 0)
- setup_frame(cpi);
-
- // Variance adaptive and in frame q adjustment experiments are mutually
- // exclusive.
- if (cpi->oxcf.aq_mode == VARIANCE_AQ) {
- vp10_vaq_frame_setup(cpi);
- } else if (cpi->oxcf.aq_mode == COMPLEXITY_AQ) {
- vp10_setup_in_frame_q_adj(cpi);
- }
-
- // transform / motion compensation build reconstruction frame
- vp10_encode_frame(cpi);
-
- // Update the skip mb flag probabilities based on the distribution
- // seen in the last encoder iteration.
- // update_base_skip_probs(cpi);
-
- vpx_clear_system_state();
-
- // Dummy pack of the bitstream using up to date stats to get an
- // accurate estimate of output frame size to determine if we need
- // to recode.
- if (cpi->sf.recode_loop >= ALLOW_RECODE_KFARFGF) {
- save_coding_context(cpi);
- vp10_pack_bitstream(cpi, dest, size);
-
- rc->projected_frame_size = (int)(*size) << 3;
- restore_coding_context(cpi);
-
- if (frame_over_shoot_limit == 0)
- frame_over_shoot_limit = 1;
- }
-
- if (cpi->oxcf.rc_mode == VPX_Q) {
- loop = 0;
- } else {
- if ((cm->frame_type == KEY_FRAME) &&
- rc->this_key_frame_forced &&
- (rc->projected_frame_size < rc->max_frame_bandwidth)) {
- int last_q = q;
- int64_t kf_err;
-
- int64_t high_err_target = cpi->ambient_err;
- int64_t low_err_target = cpi->ambient_err >> 1;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- kf_err = vp10_highbd_get_y_sse(cpi->Source, get_frame_new_buffer(cm));
- } else {
- kf_err = vp10_get_y_sse(cpi->Source, get_frame_new_buffer(cm));
- }
-#else
- kf_err = vp10_get_y_sse(cpi->Source, get_frame_new_buffer(cm));
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Prevent possible divide by zero error below for perfect KF
- kf_err += !kf_err;
-
- // The key frame is not good enough or we can afford
- // to make it better without undue risk of popping.
- if ((kf_err > high_err_target &&
- rc->projected_frame_size <= frame_over_shoot_limit) ||
- (kf_err > low_err_target &&
- rc->projected_frame_size <= frame_under_shoot_limit)) {
- // Lower q_high
- q_high = q > q_low ? q - 1 : q_low;
-
- // Adjust Q
- q = (int)((q * high_err_target) / kf_err);
- q = VPXMIN(q, (q_high + q_low) >> 1);
- } else if (kf_err < low_err_target &&
- rc->projected_frame_size >= frame_under_shoot_limit) {
- // The key frame is much better than the previous frame
- // Raise q_low
- q_low = q < q_high ? q + 1 : q_high;
-
- // Adjust Q
- q = (int)((q * low_err_target) / kf_err);
- q = VPXMIN(q, (q_high + q_low + 1) >> 1);
- }
-
- // Clamp Q to upper and lower limits:
- q = clamp(q, q_low, q_high);
-
- loop = q != last_q;
- } else if (recode_loop_test(
- cpi, frame_over_shoot_limit, frame_under_shoot_limit,
- q, VPXMAX(q_high, top_index), bottom_index)) {
- // Is the projected frame size out of range and are we allowed
- // to attempt to recode.
- int last_q = q;
- int retries = 0;
-
- if (cpi->resize_pending == 1) {
- // Change in frame size so go back around the recode loop.
- cpi->rc.frame_size_selector =
- SCALE_STEP1 - cpi->rc.frame_size_selector;
- cpi->rc.next_frame_size_selector = cpi->rc.frame_size_selector;
-
-#if CONFIG_INTERNAL_STATS
- ++cpi->tot_recode_hits;
-#endif
- ++loop_count;
- loop = 1;
- continue;
- }
-
- // Frame size out of permitted range:
- // Update correction factor & compute new Q to try...
-
- // Frame is too large
- if (rc->projected_frame_size > rc->this_frame_target) {
- // Special case if the projected size is > the max allowed.
- if (rc->projected_frame_size >= rc->max_frame_bandwidth)
- q_high = rc->worst_quality;
-
- // Raise Qlow as to at least the current value
- q_low = q < q_high ? q + 1 : q_high;
-
- if (undershoot_seen || loop_at_this_size > 1) {
- // Update rate_correction_factor unless
- vp10_rc_update_rate_correction_factors(cpi);
-
- q = (q_high + q_low + 1) / 2;
- } else {
- // Update rate_correction_factor unless
- vp10_rc_update_rate_correction_factors(cpi);
-
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- bottom_index, VPXMAX(q_high, top_index));
-
- while (q < q_low && retries < 10) {
- vp10_rc_update_rate_correction_factors(cpi);
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- bottom_index, VPXMAX(q_high, top_index));
- retries++;
- }
- }
-
- overshoot_seen = 1;
- } else {
- // Frame is too small
- q_high = q > q_low ? q - 1 : q_low;
-
- if (overshoot_seen || loop_at_this_size > 1) {
- vp10_rc_update_rate_correction_factors(cpi);
- q = (q_high + q_low) / 2;
- } else {
- vp10_rc_update_rate_correction_factors(cpi);
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- bottom_index, top_index);
- // Special case reset for qlow for constrained quality.
- // This should only trigger where there is very substantial
- // undershoot on a frame and the auto cq level is above
- // the user passsed in value.
- if (cpi->oxcf.rc_mode == VPX_CQ &&
- q < q_low) {
- q_low = q;
- }
-
- while (q > q_high && retries < 10) {
- vp10_rc_update_rate_correction_factors(cpi);
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- bottom_index, top_index);
- retries++;
- }
- }
-
- undershoot_seen = 1;
- }
-
- // Clamp Q to upper and lower limits:
- q = clamp(q, q_low, q_high);
-
- loop = (q != last_q);
- } else {
- loop = 0;
- }
- }
-
- // Special case for overlay frame.
- if (rc->is_src_frame_alt_ref &&
- rc->projected_frame_size < rc->max_frame_bandwidth)
- loop = 0;
-
- if (loop) {
- ++loop_count;
- ++loop_at_this_size;
-
-#if CONFIG_INTERNAL_STATS
- ++cpi->tot_recode_hits;
-#endif
- }
- } while (loop);
-}
-
-static int get_ref_frame_flags(const VP10_COMP *cpi) {
- const int *const map = cpi->common.ref_frame_map;
- const int gold_is_last = map[cpi->gld_fb_idx] == map[cpi->lst_fb_idx];
- const int alt_is_last = map[cpi->alt_fb_idx] == map[cpi->lst_fb_idx];
- const int gold_is_alt = map[cpi->gld_fb_idx] == map[cpi->alt_fb_idx];
- int flags = VP9_ALT_FLAG | VP9_GOLD_FLAG | VP9_LAST_FLAG;
-
- if (gold_is_last)
- flags &= ~VP9_GOLD_FLAG;
-
- if (cpi->rc.frames_till_gf_update_due == INT_MAX)
- flags &= ~VP9_GOLD_FLAG;
-
- if (alt_is_last)
- flags &= ~VP9_ALT_FLAG;
-
- if (gold_is_alt)
- flags &= ~VP9_ALT_FLAG;
-
- return flags;
-}
-
-static void set_ext_overrides(VP10_COMP *cpi) {
- // Overrides the defaults with the externally supplied values with
- // vp10_update_reference() and vp10_update_entropy() calls
- // Note: The overrides are valid only for the next frame passed
- // to encode_frame_to_data_rate() function
- if (cpi->ext_refresh_frame_context_pending) {
- cpi->common.refresh_frame_context = cpi->ext_refresh_frame_context;
- cpi->ext_refresh_frame_context_pending = 0;
- }
- if (cpi->ext_refresh_frame_flags_pending) {
- cpi->refresh_last_frame = cpi->ext_refresh_last_frame;
- cpi->refresh_golden_frame = cpi->ext_refresh_golden_frame;
- cpi->refresh_alt_ref_frame = cpi->ext_refresh_alt_ref_frame;
- cpi->ext_refresh_frame_flags_pending = 0;
- }
-}
-
-YV12_BUFFER_CONFIG *vp10_scale_if_required_fast(VP10_COMMON *cm,
- YV12_BUFFER_CONFIG *unscaled,
- YV12_BUFFER_CONFIG *scaled) {
- if (cm->mi_cols * MI_SIZE != unscaled->y_width ||
- cm->mi_rows * MI_SIZE != unscaled->y_height) {
- // For 2x2 scaling down.
- vpx_scale_frame(unscaled, scaled, unscaled->y_buffer, 9, 2, 1,
- 2, 1, 0);
- vpx_extend_frame_borders(scaled);
- return scaled;
- } else {
- return unscaled;
- }
-}
-
-YV12_BUFFER_CONFIG *vp10_scale_if_required(VP10_COMMON *cm,
- YV12_BUFFER_CONFIG *unscaled,
- YV12_BUFFER_CONFIG *scaled) {
- if (cm->mi_cols * MI_SIZE != unscaled->y_width ||
- cm->mi_rows * MI_SIZE != unscaled->y_height) {
-#if CONFIG_VP9_HIGHBITDEPTH
- scale_and_extend_frame_nonnormative(unscaled, scaled, (int)cm->bit_depth);
-#else
- scale_and_extend_frame_nonnormative(unscaled, scaled);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- return scaled;
- } else {
- return unscaled;
- }
-}
-
-static void set_arf_sign_bias(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- int arf_sign_bias;
-
- if ((cpi->oxcf.pass == 2) && cpi->multi_arf_allowed) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- arf_sign_bias = cpi->rc.source_alt_ref_active &&
- (!cpi->refresh_alt_ref_frame ||
- (gf_group->rf_level[gf_group->index] == GF_ARF_LOW));
- } else {
- arf_sign_bias =
- (cpi->rc.source_alt_ref_active && !cpi->refresh_alt_ref_frame);
- }
- cm->ref_frame_sign_bias[ALTREF_FRAME] = arf_sign_bias;
-}
-
-static int setup_interp_filter_search_mask(VP10_COMP *cpi) {
- INTERP_FILTER ifilter;
- int ref_total[MAX_REF_FRAMES] = {0};
- MV_REFERENCE_FRAME ref;
- int mask = 0;
- if (cpi->common.last_frame_type == KEY_FRAME ||
- cpi->refresh_alt_ref_frame)
- return mask;
- for (ref = LAST_FRAME; ref <= ALTREF_FRAME; ++ref)
- for (ifilter = EIGHTTAP; ifilter <= EIGHTTAP_SHARP; ++ifilter)
- ref_total[ref] += cpi->interp_filter_selected[ref][ifilter];
-
- for (ifilter = EIGHTTAP; ifilter <= EIGHTTAP_SHARP; ++ifilter) {
- if ((ref_total[LAST_FRAME] &&
- cpi->interp_filter_selected[LAST_FRAME][ifilter] == 0) &&
- (ref_total[GOLDEN_FRAME] == 0 ||
- cpi->interp_filter_selected[GOLDEN_FRAME][ifilter] * 50
- < ref_total[GOLDEN_FRAME]) &&
- (ref_total[ALTREF_FRAME] == 0 ||
- cpi->interp_filter_selected[ALTREF_FRAME][ifilter] * 50
- < ref_total[ALTREF_FRAME]))
- mask |= 1 << ifilter;
- }
- return mask;
-}
-
-static void encode_frame_to_data_rate(VP10_COMP *cpi,
- size_t *size,
- uint8_t *dest,
- unsigned int *frame_flags) {
- VP10_COMMON *const cm = &cpi->common;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- struct segmentation *const seg = &cm->seg;
- TX_SIZE t;
-
- set_ext_overrides(cpi);
- vpx_clear_system_state();
-
- // Set the arf sign bias for this frame.
- set_arf_sign_bias(cpi);
-
- // Set default state for segment based loop filter update flags.
- cm->lf.mode_ref_delta_update = 0;
-
- if (cpi->oxcf.pass == 2 &&
- cpi->sf.adaptive_interp_filter_search)
- cpi->sf.interp_filter_search_mask =
- setup_interp_filter_search_mask(cpi);
-
- // Set various flags etc to special state if it is a key frame.
- if (frame_is_intra_only(cm)) {
- // Reset the loop filter deltas and segmentation map.
- vp10_reset_segment_features(cm);
-
- // If segmentation is enabled force a map update for key frames.
- if (seg->enabled) {
- seg->update_map = 1;
- seg->update_data = 1;
- }
-
- // The alternate reference frame cannot be active for a key frame.
- cpi->rc.source_alt_ref_active = 0;
-
- cm->error_resilient_mode = oxcf->error_resilient_mode;
-
- // By default, encoder assumes decoder can use prev_mi.
- if (cm->error_resilient_mode) {
- cm->reset_frame_context = RESET_FRAME_CONTEXT_NONE;
- cm->refresh_frame_context = REFRESH_FRAME_CONTEXT_OFF;
- } else if (cm->intra_only) {
- // Only reset the current context.
- cm->reset_frame_context = RESET_FRAME_CONTEXT_CURRENT;
- }
- }
-
- // For 1 pass CBR, check if we are dropping this frame.
- // Never drop on key frame.
- if (oxcf->pass == 0 &&
- oxcf->rc_mode == VPX_CBR &&
- cm->frame_type != KEY_FRAME) {
- if (vp10_rc_drop_frame(cpi)) {
- vp10_rc_postencode_update_drop_frame(cpi);
- ++cm->current_video_frame;
- return;
- }
- }
-
- vpx_clear_system_state();
-
-#if CONFIG_INTERNAL_STATS
- memset(cpi->mode_chosen_counts, 0,
- MAX_MODES * sizeof(*cpi->mode_chosen_counts));
-#endif
-
- if (cpi->sf.recode_loop == DISALLOW_RECODE) {
- encode_without_recode_loop(cpi);
- } else {
- encode_with_recode_loop(cpi, size, dest);
- }
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
-#ifdef OUTPUT_YUV_DENOISED
- if (oxcf->noise_sensitivity > 0) {
- vp10_write_yuv_frame_420(&cpi->denoiser.running_avg_y[INTRA_FRAME],
- yuv_denoised_file);
- }
-#endif
-#endif
-#ifdef OUTPUT_YUV_SKINMAP
- if (cpi->common.current_video_frame > 1) {
- vp10_compute_skin_map(cpi, yuv_skinmap_file);
- }
-#endif
-
- // Special case code to reduce pulsing when key frames are forced at a
- // fixed interval. Note the reconstruction error if it is the frame before
- // the force key frame
- if (cpi->rc.next_key_frame_forced && cpi->rc.frames_to_key == 1) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- cpi->ambient_err = vp10_highbd_get_y_sse(cpi->Source,
- get_frame_new_buffer(cm));
- } else {
- cpi->ambient_err = vp10_get_y_sse(cpi->Source, get_frame_new_buffer(cm));
- }
-#else
- cpi->ambient_err = vp10_get_y_sse(cpi->Source, get_frame_new_buffer(cm));
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
-
- // If the encoder forced a KEY_FRAME decision
- if (cm->frame_type == KEY_FRAME)
- cpi->refresh_last_frame = 1;
-
- cm->frame_to_show = get_frame_new_buffer(cm);
- cm->frame_to_show->color_space = cm->color_space;
- cm->frame_to_show->color_range = cm->color_range;
- cm->frame_to_show->render_width = cm->render_width;
- cm->frame_to_show->render_height = cm->render_height;
-
- // Pick the loop filter level for the frame.
- loopfilter_frame(cpi, cm);
-
- // build the bitstream
- vp10_pack_bitstream(cpi, dest, size);
-
- if (cm->seg.update_map)
- update_reference_segmentation_map(cpi);
-
- if (frame_is_intra_only(cm) == 0) {
- release_scaled_references(cpi);
- }
- vp10_update_reference_frames(cpi);
-
- for (t = TX_4X4; t <= TX_32X32; t++)
- full_to_model_counts(cpi->td.counts->coef[t],
- cpi->td.rd_counts.coef_counts[t]);
-
- if (cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD) {
- vp10_adapt_coef_probs(cm);
-#if CONFIG_MISC_FIXES
- vp10_adapt_intra_frame_probs(cm);
-#else
- if (!frame_is_intra_only(cm))
- vp10_adapt_intra_frame_probs(cm);
-#endif
- }
-
- if (!frame_is_intra_only(cm)) {
- if (cm->refresh_frame_context == REFRESH_FRAME_CONTEXT_BACKWARD) {
- vp10_adapt_inter_frame_probs(cm);
- vp10_adapt_mv_probs(cm, cm->allow_high_precision_mv);
- }
- }
-
- if (cpi->refresh_golden_frame == 1)
- cpi->frame_flags |= FRAMEFLAGS_GOLDEN;
- else
- cpi->frame_flags &= ~FRAMEFLAGS_GOLDEN;
-
- if (cpi->refresh_alt_ref_frame == 1)
- cpi->frame_flags |= FRAMEFLAGS_ALTREF;
- else
- cpi->frame_flags &= ~FRAMEFLAGS_ALTREF;
-
- cpi->ref_frame_flags = get_ref_frame_flags(cpi);
-
- cm->last_frame_type = cm->frame_type;
-
- vp10_rc_postencode_update(cpi, *size);
-
-#if 0
- output_frame_level_debug_stats(cpi);
-#endif
-
- if (cm->frame_type == KEY_FRAME) {
- // Tell the caller that the frame was coded as a key frame
- *frame_flags = cpi->frame_flags | FRAMEFLAGS_KEY;
- } else {
- *frame_flags = cpi->frame_flags & ~FRAMEFLAGS_KEY;
- }
-
- // Clear the one shot update flags for segmentation map and mode/ref loop
- // filter deltas.
- cm->seg.update_map = 0;
- cm->seg.update_data = 0;
- cm->lf.mode_ref_delta_update = 0;
-
- // keep track of the last coded dimensions
- cm->last_width = cm->width;
- cm->last_height = cm->height;
-
- // reset to normal state now that we are done.
- if (!cm->show_existing_frame)
- cm->last_show_frame = cm->show_frame;
-
- if (cm->show_frame) {
- vp10_swap_mi_and_prev_mi(cm);
- // Don't increment frame counters if this was an altref buffer
- // update not a real frame
- ++cm->current_video_frame;
- }
- cm->prev_frame = cm->cur_frame;
-}
-
-static void Pass0Encode(VP10_COMP *cpi, size_t *size, uint8_t *dest,
- unsigned int *frame_flags) {
- if (cpi->oxcf.rc_mode == VPX_CBR) {
- vp10_rc_get_one_pass_cbr_params(cpi);
- } else {
- vp10_rc_get_one_pass_vbr_params(cpi);
- }
- encode_frame_to_data_rate(cpi, size, dest, frame_flags);
-}
-
-static void Pass2Encode(VP10_COMP *cpi, size_t *size,
- uint8_t *dest, unsigned int *frame_flags) {
- cpi->allow_encode_breakout = ENCODE_BREAKOUT_ENABLED;
- encode_frame_to_data_rate(cpi, size, dest, frame_flags);
-
- vp10_twopass_postencode_update(cpi);
-}
-
-static void init_ref_frame_bufs(VP10_COMMON *cm) {
- int i;
- BufferPool *const pool = cm->buffer_pool;
- cm->new_fb_idx = INVALID_IDX;
- for (i = 0; i < REF_FRAMES; ++i) {
- cm->ref_frame_map[i] = INVALID_IDX;
- pool->frame_bufs[i].ref_count = 0;
- }
-}
-
-static void check_initial_width(VP10_COMP *cpi,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- int subsampling_x, int subsampling_y) {
- VP10_COMMON *const cm = &cpi->common;
-
- if (!cpi->initial_width ||
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth != use_highbitdepth ||
-#endif
- cm->subsampling_x != subsampling_x ||
- cm->subsampling_y != subsampling_y) {
- cm->subsampling_x = subsampling_x;
- cm->subsampling_y = subsampling_y;
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth = use_highbitdepth;
-#endif
-
- alloc_raw_frame_buffers(cpi);
- init_ref_frame_bufs(cm);
- alloc_util_frame_buffers(cpi);
-
- init_motion_estimation(cpi); // TODO(agrange) This can be removed.
-
- cpi->initial_width = cm->width;
- cpi->initial_height = cm->height;
- cpi->initial_mbs = cm->MBs;
- }
-}
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
-static void setup_denoiser_buffer(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- if (cpi->oxcf.noise_sensitivity > 0 &&
- !cpi->denoiser.frame_buffer_initialized) {
- vp10_denoiser_alloc(&(cpi->denoiser), cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS);
- }
-}
-#endif
-
-int vp10_receive_raw_frame(VP10_COMP *cpi, unsigned int frame_flags,
- YV12_BUFFER_CONFIG *sd, int64_t time_stamp,
- int64_t end_time) {
- VP10_COMMON *cm = &cpi->common;
- struct vpx_usec_timer timer;
- int res = 0;
- const int subsampling_x = sd->subsampling_x;
- const int subsampling_y = sd->subsampling_y;
-#if CONFIG_VP9_HIGHBITDEPTH
- const int use_highbitdepth = sd->flags & YV12_FLAG_HIGHBITDEPTH;
- check_initial_width(cpi, use_highbitdepth, subsampling_x, subsampling_y);
-#else
- check_initial_width(cpi, subsampling_x, subsampling_y);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
- setup_denoiser_buffer(cpi);
-#endif
- vpx_usec_timer_start(&timer);
-
- if (vp10_lookahead_push(cpi->lookahead, sd, time_stamp, end_time,
-#if CONFIG_VP9_HIGHBITDEPTH
- use_highbitdepth,
-#endif // CONFIG_VP9_HIGHBITDEPTH
- frame_flags))
- res = -1;
- vpx_usec_timer_mark(&timer);
- cpi->time_receive_data += vpx_usec_timer_elapsed(&timer);
-
- if ((cm->profile == PROFILE_0 || cm->profile == PROFILE_2) &&
- (subsampling_x != 1 || subsampling_y != 1)) {
- vpx_internal_error(&cm->error, VPX_CODEC_INVALID_PARAM,
- "Non-4:2:0 color format requires profile 1 or 3");
- res = -1;
- }
- if ((cm->profile == PROFILE_1 || cm->profile == PROFILE_3) &&
- (subsampling_x == 1 && subsampling_y == 1)) {
- vpx_internal_error(&cm->error, VPX_CODEC_INVALID_PARAM,
- "4:2:0 color format requires profile 0 or 2");
- res = -1;
- }
-
- return res;
-}
-
-
-static int frame_is_reference(const VP10_COMP *cpi) {
- const VP10_COMMON *cm = &cpi->common;
-
- return cm->frame_type == KEY_FRAME ||
- cpi->refresh_last_frame ||
- cpi->refresh_golden_frame ||
- cpi->refresh_alt_ref_frame ||
- cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_OFF ||
- cm->lf.mode_ref_delta_update ||
- cm->seg.update_map ||
- cm->seg.update_data;
-}
-
-static void adjust_frame_rate(VP10_COMP *cpi,
- const struct lookahead_entry *source) {
- int64_t this_duration;
- int step = 0;
-
- if (source->ts_start == cpi->first_time_stamp_ever) {
- this_duration = source->ts_end - source->ts_start;
- step = 1;
- } else {
- int64_t last_duration = cpi->last_end_time_stamp_seen
- - cpi->last_time_stamp_seen;
-
- this_duration = source->ts_end - cpi->last_end_time_stamp_seen;
-
- // do a step update if the duration changes by 10%
- if (last_duration)
- step = (int)((this_duration - last_duration) * 10 / last_duration);
- }
-
- if (this_duration) {
- if (step) {
- vp10_new_framerate(cpi, 10000000.0 / this_duration);
- } else {
- // Average this frame's rate into the last second's average
- // frame rate. If we haven't seen 1 second yet, then average
- // over the whole interval seen.
- const double interval = VPXMIN(
- (double)(source->ts_end - cpi->first_time_stamp_ever), 10000000.0);
- double avg_duration = 10000000.0 / cpi->framerate;
- avg_duration *= (interval - avg_duration + this_duration);
- avg_duration /= interval;
-
- vp10_new_framerate(cpi, 10000000.0 / avg_duration);
- }
- }
- cpi->last_time_stamp_seen = source->ts_start;
- cpi->last_end_time_stamp_seen = source->ts_end;
-}
-
-// Returns 0 if this is not an alt ref else the offset of the source frame
-// used as the arf midpoint.
-static int get_arf_src_index(VP10_COMP *cpi) {
- RATE_CONTROL *const rc = &cpi->rc;
- int arf_src_index = 0;
- if (is_altref_enabled(cpi)) {
- if (cpi->oxcf.pass == 2) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- if (gf_group->update_type[gf_group->index] == ARF_UPDATE) {
- arf_src_index = gf_group->arf_src_offset[gf_group->index];
- }
- } else if (rc->source_alt_ref_pending) {
- arf_src_index = rc->frames_till_gf_update_due;
- }
- }
- return arf_src_index;
-}
-
-static void check_src_altref(VP10_COMP *cpi,
- const struct lookahead_entry *source) {
- RATE_CONTROL *const rc = &cpi->rc;
-
- if (cpi->oxcf.pass == 2) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- rc->is_src_frame_alt_ref =
- (gf_group->update_type[gf_group->index] == OVERLAY_UPDATE);
- } else {
- rc->is_src_frame_alt_ref = cpi->alt_ref_source &&
- (source == cpi->alt_ref_source);
- }
-
- if (rc->is_src_frame_alt_ref) {
- // Current frame is an ARF overlay frame.
- cpi->alt_ref_source = NULL;
-
- // Don't refresh the last buffer for an ARF overlay frame. It will
- // become the GF so preserve last as an alternative prediction option.
- cpi->refresh_last_frame = 0;
- }
-}
-
-#if CONFIG_INTERNAL_STATS
-extern double vp10_get_blockiness(const unsigned char *img1, int img1_pitch,
- const unsigned char *img2, int img2_pitch,
- int width, int height);
-
-static void adjust_image_stat(double y, double u, double v, double all,
- ImageStat *s) {
- s->stat[Y] += y;
- s->stat[U] += u;
- s->stat[V] += v;
- s->stat[ALL] += all;
- s->worst = VPXMIN(s->worst, all);
-}
-#endif // CONFIG_INTERNAL_STATS
-
-int vp10_get_compressed_data(VP10_COMP *cpi, unsigned int *frame_flags,
- size_t *size, uint8_t *dest,
- int64_t *time_stamp, int64_t *time_end, int flush) {
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- VP10_COMMON *const cm = &cpi->common;
- BufferPool *const pool = cm->buffer_pool;
- RATE_CONTROL *const rc = &cpi->rc;
- struct vpx_usec_timer cmptimer;
- YV12_BUFFER_CONFIG *force_src_buffer = NULL;
- struct lookahead_entry *last_source = NULL;
- struct lookahead_entry *source = NULL;
- int arf_src_index;
- int i;
-
- vpx_usec_timer_start(&cmptimer);
-
- vp10_set_high_precision_mv(cpi, ALTREF_HIGH_PRECISION_MV);
-
- // Is multi-arf enabled.
- // Note that at the moment multi_arf is only configured for 2 pass VBR
- if ((oxcf->pass == 2) && (cpi->oxcf.enable_auto_arf > 1))
- cpi->multi_arf_allowed = 1;
- else
- cpi->multi_arf_allowed = 0;
-
- // Normal defaults
- cm->reset_frame_context = RESET_FRAME_CONTEXT_NONE;
- cm->refresh_frame_context =
- oxcf->error_resilient_mode ? REFRESH_FRAME_CONTEXT_OFF :
- oxcf->frame_parallel_decoding_mode ? REFRESH_FRAME_CONTEXT_FORWARD
- : REFRESH_FRAME_CONTEXT_BACKWARD;
-
- cpi->refresh_last_frame = 1;
- cpi->refresh_golden_frame = 0;
- cpi->refresh_alt_ref_frame = 0;
-
- // Should we encode an arf frame.
- arf_src_index = get_arf_src_index(cpi);
-
- if (arf_src_index) {
- assert(arf_src_index <= rc->frames_to_key);
-
- if ((source = vp10_lookahead_peek(cpi->lookahead, arf_src_index)) != NULL) {
- cpi->alt_ref_source = source;
-
- if (oxcf->arnr_max_frames > 0) {
- // Produce the filtered ARF frame.
- vp10_temporal_filter(cpi, arf_src_index);
- vpx_extend_frame_borders(&cpi->alt_ref_buffer);
- force_src_buffer = &cpi->alt_ref_buffer;
- }
-
- cm->show_frame = 0;
- cm->intra_only = 0;
- cpi->refresh_alt_ref_frame = 1;
- cpi->refresh_golden_frame = 0;
- cpi->refresh_last_frame = 0;
- rc->is_src_frame_alt_ref = 0;
- rc->source_alt_ref_pending = 0;
- } else {
- rc->source_alt_ref_pending = 0;
- }
- }
-
- if (!source) {
- // Get last frame source.
- if (cm->current_video_frame > 0) {
- if ((last_source = vp10_lookahead_peek(cpi->lookahead, -1)) == NULL)
- return -1;
- }
-
- // Read in the source frame.
- source = vp10_lookahead_pop(cpi->lookahead, flush);
-
- if (source != NULL) {
- cm->show_frame = 1;
- cm->intra_only = 0;
-
- // Check to see if the frame should be encoded as an arf overlay.
- check_src_altref(cpi, source);
- }
- }
-
- if (source) {
- cpi->un_scaled_source = cpi->Source = force_src_buffer ? force_src_buffer
- : &source->img;
-
- cpi->unscaled_last_source = last_source != NULL ? &last_source->img : NULL;
-
- *time_stamp = source->ts_start;
- *time_end = source->ts_end;
- *frame_flags = (source->flags & VPX_EFLAG_FORCE_KF) ? FRAMEFLAGS_KEY : 0;
-
- } else {
- *size = 0;
- if (flush && oxcf->pass == 1 && !cpi->twopass.first_pass_done) {
- vp10_end_first_pass(cpi); /* get last stats packet */
- cpi->twopass.first_pass_done = 1;
- }
- return -1;
- }
-
- if (source->ts_start < cpi->first_time_stamp_ever) {
- cpi->first_time_stamp_ever = source->ts_start;
- cpi->last_end_time_stamp_seen = source->ts_start;
- }
-
- // Clear down mmx registers
- vpx_clear_system_state();
-
- // adjust frame rates based on timestamps given
- if (cm->show_frame) {
- adjust_frame_rate(cpi, source);
- }
-
- // Find a free buffer for the new frame, releasing the reference previously
- // held.
- if (cm->new_fb_idx != INVALID_IDX) {
- --pool->frame_bufs[cm->new_fb_idx].ref_count;
- }
- cm->new_fb_idx = get_free_fb(cm);
-
- if (cm->new_fb_idx == INVALID_IDX)
- return -1;
-
- cm->cur_frame = &pool->frame_bufs[cm->new_fb_idx];
-
- if (cpi->multi_arf_allowed) {
- if (cm->frame_type == KEY_FRAME) {
- init_buffer_indices(cpi);
- } else if (oxcf->pass == 2) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- cpi->alt_fb_idx = gf_group->arf_ref_idx[gf_group->index];
- }
- }
-
- // Start with a 0 size frame.
- *size = 0;
-
- cpi->frame_flags = *frame_flags;
-
- if (oxcf->pass == 2) {
- vp10_rc_get_second_pass_params(cpi);
- } else if (oxcf->pass == 1) {
- set_frame_size(cpi);
- }
-
- if (cpi->oxcf.pass != 0 || frame_is_intra_only(cm) == 1) {
- for (i = 0; i < MAX_REF_FRAMES; ++i)
- cpi->scaled_ref_idx[i] = INVALID_IDX;
- }
-
- if (oxcf->pass == 1) {
- cpi->td.mb.e_mbd.lossless[0] = is_lossless_requested(oxcf);
- vp10_first_pass(cpi, source);
- } else if (oxcf->pass == 2) {
- Pass2Encode(cpi, size, dest, frame_flags);
- } else {
- // One pass encode
- Pass0Encode(cpi, size, dest, frame_flags);
- }
-
- if (cm->refresh_frame_context != REFRESH_FRAME_CONTEXT_OFF)
- cm->frame_contexts[cm->frame_context_idx] = *cm->fc;
-
- // No frame encoded, or frame was dropped, release scaled references.
- if ((*size == 0) && (frame_is_intra_only(cm) == 0)) {
- release_scaled_references(cpi);
- }
-
- if (*size > 0) {
- cpi->droppable = !frame_is_reference(cpi);
- }
-
- vpx_usec_timer_mark(&cmptimer);
- cpi->time_compress_data += vpx_usec_timer_elapsed(&cmptimer);
-
- if (cpi->b_calculate_psnr && oxcf->pass != 1 && cm->show_frame)
- generate_psnr_packet(cpi);
-
-#if CONFIG_INTERNAL_STATS
-
- if (oxcf->pass != 1) {
- double samples = 0.0;
- cpi->bytes += (int)(*size);
-
- if (cm->show_frame) {
- cpi->count++;
-
- if (cpi->b_calculate_psnr) {
- YV12_BUFFER_CONFIG *orig = cpi->Source;
- YV12_BUFFER_CONFIG *recon = cpi->common.frame_to_show;
- YV12_BUFFER_CONFIG *pp = &cm->post_proc_buffer;
- PSNR_STATS psnr;
-#if CONFIG_VP9_HIGHBITDEPTH
- calc_highbd_psnr(orig, recon, &psnr, cpi->td.mb.e_mbd.bd,
- cpi->oxcf.input_bit_depth);
-#else
- calc_psnr(orig, recon, &psnr);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- adjust_image_stat(psnr.psnr[1], psnr.psnr[2], psnr.psnr[3],
- psnr.psnr[0], &cpi->psnr);
- cpi->total_sq_error += psnr.sse[0];
- cpi->total_samples += psnr.samples[0];
- samples = psnr.samples[0];
-
- {
- PSNR_STATS psnr2;
- double frame_ssim2 = 0, weight = 0;
-#if CONFIG_VP9_POSTPROC
- if (vpx_alloc_frame_buffer(&cm->post_proc_buffer,
- recon->y_crop_width, recon->y_crop_height,
- cm->subsampling_x, cm->subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- cm->use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS,
- cm->byte_alignment) < 0) {
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to allocate post processing buffer");
- }
-
- vp10_deblock(cm->frame_to_show, &cm->post_proc_buffer,
- cm->lf.filter_level * 10 / 6);
-#endif
- vpx_clear_system_state();
-
-#if CONFIG_VP9_HIGHBITDEPTH
- calc_highbd_psnr(orig, pp, &psnr2, cpi->td.mb.e_mbd.bd,
- cpi->oxcf.input_bit_depth);
-#else
- calc_psnr(orig, pp, &psnr2);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- cpi->totalp_sq_error += psnr2.sse[0];
- cpi->totalp_samples += psnr2.samples[0];
- adjust_image_stat(psnr2.psnr[1], psnr2.psnr[2], psnr2.psnr[3],
- psnr2.psnr[0], &cpi->psnrp);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- frame_ssim2 = vpx_highbd_calc_ssim(orig, recon, &weight,
- (int)cm->bit_depth);
- } else {
- frame_ssim2 = vpx_calc_ssim(orig, recon, &weight);
- }
-#else
- frame_ssim2 = vpx_calc_ssim(orig, recon, &weight);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- cpi->worst_ssim= VPXMIN(cpi->worst_ssim, frame_ssim2);
- cpi->summed_quality += frame_ssim2 * weight;
- cpi->summed_weights += weight;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- frame_ssim2 = vpx_highbd_calc_ssim(
- orig, &cm->post_proc_buffer, &weight, (int)cm->bit_depth);
- } else {
- frame_ssim2 = vpx_calc_ssim(orig, &cm->post_proc_buffer, &weight);
- }
-#else
- frame_ssim2 = vpx_calc_ssim(orig, &cm->post_proc_buffer, &weight);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- cpi->summedp_quality += frame_ssim2 * weight;
- cpi->summedp_weights += weight;
-#if 0
- {
- FILE *f = fopen("q_used.stt", "a");
- fprintf(f, "%5d : Y%f7.3:U%f7.3:V%f7.3:F%f7.3:S%7.3f\n",
- cpi->common.current_video_frame, y2, u2, v2,
- frame_psnr2, frame_ssim2);
- fclose(f);
- }
-#endif
- }
- }
- if (cpi->b_calculate_blockiness) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (!cm->use_highbitdepth)
-#endif
- {
- double frame_blockiness = vp10_get_blockiness(
- cpi->Source->y_buffer, cpi->Source->y_stride,
- cm->frame_to_show->y_buffer, cm->frame_to_show->y_stride,
- cpi->Source->y_width, cpi->Source->y_height);
- cpi->worst_blockiness =
- VPXMAX(cpi->worst_blockiness, frame_blockiness);
- cpi->total_blockiness += frame_blockiness;
- }
- }
-
- if (cpi->b_calculate_consistency) {
-#if CONFIG_VP9_HIGHBITDEPTH
- if (!cm->use_highbitdepth)
-#endif
- {
- double this_inconsistency = vpx_get_ssim_metrics(
- cpi->Source->y_buffer, cpi->Source->y_stride,
- cm->frame_to_show->y_buffer, cm->frame_to_show->y_stride,
- cpi->Source->y_width, cpi->Source->y_height, cpi->ssim_vars,
- &cpi->metrics, 1);
-
- const double peak = (double)((1 << cpi->oxcf.input_bit_depth) - 1);
- double consistency = vpx_sse_to_psnr(samples, peak,
- (double)cpi->total_inconsistency);
- if (consistency > 0.0)
- cpi->worst_consistency =
- VPXMIN(cpi->worst_consistency, consistency);
- cpi->total_inconsistency += this_inconsistency;
- }
- }
-
- if (cpi->b_calculate_ssimg) {
- double y, u, v, frame_all;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- frame_all = vpx_highbd_calc_ssimg(cpi->Source, cm->frame_to_show, &y,
- &u, &v, (int)cm->bit_depth);
- } else {
- frame_all = vpx_calc_ssimg(cpi->Source, cm->frame_to_show, &y, &u,
- &v);
- }
-#else
- frame_all = vpx_calc_ssimg(cpi->Source, cm->frame_to_show, &y, &u, &v);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- adjust_image_stat(y, u, v, frame_all, &cpi->ssimg);
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- if (!cm->use_highbitdepth)
-#endif
- {
- double y, u, v, frame_all;
- frame_all = vpx_calc_fastssim(cpi->Source, cm->frame_to_show, &y, &u,
- &v);
- adjust_image_stat(y, u, v, frame_all, &cpi->fastssim);
- /* TODO(JBB): add 10/12 bit support */
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- if (!cm->use_highbitdepth)
-#endif
- {
- double y, u, v, frame_all;
- frame_all = vpx_psnrhvs(cpi->Source, cm->frame_to_show, &y, &u, &v);
- adjust_image_stat(y, u, v, frame_all, &cpi->psnrhvs);
- }
- }
- }
-#endif
-
- vpx_clear_system_state();
- return 0;
-}
-
-int vp10_get_preview_raw_frame(VP10_COMP *cpi, YV12_BUFFER_CONFIG *dest,
- vp10_ppflags_t *flags) {
- VP10_COMMON *cm = &cpi->common;
-#if !CONFIG_VP9_POSTPROC
- (void)flags;
-#endif
-
- if (!cm->show_frame) {
- return -1;
- } else {
- int ret;
-#if CONFIG_VP9_POSTPROC
- ret = vp10_post_proc_frame(cm, dest, flags);
-#else
- if (cm->frame_to_show) {
- *dest = *cm->frame_to_show;
- dest->y_width = cm->width;
- dest->y_height = cm->height;
- dest->uv_width = cm->width >> cm->subsampling_x;
- dest->uv_height = cm->height >> cm->subsampling_y;
- ret = 0;
- } else {
- ret = -1;
- }
-#endif // !CONFIG_VP9_POSTPROC
- vpx_clear_system_state();
- return ret;
- }
-}
-
-int vp10_set_internal_size(VP10_COMP *cpi,
- VPX_SCALING horiz_mode, VPX_SCALING vert_mode) {
- VP10_COMMON *cm = &cpi->common;
- int hr = 0, hs = 0, vr = 0, vs = 0;
-
- if (horiz_mode > ONETWO || vert_mode > ONETWO)
- return -1;
-
- Scale2Ratio(horiz_mode, &hr, &hs);
- Scale2Ratio(vert_mode, &vr, &vs);
-
- // always go to the next whole number
- cm->width = (hs - 1 + cpi->oxcf.width * hr) / hs;
- cm->height = (vs - 1 + cpi->oxcf.height * vr) / vs;
- assert(cm->width <= cpi->initial_width);
- assert(cm->height <= cpi->initial_height);
-
- update_frame_size(cpi);
-
- return 0;
-}
-
-int vp10_set_size_literal(VP10_COMP *cpi, unsigned int width,
- unsigned int height) {
- VP10_COMMON *cm = &cpi->common;
-#if CONFIG_VP9_HIGHBITDEPTH
- check_initial_width(cpi, cm->use_highbitdepth, 1, 1);
-#else
- check_initial_width(cpi, 1, 1);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
- setup_denoiser_buffer(cpi);
-#endif
-
- if (width) {
- cm->width = width;
- if (cm->width > cpi->initial_width) {
- cm->width = cpi->initial_width;
- printf("Warning: Desired width too large, changed to %d\n", cm->width);
- }
- }
-
- if (height) {
- cm->height = height;
- if (cm->height > cpi->initial_height) {
- cm->height = cpi->initial_height;
- printf("Warning: Desired height too large, changed to %d\n", cm->height);
- }
- }
- assert(cm->width <= cpi->initial_width);
- assert(cm->height <= cpi->initial_height);
-
- update_frame_size(cpi);
-
- return 0;
-}
-
-int64_t vp10_get_y_sse(const YV12_BUFFER_CONFIG *a,
- const YV12_BUFFER_CONFIG *b) {
- assert(a->y_crop_width == b->y_crop_width);
- assert(a->y_crop_height == b->y_crop_height);
-
- return get_sse(a->y_buffer, a->y_stride, b->y_buffer, b->y_stride,
- a->y_crop_width, a->y_crop_height);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-int64_t vp10_highbd_get_y_sse(const YV12_BUFFER_CONFIG *a,
- const YV12_BUFFER_CONFIG *b) {
- assert(a->y_crop_width == b->y_crop_width);
- assert(a->y_crop_height == b->y_crop_height);
- assert((a->flags & YV12_FLAG_HIGHBITDEPTH) != 0);
- assert((b->flags & YV12_FLAG_HIGHBITDEPTH) != 0);
-
- return highbd_get_sse(a->y_buffer, a->y_stride, b->y_buffer, b->y_stride,
- a->y_crop_width, a->y_crop_height);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-int vp10_get_quantizer(VP10_COMP *cpi) {
- return cpi->common.base_qindex;
-}
-
-void vp10_apply_encoding_flags(VP10_COMP *cpi, vpx_enc_frame_flags_t flags) {
- if (flags & (VP8_EFLAG_NO_REF_LAST | VP8_EFLAG_NO_REF_GF |
- VP8_EFLAG_NO_REF_ARF)) {
- int ref = 7;
-
- if (flags & VP8_EFLAG_NO_REF_LAST)
- ref ^= VP9_LAST_FLAG;
-
- if (flags & VP8_EFLAG_NO_REF_GF)
- ref ^= VP9_GOLD_FLAG;
-
- if (flags & VP8_EFLAG_NO_REF_ARF)
- ref ^= VP9_ALT_FLAG;
-
- vp10_use_as_reference(cpi, ref);
- }
-
- if (flags & (VP8_EFLAG_NO_UPD_LAST | VP8_EFLAG_NO_UPD_GF |
- VP8_EFLAG_NO_UPD_ARF | VP8_EFLAG_FORCE_GF |
- VP8_EFLAG_FORCE_ARF)) {
- int upd = 7;
-
- if (flags & VP8_EFLAG_NO_UPD_LAST)
- upd ^= VP9_LAST_FLAG;
-
- if (flags & VP8_EFLAG_NO_UPD_GF)
- upd ^= VP9_GOLD_FLAG;
-
- if (flags & VP8_EFLAG_NO_UPD_ARF)
- upd ^= VP9_ALT_FLAG;
-
- vp10_update_reference(cpi, upd);
- }
-
- if (flags & VP8_EFLAG_NO_UPD_ENTROPY) {
- vp10_update_entropy(cpi, 0);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_ENCODER_H_
-#define VP10_ENCODER_ENCODER_H_
-
-#include <stdio.h>
-
-#include "./vpx_config.h"
-#include "vpx/vp8cx.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/ppflags.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/thread_common.h"
-#include "vp10/common/onyxc_int.h"
-
-#include "vp10/encoder/aq_cyclicrefresh.h"
-#include "vp10/encoder/context_tree.h"
-#include "vp10/encoder/encodemb.h"
-#include "vp10/encoder/firstpass.h"
-#include "vp10/encoder/lookahead.h"
-#include "vp10/encoder/mbgraph.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/speed_features.h"
-#include "vp10/encoder/tokenize.h"
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
-#include "vp10/encoder/denoiser.h"
-#endif
-
-#if CONFIG_INTERNAL_STATS
-#include "vpx_dsp/ssim.h"
-#endif
-#include "vpx_dsp/variance.h"
-#include "vpx/internal/vpx_codec_internal.h"
-#include "vpx_util/vpx_thread.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct {
- int nmvjointcost[MV_JOINTS];
- int nmvcosts[2][MV_VALS];
- int nmvcosts_hp[2][MV_VALS];
-
-#if !CONFIG_MISC_FIXES
- vpx_prob segment_pred_probs[PREDICTION_PROBS];
-#endif
-
- unsigned char *last_frame_seg_map_copy;
-
- // 0 = Intra, Last, GF, ARF
- signed char last_ref_lf_deltas[MAX_REF_FRAMES];
- // 0 = ZERO_MV, MV
- signed char last_mode_lf_deltas[MAX_MODE_LF_DELTAS];
-
- FRAME_CONTEXT fc;
-} CODING_CONTEXT;
-
-
-typedef enum {
- // encode_breakout is disabled.
- ENCODE_BREAKOUT_DISABLED = 0,
- // encode_breakout is enabled.
- ENCODE_BREAKOUT_ENABLED = 1,
- // encode_breakout is enabled with small max_thresh limit.
- ENCODE_BREAKOUT_LIMITED = 2
-} ENCODE_BREAKOUT_TYPE;
-
-typedef enum {
- NORMAL = 0,
- FOURFIVE = 1,
- THREEFIVE = 2,
- ONETWO = 3
-} VPX_SCALING;
-
-typedef enum {
- // Good Quality Fast Encoding. The encoder balances quality with the amount of
- // time it takes to encode the output. Speed setting controls how fast.
- GOOD,
-
- // The encoder places priority on the quality of the output over encoding
- // speed. The output is compressed at the highest possible quality. This
- // option takes the longest amount of time to encode. Speed setting ignored.
- BEST,
-
- // Realtime/Live Encoding. This mode is optimized for realtime encoding (for
- // example, capturing a television signal or feed from a live camera). Speed
- // setting controls how fast.
- REALTIME
-} MODE;
-
-typedef enum {
- FRAMEFLAGS_KEY = 1 << 0,
- FRAMEFLAGS_GOLDEN = 1 << 1,
- FRAMEFLAGS_ALTREF = 1 << 2,
-} FRAMETYPE_FLAGS;
-
-typedef enum {
- NO_AQ = 0,
- VARIANCE_AQ = 1,
- COMPLEXITY_AQ = 2,
- CYCLIC_REFRESH_AQ = 3,
- AQ_MODE_COUNT // This should always be the last member of the enum
-} AQ_MODE;
-
-typedef enum {
- RESIZE_NONE = 0, // No frame resizing allowed.
- RESIZE_FIXED = 1, // All frames are coded at the specified dimension.
- RESIZE_DYNAMIC = 2 // Coded size of each frame is determined by the codec.
-} RESIZE_TYPE;
-
-typedef struct VP10EncoderConfig {
- BITSTREAM_PROFILE profile;
- vpx_bit_depth_t bit_depth; // Codec bit-depth.
- int width; // width of data passed to the compressor
- int height; // height of data passed to the compressor
- unsigned int input_bit_depth; // Input bit depth.
- double init_framerate; // set to passed in framerate
- int64_t target_bandwidth; // bandwidth to be used in kilobits per second
-
- int noise_sensitivity; // pre processing blur: recommendation 0
- int sharpness; // sharpening output: recommendation 0:
- int speed;
- // maximum allowed bitrate for any intra frame in % of bitrate target.
- unsigned int rc_max_intra_bitrate_pct;
- // maximum allowed bitrate for any inter frame in % of bitrate target.
- unsigned int rc_max_inter_bitrate_pct;
- // percent of rate boost for golden frame in CBR mode.
- unsigned int gf_cbr_boost_pct;
-
- MODE mode;
- int pass;
-
- // Key Framing Operations
- int auto_key; // autodetect cut scenes and set the keyframes
- int key_freq; // maximum distance to key frame.
-
- int lag_in_frames; // how many frames lag before we start encoding
-
- // ----------------------------------------------------------------
- // DATARATE CONTROL OPTIONS
-
- // vbr, cbr, constrained quality or constant quality
- enum vpx_rc_mode rc_mode;
-
- // buffer targeting aggressiveness
- int under_shoot_pct;
- int over_shoot_pct;
-
- // buffering parameters
- int64_t starting_buffer_level_ms;
- int64_t optimal_buffer_level_ms;
- int64_t maximum_buffer_size_ms;
-
- // Frame drop threshold.
- int drop_frames_water_mark;
-
- // controlling quality
- int fixed_q;
- int worst_allowed_q;
- int best_allowed_q;
- int cq_level;
- AQ_MODE aq_mode; // Adaptive Quantization mode
-
- // Internal frame size scaling.
- RESIZE_TYPE resize_mode;
- int scaled_frame_width;
- int scaled_frame_height;
-
- // Enable feature to reduce the frame quantization every x frames.
- int frame_periodic_boost;
-
- // two pass datarate control
- int two_pass_vbrbias; // two pass datarate control tweaks
- int two_pass_vbrmin_section;
- int two_pass_vbrmax_section;
- // END DATARATE CONTROL OPTIONS
- // ----------------------------------------------------------------
-
- int enable_auto_arf;
-
- int encode_breakout; // early breakout : for video conf recommend 800
-
- /* Bitfield defining the error resiliency features to enable.
- * Can provide decodable frames after losses in previous
- * frames and decodable partitions after losses in the same frame.
- */
- unsigned int error_resilient_mode;
-
- /* Bitfield defining the parallel decoding mode where the
- * decoding in successive frames may be conducted in parallel
- * just by decoding the frame headers.
- */
- unsigned int frame_parallel_decoding_mode;
-
- int arnr_max_frames;
- int arnr_strength;
-
- int min_gf_interval;
- int max_gf_interval;
-
- int tile_columns;
- int tile_rows;
-
- int max_threads;
-
- vpx_fixed_buf_t two_pass_stats_in;
- struct vpx_codec_pkt_list *output_pkt_list;
-
-#if CONFIG_FP_MB_STATS
- vpx_fixed_buf_t firstpass_mb_stats_in;
-#endif
-
- vp8e_tuning tuning;
- vp9e_tune_content content;
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth;
-#endif
- vpx_color_space_t color_space;
- int color_range;
- int render_width;
- int render_height;
-} VP10EncoderConfig;
-
-static INLINE int is_lossless_requested(const VP10EncoderConfig *cfg) {
- return cfg->best_allowed_q == 0 && cfg->worst_allowed_q == 0;
-}
-
-// TODO(jingning) All spatially adaptive variables should go to TileDataEnc.
-typedef struct TileDataEnc {
- TileInfo tile_info;
- int thresh_freq_fact[BLOCK_SIZES][MAX_MODES];
- int mode_map[BLOCK_SIZES][MAX_MODES];
-} TileDataEnc;
-
-typedef struct RD_COUNTS {
- vp10_coeff_count coef_counts[TX_SIZES][PLANE_TYPES];
- int64_t comp_pred_diff[REFERENCE_MODES];
- int64_t filter_diff[SWITCHABLE_FILTER_CONTEXTS];
-} RD_COUNTS;
-
-typedef struct ThreadData {
- MACROBLOCK mb;
- RD_COUNTS rd_counts;
- FRAME_COUNTS *counts;
-
- PICK_MODE_CONTEXT *leaf_tree;
- PC_TREE *pc_tree;
- PC_TREE *pc_root;
-} ThreadData;
-
-struct EncWorkerData;
-
-typedef struct ActiveMap {
- int enabled;
- int update;
- unsigned char *map;
-} ActiveMap;
-
-typedef enum {
- Y,
- U,
- V,
- ALL
-} STAT_TYPE;
-
-typedef struct IMAGE_STAT {
- double stat[ALL+1];
- double worst;
-} ImageStat;
-
-typedef struct VP10_COMP {
- QUANTS quants;
- ThreadData td;
- MB_MODE_INFO_EXT *mbmi_ext_base;
- DECLARE_ALIGNED(16, int16_t, y_dequant[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, uv_dequant[QINDEX_RANGE][8]);
- VP10_COMMON common;
- VP10EncoderConfig oxcf;
- struct lookahead_ctx *lookahead;
- struct lookahead_entry *alt_ref_source;
-
- YV12_BUFFER_CONFIG *Source;
- YV12_BUFFER_CONFIG *Last_Source; // NULL for first frame and alt_ref frames
- YV12_BUFFER_CONFIG *un_scaled_source;
- YV12_BUFFER_CONFIG scaled_source;
- YV12_BUFFER_CONFIG *unscaled_last_source;
- YV12_BUFFER_CONFIG scaled_last_source;
-
- TileDataEnc *tile_data;
- int allocated_tiles; // Keep track of memory allocated for tiles.
-
- // For a still frame, this flag is set to 1 to skip partition search.
- int partition_search_skippable_frame;
-
- int scaled_ref_idx[MAX_REF_FRAMES];
- int lst_fb_idx;
- int gld_fb_idx;
- int alt_fb_idx;
-
- int refresh_last_frame;
- int refresh_golden_frame;
- int refresh_alt_ref_frame;
-
- int ext_refresh_frame_flags_pending;
- int ext_refresh_last_frame;
- int ext_refresh_golden_frame;
- int ext_refresh_alt_ref_frame;
-
- int ext_refresh_frame_context_pending;
- int ext_refresh_frame_context;
-
- YV12_BUFFER_CONFIG last_frame_uf;
-
- TOKENEXTRA *tile_tok[4][1 << 6];
- unsigned int tok_count[4][1 << 6];
-
- // Ambient reconstruction err target for force key frames
- int64_t ambient_err;
-
- RD_OPT rd;
-
- CODING_CONTEXT coding_context;
-
- int *nmvcosts[2];
- int *nmvcosts_hp[2];
- int *nmvsadcosts[2];
- int *nmvsadcosts_hp[2];
-
- int64_t last_time_stamp_seen;
- int64_t last_end_time_stamp_seen;
- int64_t first_time_stamp_ever;
-
- RATE_CONTROL rc;
- double framerate;
-
- int interp_filter_selected[MAX_REF_FRAMES][SWITCHABLE];
-
- struct vpx_codec_pkt_list *output_pkt_list;
-
- MBGRAPH_FRAME_STATS mbgraph_stats[MAX_LAG_BUFFERS];
- int mbgraph_n_frames; // number of frames filled in the above
- int static_mb_pct; // % forced skip mbs by segmentation
- int ref_frame_flags;
-
- SPEED_FEATURES sf;
-
- unsigned int max_mv_magnitude;
- int mv_step_param;
-
- int allow_comp_inter_inter;
-
- // Default value is 1. From first pass stats, encode_breakout may be disabled.
- ENCODE_BREAKOUT_TYPE allow_encode_breakout;
-
- // Get threshold from external input. A suggested threshold is 800 for HD
- // clips, and 300 for < HD clips.
- int encode_breakout;
-
- unsigned char *segmentation_map;
-
- // segment threashold for encode breakout
- int segment_encode_breakout[MAX_SEGMENTS];
-
- CYCLIC_REFRESH *cyclic_refresh;
- ActiveMap active_map;
-
- fractional_mv_step_fp *find_fractional_mv_step;
- vp10_full_search_fn_t full_search_sad;
- vp10_diamond_search_fn_t diamond_search_sad;
- vp9_variance_fn_ptr_t fn_ptr[BLOCK_SIZES];
- uint64_t time_receive_data;
- uint64_t time_compress_data;
- uint64_t time_pick_lpf;
- uint64_t time_encode_sb_row;
-
-#if CONFIG_FP_MB_STATS
- int use_fp_mb_stats;
-#endif
-
- TWO_PASS twopass;
-
- YV12_BUFFER_CONFIG alt_ref_buffer;
-
-
-#if CONFIG_INTERNAL_STATS
- unsigned int mode_chosen_counts[MAX_MODES];
-
- int count;
- uint64_t total_sq_error;
- uint64_t total_samples;
- ImageStat psnr;
-
- uint64_t totalp_sq_error;
- uint64_t totalp_samples;
- ImageStat psnrp;
-
- double total_blockiness;
- double worst_blockiness;
-
- int bytes;
- double summed_quality;
- double summed_weights;
- double summedp_quality;
- double summedp_weights;
- unsigned int tot_recode_hits;
- double worst_ssim;
-
- ImageStat ssimg;
- ImageStat fastssim;
- ImageStat psnrhvs;
-
- int b_calculate_ssimg;
- int b_calculate_blockiness;
-
- int b_calculate_consistency;
-
- double total_inconsistency;
- double worst_consistency;
- Ssimv *ssim_vars;
- Metrics metrics;
-#endif
- int b_calculate_psnr;
-
- int droppable;
-
- int initial_width;
- int initial_height;
- int initial_mbs; // Number of MBs in the full-size frame; to be used to
- // normalize the firstpass stats. This will differ from the
- // number of MBs in the current frame when the frame is
- // scaled.
-
- // Store frame variance info in SOURCE_VAR_BASED_PARTITION search type.
- diff *source_diff_var;
- // The threshold used in SOURCE_VAR_BASED_PARTITION search type.
- unsigned int source_var_thresh;
- int frames_till_next_var_check;
-
- int frame_flags;
-
- search_site_config ss_cfg;
-
- int mbmode_cost[INTRA_MODES];
- unsigned int inter_mode_cost[INTER_MODE_CONTEXTS][INTER_MODES];
- int intra_uv_mode_cost[INTRA_MODES];
- int y_mode_costs[INTRA_MODES][INTRA_MODES][INTRA_MODES];
- int switchable_interp_costs[SWITCHABLE_FILTER_CONTEXTS][SWITCHABLE_FILTERS];
- int partition_cost[PARTITION_CONTEXTS][PARTITION_TYPES];
- int palette_y_size_cost[PALETTE_BLOCK_SIZES][PALETTE_SIZES];
- int palette_uv_size_cost[PALETTE_BLOCK_SIZES][PALETTE_SIZES];
- int palette_y_color_cost[PALETTE_MAX_SIZE - 1][PALETTE_COLOR_CONTEXTS]
- [PALETTE_COLORS];
- int palette_uv_color_cost[PALETTE_MAX_SIZE - 1][PALETTE_COLOR_CONTEXTS]
- [PALETTE_COLORS];
-
- int multi_arf_allowed;
- int multi_arf_enabled;
- int multi_arf_last_grp_enabled;
-
-#if CONFIG_VP9_TEMPORAL_DENOISING
- VP9_DENOISER denoiser;
-#endif
-
- int resize_pending;
- int resize_state;
- int resize_scale_num;
- int resize_scale_den;
- int resize_avg_qp;
- int resize_buffer_underflow;
- int resize_count;
-
- // VAR_BASED_PARTITION thresholds
- // 0 - threshold_64x64; 1 - threshold_32x32;
- // 2 - threshold_16x16; 3 - vbp_threshold_8x8;
- int64_t vbp_thresholds[4];
- int64_t vbp_threshold_minmax;
- int64_t vbp_threshold_sad;
- BLOCK_SIZE vbp_bsize_min;
-
- // Multi-threading
- int num_workers;
- VPxWorker *workers;
- struct EncWorkerData *tile_thr_data;
- VP9LfSync lf_row_sync;
-} VP10_COMP;
-
-void vp10_initialize_enc(void);
-
-struct VP10_COMP *vp10_create_compressor(VP10EncoderConfig *oxcf,
- BufferPool *const pool);
-void vp10_remove_compressor(VP10_COMP *cpi);
-
-void vp10_change_config(VP10_COMP *cpi, const VP10EncoderConfig *oxcf);
-
- // receive a frames worth of data. caller can assume that a copy of this
- // frame is made and not just a copy of the pointer..
-int vp10_receive_raw_frame(VP10_COMP *cpi, unsigned int frame_flags,
- YV12_BUFFER_CONFIG *sd, int64_t time_stamp,
- int64_t end_time_stamp);
-
-int vp10_get_compressed_data(VP10_COMP *cpi, unsigned int *frame_flags,
- size_t *size, uint8_t *dest,
- int64_t *time_stamp, int64_t *time_end, int flush);
-
-int vp10_get_preview_raw_frame(VP10_COMP *cpi, YV12_BUFFER_CONFIG *dest,
- vp10_ppflags_t *flags);
-
-int vp10_use_as_reference(VP10_COMP *cpi, int ref_frame_flags);
-
-void vp10_update_reference(VP10_COMP *cpi, int ref_frame_flags);
-
-int vp10_copy_reference_enc(VP10_COMP *cpi, VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd);
-
-int vp10_set_reference_enc(VP10_COMP *cpi, VP9_REFFRAME ref_frame_flag,
- YV12_BUFFER_CONFIG *sd);
-
-int vp10_update_entropy(VP10_COMP *cpi, int update);
-
-int vp10_set_active_map(VP10_COMP *cpi, unsigned char *map, int rows, int cols);
-
-int vp10_get_active_map(VP10_COMP *cpi, unsigned char *map, int rows, int cols);
-
-int vp10_set_internal_size(VP10_COMP *cpi,
- VPX_SCALING horiz_mode, VPX_SCALING vert_mode);
-
-int vp10_set_size_literal(VP10_COMP *cpi, unsigned int width,
- unsigned int height);
-
-int vp10_get_quantizer(struct VP10_COMP *cpi);
-
-static INLINE int frame_is_kf_gf_arf(const VP10_COMP *cpi) {
- return frame_is_intra_only(&cpi->common) ||
- cpi->refresh_alt_ref_frame ||
- (cpi->refresh_golden_frame && !cpi->rc.is_src_frame_alt_ref);
-}
-
-static INLINE int get_ref_frame_map_idx(const VP10_COMP *cpi,
- MV_REFERENCE_FRAME ref_frame) {
- if (ref_frame == LAST_FRAME) {
- return cpi->lst_fb_idx;
- } else if (ref_frame == GOLDEN_FRAME) {
- return cpi->gld_fb_idx;
- } else {
- return cpi->alt_fb_idx;
- }
-}
-
-static INLINE int get_ref_frame_buf_idx(const VP10_COMP *const cpi,
- int ref_frame) {
- const VP10_COMMON *const cm = &cpi->common;
- const int map_idx = get_ref_frame_map_idx(cpi, ref_frame);
- return (map_idx != INVALID_IDX) ? cm->ref_frame_map[map_idx] : INVALID_IDX;
-}
-
-static INLINE YV12_BUFFER_CONFIG *get_ref_frame_buffer(
- VP10_COMP *cpi, MV_REFERENCE_FRAME ref_frame) {
- VP10_COMMON *const cm = &cpi->common;
- const int buf_idx = get_ref_frame_buf_idx(cpi, ref_frame);
- return
- buf_idx != INVALID_IDX ? &cm->buffer_pool->frame_bufs[buf_idx].buf : NULL;
-}
-
-static INLINE int get_token_alloc(int mb_rows, int mb_cols) {
- // TODO(JBB): double check we can't exceed this token count if we have a
- // 32x32 transform crossing a boundary at a multiple of 16.
- // mb_rows, cols are in units of 16 pixels. We assume 3 planes all at full
- // resolution. We assume up to 1 token per pixel, and then allow
- // a head room of 1 EOSB token per 8x8 block per plane.
- return mb_rows * mb_cols * (16 * 16 + 4) * 3;
-}
-
-// Get the allocated token size for a tile. It does the same calculation as in
-// the frame token allocation.
-static INLINE int allocated_tokens(TileInfo tile) {
- int tile_mb_rows = (tile.mi_row_end - tile.mi_row_start + 1) >> 1;
- int tile_mb_cols = (tile.mi_col_end - tile.mi_col_start + 1) >> 1;
-
- return get_token_alloc(tile_mb_rows, tile_mb_cols);
-}
-
-int64_t vp10_get_y_sse(const YV12_BUFFER_CONFIG *a, const YV12_BUFFER_CONFIG *b);
-#if CONFIG_VP9_HIGHBITDEPTH
-int64_t vp10_highbd_get_y_sse(const YV12_BUFFER_CONFIG *a,
- const YV12_BUFFER_CONFIG *b);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_alloc_compressor_data(VP10_COMP *cpi);
-
-void vp10_scale_references(VP10_COMP *cpi);
-
-void vp10_update_reference_frames(VP10_COMP *cpi);
-
-void vp10_set_high_precision_mv(VP10_COMP *cpi, int allow_high_precision_mv);
-
-YV12_BUFFER_CONFIG *vp10_scale_if_required_fast(VP10_COMMON *cm,
- YV12_BUFFER_CONFIG *unscaled,
- YV12_BUFFER_CONFIG *scaled);
-
-YV12_BUFFER_CONFIG *vp10_scale_if_required(VP10_COMMON *cm,
- YV12_BUFFER_CONFIG *unscaled,
- YV12_BUFFER_CONFIG *scaled);
-
-void vp10_apply_encoding_flags(VP10_COMP *cpi, vpx_enc_frame_flags_t flags);
-
-static INLINE int is_altref_enabled(const VP10_COMP *const cpi) {
- return cpi->oxcf.mode != REALTIME && cpi->oxcf.lag_in_frames > 0 &&
- cpi->oxcf.enable_auto_arf;
-}
-
-static INLINE void set_ref_ptrs(VP10_COMMON *cm, MACROBLOCKD *xd,
- MV_REFERENCE_FRAME ref0,
- MV_REFERENCE_FRAME ref1) {
- xd->block_refs[0] = &cm->frame_refs[ref0 >= LAST_FRAME ? ref0 - LAST_FRAME
- : 0];
- xd->block_refs[1] = &cm->frame_refs[ref1 >= LAST_FRAME ? ref1 - LAST_FRAME
- : 0];
-}
-
-static INLINE int get_chessboard_index(const int frame_index) {
- return frame_index & 0x1;
-}
-
-static INLINE int *cond_cost_list(const struct VP10_COMP *cpi, int *cost_list) {
- return cpi->sf.mv.subpel_search_method != SUBPEL_TREE ? cost_list : NULL;
-}
-
-void vp10_new_framerate(VP10_COMP *cpi, double framerate);
-
-#define LAYER_IDS_TO_IDX(sl, tl, num_tl) ((sl) * (num_tl) + (tl))
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_ENCODER_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/encoder/encodeframe.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/ethread.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-
-static void accumulate_rd_opt(ThreadData *td, ThreadData *td_t) {
- int i, j, k, l, m, n;
-
- for (i = 0; i < REFERENCE_MODES; i++)
- td->rd_counts.comp_pred_diff[i] += td_t->rd_counts.comp_pred_diff[i];
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- td->rd_counts.filter_diff[i] += td_t->rd_counts.filter_diff[i];
-
- for (i = 0; i < TX_SIZES; i++)
- for (j = 0; j < PLANE_TYPES; j++)
- for (k = 0; k < REF_TYPES; k++)
- for (l = 0; l < COEF_BANDS; l++)
- for (m = 0; m < COEFF_CONTEXTS; m++)
- for (n = 0; n < ENTROPY_TOKENS; n++)
- td->rd_counts.coef_counts[i][j][k][l][m][n] +=
- td_t->rd_counts.coef_counts[i][j][k][l][m][n];
-}
-
-static int enc_worker_hook(EncWorkerData *const thread_data, void *unused) {
- VP10_COMP *const cpi = thread_data->cpi;
- const VP10_COMMON *const cm = &cpi->common;
- const int tile_cols = 1 << cm->log2_tile_cols;
- const int tile_rows = 1 << cm->log2_tile_rows;
- int t;
-
- (void) unused;
-
- for (t = thread_data->start; t < tile_rows * tile_cols;
- t += cpi->num_workers) {
- int tile_row = t / tile_cols;
- int tile_col = t % tile_cols;
-
- vp10_encode_tile(cpi, thread_data->td, tile_row, tile_col);
- }
-
- return 0;
-}
-
-void vp10_encode_tiles_mt(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- const int tile_cols = 1 << cm->log2_tile_cols;
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- const int num_workers = VPXMIN(cpi->oxcf.max_threads, tile_cols);
- int i;
-
- vp10_init_tile_data(cpi);
-
- // Only run once to create threads and allocate thread data.
- if (cpi->num_workers == 0) {
- int allocated_workers = num_workers;
-
- CHECK_MEM_ERROR(cm, cpi->workers,
- vpx_malloc(allocated_workers * sizeof(*cpi->workers)));
-
- CHECK_MEM_ERROR(cm, cpi->tile_thr_data,
- vpx_calloc(allocated_workers,
- sizeof(*cpi->tile_thr_data)));
-
- for (i = 0; i < allocated_workers; i++) {
- VPxWorker *const worker = &cpi->workers[i];
- EncWorkerData *thread_data = &cpi->tile_thr_data[i];
-
- ++cpi->num_workers;
- winterface->init(worker);
-
- if (i < allocated_workers - 1) {
- thread_data->cpi = cpi;
-
- // Allocate thread data.
- CHECK_MEM_ERROR(cm, thread_data->td,
- vpx_memalign(32, sizeof(*thread_data->td)));
- vp10_zero(*thread_data->td);
-
- // Set up pc_tree.
- thread_data->td->leaf_tree = NULL;
- thread_data->td->pc_tree = NULL;
- vp10_setup_pc_tree(cm, thread_data->td);
-
- // Allocate frame counters in thread data.
- CHECK_MEM_ERROR(cm, thread_data->td->counts,
- vpx_calloc(1, sizeof(*thread_data->td->counts)));
-
- // Create threads
- if (!winterface->reset(worker))
- vpx_internal_error(&cm->error, VPX_CODEC_ERROR,
- "Tile encoder thread creation failed");
- } else {
- // Main thread acts as a worker and uses the thread data in cpi.
- thread_data->cpi = cpi;
- thread_data->td = &cpi->td;
- }
-
- winterface->sync(worker);
- }
- }
-
- for (i = 0; i < num_workers; i++) {
- VPxWorker *const worker = &cpi->workers[i];
- EncWorkerData *thread_data;
-
- worker->hook = (VPxWorkerHook)enc_worker_hook;
- worker->data1 = &cpi->tile_thr_data[i];
- worker->data2 = NULL;
- thread_data = (EncWorkerData*)worker->data1;
-
- // Before encoding a frame, copy the thread data from cpi.
- if (thread_data->td != &cpi->td) {
- thread_data->td->mb = cpi->td.mb;
- thread_data->td->rd_counts = cpi->td.rd_counts;
- }
- if (thread_data->td->counts != &cpi->common.counts) {
- memcpy(thread_data->td->counts, &cpi->common.counts,
- sizeof(cpi->common.counts));
- }
-
- // Allocate buffers used by palette coding mode.
- if (cpi->common.allow_screen_content_tools && i < num_workers - 1) {
- MACROBLOCK *x = &thread_data->td->mb;
- CHECK_MEM_ERROR(cm, x->palette_buffer,
- vpx_memalign(16, sizeof(*x->palette_buffer)));
- }
- }
-
- // Encode a frame
- for (i = 0; i < num_workers; i++) {
- VPxWorker *const worker = &cpi->workers[i];
- EncWorkerData *const thread_data = (EncWorkerData*)worker->data1;
-
- // Set the starting tile for each thread.
- thread_data->start = i;
-
- if (i == cpi->num_workers - 1)
- winterface->execute(worker);
- else
- winterface->launch(worker);
- }
-
- // Encoding ends.
- for (i = 0; i < num_workers; i++) {
- VPxWorker *const worker = &cpi->workers[i];
- winterface->sync(worker);
- }
-
- for (i = 0; i < num_workers; i++) {
- VPxWorker *const worker = &cpi->workers[i];
- EncWorkerData *const thread_data = (EncWorkerData*)worker->data1;
-
- // Accumulate counters.
- if (i < cpi->num_workers - 1) {
- vp10_accumulate_frame_counts(cm, thread_data->td->counts, 0);
- accumulate_rd_opt(&cpi->td, thread_data->td);
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_ETHREAD_H_
-#define VP10_ENCODER_ETHREAD_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10_COMP;
-struct ThreadData;
-
-typedef struct EncWorkerData {
- struct VP10_COMP *cpi;
- struct ThreadData *td;
- int start;
-} EncWorkerData;
-
-void vp10_encode_tiles_mt(struct VP10_COMP *cpi);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_ETHREAD_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/common.h"
-#include "vp10/encoder/extend.h"
-
-static void copy_and_extend_plane(const uint8_t *src, int src_pitch,
- uint8_t *dst, int dst_pitch,
- int w, int h,
- int extend_top, int extend_left,
- int extend_bottom, int extend_right) {
- int i, linesize;
-
- // copy the left and right most columns out
- const uint8_t *src_ptr1 = src;
- const uint8_t *src_ptr2 = src + w - 1;
- uint8_t *dst_ptr1 = dst - extend_left;
- uint8_t *dst_ptr2 = dst + w;
-
- for (i = 0; i < h; i++) {
- memset(dst_ptr1, src_ptr1[0], extend_left);
- memcpy(dst_ptr1 + extend_left, src_ptr1, w);
- memset(dst_ptr2, src_ptr2[0], extend_right);
- src_ptr1 += src_pitch;
- src_ptr2 += src_pitch;
- dst_ptr1 += dst_pitch;
- dst_ptr2 += dst_pitch;
- }
-
- // Now copy the top and bottom lines into each line of the respective
- // borders
- src_ptr1 = dst - extend_left;
- src_ptr2 = dst + dst_pitch * (h - 1) - extend_left;
- dst_ptr1 = dst + dst_pitch * (-extend_top) - extend_left;
- dst_ptr2 = dst + dst_pitch * (h) - extend_left;
- linesize = extend_left + extend_right + w;
-
- for (i = 0; i < extend_top; i++) {
- memcpy(dst_ptr1, src_ptr1, linesize);
- dst_ptr1 += dst_pitch;
- }
-
- for (i = 0; i < extend_bottom; i++) {
- memcpy(dst_ptr2, src_ptr2, linesize);
- dst_ptr2 += dst_pitch;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void highbd_copy_and_extend_plane(const uint8_t *src8, int src_pitch,
- uint8_t *dst8, int dst_pitch,
- int w, int h,
- int extend_top, int extend_left,
- int extend_bottom, int extend_right) {
- int i, linesize;
- uint16_t *src = CONVERT_TO_SHORTPTR(src8);
- uint16_t *dst = CONVERT_TO_SHORTPTR(dst8);
-
- // copy the left and right most columns out
- const uint16_t *src_ptr1 = src;
- const uint16_t *src_ptr2 = src + w - 1;
- uint16_t *dst_ptr1 = dst - extend_left;
- uint16_t *dst_ptr2 = dst + w;
-
- for (i = 0; i < h; i++) {
- vpx_memset16(dst_ptr1, src_ptr1[0], extend_left);
- memcpy(dst_ptr1 + extend_left, src_ptr1, w * sizeof(src_ptr1[0]));
- vpx_memset16(dst_ptr2, src_ptr2[0], extend_right);
- src_ptr1 += src_pitch;
- src_ptr2 += src_pitch;
- dst_ptr1 += dst_pitch;
- dst_ptr2 += dst_pitch;
- }
-
- // Now copy the top and bottom lines into each line of the respective
- // borders
- src_ptr1 = dst - extend_left;
- src_ptr2 = dst + dst_pitch * (h - 1) - extend_left;
- dst_ptr1 = dst + dst_pitch * (-extend_top) - extend_left;
- dst_ptr2 = dst + dst_pitch * (h) - extend_left;
- linesize = extend_left + extend_right + w;
-
- for (i = 0; i < extend_top; i++) {
- memcpy(dst_ptr1, src_ptr1, linesize * sizeof(src_ptr1[0]));
- dst_ptr1 += dst_pitch;
- }
-
- for (i = 0; i < extend_bottom; i++) {
- memcpy(dst_ptr2, src_ptr2, linesize * sizeof(src_ptr2[0]));
- dst_ptr2 += dst_pitch;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_copy_and_extend_frame(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst) {
- // Extend src frame in buffer
- // Altref filtering assumes 16 pixel extension
- const int et_y = 16;
- const int el_y = 16;
- // Motion estimation may use src block variance with the block size up
- // to 64x64, so the right and bottom need to be extended to 64 multiple
- // or up to 16, whichever is greater.
- const int er_y =
- VPXMAX(src->y_width + 16, ALIGN_POWER_OF_TWO(src->y_width, 6)) -
- src->y_crop_width;
- const int eb_y =
- VPXMAX(src->y_height + 16, ALIGN_POWER_OF_TWO(src->y_height, 6)) -
- src->y_crop_height;
- const int uv_width_subsampling = (src->uv_width != src->y_width);
- const int uv_height_subsampling = (src->uv_height != src->y_height);
- const int et_uv = et_y >> uv_height_subsampling;
- const int el_uv = el_y >> uv_width_subsampling;
- const int eb_uv = eb_y >> uv_height_subsampling;
- const int er_uv = er_y >> uv_width_subsampling;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (src->flags & YV12_FLAG_HIGHBITDEPTH) {
- highbd_copy_and_extend_plane(src->y_buffer, src->y_stride,
- dst->y_buffer, dst->y_stride,
- src->y_crop_width, src->y_crop_height,
- et_y, el_y, eb_y, er_y);
-
- highbd_copy_and_extend_plane(src->u_buffer, src->uv_stride,
- dst->u_buffer, dst->uv_stride,
- src->uv_crop_width, src->uv_crop_height,
- et_uv, el_uv, eb_uv, er_uv);
-
- highbd_copy_and_extend_plane(src->v_buffer, src->uv_stride,
- dst->v_buffer, dst->uv_stride,
- src->uv_crop_width, src->uv_crop_height,
- et_uv, el_uv, eb_uv, er_uv);
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- copy_and_extend_plane(src->y_buffer, src->y_stride,
- dst->y_buffer, dst->y_stride,
- src->y_crop_width, src->y_crop_height,
- et_y, el_y, eb_y, er_y);
-
- copy_and_extend_plane(src->u_buffer, src->uv_stride,
- dst->u_buffer, dst->uv_stride,
- src->uv_crop_width, src->uv_crop_height,
- et_uv, el_uv, eb_uv, er_uv);
-
- copy_and_extend_plane(src->v_buffer, src->uv_stride,
- dst->v_buffer, dst->uv_stride,
- src->uv_crop_width, src->uv_crop_height,
- et_uv, el_uv, eb_uv, er_uv);
-}
-
-void vp10_copy_and_extend_frame_with_rect(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst,
- int srcy, int srcx,
- int srch, int srcw) {
- // If the side is not touching the bounder then don't extend.
- const int et_y = srcy ? 0 : dst->border;
- const int el_y = srcx ? 0 : dst->border;
- const int eb_y = srcy + srch != src->y_height ? 0 :
- dst->border + dst->y_height - src->y_height;
- const int er_y = srcx + srcw != src->y_width ? 0 :
- dst->border + dst->y_width - src->y_width;
- const int src_y_offset = srcy * src->y_stride + srcx;
- const int dst_y_offset = srcy * dst->y_stride + srcx;
-
- const int et_uv = ROUND_POWER_OF_TWO(et_y, 1);
- const int el_uv = ROUND_POWER_OF_TWO(el_y, 1);
- const int eb_uv = ROUND_POWER_OF_TWO(eb_y, 1);
- const int er_uv = ROUND_POWER_OF_TWO(er_y, 1);
- const int src_uv_offset = ((srcy * src->uv_stride) >> 1) + (srcx >> 1);
- const int dst_uv_offset = ((srcy * dst->uv_stride) >> 1) + (srcx >> 1);
- const int srch_uv = ROUND_POWER_OF_TWO(srch, 1);
- const int srcw_uv = ROUND_POWER_OF_TWO(srcw, 1);
-
- copy_and_extend_plane(src->y_buffer + src_y_offset, src->y_stride,
- dst->y_buffer + dst_y_offset, dst->y_stride,
- srcw, srch,
- et_y, el_y, eb_y, er_y);
-
- copy_and_extend_plane(src->u_buffer + src_uv_offset, src->uv_stride,
- dst->u_buffer + dst_uv_offset, dst->uv_stride,
- srcw_uv, srch_uv,
- et_uv, el_uv, eb_uv, er_uv);
-
- copy_and_extend_plane(src->v_buffer + src_uv_offset, src->uv_stride,
- dst->v_buffer + dst_uv_offset, dst->uv_stride,
- srcw_uv, srch_uv,
- et_uv, el_uv, eb_uv, er_uv);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_EXTEND_H_
-#define VP10_ENCODER_EXTEND_H_
-
-#include "vpx_scale/yv12config.h"
-#include "vpx/vpx_integer.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-
-void vp10_copy_and_extend_frame(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst);
-
-void vp10_copy_and_extend_frame_with_rect(const YV12_BUFFER_CONFIG *src,
- YV12_BUFFER_CONFIG *dst,
- int srcy, int srcx,
- int srch, int srcw);
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_EXTEND_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-#include <math.h>
-#include <stdio.h>
-
-#include "./vpx_dsp_rtcd.h"
-#include "./vpx_scale_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-#include "vpx_scale/vpx_scale.h"
-#include "vpx_scale/yv12config.h"
-
-#include "vp10/common/entropymv.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconinter.h" // vp10_setup_dst_planes()
-#include "vp10/encoder/aq_variance.h"
-#include "vp10/encoder/block.h"
-#include "vp10/encoder/encodeframe.h"
-#include "vp10/encoder/encodemb.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/extend.h"
-#include "vp10/encoder/firstpass.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/rd.h"
-#include "vpx_dsp/variance.h"
-
-#define OUTPUT_FPF 0
-#define ARF_STATS_OUTPUT 0
-
-#define GROUP_ADAPTIVE_MAXQ 1
-
-#define BOOST_BREAKOUT 12.5
-#define BOOST_FACTOR 12.5
-#define ERR_DIVISOR 128.0
-#define FACTOR_PT_LOW 0.70
-#define FACTOR_PT_HIGH 0.90
-#define FIRST_PASS_Q 10.0
-#define GF_MAX_BOOST 96.0
-#define INTRA_MODE_PENALTY 1024
-#define KF_MAX_BOOST 128.0
-#define MIN_ARF_GF_BOOST 240
-#define MIN_DECAY_FACTOR 0.01
-#define MIN_KF_BOOST 300
-#define NEW_MV_MODE_PENALTY 32
-#define DARK_THRESH 64
-#define DEFAULT_GRP_WEIGHT 1.0
-#define RC_FACTOR_MIN 0.75
-#define RC_FACTOR_MAX 1.75
-
-
-#define NCOUNT_INTRA_THRESH 8192
-#define NCOUNT_INTRA_FACTOR 3
-#define NCOUNT_FRAME_II_THRESH 5.0
-
-#define DOUBLE_DIVIDE_CHECK(x) ((x) < 0 ? (x) - 0.000001 : (x) + 0.000001)
-
-#if ARF_STATS_OUTPUT
-unsigned int arf_count = 0;
-#endif
-
-// Resets the first pass file to the given position using a relative seek from
-// the current position.
-static void reset_fpf_position(TWO_PASS *p,
- const FIRSTPASS_STATS *position) {
- p->stats_in = position;
-}
-
-// Read frame stats at an offset from the current position.
-static const FIRSTPASS_STATS *read_frame_stats(const TWO_PASS *p, int offset) {
- if ((offset >= 0 && p->stats_in + offset >= p->stats_in_end) ||
- (offset < 0 && p->stats_in + offset < p->stats_in_start)) {
- return NULL;
- }
-
- return &p->stats_in[offset];
-}
-
-static int input_stats(TWO_PASS *p, FIRSTPASS_STATS *fps) {
- if (p->stats_in >= p->stats_in_end)
- return EOF;
-
- *fps = *p->stats_in;
- ++p->stats_in;
- return 1;
-}
-
-static void output_stats(FIRSTPASS_STATS *stats,
- struct vpx_codec_pkt_list *pktlist) {
- struct vpx_codec_cx_pkt pkt;
- pkt.kind = VPX_CODEC_STATS_PKT;
- pkt.data.twopass_stats.buf = stats;
- pkt.data.twopass_stats.sz = sizeof(FIRSTPASS_STATS);
- vpx_codec_pkt_list_add(pktlist, &pkt);
-
-// TEMP debug code
-#if OUTPUT_FPF
- {
- FILE *fpfile;
- fpfile = fopen("firstpass.stt", "a");
-
- fprintf(fpfile, "%12.0lf %12.4lf %12.0lf %12.0lf %12.0lf %12.4lf %12.4lf"
- "%12.4lf %12.4lf %12.4lf %12.4lf %12.4lf %12.4lf %12.4lf %12.4lf"
- "%12.4lf %12.4lf %12.0lf %12.0lf %12.0lf %12.4lf\n",
- stats->frame,
- stats->weight,
- stats->intra_error,
- stats->coded_error,
- stats->sr_coded_error,
- stats->pcnt_inter,
- stats->pcnt_motion,
- stats->pcnt_second_ref,
- stats->pcnt_neutral,
- stats->intra_skip_pct,
- stats->inactive_zone_rows,
- stats->inactive_zone_cols,
- stats->MVr,
- stats->mvr_abs,
- stats->MVc,
- stats->mvc_abs,
- stats->MVrv,
- stats->MVcv,
- stats->mv_in_out_count,
- stats->new_mv_count,
- stats->count,
- stats->duration);
- fclose(fpfile);
- }
-#endif
-}
-
-#if CONFIG_FP_MB_STATS
-static void output_fpmb_stats(uint8_t *this_frame_mb_stats,
- VP10_COMMON *cm,
- struct vpx_codec_pkt_list *pktlist) {
- struct vpx_codec_cx_pkt pkt;
- pkt.kind = VPX_CODEC_FPMB_STATS_PKT;
- pkt.data.firstpass_mb_stats.buf = this_frame_mb_stats;
- pkt.data.firstpass_mb_stats.sz = cm->initial_mbs * sizeof(uint8_t);
- vpx_codec_pkt_list_add(pktlist, &pkt);
-}
-#endif
-
-static void zero_stats(FIRSTPASS_STATS *section) {
- section->frame = 0.0;
- section->weight = 0.0;
- section->intra_error = 0.0;
- section->coded_error = 0.0;
- section->sr_coded_error = 0.0;
- section->pcnt_inter = 0.0;
- section->pcnt_motion = 0.0;
- section->pcnt_second_ref = 0.0;
- section->pcnt_neutral = 0.0;
- section->intra_skip_pct = 0.0;
- section->inactive_zone_rows = 0.0;
- section->inactive_zone_cols = 0.0;
- section->MVr = 0.0;
- section->mvr_abs = 0.0;
- section->MVc = 0.0;
- section->mvc_abs = 0.0;
- section->MVrv = 0.0;
- section->MVcv = 0.0;
- section->mv_in_out_count = 0.0;
- section->new_mv_count = 0.0;
- section->count = 0.0;
- section->duration = 1.0;
-}
-
-static void accumulate_stats(FIRSTPASS_STATS *section,
- const FIRSTPASS_STATS *frame) {
- section->frame += frame->frame;
- section->weight += frame->weight;
- section->intra_error += frame->intra_error;
- section->coded_error += frame->coded_error;
- section->sr_coded_error += frame->sr_coded_error;
- section->pcnt_inter += frame->pcnt_inter;
- section->pcnt_motion += frame->pcnt_motion;
- section->pcnt_second_ref += frame->pcnt_second_ref;
- section->pcnt_neutral += frame->pcnt_neutral;
- section->intra_skip_pct += frame->intra_skip_pct;
- section->inactive_zone_rows += frame->inactive_zone_rows;
- section->inactive_zone_cols += frame->inactive_zone_cols;
- section->MVr += frame->MVr;
- section->mvr_abs += frame->mvr_abs;
- section->MVc += frame->MVc;
- section->mvc_abs += frame->mvc_abs;
- section->MVrv += frame->MVrv;
- section->MVcv += frame->MVcv;
- section->mv_in_out_count += frame->mv_in_out_count;
- section->new_mv_count += frame->new_mv_count;
- section->count += frame->count;
- section->duration += frame->duration;
-}
-
-static void subtract_stats(FIRSTPASS_STATS *section,
- const FIRSTPASS_STATS *frame) {
- section->frame -= frame->frame;
- section->weight -= frame->weight;
- section->intra_error -= frame->intra_error;
- section->coded_error -= frame->coded_error;
- section->sr_coded_error -= frame->sr_coded_error;
- section->pcnt_inter -= frame->pcnt_inter;
- section->pcnt_motion -= frame->pcnt_motion;
- section->pcnt_second_ref -= frame->pcnt_second_ref;
- section->pcnt_neutral -= frame->pcnt_neutral;
- section->intra_skip_pct -= frame->intra_skip_pct;
- section->inactive_zone_rows -= frame->inactive_zone_rows;
- section->inactive_zone_cols -= frame->inactive_zone_cols;
- section->MVr -= frame->MVr;
- section->mvr_abs -= frame->mvr_abs;
- section->MVc -= frame->MVc;
- section->mvc_abs -= frame->mvc_abs;
- section->MVrv -= frame->MVrv;
- section->MVcv -= frame->MVcv;
- section->mv_in_out_count -= frame->mv_in_out_count;
- section->new_mv_count -= frame->new_mv_count;
- section->count -= frame->count;
- section->duration -= frame->duration;
-}
-
-// Calculate an active area of the image that discounts formatting
-// bars and partially discounts other 0 energy areas.
-#define MIN_ACTIVE_AREA 0.5
-#define MAX_ACTIVE_AREA 1.0
-static double calculate_active_area(const VP10_COMP *cpi,
- const FIRSTPASS_STATS *this_frame)
-{
- double active_pct;
-
- active_pct = 1.0 -
- ((this_frame->intra_skip_pct / 2) +
- ((this_frame->inactive_zone_rows * 2) / (double)cpi->common.mb_rows));
- return fclamp(active_pct, MIN_ACTIVE_AREA, MAX_ACTIVE_AREA);
-}
-
-// Calculate a modified Error used in distributing bits between easier and
-// harder frames.
-#define ACT_AREA_CORRECTION 0.5
-static double calculate_modified_err(const VP10_COMP *cpi,
- const TWO_PASS *twopass,
- const VP10EncoderConfig *oxcf,
- const FIRSTPASS_STATS *this_frame) {
- const FIRSTPASS_STATS *const stats = &twopass->total_stats;
- const double av_weight = stats->weight / stats->count;
- const double av_err = (stats->coded_error * av_weight) / stats->count;
- double modified_error =
- av_err * pow(this_frame->coded_error * this_frame->weight /
- DOUBLE_DIVIDE_CHECK(av_err), oxcf->two_pass_vbrbias / 100.0);
-
- // Correction for active area. Frames with a reduced active area
- // (eg due to formatting bars) have a higher error per mb for the
- // remaining active MBs. The correction here assumes that coding
- // 0.5N blocks of complexity 2X is a little easier than coding N
- // blocks of complexity X.
- modified_error *=
- pow(calculate_active_area(cpi, this_frame), ACT_AREA_CORRECTION);
-
- return fclamp(modified_error,
- twopass->modified_error_min, twopass->modified_error_max);
-}
-
-// This function returns the maximum target rate per frame.
-static int frame_max_bits(const RATE_CONTROL *rc,
- const VP10EncoderConfig *oxcf) {
- int64_t max_bits = ((int64_t)rc->avg_frame_bandwidth *
- (int64_t)oxcf->two_pass_vbrmax_section) / 100;
- if (max_bits < 0)
- max_bits = 0;
- else if (max_bits > rc->max_frame_bandwidth)
- max_bits = rc->max_frame_bandwidth;
-
- return (int)max_bits;
-}
-
-void vp10_init_first_pass(VP10_COMP *cpi) {
- zero_stats(&cpi->twopass.total_stats);
-}
-
-void vp10_end_first_pass(VP10_COMP *cpi) {
- output_stats(&cpi->twopass.total_stats, cpi->output_pkt_list);
-}
-
-static vpx_variance_fn_t get_block_variance_fn(BLOCK_SIZE bsize) {
- switch (bsize) {
- case BLOCK_8X8:
- return vpx_mse8x8;
- case BLOCK_16X8:
- return vpx_mse16x8;
- case BLOCK_8X16:
- return vpx_mse8x16;
- default:
- return vpx_mse16x16;
- }
-}
-
-static unsigned int get_prediction_error(BLOCK_SIZE bsize,
- const struct buf_2d *src,
- const struct buf_2d *ref) {
- unsigned int sse;
- const vpx_variance_fn_t fn = get_block_variance_fn(bsize);
- fn(src->buf, src->stride, ref->buf, ref->stride, &sse);
- return sse;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static vpx_variance_fn_t highbd_get_block_variance_fn(BLOCK_SIZE bsize,
- int bd) {
- switch (bd) {
- default:
- switch (bsize) {
- case BLOCK_8X8:
- return vpx_highbd_8_mse8x8;
- case BLOCK_16X8:
- return vpx_highbd_8_mse16x8;
- case BLOCK_8X16:
- return vpx_highbd_8_mse8x16;
- default:
- return vpx_highbd_8_mse16x16;
- }
- break;
- case 10:
- switch (bsize) {
- case BLOCK_8X8:
- return vpx_highbd_10_mse8x8;
- case BLOCK_16X8:
- return vpx_highbd_10_mse16x8;
- case BLOCK_8X16:
- return vpx_highbd_10_mse8x16;
- default:
- return vpx_highbd_10_mse16x16;
- }
- break;
- case 12:
- switch (bsize) {
- case BLOCK_8X8:
- return vpx_highbd_12_mse8x8;
- case BLOCK_16X8:
- return vpx_highbd_12_mse16x8;
- case BLOCK_8X16:
- return vpx_highbd_12_mse8x16;
- default:
- return vpx_highbd_12_mse16x16;
- }
- break;
- }
-}
-
-static unsigned int highbd_get_prediction_error(BLOCK_SIZE bsize,
- const struct buf_2d *src,
- const struct buf_2d *ref,
- int bd) {
- unsigned int sse;
- const vpx_variance_fn_t fn = highbd_get_block_variance_fn(bsize, bd);
- fn(src->buf, src->stride, ref->buf, ref->stride, &sse);
- return sse;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-// Refine the motion search range according to the frame dimension
-// for first pass test.
-static int get_search_range(const VP10_COMP *cpi) {
- int sr = 0;
- const int dim = VPXMIN(cpi->initial_width, cpi->initial_height);
-
- while ((dim << sr) < MAX_FULL_PEL_VAL)
- ++sr;
- return sr;
-}
-
-static void first_pass_motion_search(VP10_COMP *cpi, MACROBLOCK *x,
- const MV *ref_mv, MV *best_mv,
- int *best_motion_err) {
- MACROBLOCKD *const xd = &x->e_mbd;
- MV tmp_mv = {0, 0};
- MV ref_mv_full = {ref_mv->row >> 3, ref_mv->col >> 3};
- int num00, tmp_err, n;
- const BLOCK_SIZE bsize = xd->mi[0]->mbmi.sb_type;
- vp9_variance_fn_ptr_t v_fn_ptr = cpi->fn_ptr[bsize];
- const int new_mv_mode_penalty = NEW_MV_MODE_PENALTY;
-
- int step_param = 3;
- int further_steps = (MAX_MVSEARCH_STEPS - 1) - step_param;
- const int sr = get_search_range(cpi);
- step_param += sr;
- further_steps -= sr;
-
- // Override the default variance function to use MSE.
- v_fn_ptr.vf = get_block_variance_fn(bsize);
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- v_fn_ptr.vf = highbd_get_block_variance_fn(bsize, xd->bd);
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Center the initial step/diamond search on best mv.
- tmp_err = cpi->diamond_search_sad(x, &cpi->ss_cfg, &ref_mv_full, &tmp_mv,
- step_param,
- x->sadperbit16, &num00, &v_fn_ptr, ref_mv);
- if (tmp_err < INT_MAX)
- tmp_err = vp10_get_mvpred_var(x, &tmp_mv, ref_mv, &v_fn_ptr, 1);
- if (tmp_err < INT_MAX - new_mv_mode_penalty)
- tmp_err += new_mv_mode_penalty;
-
- if (tmp_err < *best_motion_err) {
- *best_motion_err = tmp_err;
- *best_mv = tmp_mv;
- }
-
- // Carry out further step/diamond searches as necessary.
- n = num00;
- num00 = 0;
-
- while (n < further_steps) {
- ++n;
-
- if (num00) {
- --num00;
- } else {
- tmp_err = cpi->diamond_search_sad(x, &cpi->ss_cfg, &ref_mv_full, &tmp_mv,
- step_param + n, x->sadperbit16,
- &num00, &v_fn_ptr, ref_mv);
- if (tmp_err < INT_MAX)
- tmp_err = vp10_get_mvpred_var(x, &tmp_mv, ref_mv, &v_fn_ptr, 1);
- if (tmp_err < INT_MAX - new_mv_mode_penalty)
- tmp_err += new_mv_mode_penalty;
-
- if (tmp_err < *best_motion_err) {
- *best_motion_err = tmp_err;
- *best_mv = tmp_mv;
- }
- }
- }
-}
-
-static BLOCK_SIZE get_bsize(const VP10_COMMON *cm, int mb_row, int mb_col) {
- if (2 * mb_col + 1 < cm->mi_cols) {
- return 2 * mb_row + 1 < cm->mi_rows ? BLOCK_16X16
- : BLOCK_16X8;
- } else {
- return 2 * mb_row + 1 < cm->mi_rows ? BLOCK_8X16
- : BLOCK_8X8;
- }
-}
-
-static int find_fp_qindex(vpx_bit_depth_t bit_depth) {
- int i;
-
- for (i = 0; i < QINDEX_RANGE; ++i)
- if (vp10_convert_qindex_to_q(i, bit_depth) >= FIRST_PASS_Q)
- break;
-
- if (i == QINDEX_RANGE)
- i--;
-
- return i;
-}
-
-static void set_first_pass_params(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- if (!cpi->refresh_alt_ref_frame &&
- (cm->current_video_frame == 0 ||
- (cpi->frame_flags & FRAMEFLAGS_KEY))) {
- cm->frame_type = KEY_FRAME;
- } else {
- cm->frame_type = INTER_FRAME;
- }
- // Do not use periodic key frames.
- cpi->rc.frames_to_key = INT_MAX;
-}
-
-#define UL_INTRA_THRESH 50
-#define INVALID_ROW -1
-void vp10_first_pass(VP10_COMP *cpi, const struct lookahead_entry *source) {
- int mb_row, mb_col;
- MACROBLOCK *const x = &cpi->td.mb;
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- TileInfo tile;
- struct macroblock_plane *const p = x->plane;
- struct macroblockd_plane *const pd = xd->plane;
- const PICK_MODE_CONTEXT *ctx = &cpi->td.pc_root->none;
- int i;
-
- int recon_yoffset, recon_uvoffset;
- int64_t intra_error = 0;
- int64_t coded_error = 0;
- int64_t sr_coded_error = 0;
-
- int sum_mvr = 0, sum_mvc = 0;
- int sum_mvr_abs = 0, sum_mvc_abs = 0;
- int64_t sum_mvrs = 0, sum_mvcs = 0;
- int mvcount = 0;
- int intercount = 0;
- int second_ref_count = 0;
- const int intrapenalty = INTRA_MODE_PENALTY;
- double neutral_count;
- int intra_skip_count = 0;
- int image_data_start_row = INVALID_ROW;
- int new_mv_count = 0;
- int sum_in_vectors = 0;
- MV lastmv = {0, 0};
- TWO_PASS *twopass = &cpi->twopass;
- const MV zero_mv = {0, 0};
- int recon_y_stride, recon_uv_stride, uv_mb_height;
-
- YV12_BUFFER_CONFIG *const lst_yv12 = get_ref_frame_buffer(cpi, LAST_FRAME);
- YV12_BUFFER_CONFIG *gld_yv12 = get_ref_frame_buffer(cpi, GOLDEN_FRAME);
- YV12_BUFFER_CONFIG *const new_yv12 = get_frame_new_buffer(cm);
- const YV12_BUFFER_CONFIG *first_ref_buf = lst_yv12;
- double intra_factor;
- double brightness_factor;
- BufferPool *const pool = cm->buffer_pool;
-
- // First pass code requires valid last and new frame buffers.
- assert(new_yv12 != NULL);
- assert(frame_is_intra_only(cm) || (lst_yv12 != NULL));
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- vp10_zero_array(cpi->twopass.frame_mb_stats_buf, cm->initial_mbs);
- }
-#endif
-
- vpx_clear_system_state();
-
- intra_factor = 0.0;
- brightness_factor = 0.0;
- neutral_count = 0.0;
-
- set_first_pass_params(cpi);
- vp10_set_quantizer(cm, find_fp_qindex(cm->bit_depth));
-
- vp10_setup_block_planes(&x->e_mbd, cm->subsampling_x, cm->subsampling_y);
-
- vp10_setup_src_planes(x, cpi->Source, 0, 0);
- vp10_setup_dst_planes(xd->plane, new_yv12, 0, 0);
-
- if (!frame_is_intra_only(cm)) {
- vp10_setup_pre_planes(xd, 0, first_ref_buf, 0, 0, NULL);
- }
-
- xd->mi = cm->mi_grid_visible;
- xd->mi[0] = cm->mi;
-
- vp10_frame_init_quantizer(cpi);
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- p[i].coeff = ctx->coeff_pbuf[i][1];
- p[i].qcoeff = ctx->qcoeff_pbuf[i][1];
- pd[i].dqcoeff = ctx->dqcoeff_pbuf[i][1];
- p[i].eobs = ctx->eobs_pbuf[i][1];
- }
- x->skip_recode = 0;
-
- vp10_init_mv_probs(cm);
- vp10_initialize_rd_consts(cpi);
-
- // Tiling is ignored in the first pass.
- vp10_tile_init(&tile, cm, 0, 0);
-
- recon_y_stride = new_yv12->y_stride;
- recon_uv_stride = new_yv12->uv_stride;
- uv_mb_height = 16 >> (new_yv12->y_height > new_yv12->uv_height);
-
- for (mb_row = 0; mb_row < cm->mb_rows; ++mb_row) {
- MV best_ref_mv = {0, 0};
-
- // Reset above block coeffs.
- xd->up_available = (mb_row != 0);
- recon_yoffset = (mb_row * recon_y_stride * 16);
- recon_uvoffset = (mb_row * recon_uv_stride * uv_mb_height);
-
- // Set up limit values for motion vectors to prevent them extending
- // outside the UMV borders.
- x->mv_row_min = -((mb_row * 16) + BORDER_MV_PIXELS_B16);
- x->mv_row_max = ((cm->mb_rows - 1 - mb_row) * 16)
- + BORDER_MV_PIXELS_B16;
-
- for (mb_col = 0; mb_col < cm->mb_cols; ++mb_col) {
- int this_error;
- const int use_dc_pred = (mb_col || mb_row) && (!mb_col || !mb_row);
- const BLOCK_SIZE bsize = get_bsize(cm, mb_row, mb_col);
- double log_intra;
- int level_sample;
-
-#if CONFIG_FP_MB_STATS
- const int mb_index = mb_row * cm->mb_cols + mb_col;
-#endif
-
- vpx_clear_system_state();
-
- xd->plane[0].dst.buf = new_yv12->y_buffer + recon_yoffset;
- xd->plane[1].dst.buf = new_yv12->u_buffer + recon_uvoffset;
- xd->plane[2].dst.buf = new_yv12->v_buffer + recon_uvoffset;
- xd->left_available = (mb_col != 0);
- xd->mi[0]->mbmi.sb_type = bsize;
- xd->mi[0]->mbmi.ref_frame[0] = INTRA_FRAME;
- set_mi_row_col(xd, &tile,
- mb_row << 1, num_8x8_blocks_high_lookup[bsize],
- mb_col << 1, num_8x8_blocks_wide_lookup[bsize],
- cm->mi_rows, cm->mi_cols);
-
- // Do intra 16x16 prediction.
- xd->mi[0]->mbmi.segment_id = 0;
- xd->mi[0]->mbmi.mode = DC_PRED;
- xd->mi[0]->mbmi.tx_size = use_dc_pred ?
- (bsize >= BLOCK_16X16 ? TX_16X16 : TX_8X8) : TX_4X4;
- vp10_encode_intra_block_plane(x, bsize, 0);
- this_error = vpx_get_mb_ss(x->plane[0].src_diff);
-
- // Keep a record of blocks that have almost no intra error residual
- // (i.e. are in effect completely flat and untextured in the intra
- // domain). In natural videos this is uncommon, but it is much more
- // common in animations, graphics and screen content, so may be used
- // as a signal to detect these types of content.
- if (this_error < UL_INTRA_THRESH) {
- ++intra_skip_count;
- } else if ((mb_col > 0) && (image_data_start_row == INVALID_ROW)) {
- image_data_start_row = mb_row;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- switch (cm->bit_depth) {
- case VPX_BITS_8:
- break;
- case VPX_BITS_10:
- this_error >>= 4;
- break;
- case VPX_BITS_12:
- this_error >>= 8;
- break;
- default:
- assert(0 && "cm->bit_depth should be VPX_BITS_8, "
- "VPX_BITS_10 or VPX_BITS_12");
- return;
- }
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- vpx_clear_system_state();
- log_intra = log(this_error + 1.0);
- if (log_intra < 10.0)
- intra_factor += 1.0 + ((10.0 - log_intra) * 0.05);
- else
- intra_factor += 1.0;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth)
- level_sample = CONVERT_TO_SHORTPTR(x->plane[0].src.buf)[0];
- else
- level_sample = x->plane[0].src.buf[0];
-#else
- level_sample = x->plane[0].src.buf[0];
-#endif
- if ((level_sample < DARK_THRESH) && (log_intra < 9.0))
- brightness_factor += 1.0 + (0.01 * (DARK_THRESH - level_sample));
- else
- brightness_factor += 1.0;
-
- // Intrapenalty below deals with situations where the intra and inter
- // error scores are very low (e.g. a plain black frame).
- // We do not have special cases in first pass for 0,0 and nearest etc so
- // all inter modes carry an overhead cost estimate for the mv.
- // When the error score is very low this causes us to pick all or lots of
- // INTRA modes and throw lots of key frames.
- // This penalty adds a cost matching that of a 0,0 mv to the intra case.
- this_error += intrapenalty;
-
- // Accumulate the intra error.
- intra_error += (int64_t)this_error;
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- // initialization
- cpi->twopass.frame_mb_stats_buf[mb_index] = 0;
- }
-#endif
-
- // Set up limit values for motion vectors to prevent them extending
- // outside the UMV borders.
- x->mv_col_min = -((mb_col * 16) + BORDER_MV_PIXELS_B16);
- x->mv_col_max = ((cm->mb_cols - 1 - mb_col) * 16) + BORDER_MV_PIXELS_B16;
-
- // Other than for the first frame do a motion search.
- if (cm->current_video_frame > 0) {
- int tmp_err, motion_error, raw_motion_error;
- // Assume 0,0 motion with no mv overhead.
- MV mv = {0, 0} , tmp_mv = {0, 0};
- struct buf_2d unscaled_last_source_buf_2d;
-
- xd->plane[0].pre[0].buf = first_ref_buf->y_buffer + recon_yoffset;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- motion_error = highbd_get_prediction_error(
- bsize, &x->plane[0].src, &xd->plane[0].pre[0], xd->bd);
- } else {
- motion_error = get_prediction_error(
- bsize, &x->plane[0].src, &xd->plane[0].pre[0]);
- }
-#else
- motion_error = get_prediction_error(
- bsize, &x->plane[0].src, &xd->plane[0].pre[0]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Compute the motion error of the 0,0 motion using the last source
- // frame as the reference. Skip the further motion search on
- // reconstructed frame if this error is small.
- unscaled_last_source_buf_2d.buf =
- cpi->unscaled_last_source->y_buffer + recon_yoffset;
- unscaled_last_source_buf_2d.stride =
- cpi->unscaled_last_source->y_stride;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- raw_motion_error = highbd_get_prediction_error(
- bsize, &x->plane[0].src, &unscaled_last_source_buf_2d, xd->bd);
- } else {
- raw_motion_error = get_prediction_error(
- bsize, &x->plane[0].src, &unscaled_last_source_buf_2d);
- }
-#else
- raw_motion_error = get_prediction_error(
- bsize, &x->plane[0].src, &unscaled_last_source_buf_2d);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // TODO(pengchong): Replace the hard-coded threshold
- if (raw_motion_error > 25) {
- // Test last reference frame using the previous best mv as the
- // starting point (best reference) for the search.
- first_pass_motion_search(cpi, x, &best_ref_mv, &mv, &motion_error);
-
- // If the current best reference mv is not centered on 0,0 then do a
- // 0,0 based search as well.
- if (!is_zero_mv(&best_ref_mv)) {
- tmp_err = INT_MAX;
- first_pass_motion_search(cpi, x, &zero_mv, &tmp_mv, &tmp_err);
-
- if (tmp_err < motion_error) {
- motion_error = tmp_err;
- mv = tmp_mv;
- }
- }
-
- // Search in an older reference frame.
- if ((cm->current_video_frame > 1) && gld_yv12 != NULL) {
- // Assume 0,0 motion with no mv overhead.
- int gf_motion_error;
-
- xd->plane[0].pre[0].buf = gld_yv12->y_buffer + recon_yoffset;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- gf_motion_error = highbd_get_prediction_error(
- bsize, &x->plane[0].src, &xd->plane[0].pre[0], xd->bd);
- } else {
- gf_motion_error = get_prediction_error(
- bsize, &x->plane[0].src, &xd->plane[0].pre[0]);
- }
-#else
- gf_motion_error = get_prediction_error(
- bsize, &x->plane[0].src, &xd->plane[0].pre[0]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- first_pass_motion_search(cpi, x, &zero_mv, &tmp_mv,
- &gf_motion_error);
-
- if (gf_motion_error < motion_error && gf_motion_error < this_error)
- ++second_ref_count;
-
- // Reset to last frame as reference buffer.
- xd->plane[0].pre[0].buf = first_ref_buf->y_buffer + recon_yoffset;
- xd->plane[1].pre[0].buf = first_ref_buf->u_buffer + recon_uvoffset;
- xd->plane[2].pre[0].buf = first_ref_buf->v_buffer + recon_uvoffset;
-
- // In accumulating a score for the older reference frame take the
- // best of the motion predicted score and the intra coded error
- // (just as will be done for) accumulation of "coded_error" for
- // the last frame.
- if (gf_motion_error < this_error)
- sr_coded_error += gf_motion_error;
- else
- sr_coded_error += this_error;
- } else {
- sr_coded_error += motion_error;
- }
- } else {
- sr_coded_error += motion_error;
- }
-
- // Start by assuming that intra mode is best.
- best_ref_mv.row = 0;
- best_ref_mv.col = 0;
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- // intra predication statistics
- cpi->twopass.frame_mb_stats_buf[mb_index] = 0;
- cpi->twopass.frame_mb_stats_buf[mb_index] |= FPMB_DCINTRA_MASK;
- cpi->twopass.frame_mb_stats_buf[mb_index] |= FPMB_MOTION_ZERO_MASK;
- if (this_error > FPMB_ERROR_LARGE_TH) {
- cpi->twopass.frame_mb_stats_buf[mb_index] |= FPMB_ERROR_LARGE_MASK;
- } else if (this_error < FPMB_ERROR_SMALL_TH) {
- cpi->twopass.frame_mb_stats_buf[mb_index] |= FPMB_ERROR_SMALL_MASK;
- }
- }
-#endif
-
- if (motion_error <= this_error) {
- vpx_clear_system_state();
-
- // Keep a count of cases where the inter and intra were very close
- // and very low. This helps with scene cut detection for example in
- // cropped clips with black bars at the sides or top and bottom.
- if (((this_error - intrapenalty) * 9 <= motion_error * 10) &&
- (this_error < (2 * intrapenalty))) {
- neutral_count += 1.0;
- // Also track cases where the intra is not much worse than the inter
- // and use this in limiting the GF/arf group length.
- } else if ((this_error > NCOUNT_INTRA_THRESH) &&
- (this_error < (NCOUNT_INTRA_FACTOR * motion_error))) {
- neutral_count += (double)motion_error /
- DOUBLE_DIVIDE_CHECK((double)this_error);
- }
-
- mv.row *= 8;
- mv.col *= 8;
- this_error = motion_error;
- xd->mi[0]->mbmi.mode = NEWMV;
- xd->mi[0]->mbmi.mv[0].as_mv = mv;
- xd->mi[0]->mbmi.tx_size = TX_4X4;
- xd->mi[0]->mbmi.ref_frame[0] = LAST_FRAME;
- xd->mi[0]->mbmi.ref_frame[1] = NONE;
- vp10_build_inter_predictors_sby(xd, mb_row << 1, mb_col << 1, bsize);
- vp10_encode_sby_pass1(x, bsize);
- sum_mvr += mv.row;
- sum_mvr_abs += abs(mv.row);
- sum_mvc += mv.col;
- sum_mvc_abs += abs(mv.col);
- sum_mvrs += mv.row * mv.row;
- sum_mvcs += mv.col * mv.col;
- ++intercount;
-
- best_ref_mv = mv;
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- // inter predication statistics
- cpi->twopass.frame_mb_stats_buf[mb_index] = 0;
- cpi->twopass.frame_mb_stats_buf[mb_index] &= ~FPMB_DCINTRA_MASK;
- cpi->twopass.frame_mb_stats_buf[mb_index] |= FPMB_MOTION_ZERO_MASK;
- if (this_error > FPMB_ERROR_LARGE_TH) {
- cpi->twopass.frame_mb_stats_buf[mb_index] |=
- FPMB_ERROR_LARGE_MASK;
- } else if (this_error < FPMB_ERROR_SMALL_TH) {
- cpi->twopass.frame_mb_stats_buf[mb_index] |=
- FPMB_ERROR_SMALL_MASK;
- }
- }
-#endif
-
- if (!is_zero_mv(&mv)) {
- ++mvcount;
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- cpi->twopass.frame_mb_stats_buf[mb_index] &=
- ~FPMB_MOTION_ZERO_MASK;
- // check estimated motion direction
- if (mv.as_mv.col > 0 && mv.as_mv.col >= abs(mv.as_mv.row)) {
- // right direction
- cpi->twopass.frame_mb_stats_buf[mb_index] |=
- FPMB_MOTION_RIGHT_MASK;
- } else if (mv.as_mv.row < 0 &&
- abs(mv.as_mv.row) >= abs(mv.as_mv.col)) {
- // up direction
- cpi->twopass.frame_mb_stats_buf[mb_index] |=
- FPMB_MOTION_UP_MASK;
- } else if (mv.as_mv.col < 0 &&
- abs(mv.as_mv.col) >= abs(mv.as_mv.row)) {
- // left direction
- cpi->twopass.frame_mb_stats_buf[mb_index] |=
- FPMB_MOTION_LEFT_MASK;
- } else {
- // down direction
- cpi->twopass.frame_mb_stats_buf[mb_index] |=
- FPMB_MOTION_DOWN_MASK;
- }
- }
-#endif
-
- // Non-zero vector, was it different from the last non zero vector?
- if (!is_equal_mv(&mv, &lastmv))
- ++new_mv_count;
- lastmv = mv;
-
- // Does the row vector point inwards or outwards?
- if (mb_row < cm->mb_rows / 2) {
- if (mv.row > 0)
- --sum_in_vectors;
- else if (mv.row < 0)
- ++sum_in_vectors;
- } else if (mb_row > cm->mb_rows / 2) {
- if (mv.row > 0)
- ++sum_in_vectors;
- else if (mv.row < 0)
- --sum_in_vectors;
- }
-
- // Does the col vector point inwards or outwards?
- if (mb_col < cm->mb_cols / 2) {
- if (mv.col > 0)
- --sum_in_vectors;
- else if (mv.col < 0)
- ++sum_in_vectors;
- } else if (mb_col > cm->mb_cols / 2) {
- if (mv.col > 0)
- ++sum_in_vectors;
- else if (mv.col < 0)
- --sum_in_vectors;
- }
- }
- }
- } else {
- sr_coded_error += (int64_t)this_error;
- }
- coded_error += (int64_t)this_error;
-
- // Adjust to the next column of MBs.
- x->plane[0].src.buf += 16;
- x->plane[1].src.buf += uv_mb_height;
- x->plane[2].src.buf += uv_mb_height;
-
- recon_yoffset += 16;
- recon_uvoffset += uv_mb_height;
- }
-
- // Adjust to the next row of MBs.
- x->plane[0].src.buf += 16 * x->plane[0].src.stride - 16 * cm->mb_cols;
- x->plane[1].src.buf += uv_mb_height * x->plane[1].src.stride -
- uv_mb_height * cm->mb_cols;
- x->plane[2].src.buf += uv_mb_height * x->plane[1].src.stride -
- uv_mb_height * cm->mb_cols;
-
- vpx_clear_system_state();
- }
-
- // Clamp the image start to rows/2. This number of rows is discarded top
- // and bottom as dead data so rows / 2 means the frame is blank.
- if ((image_data_start_row > cm->mb_rows / 2) ||
- (image_data_start_row == INVALID_ROW)) {
- image_data_start_row = cm->mb_rows / 2;
- }
- // Exclude any image dead zone
- if (image_data_start_row > 0) {
- intra_skip_count =
- VPXMAX(0, intra_skip_count - (image_data_start_row * cm->mb_cols * 2));
- }
-
- {
- FIRSTPASS_STATS fps;
- // The minimum error here insures some bit allocation to frames even
- // in static regions. The allocation per MB declines for larger formats
- // where the typical "real" energy per MB also falls.
- // Initial estimate here uses sqrt(mbs) to define the min_err, where the
- // number of mbs is proportional to the image area.
- const int num_mbs = (cpi->oxcf.resize_mode != RESIZE_NONE)
- ? cpi->initial_mbs : cpi->common.MBs;
- const double min_err = 200 * sqrt(num_mbs);
-
- intra_factor = intra_factor / (double)num_mbs;
- brightness_factor = brightness_factor / (double)num_mbs;
- fps.weight = intra_factor * brightness_factor;
-
- fps.frame = cm->current_video_frame;
- fps.coded_error = (double)(coded_error >> 8) + min_err;
- fps.sr_coded_error = (double)(sr_coded_error >> 8) + min_err;
- fps.intra_error = (double)(intra_error >> 8) + min_err;
- fps.count = 1.0;
- fps.pcnt_inter = (double)intercount / num_mbs;
- fps.pcnt_second_ref = (double)second_ref_count / num_mbs;
- fps.pcnt_neutral = (double)neutral_count / num_mbs;
- fps.intra_skip_pct = (double)intra_skip_count / num_mbs;
- fps.inactive_zone_rows = (double)image_data_start_row;
- fps.inactive_zone_cols = (double)0; // TODO(paulwilkins): fix
-
- if (mvcount > 0) {
- fps.MVr = (double)sum_mvr / mvcount;
- fps.mvr_abs = (double)sum_mvr_abs / mvcount;
- fps.MVc = (double)sum_mvc / mvcount;
- fps.mvc_abs = (double)sum_mvc_abs / mvcount;
- fps.MVrv = ((double)sum_mvrs -
- ((double)sum_mvr * sum_mvr / mvcount)) / mvcount;
- fps.MVcv = ((double)sum_mvcs -
- ((double)sum_mvc * sum_mvc / mvcount)) / mvcount;
- fps.mv_in_out_count = (double)sum_in_vectors / (mvcount * 2);
- fps.new_mv_count = new_mv_count;
- fps.pcnt_motion = (double)mvcount / num_mbs;
- } else {
- fps.MVr = 0.0;
- fps.mvr_abs = 0.0;
- fps.MVc = 0.0;
- fps.mvc_abs = 0.0;
- fps.MVrv = 0.0;
- fps.MVcv = 0.0;
- fps.mv_in_out_count = 0.0;
- fps.new_mv_count = 0.0;
- fps.pcnt_motion = 0.0;
- }
-
- // TODO(paulwilkins): Handle the case when duration is set to 0, or
- // something less than the full time between subsequent values of
- // cpi->source_time_stamp.
- fps.duration = (double)(source->ts_end - source->ts_start);
-
- // Don't want to do output stats with a stack variable!
- twopass->this_frame_stats = fps;
- output_stats(&twopass->this_frame_stats, cpi->output_pkt_list);
- accumulate_stats(&twopass->total_stats, &fps);
-
-#if CONFIG_FP_MB_STATS
- if (cpi->use_fp_mb_stats) {
- output_fpmb_stats(twopass->frame_mb_stats_buf, cm, cpi->output_pkt_list);
- }
-#endif
- }
-
- // Copy the previous Last Frame back into gf and and arf buffers if
- // the prediction is good enough... but also don't allow it to lag too far.
- if ((twopass->sr_update_lag > 3) ||
- ((cm->current_video_frame > 0) &&
- (twopass->this_frame_stats.pcnt_inter > 0.20) &&
- ((twopass->this_frame_stats.intra_error /
- DOUBLE_DIVIDE_CHECK(twopass->this_frame_stats.coded_error)) > 2.0))) {
- if (gld_yv12 != NULL) {
- ref_cnt_fb(pool->frame_bufs, &cm->ref_frame_map[cpi->gld_fb_idx],
- cm->ref_frame_map[cpi->lst_fb_idx]);
- }
- twopass->sr_update_lag = 1;
- } else {
- ++twopass->sr_update_lag;
- }
-
- vpx_extend_frame_borders(new_yv12);
-
- // The frame we just compressed now becomes the last frame.
- ref_cnt_fb(pool->frame_bufs, &cm->ref_frame_map[cpi->lst_fb_idx],
- cm->new_fb_idx);
-
- // Special case for the first frame. Copy into the GF buffer as a second
- // reference.
- if (cm->current_video_frame == 0 && cpi->gld_fb_idx != INVALID_IDX) {
- ref_cnt_fb(pool->frame_bufs, &cm->ref_frame_map[cpi->gld_fb_idx],
- cm->ref_frame_map[cpi->lst_fb_idx]);
- }
-
- // Use this to see what the first pass reconstruction looks like.
- if (0) {
- char filename[512];
- FILE *recon_file;
- snprintf(filename, sizeof(filename), "enc%04d.yuv",
- (int)cm->current_video_frame);
-
- if (cm->current_video_frame == 0)
- recon_file = fopen(filename, "wb");
- else
- recon_file = fopen(filename, "ab");
-
- (void)fwrite(lst_yv12->buffer_alloc, lst_yv12->frame_size, 1, recon_file);
- fclose(recon_file);
- }
-
- ++cm->current_video_frame;
-}
-
-static double calc_correction_factor(double err_per_mb,
- double err_divisor,
- double pt_low,
- double pt_high,
- int q,
- vpx_bit_depth_t bit_depth) {
- const double error_term = err_per_mb / err_divisor;
-
- // Adjustment based on actual quantizer to power term.
- const double power_term =
- VPXMIN(vp10_convert_qindex_to_q(q, bit_depth) * 0.01 + pt_low, pt_high);
-
- // Calculate correction factor.
- if (power_term < 1.0)
- assert(error_term >= 0.0);
-
- return fclamp(pow(error_term, power_term), 0.05, 5.0);
-}
-
-// Larger image formats are expected to be a little harder to code relatively
-// given the same prediction error score. This in part at least relates to the
-// increased size and hence coding cost of motion vectors.
-#define EDIV_SIZE_FACTOR 800
-
-static int get_twopass_worst_quality(const VP10_COMP *cpi,
- const double section_err,
- double inactive_zone,
- int section_target_bandwidth,
- double group_weight_factor) {
- const RATE_CONTROL *const rc = &cpi->rc;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
-
- inactive_zone = fclamp(inactive_zone, 0.0, 1.0);
-
- if (section_target_bandwidth <= 0) {
- return rc->worst_quality; // Highest value allowed
- } else {
- const int num_mbs = (cpi->oxcf.resize_mode != RESIZE_NONE)
- ? cpi->initial_mbs : cpi->common.MBs;
- const int active_mbs = VPXMAX(1, num_mbs - (int)(num_mbs * inactive_zone));
- const double av_err_per_mb = section_err / active_mbs;
- const double speed_term = 1.0 + 0.04 * oxcf->speed;
- const double ediv_size_correction = (double)num_mbs / EDIV_SIZE_FACTOR;
- const int target_norm_bits_per_mb = ((uint64_t)section_target_bandwidth <<
- BPER_MB_NORMBITS) / active_mbs;
-
- int q;
-
- // Try and pick a max Q that will be high enough to encode the
- // content at the given rate.
- for (q = rc->best_quality; q < rc->worst_quality; ++q) {
- const double factor =
- calc_correction_factor(av_err_per_mb,
- ERR_DIVISOR - ediv_size_correction,
- FACTOR_PT_LOW, FACTOR_PT_HIGH, q,
- cpi->common.bit_depth);
- const int bits_per_mb =
- vp10_rc_bits_per_mb(INTER_FRAME, q,
- factor * speed_term * group_weight_factor,
- cpi->common.bit_depth);
- if (bits_per_mb <= target_norm_bits_per_mb)
- break;
- }
-
- // Restriction on active max q for constrained quality mode.
- if (cpi->oxcf.rc_mode == VPX_CQ)
- q = VPXMAX(q, oxcf->cq_level);
- return q;
- }
-}
-
-static void setup_rf_level_maxq(VP10_COMP *cpi) {
- int i;
- RATE_CONTROL *const rc = &cpi->rc;
- for (i = INTER_NORMAL; i < RATE_FACTOR_LEVELS; ++i) {
- int qdelta = vp10_frame_type_qdelta(cpi, i, rc->worst_quality);
- rc->rf_level_maxq[i] = VPXMAX(rc->worst_quality + qdelta, rc->best_quality);
- }
-}
-
-void vp10_init_subsampling(VP10_COMP *cpi) {
- const VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- const int w = cm->width;
- const int h = cm->height;
- int i;
-
- for (i = 0; i < FRAME_SCALE_STEPS; ++i) {
- // Note: Frames with odd-sized dimensions may result from this scaling.
- rc->frame_width[i] = (w * 16) / frame_scale_factor[i];
- rc->frame_height[i] = (h * 16) / frame_scale_factor[i];
- }
-
- setup_rf_level_maxq(cpi);
-}
-
-void vp10_calculate_coded_size(VP10_COMP *cpi,
- int *scaled_frame_width,
- int *scaled_frame_height) {
- RATE_CONTROL *const rc = &cpi->rc;
- *scaled_frame_width = rc->frame_width[rc->frame_size_selector];
- *scaled_frame_height = rc->frame_height[rc->frame_size_selector];
-}
-
-void vp10_init_second_pass(VP10_COMP *cpi) {
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- TWO_PASS *const twopass = &cpi->twopass;
- double frame_rate;
- FIRSTPASS_STATS *stats;
-
- zero_stats(&twopass->total_stats);
- zero_stats(&twopass->total_left_stats);
-
- if (!twopass->stats_in_end)
- return;
-
- stats = &twopass->total_stats;
-
- *stats = *twopass->stats_in_end;
- twopass->total_left_stats = *stats;
-
- frame_rate = 10000000.0 * stats->count / stats->duration;
- // Each frame can have a different duration, as the frame rate in the source
- // isn't guaranteed to be constant. The frame rate prior to the first frame
- // encoded in the second pass is a guess. However, the sum duration is not.
- // It is calculated based on the actual durations of all frames from the
- // first pass.
- vp10_new_framerate(cpi, frame_rate);
- twopass->bits_left = (int64_t)(stats->duration * oxcf->target_bandwidth /
- 10000000.0);
-
- // This variable monitors how far behind the second ref update is lagging.
- twopass->sr_update_lag = 1;
-
- // Scan the first pass file and calculate a modified total error based upon
- // the bias/power function used to allocate bits.
- {
- const double avg_error = stats->coded_error /
- DOUBLE_DIVIDE_CHECK(stats->count);
- const FIRSTPASS_STATS *s = twopass->stats_in;
- double modified_error_total = 0.0;
- twopass->modified_error_min = (avg_error *
- oxcf->two_pass_vbrmin_section) / 100;
- twopass->modified_error_max = (avg_error *
- oxcf->two_pass_vbrmax_section) / 100;
- while (s < twopass->stats_in_end) {
- modified_error_total += calculate_modified_err(cpi, twopass, oxcf, s);
- ++s;
- }
- twopass->modified_error_left = modified_error_total;
- }
-
- // Reset the vbr bits off target counters
- cpi->rc.vbr_bits_off_target = 0;
- cpi->rc.vbr_bits_off_target_fast = 0;
-
- cpi->rc.rate_error_estimate = 0;
-
- // Static sequence monitor variables.
- twopass->kf_zeromotion_pct = 100;
- twopass->last_kfgroup_zeromotion_pct = 100;
-
- if (oxcf->resize_mode != RESIZE_NONE) {
- vp10_init_subsampling(cpi);
- }
-}
-
-#define SR_DIFF_PART 0.0015
-#define MOTION_AMP_PART 0.003
-#define INTRA_PART 0.005
-#define DEFAULT_DECAY_LIMIT 0.75
-#define LOW_SR_DIFF_TRHESH 0.1
-#define SR_DIFF_MAX 128.0
-
-static double get_sr_decay_rate(const VP10_COMP *cpi,
- const FIRSTPASS_STATS *frame) {
- const int num_mbs = (cpi->oxcf.resize_mode != RESIZE_NONE)
- ? cpi->initial_mbs : cpi->common.MBs;
- double sr_diff =
- (frame->sr_coded_error - frame->coded_error) / num_mbs;
- double sr_decay = 1.0;
- double modified_pct_inter;
- double modified_pcnt_intra;
- const double motion_amplitude_factor =
- frame->pcnt_motion * ((frame->mvc_abs + frame->mvr_abs) / 2);
-
- modified_pct_inter = frame->pcnt_inter;
- if ((frame->intra_error / DOUBLE_DIVIDE_CHECK(frame->coded_error)) <
- (double)NCOUNT_FRAME_II_THRESH) {
- modified_pct_inter = frame->pcnt_inter - frame->pcnt_neutral;
- }
- modified_pcnt_intra = 100 * (1.0 - modified_pct_inter);
-
-
- if ((sr_diff > LOW_SR_DIFF_TRHESH)) {
- sr_diff = VPXMIN(sr_diff, SR_DIFF_MAX);
- sr_decay = 1.0 - (SR_DIFF_PART * sr_diff) -
- (MOTION_AMP_PART * motion_amplitude_factor) -
- (INTRA_PART * modified_pcnt_intra);
- }
- return VPXMAX(sr_decay, VPXMIN(DEFAULT_DECAY_LIMIT, modified_pct_inter));
-}
-
-// This function gives an estimate of how badly we believe the prediction
-// quality is decaying from frame to frame.
-static double get_zero_motion_factor(const VP10_COMP *cpi,
- const FIRSTPASS_STATS *frame) {
- const double zero_motion_pct = frame->pcnt_inter -
- frame->pcnt_motion;
- double sr_decay = get_sr_decay_rate(cpi, frame);
- return VPXMIN(sr_decay, zero_motion_pct);
-}
-
-#define ZM_POWER_FACTOR 0.75
-
-static double get_prediction_decay_rate(const VP10_COMP *cpi,
- const FIRSTPASS_STATS *next_frame) {
- const double sr_decay_rate = get_sr_decay_rate(cpi, next_frame);
- const double zero_motion_factor =
- (0.95 * pow((next_frame->pcnt_inter - next_frame->pcnt_motion),
- ZM_POWER_FACTOR));
-
- return VPXMAX(zero_motion_factor,
- (sr_decay_rate + ((1.0 - sr_decay_rate) * zero_motion_factor)));
-}
-
-// Function to test for a condition where a complex transition is followed
-// by a static section. For example in slide shows where there is a fade
-// between slides. This is to help with more optimal kf and gf positioning.
-static int detect_transition_to_still(VP10_COMP *cpi,
- int frame_interval, int still_interval,
- double loop_decay_rate,
- double last_decay_rate) {
- TWO_PASS *const twopass = &cpi->twopass;
- RATE_CONTROL *const rc = &cpi->rc;
-
- // Break clause to detect very still sections after motion
- // For example a static image after a fade or other transition
- // instead of a clean scene cut.
- if (frame_interval > rc->min_gf_interval &&
- loop_decay_rate >= 0.999 &&
- last_decay_rate < 0.9) {
- int j;
-
- // Look ahead a few frames to see if static condition persists...
- for (j = 0; j < still_interval; ++j) {
- const FIRSTPASS_STATS *stats = &twopass->stats_in[j];
- if (stats >= twopass->stats_in_end)
- break;
-
- if (stats->pcnt_inter - stats->pcnt_motion < 0.999)
- break;
- }
-
- // Only if it does do we signal a transition to still.
- return j == still_interval;
- }
-
- return 0;
-}
-
-// This function detects a flash through the high relative pcnt_second_ref
-// score in the frame following a flash frame. The offset passed in should
-// reflect this.
-static int detect_flash(const TWO_PASS *twopass, int offset) {
- const FIRSTPASS_STATS *const next_frame = read_frame_stats(twopass, offset);
-
- // What we are looking for here is a situation where there is a
- // brief break in prediction (such as a flash) but subsequent frames
- // are reasonably well predicted by an earlier (pre flash) frame.
- // The recovery after a flash is indicated by a high pcnt_second_ref
- // compared to pcnt_inter.
- return next_frame != NULL &&
- next_frame->pcnt_second_ref > next_frame->pcnt_inter &&
- next_frame->pcnt_second_ref >= 0.5;
-}
-
-// Update the motion related elements to the GF arf boost calculation.
-static void accumulate_frame_motion_stats(const FIRSTPASS_STATS *stats,
- double *mv_in_out,
- double *mv_in_out_accumulator,
- double *abs_mv_in_out_accumulator,
- double *mv_ratio_accumulator) {
- const double pct = stats->pcnt_motion;
-
- // Accumulate Motion In/Out of frame stats.
- *mv_in_out = stats->mv_in_out_count * pct;
- *mv_in_out_accumulator += *mv_in_out;
- *abs_mv_in_out_accumulator += fabs(*mv_in_out);
-
- // Accumulate a measure of how uniform (or conversely how random) the motion
- // field is (a ratio of abs(mv) / mv).
- if (pct > 0.05) {
- const double mvr_ratio = fabs(stats->mvr_abs) /
- DOUBLE_DIVIDE_CHECK(fabs(stats->MVr));
- const double mvc_ratio = fabs(stats->mvc_abs) /
- DOUBLE_DIVIDE_CHECK(fabs(stats->MVc));
-
- *mv_ratio_accumulator += pct * (mvr_ratio < stats->mvr_abs ?
- mvr_ratio : stats->mvr_abs);
- *mv_ratio_accumulator += pct * (mvc_ratio < stats->mvc_abs ?
- mvc_ratio : stats->mvc_abs);
- }
-}
-
-#define BASELINE_ERR_PER_MB 1000.0
-static double calc_frame_boost(VP10_COMP *cpi,
- const FIRSTPASS_STATS *this_frame,
- double this_frame_mv_in_out,
- double max_boost) {
- double frame_boost;
- const double lq =
- vp10_convert_qindex_to_q(cpi->rc.avg_frame_qindex[INTER_FRAME],
- cpi->common.bit_depth);
- const double boost_q_correction = VPXMIN((0.5 + (lq * 0.015)), 1.5);
- int num_mbs = (cpi->oxcf.resize_mode != RESIZE_NONE)
- ? cpi->initial_mbs : cpi->common.MBs;
-
- // Correct for any inactive region in the image
- num_mbs = (int)VPXMAX(1, num_mbs * calculate_active_area(cpi, this_frame));
-
- // Underlying boost factor is based on inter error ratio.
- frame_boost = (BASELINE_ERR_PER_MB * num_mbs) /
- DOUBLE_DIVIDE_CHECK(this_frame->coded_error);
- frame_boost = frame_boost * BOOST_FACTOR * boost_q_correction;
-
- // Increase boost for frames where new data coming into frame (e.g. zoom out).
- // Slightly reduce boost if there is a net balance of motion out of the frame
- // (zoom in). The range for this_frame_mv_in_out is -1.0 to +1.0.
- if (this_frame_mv_in_out > 0.0)
- frame_boost += frame_boost * (this_frame_mv_in_out * 2.0);
- // In the extreme case the boost is halved.
- else
- frame_boost += frame_boost * (this_frame_mv_in_out / 2.0);
-
- return VPXMIN(frame_boost, max_boost * boost_q_correction);
-}
-
-static int calc_arf_boost(VP10_COMP *cpi, int offset,
- int f_frames, int b_frames,
- int *f_boost, int *b_boost) {
- TWO_PASS *const twopass = &cpi->twopass;
- int i;
- double boost_score = 0.0;
- double mv_ratio_accumulator = 0.0;
- double decay_accumulator = 1.0;
- double this_frame_mv_in_out = 0.0;
- double mv_in_out_accumulator = 0.0;
- double abs_mv_in_out_accumulator = 0.0;
- int arf_boost;
- int flash_detected = 0;
-
- // Search forward from the proposed arf/next gf position.
- for (i = 0; i < f_frames; ++i) {
- const FIRSTPASS_STATS *this_frame = read_frame_stats(twopass, i + offset);
- if (this_frame == NULL)
- break;
-
- // Update the motion related elements to the boost calculation.
- accumulate_frame_motion_stats(this_frame,
- &this_frame_mv_in_out, &mv_in_out_accumulator,
- &abs_mv_in_out_accumulator,
- &mv_ratio_accumulator);
-
- // We want to discount the flash frame itself and the recovery
- // frame that follows as both will have poor scores.
- flash_detected = detect_flash(twopass, i + offset) ||
- detect_flash(twopass, i + offset + 1);
-
- // Accumulate the effect of prediction quality decay.
- if (!flash_detected) {
- decay_accumulator *= get_prediction_decay_rate(cpi, this_frame);
- decay_accumulator = decay_accumulator < MIN_DECAY_FACTOR
- ? MIN_DECAY_FACTOR : decay_accumulator;
- }
-
- boost_score += decay_accumulator * calc_frame_boost(cpi, this_frame,
- this_frame_mv_in_out,
- GF_MAX_BOOST);
- }
-
- *f_boost = (int)boost_score;
-
- // Reset for backward looking loop.
- boost_score = 0.0;
- mv_ratio_accumulator = 0.0;
- decay_accumulator = 1.0;
- this_frame_mv_in_out = 0.0;
- mv_in_out_accumulator = 0.0;
- abs_mv_in_out_accumulator = 0.0;
-
- // Search backward towards last gf position.
- for (i = -1; i >= -b_frames; --i) {
- const FIRSTPASS_STATS *this_frame = read_frame_stats(twopass, i + offset);
- if (this_frame == NULL)
- break;
-
- // Update the motion related elements to the boost calculation.
- accumulate_frame_motion_stats(this_frame,
- &this_frame_mv_in_out, &mv_in_out_accumulator,
- &abs_mv_in_out_accumulator,
- &mv_ratio_accumulator);
-
- // We want to discount the the flash frame itself and the recovery
- // frame that follows as both will have poor scores.
- flash_detected = detect_flash(twopass, i + offset) ||
- detect_flash(twopass, i + offset + 1);
-
- // Cumulative effect of prediction quality decay.
- if (!flash_detected) {
- decay_accumulator *= get_prediction_decay_rate(cpi, this_frame);
- decay_accumulator = decay_accumulator < MIN_DECAY_FACTOR
- ? MIN_DECAY_FACTOR : decay_accumulator;
- }
-
- boost_score += decay_accumulator * calc_frame_boost(cpi, this_frame,
- this_frame_mv_in_out,
- GF_MAX_BOOST);
- }
- *b_boost = (int)boost_score;
-
- arf_boost = (*f_boost + *b_boost);
- if (arf_boost < ((b_frames + f_frames) * 20))
- arf_boost = ((b_frames + f_frames) * 20);
- arf_boost = VPXMAX(arf_boost, MIN_ARF_GF_BOOST);
-
- return arf_boost;
-}
-
-// Calculate a section intra ratio used in setting max loop filter.
-static int calculate_section_intra_ratio(const FIRSTPASS_STATS *begin,
- const FIRSTPASS_STATS *end,
- int section_length) {
- const FIRSTPASS_STATS *s = begin;
- double intra_error = 0.0;
- double coded_error = 0.0;
- int i = 0;
-
- while (s < end && i < section_length) {
- intra_error += s->intra_error;
- coded_error += s->coded_error;
- ++s;
- ++i;
- }
-
- return (int)(intra_error / DOUBLE_DIVIDE_CHECK(coded_error));
-}
-
-// Calculate the total bits to allocate in this GF/ARF group.
-static int64_t calculate_total_gf_group_bits(VP10_COMP *cpi,
- double gf_group_err) {
- const RATE_CONTROL *const rc = &cpi->rc;
- const TWO_PASS *const twopass = &cpi->twopass;
- const int max_bits = frame_max_bits(rc, &cpi->oxcf);
- int64_t total_group_bits;
-
- // Calculate the bits to be allocated to the group as a whole.
- if ((twopass->kf_group_bits > 0) && (twopass->kf_group_error_left > 0)) {
- total_group_bits = (int64_t)(twopass->kf_group_bits *
- (gf_group_err / twopass->kf_group_error_left));
- } else {
- total_group_bits = 0;
- }
-
- // Clamp odd edge cases.
- total_group_bits = (total_group_bits < 0) ?
- 0 : (total_group_bits > twopass->kf_group_bits) ?
- twopass->kf_group_bits : total_group_bits;
-
- // Clip based on user supplied data rate variability limit.
- if (total_group_bits > (int64_t)max_bits * rc->baseline_gf_interval)
- total_group_bits = (int64_t)max_bits * rc->baseline_gf_interval;
-
- return total_group_bits;
-}
-
-// Calculate the number bits extra to assign to boosted frames in a group.
-static int calculate_boost_bits(int frame_count,
- int boost, int64_t total_group_bits) {
- int allocation_chunks;
-
- // return 0 for invalid inputs (could arise e.g. through rounding errors)
- if (!boost || (total_group_bits <= 0) || (frame_count <= 0) )
- return 0;
-
- allocation_chunks = (frame_count * 100) + boost;
-
- // Prevent overflow.
- if (boost > 1023) {
- int divisor = boost >> 10;
- boost /= divisor;
- allocation_chunks /= divisor;
- }
-
- // Calculate the number of extra bits for use in the boosted frame or frames.
- return VPXMAX((int)(((int64_t)boost * total_group_bits) / allocation_chunks),
- 0);
-}
-
-// Current limit on maximum number of active arfs in a GF/ARF group.
-#define MAX_ACTIVE_ARFS 2
-#define ARF_SLOT1 2
-#define ARF_SLOT2 3
-// This function indirects the choice of buffers for arfs.
-// At the moment the values are fixed but this may change as part of
-// the integration process with other codec features that swap buffers around.
-static void get_arf_buffer_indices(unsigned char *arf_buffer_indices) {
- arf_buffer_indices[0] = ARF_SLOT1;
- arf_buffer_indices[1] = ARF_SLOT2;
-}
-
-static void allocate_gf_group_bits(VP10_COMP *cpi, int64_t gf_group_bits,
- double group_error, int gf_arf_bits) {
- RATE_CONTROL *const rc = &cpi->rc;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- TWO_PASS *const twopass = &cpi->twopass;
- GF_GROUP *const gf_group = &twopass->gf_group;
- FIRSTPASS_STATS frame_stats;
- int i;
- int frame_index = 1;
- int target_frame_size;
- int key_frame;
- const int max_bits = frame_max_bits(&cpi->rc, &cpi->oxcf);
- int64_t total_group_bits = gf_group_bits;
- double modified_err = 0.0;
- double err_fraction;
- int mid_boost_bits = 0;
- int mid_frame_idx;
- unsigned char arf_buffer_indices[MAX_ACTIVE_ARFS];
- int alt_frame_index = frame_index;
-
- key_frame = cpi->common.frame_type == KEY_FRAME;
-
- get_arf_buffer_indices(arf_buffer_indices);
-
- // For key frames the frame target rate is already set and it
- // is also the golden frame.
- if (!key_frame) {
- if (rc->source_alt_ref_active) {
- gf_group->update_type[0] = OVERLAY_UPDATE;
- gf_group->rf_level[0] = INTER_NORMAL;
- gf_group->bit_allocation[0] = 0;
- gf_group->arf_update_idx[0] = arf_buffer_indices[0];
- gf_group->arf_ref_idx[0] = arf_buffer_indices[0];
- } else {
- gf_group->update_type[0] = GF_UPDATE;
- gf_group->rf_level[0] = GF_ARF_STD;
- gf_group->bit_allocation[0] = gf_arf_bits;
- gf_group->arf_update_idx[0] = arf_buffer_indices[0];
- gf_group->arf_ref_idx[0] = arf_buffer_indices[0];
- }
-
- // Step over the golden frame / overlay frame
- if (EOF == input_stats(twopass, &frame_stats))
- return;
- }
-
- // Deduct the boost bits for arf (or gf if it is not a key frame)
- // from the group total.
- if (rc->source_alt_ref_pending || !key_frame)
- total_group_bits -= gf_arf_bits;
-
- // Store the bits to spend on the ARF if there is one.
- if (rc->source_alt_ref_pending) {
- gf_group->update_type[alt_frame_index] = ARF_UPDATE;
- gf_group->rf_level[alt_frame_index] = GF_ARF_STD;
- gf_group->bit_allocation[alt_frame_index] = gf_arf_bits;
-
- gf_group->arf_src_offset[alt_frame_index] =
- (unsigned char)(rc->baseline_gf_interval - 1);
-
- gf_group->arf_update_idx[alt_frame_index] = arf_buffer_indices[0];
- gf_group->arf_ref_idx[alt_frame_index] =
- arf_buffer_indices[cpi->multi_arf_last_grp_enabled &&
- rc->source_alt_ref_active];
- ++frame_index;
-
- if (cpi->multi_arf_enabled) {
- // Set aside a slot for a level 1 arf.
- gf_group->update_type[frame_index] = ARF_UPDATE;
- gf_group->rf_level[frame_index] = GF_ARF_LOW;
- gf_group->arf_src_offset[frame_index] =
- (unsigned char)((rc->baseline_gf_interval >> 1) - 1);
- gf_group->arf_update_idx[frame_index] = arf_buffer_indices[1];
- gf_group->arf_ref_idx[frame_index] = arf_buffer_indices[0];
- ++frame_index;
- }
- }
-
- // Define middle frame
- mid_frame_idx = frame_index + (rc->baseline_gf_interval >> 1) - 1;
-
- // Allocate bits to the other frames in the group.
- for (i = 0; i < rc->baseline_gf_interval - rc->source_alt_ref_pending; ++i) {
- int arf_idx = 0;
- if (EOF == input_stats(twopass, &frame_stats))
- break;
-
- modified_err = calculate_modified_err(cpi, twopass, oxcf, &frame_stats);
-
- if (group_error > 0)
- err_fraction = modified_err / DOUBLE_DIVIDE_CHECK(group_error);
- else
- err_fraction = 0.0;
-
- target_frame_size = (int)((double)total_group_bits * err_fraction);
-
- if (rc->source_alt_ref_pending && cpi->multi_arf_enabled) {
- mid_boost_bits += (target_frame_size >> 4);
- target_frame_size -= (target_frame_size >> 4);
-
- if (frame_index <= mid_frame_idx)
- arf_idx = 1;
- }
- gf_group->arf_update_idx[frame_index] = arf_buffer_indices[arf_idx];
- gf_group->arf_ref_idx[frame_index] = arf_buffer_indices[arf_idx];
-
- target_frame_size = clamp(target_frame_size, 0,
- VPXMIN(max_bits, (int)total_group_bits));
-
- gf_group->update_type[frame_index] = LF_UPDATE;
- gf_group->rf_level[frame_index] = INTER_NORMAL;
-
- gf_group->bit_allocation[frame_index] = target_frame_size;
- ++frame_index;
- }
-
- // Note:
- // We need to configure the frame at the end of the sequence + 1 that will be
- // the start frame for the next group. Otherwise prior to the call to
- // vp10_rc_get_second_pass_params() the data will be undefined.
- gf_group->arf_update_idx[frame_index] = arf_buffer_indices[0];
- gf_group->arf_ref_idx[frame_index] = arf_buffer_indices[0];
-
- if (rc->source_alt_ref_pending) {
- gf_group->update_type[frame_index] = OVERLAY_UPDATE;
- gf_group->rf_level[frame_index] = INTER_NORMAL;
-
- // Final setup for second arf and its overlay.
- if (cpi->multi_arf_enabled) {
- gf_group->bit_allocation[2] =
- gf_group->bit_allocation[mid_frame_idx] + mid_boost_bits;
- gf_group->update_type[mid_frame_idx] = OVERLAY_UPDATE;
- gf_group->bit_allocation[mid_frame_idx] = 0;
- }
- } else {
- gf_group->update_type[frame_index] = GF_UPDATE;
- gf_group->rf_level[frame_index] = GF_ARF_STD;
- }
-
- // Note whether multi-arf was enabled this group for next time.
- cpi->multi_arf_last_grp_enabled = cpi->multi_arf_enabled;
-}
-
-// Analyse and define a gf/arf group.
-static void define_gf_group(VP10_COMP *cpi, FIRSTPASS_STATS *this_frame) {
- VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- VP10EncoderConfig *const oxcf = &cpi->oxcf;
- TWO_PASS *const twopass = &cpi->twopass;
- FIRSTPASS_STATS next_frame;
- const FIRSTPASS_STATS *const start_pos = twopass->stats_in;
- int i;
-
- double boost_score = 0.0;
- double old_boost_score = 0.0;
- double gf_group_err = 0.0;
-#if GROUP_ADAPTIVE_MAXQ
- double gf_group_raw_error = 0.0;
-#endif
- double gf_group_skip_pct = 0.0;
- double gf_group_inactive_zone_rows = 0.0;
- double gf_first_frame_err = 0.0;
- double mod_frame_err = 0.0;
-
- double mv_ratio_accumulator = 0.0;
- double decay_accumulator = 1.0;
- double zero_motion_accumulator = 1.0;
-
- double loop_decay_rate = 1.00;
- double last_loop_decay_rate = 1.00;
-
- double this_frame_mv_in_out = 0.0;
- double mv_in_out_accumulator = 0.0;
- double abs_mv_in_out_accumulator = 0.0;
- double mv_ratio_accumulator_thresh;
- unsigned int allow_alt_ref = is_altref_enabled(cpi);
-
- int f_boost = 0;
- int b_boost = 0;
- int flash_detected;
- int active_max_gf_interval;
- int active_min_gf_interval;
- int64_t gf_group_bits;
- double gf_group_error_left;
- int gf_arf_bits;
- const int is_key_frame = frame_is_intra_only(cm);
- const int arf_active_or_kf = is_key_frame || rc->source_alt_ref_active;
-
- // Reset the GF group data structures unless this is a key
- // frame in which case it will already have been done.
- if (is_key_frame == 0) {
- vp10_zero(twopass->gf_group);
- }
-
- vpx_clear_system_state();
- vp10_zero(next_frame);
-
- // Load stats for the current frame.
- mod_frame_err = calculate_modified_err(cpi, twopass, oxcf, this_frame);
-
- // Note the error of the frame at the start of the group. This will be
- // the GF frame error if we code a normal gf.
- gf_first_frame_err = mod_frame_err;
-
- // If this is a key frame or the overlay from a previous arf then
- // the error score / cost of this frame has already been accounted for.
- if (arf_active_or_kf) {
- gf_group_err -= gf_first_frame_err;
-#if GROUP_ADAPTIVE_MAXQ
- gf_group_raw_error -= this_frame->coded_error;
-#endif
- gf_group_skip_pct -= this_frame->intra_skip_pct;
- gf_group_inactive_zone_rows -= this_frame->inactive_zone_rows;
- }
-
- // Motion breakout threshold for loop below depends on image size.
- mv_ratio_accumulator_thresh =
- (cpi->initial_height + cpi->initial_width) / 4.0;
-
- // Set a maximum and minimum interval for the GF group.
- // If the image appears almost completely static we can extend beyond this.
- {
- int int_max_q =
- (int)(vp10_convert_qindex_to_q(twopass->active_worst_quality,
- cpi->common.bit_depth));
- int int_lbq =
- (int)(vp10_convert_qindex_to_q(rc->last_boosted_qindex,
- cpi->common.bit_depth));
- active_min_gf_interval = rc->min_gf_interval + VPXMIN(2, int_max_q / 200);
- if (active_min_gf_interval > rc->max_gf_interval)
- active_min_gf_interval = rc->max_gf_interval;
-
- if (cpi->multi_arf_allowed) {
- active_max_gf_interval = rc->max_gf_interval;
- } else {
- // The value chosen depends on the active Q range. At low Q we have
- // bits to spare and are better with a smaller interval and smaller boost.
- // At high Q when there are few bits to spare we are better with a longer
- // interval to spread the cost of the GF.
- active_max_gf_interval = 12 + VPXMIN(4, (int_lbq / 6));
- if (active_max_gf_interval < active_min_gf_interval)
- active_max_gf_interval = active_min_gf_interval;
-
- if (active_max_gf_interval > rc->max_gf_interval)
- active_max_gf_interval = rc->max_gf_interval;
- if (active_max_gf_interval < active_min_gf_interval)
- active_max_gf_interval = active_min_gf_interval;
- }
- }
-
- i = 0;
- while (i < rc->static_scene_max_gf_interval && i < rc->frames_to_key) {
- ++i;
-
- // Accumulate error score of frames in this gf group.
- mod_frame_err = calculate_modified_err(cpi, twopass, oxcf, this_frame);
- gf_group_err += mod_frame_err;
-#if GROUP_ADAPTIVE_MAXQ
- gf_group_raw_error += this_frame->coded_error;
-#endif
- gf_group_skip_pct += this_frame->intra_skip_pct;
- gf_group_inactive_zone_rows += this_frame->inactive_zone_rows;
-
- if (EOF == input_stats(twopass, &next_frame))
- break;
-
- // Test for the case where there is a brief flash but the prediction
- // quality back to an earlier frame is then restored.
- flash_detected = detect_flash(twopass, 0);
-
- // Update the motion related elements to the boost calculation.
- accumulate_frame_motion_stats(&next_frame,
- &this_frame_mv_in_out, &mv_in_out_accumulator,
- &abs_mv_in_out_accumulator,
- &mv_ratio_accumulator);
-
- // Accumulate the effect of prediction quality decay.
- if (!flash_detected) {
- last_loop_decay_rate = loop_decay_rate;
- loop_decay_rate = get_prediction_decay_rate(cpi, &next_frame);
-
- decay_accumulator = decay_accumulator * loop_decay_rate;
-
- // Monitor for static sections.
- zero_motion_accumulator = VPXMIN(
- zero_motion_accumulator, get_zero_motion_factor(cpi, &next_frame));
-
- // Break clause to detect very still sections after motion. For example,
- // a static image after a fade or other transition.
- if (detect_transition_to_still(cpi, i, 5, loop_decay_rate,
- last_loop_decay_rate)) {
- allow_alt_ref = 0;
- break;
- }
- }
-
- // Calculate a boost number for this frame.
- boost_score += decay_accumulator * calc_frame_boost(cpi, &next_frame,
- this_frame_mv_in_out,
- GF_MAX_BOOST);
-
- // Break out conditions.
- if (
- // Break at active_max_gf_interval unless almost totally static.
- (i >= (active_max_gf_interval + arf_active_or_kf) &&
- zero_motion_accumulator < 0.995) ||
- (
- // Don't break out with a very short interval.
- (i >= active_min_gf_interval + arf_active_or_kf) &&
- (!flash_detected) &&
- ((mv_ratio_accumulator > mv_ratio_accumulator_thresh) ||
- (abs_mv_in_out_accumulator > 3.0) ||
- (mv_in_out_accumulator < -2.0) ||
- ((boost_score - old_boost_score) < BOOST_BREAKOUT)))) {
- boost_score = old_boost_score;
- break;
- }
-
- *this_frame = next_frame;
- old_boost_score = boost_score;
- }
-
- twopass->gf_zeromotion_pct = (int)(zero_motion_accumulator * 1000.0);
-
- // Was the group length constrained by the requirement for a new KF?
- rc->constrained_gf_group = (i >= rc->frames_to_key) ? 1 : 0;
-
- // Should we use the alternate reference frame.
- if (allow_alt_ref &&
- (i < cpi->oxcf.lag_in_frames) &&
- (i >= rc->min_gf_interval)) {
- // Calculate the boost for alt ref.
- rc->gfu_boost = calc_arf_boost(cpi, 0, (i - 1), (i - 1), &f_boost,
- &b_boost);
- rc->source_alt_ref_pending = 1;
-
- // Test to see if multi arf is appropriate.
- cpi->multi_arf_enabled =
- (cpi->multi_arf_allowed && (rc->baseline_gf_interval >= 6) &&
- (zero_motion_accumulator < 0.995)) ? 1 : 0;
- } else {
- rc->gfu_boost = VPXMAX((int)boost_score, MIN_ARF_GF_BOOST);
- rc->source_alt_ref_pending = 0;
- }
-
- // Set the interval until the next gf.
- rc->baseline_gf_interval = i - (is_key_frame || rc->source_alt_ref_pending);
-
- rc->frames_till_gf_update_due = rc->baseline_gf_interval;
-
- // Reset the file position.
- reset_fpf_position(twopass, start_pos);
-
- // Calculate the bits to be allocated to the gf/arf group as a whole
- gf_group_bits = calculate_total_gf_group_bits(cpi, gf_group_err);
-
-#if GROUP_ADAPTIVE_MAXQ
- // Calculate an estimate of the maxq needed for the group.
- // We are more agressive about correcting for sections
- // where there could be significant overshoot than for easier
- // sections where we do not wish to risk creating an overshoot
- // of the allocated bit budget.
- if ((cpi->oxcf.rc_mode != VPX_Q) && (rc->baseline_gf_interval > 1)) {
- const int vbr_group_bits_per_frame =
- (int)(gf_group_bits / rc->baseline_gf_interval);
- const double group_av_err = gf_group_raw_error / rc->baseline_gf_interval;
- const double group_av_skip_pct =
- gf_group_skip_pct / rc->baseline_gf_interval;
- const double group_av_inactive_zone =
- ((gf_group_inactive_zone_rows * 2) /
- (rc->baseline_gf_interval * (double)cm->mb_rows));
-
- int tmp_q;
- // rc factor is a weight factor that corrects for local rate control drift.
- double rc_factor = 1.0;
- if (rc->rate_error_estimate > 0) {
- rc_factor = VPXMAX(RC_FACTOR_MIN,
- (double)(100 - rc->rate_error_estimate) / 100.0);
- } else {
- rc_factor = VPXMIN(RC_FACTOR_MAX,
- (double)(100 - rc->rate_error_estimate) / 100.0);
- }
- tmp_q =
- get_twopass_worst_quality(cpi, group_av_err,
- (group_av_skip_pct + group_av_inactive_zone),
- vbr_group_bits_per_frame,
- twopass->kfgroup_inter_fraction * rc_factor);
- twopass->active_worst_quality =
- VPXMAX(tmp_q, twopass->active_worst_quality >> 1);
- }
-#endif
-
- // Calculate the extra bits to be used for boosted frame(s)
- gf_arf_bits = calculate_boost_bits(rc->baseline_gf_interval,
- rc->gfu_boost, gf_group_bits);
-
- // Adjust KF group bits and error remaining.
- twopass->kf_group_error_left -= (int64_t)gf_group_err;
-
- // If this is an arf update we want to remove the score for the overlay
- // frame at the end which will usually be very cheap to code.
- // The overlay frame has already, in effect, been coded so we want to spread
- // the remaining bits among the other frames.
- // For normal GFs remove the score for the GF itself unless this is
- // also a key frame in which case it has already been accounted for.
- if (rc->source_alt_ref_pending) {
- gf_group_error_left = gf_group_err - mod_frame_err;
- } else if (is_key_frame == 0) {
- gf_group_error_left = gf_group_err - gf_first_frame_err;
- } else {
- gf_group_error_left = gf_group_err;
- }
-
- // Allocate bits to each of the frames in the GF group.
- allocate_gf_group_bits(cpi, gf_group_bits, gf_group_error_left, gf_arf_bits);
-
- // Reset the file position.
- reset_fpf_position(twopass, start_pos);
-
- // Calculate a section intra ratio used in setting max loop filter.
- if (cpi->common.frame_type != KEY_FRAME) {
- twopass->section_intra_rating =
- calculate_section_intra_ratio(start_pos, twopass->stats_in_end,
- rc->baseline_gf_interval);
- }
-
- if (oxcf->resize_mode == RESIZE_DYNAMIC) {
- // Default to starting GF groups at normal frame size.
- cpi->rc.next_frame_size_selector = UNSCALED;
- }
-}
-
-// Threshold for use of the lagging second reference frame. High second ref
-// usage may point to a transient event like a flash or occlusion rather than
-// a real scene cut.
-#define SECOND_REF_USEAGE_THRESH 0.1
-// Minimum % intra coding observed in first pass (1.0 = 100%)
-#define MIN_INTRA_LEVEL 0.25
-// Minimum ratio between the % of intra coding and inter coding in the first
-// pass after discounting neutral blocks (discounting neutral blocks in this
-// way helps catch scene cuts in clips with very flat areas or letter box
-// format clips with image padding.
-#define INTRA_VS_INTER_THRESH 2.0
-// Hard threshold where the first pass chooses intra for almost all blocks.
-// In such a case even if the frame is not a scene cut coding a key frame
-// may be a good option.
-#define VERY_LOW_INTER_THRESH 0.05
-// Maximum threshold for the relative ratio of intra error score vs best
-// inter error score.
-#define KF_II_ERR_THRESHOLD 2.5
-// In real scene cuts there is almost always a sharp change in the intra
-// or inter error score.
-#define ERR_CHANGE_THRESHOLD 0.4
-// For real scene cuts we expect an improvment in the intra inter error
-// ratio in the next frame.
-#define II_IMPROVEMENT_THRESHOLD 3.5
-#define KF_II_MAX 128.0
-
-static int test_candidate_kf(TWO_PASS *twopass,
- const FIRSTPASS_STATS *last_frame,
- const FIRSTPASS_STATS *this_frame,
- const FIRSTPASS_STATS *next_frame) {
- int is_viable_kf = 0;
- double pcnt_intra = 1.0 - this_frame->pcnt_inter;
- double modified_pcnt_inter =
- this_frame->pcnt_inter - this_frame->pcnt_neutral;
-
- // Does the frame satisfy the primary criteria of a key frame?
- // See above for an explanation of the test criteria.
- // If so, then examine how well it predicts subsequent frames.
- if ((this_frame->pcnt_second_ref < SECOND_REF_USEAGE_THRESH) &&
- (next_frame->pcnt_second_ref < SECOND_REF_USEAGE_THRESH) &&
- ((this_frame->pcnt_inter < VERY_LOW_INTER_THRESH) ||
- ((pcnt_intra > MIN_INTRA_LEVEL) &&
- (pcnt_intra > (INTRA_VS_INTER_THRESH * modified_pcnt_inter)) &&
- ((this_frame->intra_error /
- DOUBLE_DIVIDE_CHECK(this_frame->coded_error)) <
- KF_II_ERR_THRESHOLD) &&
- ((fabs(last_frame->coded_error - this_frame->coded_error) /
- DOUBLE_DIVIDE_CHECK(this_frame->coded_error) >
- ERR_CHANGE_THRESHOLD) ||
- (fabs(last_frame->intra_error - this_frame->intra_error) /
- DOUBLE_DIVIDE_CHECK(this_frame->intra_error) >
- ERR_CHANGE_THRESHOLD) ||
- ((next_frame->intra_error /
- DOUBLE_DIVIDE_CHECK(next_frame->coded_error)) >
- II_IMPROVEMENT_THRESHOLD))))) {
- int i;
- const FIRSTPASS_STATS *start_pos = twopass->stats_in;
- FIRSTPASS_STATS local_next_frame = *next_frame;
- double boost_score = 0.0;
- double old_boost_score = 0.0;
- double decay_accumulator = 1.0;
-
- // Examine how well the key frame predicts subsequent frames.
- for (i = 0; i < 16; ++i) {
- double next_iiratio = (BOOST_FACTOR * local_next_frame.intra_error /
- DOUBLE_DIVIDE_CHECK(local_next_frame.coded_error));
-
- if (next_iiratio > KF_II_MAX)
- next_iiratio = KF_II_MAX;
-
- // Cumulative effect of decay in prediction quality.
- if (local_next_frame.pcnt_inter > 0.85)
- decay_accumulator *= local_next_frame.pcnt_inter;
- else
- decay_accumulator *= (0.85 + local_next_frame.pcnt_inter) / 2.0;
-
- // Keep a running total.
- boost_score += (decay_accumulator * next_iiratio);
-
- // Test various breakout clauses.
- if ((local_next_frame.pcnt_inter < 0.05) ||
- (next_iiratio < 1.5) ||
- (((local_next_frame.pcnt_inter -
- local_next_frame.pcnt_neutral) < 0.20) &&
- (next_iiratio < 3.0)) ||
- ((boost_score - old_boost_score) < 3.0) ||
- (local_next_frame.intra_error < 200)) {
- break;
- }
-
- old_boost_score = boost_score;
-
- // Get the next frame details
- if (EOF == input_stats(twopass, &local_next_frame))
- break;
- }
-
- // If there is tolerable prediction for at least the next 3 frames then
- // break out else discard this potential key frame and move on
- if (boost_score > 30.0 && (i > 3)) {
- is_viable_kf = 1;
- } else {
- // Reset the file position
- reset_fpf_position(twopass, start_pos);
-
- is_viable_kf = 0;
- }
- }
-
- return is_viable_kf;
-}
-
-static void find_next_key_frame(VP10_COMP *cpi, FIRSTPASS_STATS *this_frame) {
- int i, j;
- RATE_CONTROL *const rc = &cpi->rc;
- TWO_PASS *const twopass = &cpi->twopass;
- GF_GROUP *const gf_group = &twopass->gf_group;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- const FIRSTPASS_STATS first_frame = *this_frame;
- const FIRSTPASS_STATS *const start_position = twopass->stats_in;
- FIRSTPASS_STATS next_frame;
- FIRSTPASS_STATS last_frame;
- int kf_bits = 0;
- int loop_decay_counter = 0;
- double decay_accumulator = 1.0;
- double av_decay_accumulator = 0.0;
- double zero_motion_accumulator = 1.0;
- double boost_score = 0.0;
- double kf_mod_err = 0.0;
- double kf_group_err = 0.0;
- double recent_loop_decay[8] = {1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0};
-
- vp10_zero(next_frame);
-
- cpi->common.frame_type = KEY_FRAME;
-
- // Reset the GF group data structures.
- vp10_zero(*gf_group);
-
- // Is this a forced key frame by interval.
- rc->this_key_frame_forced = rc->next_key_frame_forced;
-
- // Clear the alt ref active flag and last group multi arf flags as they
- // can never be set for a key frame.
- rc->source_alt_ref_active = 0;
- cpi->multi_arf_last_grp_enabled = 0;
-
- // KF is always a GF so clear frames till next gf counter.
- rc->frames_till_gf_update_due = 0;
-
- rc->frames_to_key = 1;
-
- twopass->kf_group_bits = 0; // Total bits available to kf group
- twopass->kf_group_error_left = 0; // Group modified error score.
-
- kf_mod_err = calculate_modified_err(cpi, twopass, oxcf, this_frame);
-
- // Find the next keyframe.
- i = 0;
- while (twopass->stats_in < twopass->stats_in_end &&
- rc->frames_to_key < cpi->oxcf.key_freq) {
- // Accumulate kf group error.
- kf_group_err += calculate_modified_err(cpi, twopass, oxcf, this_frame);
-
- // Load the next frame's stats.
- last_frame = *this_frame;
- input_stats(twopass, this_frame);
-
- // Provided that we are not at the end of the file...
- if (cpi->oxcf.auto_key && twopass->stats_in < twopass->stats_in_end) {
- double loop_decay_rate;
-
- // Check for a scene cut.
- if (test_candidate_kf(twopass, &last_frame, this_frame,
- twopass->stats_in))
- break;
-
- // How fast is the prediction quality decaying?
- loop_decay_rate = get_prediction_decay_rate(cpi, twopass->stats_in);
-
- // We want to know something about the recent past... rather than
- // as used elsewhere where we are concerned with decay in prediction
- // quality since the last GF or KF.
- recent_loop_decay[i % 8] = loop_decay_rate;
- decay_accumulator = 1.0;
- for (j = 0; j < 8; ++j)
- decay_accumulator *= recent_loop_decay[j];
-
- // Special check for transition or high motion followed by a
- // static scene.
- if (detect_transition_to_still(cpi, i, cpi->oxcf.key_freq - i,
- loop_decay_rate, decay_accumulator))
- break;
-
- // Step on to the next frame.
- ++rc->frames_to_key;
-
- // If we don't have a real key frame within the next two
- // key_freq intervals then break out of the loop.
- if (rc->frames_to_key >= 2 * cpi->oxcf.key_freq)
- break;
- } else {
- ++rc->frames_to_key;
- }
- ++i;
- }
-
- // If there is a max kf interval set by the user we must obey it.
- // We already breakout of the loop above at 2x max.
- // This code centers the extra kf if the actual natural interval
- // is between 1x and 2x.
- if (cpi->oxcf.auto_key &&
- rc->frames_to_key > cpi->oxcf.key_freq) {
- FIRSTPASS_STATS tmp_frame = first_frame;
-
- rc->frames_to_key /= 2;
-
- // Reset to the start of the group.
- reset_fpf_position(twopass, start_position);
-
- kf_group_err = 0.0;
-
- // Rescan to get the correct error data for the forced kf group.
- for (i = 0; i < rc->frames_to_key; ++i) {
- kf_group_err += calculate_modified_err(cpi, twopass, oxcf, &tmp_frame);
- input_stats(twopass, &tmp_frame);
- }
- rc->next_key_frame_forced = 1;
- } else if (twopass->stats_in == twopass->stats_in_end ||
- rc->frames_to_key >= cpi->oxcf.key_freq) {
- rc->next_key_frame_forced = 1;
- } else {
- rc->next_key_frame_forced = 0;
- }
-
- // Special case for the last key frame of the file.
- if (twopass->stats_in >= twopass->stats_in_end) {
- // Accumulate kf group error.
- kf_group_err += calculate_modified_err(cpi, twopass, oxcf, this_frame);
- }
-
- // Calculate the number of bits that should be assigned to the kf group.
- if (twopass->bits_left > 0 && twopass->modified_error_left > 0.0) {
- // Maximum number of bits for a single normal frame (not key frame).
- const int max_bits = frame_max_bits(rc, &cpi->oxcf);
-
- // Maximum number of bits allocated to the key frame group.
- int64_t max_grp_bits;
-
- // Default allocation based on bits left and relative
- // complexity of the section.
- twopass->kf_group_bits = (int64_t)(twopass->bits_left *
- (kf_group_err / twopass->modified_error_left));
-
- // Clip based on maximum per frame rate defined by the user.
- max_grp_bits = (int64_t)max_bits * (int64_t)rc->frames_to_key;
- if (twopass->kf_group_bits > max_grp_bits)
- twopass->kf_group_bits = max_grp_bits;
- } else {
- twopass->kf_group_bits = 0;
- }
- twopass->kf_group_bits = VPXMAX(0, twopass->kf_group_bits);
-
- // Reset the first pass file position.
- reset_fpf_position(twopass, start_position);
-
- // Scan through the kf group collating various stats used to determine
- // how many bits to spend on it.
- decay_accumulator = 1.0;
- boost_score = 0.0;
- for (i = 0; i < (rc->frames_to_key - 1); ++i) {
- if (EOF == input_stats(twopass, &next_frame))
- break;
-
- // Monitor for static sections.
- zero_motion_accumulator = VPXMIN(
- zero_motion_accumulator, get_zero_motion_factor(cpi, &next_frame));
-
- // Not all frames in the group are necessarily used in calculating boost.
- if ((i <= rc->max_gf_interval) ||
- ((i <= (rc->max_gf_interval * 4)) && (decay_accumulator > 0.5))) {
- const double frame_boost =
- calc_frame_boost(cpi, this_frame, 0, KF_MAX_BOOST);
-
- // How fast is prediction quality decaying.
- if (!detect_flash(twopass, 0)) {
- const double loop_decay_rate =
- get_prediction_decay_rate(cpi, &next_frame);
- decay_accumulator *= loop_decay_rate;
- decay_accumulator = VPXMAX(decay_accumulator, MIN_DECAY_FACTOR);
- av_decay_accumulator += decay_accumulator;
- ++loop_decay_counter;
- }
- boost_score += (decay_accumulator * frame_boost);
- }
- }
- av_decay_accumulator /= (double)loop_decay_counter;
-
- reset_fpf_position(twopass, start_position);
-
- // Store the zero motion percentage
- twopass->kf_zeromotion_pct = (int)(zero_motion_accumulator * 100.0);
-
- // Calculate a section intra ratio used in setting max loop filter.
- twopass->section_intra_rating =
- calculate_section_intra_ratio(start_position, twopass->stats_in_end,
- rc->frames_to_key);
-
- // Apply various clamps for min and max boost
- rc->kf_boost = (int)(av_decay_accumulator * boost_score);
- rc->kf_boost = VPXMAX(rc->kf_boost, (rc->frames_to_key * 3));
- rc->kf_boost = VPXMAX(rc->kf_boost, MIN_KF_BOOST);
-
- // Work out how many bits to allocate for the key frame itself.
- kf_bits = calculate_boost_bits((rc->frames_to_key - 1),
- rc->kf_boost, twopass->kf_group_bits);
-
- // Work out the fraction of the kf group bits reserved for the inter frames
- // within the group after discounting the bits for the kf itself.
- if (twopass->kf_group_bits) {
- twopass->kfgroup_inter_fraction =
- (double)(twopass->kf_group_bits - kf_bits) /
- (double)twopass->kf_group_bits;
- } else {
- twopass->kfgroup_inter_fraction = 1.0;
- }
-
- twopass->kf_group_bits -= kf_bits;
-
- // Save the bits to spend on the key frame.
- gf_group->bit_allocation[0] = kf_bits;
- gf_group->update_type[0] = KF_UPDATE;
- gf_group->rf_level[0] = KF_STD;
-
- // Note the total error score of the kf group minus the key frame itself.
- twopass->kf_group_error_left = (int)(kf_group_err - kf_mod_err);
-
- // Adjust the count of total modified error left.
- // The count of bits left is adjusted elsewhere based on real coded frame
- // sizes.
- twopass->modified_error_left -= kf_group_err;
-
- if (oxcf->resize_mode == RESIZE_DYNAMIC) {
- // Default to normal-sized frame on keyframes.
- cpi->rc.next_frame_size_selector = UNSCALED;
- }
-}
-
-// Define the reference buffers that will be updated post encode.
-static void configure_buffer_updates(VP10_COMP *cpi) {
- TWO_PASS *const twopass = &cpi->twopass;
-
- cpi->rc.is_src_frame_alt_ref = 0;
- switch (twopass->gf_group.update_type[twopass->gf_group.index]) {
- case KF_UPDATE:
- cpi->refresh_last_frame = 1;
- cpi->refresh_golden_frame = 1;
- cpi->refresh_alt_ref_frame = 1;
- break;
- case LF_UPDATE:
- cpi->refresh_last_frame = 1;
- cpi->refresh_golden_frame = 0;
- cpi->refresh_alt_ref_frame = 0;
- break;
- case GF_UPDATE:
- cpi->refresh_last_frame = 1;
- cpi->refresh_golden_frame = 1;
- cpi->refresh_alt_ref_frame = 0;
- break;
- case OVERLAY_UPDATE:
- cpi->refresh_last_frame = 0;
- cpi->refresh_golden_frame = 1;
- cpi->refresh_alt_ref_frame = 0;
- cpi->rc.is_src_frame_alt_ref = 1;
- break;
- case ARF_UPDATE:
- cpi->refresh_last_frame = 0;
- cpi->refresh_golden_frame = 0;
- cpi->refresh_alt_ref_frame = 1;
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static int is_skippable_frame(const VP10_COMP *cpi) {
- // If the current frame does not have non-zero motion vector detected in the
- // first pass, and so do its previous and forward frames, then this frame
- // can be skipped for partition check, and the partition size is assigned
- // according to the variance
- const TWO_PASS *const twopass = &cpi->twopass;
-
- return (!frame_is_intra_only(&cpi->common) &&
- twopass->stats_in - 2 > twopass->stats_in_start &&
- twopass->stats_in < twopass->stats_in_end &&
- (twopass->stats_in - 1)->pcnt_inter - (twopass->stats_in - 1)->pcnt_motion
- == 1 &&
- (twopass->stats_in - 2)->pcnt_inter - (twopass->stats_in - 2)->pcnt_motion
- == 1 &&
- twopass->stats_in->pcnt_inter - twopass->stats_in->pcnt_motion == 1);
-}
-
-void vp10_rc_get_second_pass_params(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- TWO_PASS *const twopass = &cpi->twopass;
- GF_GROUP *const gf_group = &twopass->gf_group;
- int frames_left;
- FIRSTPASS_STATS this_frame;
-
- int target_rate;
-
- frames_left = (int)(twopass->total_stats.count -
- cm->current_video_frame);
-
- if (!twopass->stats_in)
- return;
-
- // If this is an arf frame then we dont want to read the stats file or
- // advance the input pointer as we already have what we need.
- if (gf_group->update_type[gf_group->index] == ARF_UPDATE) {
- int target_rate;
- configure_buffer_updates(cpi);
- target_rate = gf_group->bit_allocation[gf_group->index];
- target_rate = vp10_rc_clamp_pframe_target_size(cpi, target_rate);
- rc->base_frame_target = target_rate;
-
- cm->frame_type = INTER_FRAME;
-
- // Do the firstpass stats indicate that this frame is skippable for the
- // partition search?
- if (cpi->sf.allow_partition_search_skip && cpi->oxcf.pass == 2) {
- cpi->partition_search_skippable_frame = is_skippable_frame(cpi);
- }
-
- return;
- }
-
- vpx_clear_system_state();
-
- if (cpi->oxcf.rc_mode == VPX_Q) {
- twopass->active_worst_quality = cpi->oxcf.cq_level;
- } else if (cm->current_video_frame == 0) {
- // Special case code for first frame.
- const int section_target_bandwidth = (int)(twopass->bits_left /
- frames_left);
- const double section_length = twopass->total_left_stats.count;
- const double section_error =
- twopass->total_left_stats.coded_error / section_length;
- const double section_intra_skip =
- twopass->total_left_stats.intra_skip_pct / section_length;
- const double section_inactive_zone =
- (twopass->total_left_stats.inactive_zone_rows * 2) /
- ((double)cm->mb_rows * section_length);
- const int tmp_q =
- get_twopass_worst_quality(cpi, section_error,
- section_intra_skip + section_inactive_zone,
- section_target_bandwidth, DEFAULT_GRP_WEIGHT);
-
- twopass->active_worst_quality = tmp_q;
- twopass->baseline_active_worst_quality = tmp_q;
- rc->ni_av_qi = tmp_q;
- rc->last_q[INTER_FRAME] = tmp_q;
- rc->avg_q = vp10_convert_qindex_to_q(tmp_q, cm->bit_depth);
- rc->avg_frame_qindex[INTER_FRAME] = tmp_q;
- rc->last_q[KEY_FRAME] = (tmp_q + cpi->oxcf.best_allowed_q) / 2;
- rc->avg_frame_qindex[KEY_FRAME] = rc->last_q[KEY_FRAME];
- }
- vp10_zero(this_frame);
- if (EOF == input_stats(twopass, &this_frame))
- return;
-
- // Set the frame content type flag.
- if (this_frame.intra_skip_pct >= FC_ANIMATION_THRESH)
- twopass->fr_content_type = FC_GRAPHICS_ANIMATION;
- else
- twopass->fr_content_type = FC_NORMAL;
-
- // Keyframe and section processing.
- if (rc->frames_to_key == 0 || (cpi->frame_flags & FRAMEFLAGS_KEY)) {
- FIRSTPASS_STATS this_frame_copy;
- this_frame_copy = this_frame;
- // Define next KF group and assign bits to it.
- find_next_key_frame(cpi, &this_frame);
- this_frame = this_frame_copy;
- } else {
- cm->frame_type = INTER_FRAME;
- }
-
- // Define a new GF/ARF group. (Should always enter here for key frames).
- if (rc->frames_till_gf_update_due == 0) {
- define_gf_group(cpi, &this_frame);
-
- rc->frames_till_gf_update_due = rc->baseline_gf_interval;
-
-#if ARF_STATS_OUTPUT
- {
- FILE *fpfile;
- fpfile = fopen("arf.stt", "a");
- ++arf_count;
- fprintf(fpfile, "%10d %10ld %10d %10d %10ld\n",
- cm->current_video_frame, rc->frames_till_gf_update_due,
- rc->kf_boost, arf_count, rc->gfu_boost);
-
- fclose(fpfile);
- }
-#endif
- }
-
- configure_buffer_updates(cpi);
-
- // Do the firstpass stats indicate that this frame is skippable for the
- // partition search?
- if (cpi->sf.allow_partition_search_skip && cpi->oxcf.pass == 2) {
- cpi->partition_search_skippable_frame = is_skippable_frame(cpi);
- }
-
- target_rate = gf_group->bit_allocation[gf_group->index];
- if (cpi->common.frame_type == KEY_FRAME)
- target_rate = vp10_rc_clamp_iframe_target_size(cpi, target_rate);
- else
- target_rate = vp10_rc_clamp_pframe_target_size(cpi, target_rate);
-
- rc->base_frame_target = target_rate;
-
- {
- const int num_mbs = (cpi->oxcf.resize_mode != RESIZE_NONE)
- ? cpi->initial_mbs : cpi->common.MBs;
- // The multiplication by 256 reverses a scaling factor of (>> 8)
- // applied when combining MB error values for the frame.
- twopass->mb_av_energy =
- log(((this_frame.intra_error * 256.0) / num_mbs) + 1.0);
- }
-
- // Update the total stats remaining structure.
- subtract_stats(&twopass->total_left_stats, &this_frame);
-}
-
-#define MINQ_ADJ_LIMIT 48
-#define MINQ_ADJ_LIMIT_CQ 20
-#define HIGH_UNDERSHOOT_RATIO 2
-void vp10_twopass_postencode_update(VP10_COMP *cpi) {
- TWO_PASS *const twopass = &cpi->twopass;
- RATE_CONTROL *const rc = &cpi->rc;
- const int bits_used = rc->base_frame_target;
-
- // VBR correction is done through rc->vbr_bits_off_target. Based on the
- // sign of this value, a limited % adjustment is made to the target rate
- // of subsequent frames, to try and push it back towards 0. This method
- // is designed to prevent extreme behaviour at the end of a clip
- // or group of frames.
- rc->vbr_bits_off_target += rc->base_frame_target - rc->projected_frame_size;
- twopass->bits_left = VPXMAX(twopass->bits_left - bits_used, 0);
-
- // Calculate the pct rc error.
- if (rc->total_actual_bits) {
- rc->rate_error_estimate =
- (int)((rc->vbr_bits_off_target * 100) / rc->total_actual_bits);
- rc->rate_error_estimate = clamp(rc->rate_error_estimate, -100, 100);
- } else {
- rc->rate_error_estimate = 0;
- }
-
- if (cpi->common.frame_type != KEY_FRAME) {
- twopass->kf_group_bits -= bits_used;
- twopass->last_kfgroup_zeromotion_pct = twopass->kf_zeromotion_pct;
- }
- twopass->kf_group_bits = VPXMAX(twopass->kf_group_bits, 0);
-
- // Increment the gf group index ready for the next frame.
- ++twopass->gf_group.index;
-
- // If the rate control is drifting consider adjustment to min or maxq.
- if ((cpi->oxcf.rc_mode != VPX_Q) &&
- (cpi->twopass.gf_zeromotion_pct < VLOW_MOTION_THRESHOLD) &&
- !cpi->rc.is_src_frame_alt_ref) {
- const int maxq_adj_limit =
- rc->worst_quality - twopass->active_worst_quality;
- const int minq_adj_limit =
- (cpi->oxcf.rc_mode == VPX_CQ ? MINQ_ADJ_LIMIT_CQ : MINQ_ADJ_LIMIT);
-
- // Undershoot.
- if (rc->rate_error_estimate > cpi->oxcf.under_shoot_pct) {
- --twopass->extend_maxq;
- if (rc->rolling_target_bits >= rc->rolling_actual_bits)
- ++twopass->extend_minq;
- // Overshoot.
- } else if (rc->rate_error_estimate < -cpi->oxcf.over_shoot_pct) {
- --twopass->extend_minq;
- if (rc->rolling_target_bits < rc->rolling_actual_bits)
- ++twopass->extend_maxq;
- } else {
- // Adjustment for extreme local overshoot.
- if (rc->projected_frame_size > (2 * rc->base_frame_target) &&
- rc->projected_frame_size > (2 * rc->avg_frame_bandwidth))
- ++twopass->extend_maxq;
-
- // Unwind undershoot or overshoot adjustment.
- if (rc->rolling_target_bits < rc->rolling_actual_bits)
- --twopass->extend_minq;
- else if (rc->rolling_target_bits > rc->rolling_actual_bits)
- --twopass->extend_maxq;
- }
-
- twopass->extend_minq = clamp(twopass->extend_minq, 0, minq_adj_limit);
- twopass->extend_maxq = clamp(twopass->extend_maxq, 0, maxq_adj_limit);
-
- // If there is a big and undexpected undershoot then feed the extra
- // bits back in quickly. One situation where this may happen is if a
- // frame is unexpectedly almost perfectly predicted by the ARF or GF
- // but not very well predcited by the previous frame.
- if (!frame_is_kf_gf_arf(cpi) && !cpi->rc.is_src_frame_alt_ref) {
- int fast_extra_thresh = rc->base_frame_target / HIGH_UNDERSHOOT_RATIO;
- if (rc->projected_frame_size < fast_extra_thresh) {
- rc->vbr_bits_off_target_fast +=
- fast_extra_thresh - rc->projected_frame_size;
- rc->vbr_bits_off_target_fast =
- VPXMIN(rc->vbr_bits_off_target_fast, (4 * rc->avg_frame_bandwidth));
-
- // Fast adaptation of minQ if necessary to use up the extra bits.
- if (rc->avg_frame_bandwidth) {
- twopass->extend_minq_fast =
- (int)(rc->vbr_bits_off_target_fast * 8 / rc->avg_frame_bandwidth);
- }
- twopass->extend_minq_fast = VPXMIN(
- twopass->extend_minq_fast, minq_adj_limit - twopass->extend_minq);
- } else if (rc->vbr_bits_off_target_fast) {
- twopass->extend_minq_fast = VPXMIN(
- twopass->extend_minq_fast, minq_adj_limit - twopass->extend_minq);
- } else {
- twopass->extend_minq_fast = 0;
- }
- }
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_FIRSTPASS_H_
-#define VP10_ENCODER_FIRSTPASS_H_
-
-#include "vp10/encoder/lookahead.h"
-#include "vp10/encoder/ratectrl.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#if CONFIG_FP_MB_STATS
-
-#define FPMB_DCINTRA_MASK 0x01
-
-#define FPMB_MOTION_ZERO_MASK 0x02
-#define FPMB_MOTION_LEFT_MASK 0x04
-#define FPMB_MOTION_RIGHT_MASK 0x08
-#define FPMB_MOTION_UP_MASK 0x10
-#define FPMB_MOTION_DOWN_MASK 0x20
-
-#define FPMB_ERROR_SMALL_MASK 0x40
-#define FPMB_ERROR_LARGE_MASK 0x80
-#define FPMB_ERROR_SMALL_TH 2000
-#define FPMB_ERROR_LARGE_TH 48000
-
-typedef struct {
- uint8_t *mb_stats_start;
- uint8_t *mb_stats_end;
-} FIRSTPASS_MB_STATS;
-#endif
-
-#define VLOW_MOTION_THRESHOLD 950
-
-typedef struct {
- double frame;
- double weight;
- double intra_error;
- double coded_error;
- double sr_coded_error;
- double pcnt_inter;
- double pcnt_motion;
- double pcnt_second_ref;
- double pcnt_neutral;
- double intra_skip_pct;
- double inactive_zone_rows; // Image mask rows top and bottom.
- double inactive_zone_cols; // Image mask columns at left and right edges.
- double MVr;
- double mvr_abs;
- double MVc;
- double mvc_abs;
- double MVrv;
- double MVcv;
- double mv_in_out_count;
- double new_mv_count;
- double duration;
- double count;
-} FIRSTPASS_STATS;
-
-typedef enum {
- KF_UPDATE = 0,
- LF_UPDATE = 1,
- GF_UPDATE = 2,
- ARF_UPDATE = 3,
- OVERLAY_UPDATE = 4,
- FRAME_UPDATE_TYPES = 5
-} FRAME_UPDATE_TYPE;
-
-#define FC_ANIMATION_THRESH 0.15
-typedef enum {
- FC_NORMAL = 0,
- FC_GRAPHICS_ANIMATION = 1,
- FRAME_CONTENT_TYPES = 2
-} FRAME_CONTENT_TYPE;
-
-typedef struct {
- unsigned char index;
- RATE_FACTOR_LEVEL rf_level[(MAX_LAG_BUFFERS * 2) + 1];
- FRAME_UPDATE_TYPE update_type[(MAX_LAG_BUFFERS * 2) + 1];
- unsigned char arf_src_offset[(MAX_LAG_BUFFERS * 2) + 1];
- unsigned char arf_update_idx[(MAX_LAG_BUFFERS * 2) + 1];
- unsigned char arf_ref_idx[(MAX_LAG_BUFFERS * 2) + 1];
- int bit_allocation[(MAX_LAG_BUFFERS * 2) + 1];
-} GF_GROUP;
-
-typedef struct {
- unsigned int section_intra_rating;
- FIRSTPASS_STATS total_stats;
- FIRSTPASS_STATS this_frame_stats;
- const FIRSTPASS_STATS *stats_in;
- const FIRSTPASS_STATS *stats_in_start;
- const FIRSTPASS_STATS *stats_in_end;
- FIRSTPASS_STATS total_left_stats;
- int first_pass_done;
- int64_t bits_left;
- double modified_error_min;
- double modified_error_max;
- double modified_error_left;
- double mb_av_energy;
-
-#if CONFIG_FP_MB_STATS
- uint8_t *frame_mb_stats_buf;
- uint8_t *this_frame_mb_stats;
- FIRSTPASS_MB_STATS firstpass_mb_stats;
-#endif
- // An indication of the content type of the current frame
- FRAME_CONTENT_TYPE fr_content_type;
-
- // Projected total bits available for a key frame group of frames
- int64_t kf_group_bits;
-
- // Error score of frames still to be coded in kf group
- int64_t kf_group_error_left;
-
- // The fraction for a kf groups total bits allocated to the inter frames
- double kfgroup_inter_fraction;
-
- int sr_update_lag;
-
- int kf_zeromotion_pct;
- int last_kfgroup_zeromotion_pct;
- int gf_zeromotion_pct;
- int active_worst_quality;
- int baseline_active_worst_quality;
- int extend_minq;
- int extend_maxq;
- int extend_minq_fast;
-
- GF_GROUP gf_group;
-} TWO_PASS;
-
-struct VP10_COMP;
-
-void vp10_init_first_pass(struct VP10_COMP *cpi);
-void vp10_rc_get_first_pass_params(struct VP10_COMP *cpi);
-void vp10_first_pass(struct VP10_COMP *cpi,
- const struct lookahead_entry *source);
-void vp10_end_first_pass(struct VP10_COMP *cpi);
-
-void vp10_init_second_pass(struct VP10_COMP *cpi);
-void vp10_rc_get_second_pass_params(struct VP10_COMP *cpi);
-void vp10_twopass_postencode_update(struct VP10_COMP *cpi);
-
-// Post encode update of the rate control parameters for 2-pass
-void vp10_twopass_postencode_update(struct VP10_COMP *cpi);
-
-void vp10_init_subsampling(struct VP10_COMP *cpi);
-
-void vp10_calculate_coded_size(struct VP10_COMP *cpi,
- int *scaled_frame_width,
- int *scaled_frame_height);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_FIRSTPASS_H_
+++ /dev/null
-/*
- * Copyright (c) 2011 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#include <assert.h>
-#include <stdlib.h>
-
-#include "./vpx_config.h"
-
-#include "vp10/common/common.h"
-
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/extend.h"
-#include "vp10/encoder/lookahead.h"
-
-/* Return the buffer at the given absolute index and increment the index */
-static struct lookahead_entry *pop(struct lookahead_ctx *ctx,
- unsigned int *idx) {
- unsigned int index = *idx;
- struct lookahead_entry *buf = ctx->buf + index;
-
- assert(index < ctx->max_sz);
- if (++index >= ctx->max_sz)
- index -= ctx->max_sz;
- *idx = index;
- return buf;
-}
-
-
-void vp10_lookahead_destroy(struct lookahead_ctx *ctx) {
- if (ctx) {
- if (ctx->buf) {
- unsigned int i;
-
- for (i = 0; i < ctx->max_sz; i++)
- vpx_free_frame_buffer(&ctx->buf[i].img);
- free(ctx->buf);
- }
- free(ctx);
- }
-}
-
-
-struct lookahead_ctx *vp10_lookahead_init(unsigned int width,
- unsigned int height,
- unsigned int subsampling_x,
- unsigned int subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- unsigned int depth) {
- struct lookahead_ctx *ctx = NULL;
-
- // Clamp the lookahead queue depth
- depth = clamp(depth, 1, MAX_LAG_BUFFERS);
-
- // Allocate memory to keep previous source frames available.
- depth += MAX_PRE_FRAMES;
-
- // Allocate the lookahead structures
- ctx = calloc(1, sizeof(*ctx));
- if (ctx) {
- const int legacy_byte_alignment = 0;
- unsigned int i;
- ctx->max_sz = depth;
- ctx->buf = calloc(depth, sizeof(*ctx->buf));
- if (!ctx->buf)
- goto bail;
- for (i = 0; i < depth; i++)
- if (vpx_alloc_frame_buffer(&ctx->buf[i].img,
- width, height, subsampling_x, subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS,
- legacy_byte_alignment))
- goto bail;
- }
- return ctx;
- bail:
- vp10_lookahead_destroy(ctx);
- return NULL;
-}
-
-#define USE_PARTIAL_COPY 0
-
-int vp10_lookahead_push(struct lookahead_ctx *ctx, YV12_BUFFER_CONFIG *src,
- int64_t ts_start, int64_t ts_end,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- unsigned int flags) {
- struct lookahead_entry *buf;
-#if USE_PARTIAL_COPY
- int row, col, active_end;
- int mb_rows = (src->y_height + 15) >> 4;
- int mb_cols = (src->y_width + 15) >> 4;
-#endif
- int width = src->y_crop_width;
- int height = src->y_crop_height;
- int uv_width = src->uv_crop_width;
- int uv_height = src->uv_crop_height;
- int subsampling_x = src->subsampling_x;
- int subsampling_y = src->subsampling_y;
- int larger_dimensions, new_dimensions;
-
- if (ctx->sz + 1 + MAX_PRE_FRAMES > ctx->max_sz)
- return 1;
- ctx->sz++;
- buf = pop(ctx, &ctx->write_idx);
-
- new_dimensions = width != buf->img.y_crop_width ||
- height != buf->img.y_crop_height ||
- uv_width != buf->img.uv_crop_width ||
- uv_height != buf->img.uv_crop_height;
- larger_dimensions = width > buf->img.y_width ||
- height > buf->img.y_height ||
- uv_width > buf->img.uv_width ||
- uv_height > buf->img.uv_height;
- assert(!larger_dimensions || new_dimensions);
-
-#if USE_PARTIAL_COPY
- // TODO(jkoleszar): This is disabled for now, as
- // vp10_copy_and_extend_frame_with_rect is not subsampling/alpha aware.
-
- // Only do this partial copy if the following conditions are all met:
- // 1. Lookahead queue has has size of 1.
- // 2. Active map is provided.
- // 3. This is not a key frame, golden nor altref frame.
- if (!new_dimensions && ctx->max_sz == 1 && active_map && !flags) {
- for (row = 0; row < mb_rows; ++row) {
- col = 0;
-
- while (1) {
- // Find the first active macroblock in this row.
- for (; col < mb_cols; ++col) {
- if (active_map[col])
- break;
- }
-
- // No more active macroblock in this row.
- if (col == mb_cols)
- break;
-
- // Find the end of active region in this row.
- active_end = col;
-
- for (; active_end < mb_cols; ++active_end) {
- if (!active_map[active_end])
- break;
- }
-
- // Only copy this active region.
- vp10_copy_and_extend_frame_with_rect(src, &buf->img,
- row << 4,
- col << 4, 16,
- (active_end - col) << 4);
-
- // Start again from the end of this active region.
- col = active_end;
- }
-
- active_map += mb_cols;
- }
- } else {
-#endif
- if (larger_dimensions) {
- YV12_BUFFER_CONFIG new_img;
- memset(&new_img, 0, sizeof(new_img));
- if (vpx_alloc_frame_buffer(&new_img,
- width, height, subsampling_x, subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- use_highbitdepth,
-#endif
- VP9_ENC_BORDER_IN_PIXELS,
- 0))
- return 1;
- vpx_free_frame_buffer(&buf->img);
- buf->img = new_img;
- } else if (new_dimensions) {
- buf->img.y_crop_width = src->y_crop_width;
- buf->img.y_crop_height = src->y_crop_height;
- buf->img.uv_crop_width = src->uv_crop_width;
- buf->img.uv_crop_height = src->uv_crop_height;
- buf->img.subsampling_x = src->subsampling_x;
- buf->img.subsampling_y = src->subsampling_y;
- }
- // Partial copy not implemented yet
- vp10_copy_and_extend_frame(src, &buf->img);
-#if USE_PARTIAL_COPY
- }
-#endif
-
- buf->ts_start = ts_start;
- buf->ts_end = ts_end;
- buf->flags = flags;
- return 0;
-}
-
-
-struct lookahead_entry *vp10_lookahead_pop(struct lookahead_ctx *ctx,
- int drain) {
- struct lookahead_entry *buf = NULL;
-
- if (ctx->sz && (drain || ctx->sz == ctx->max_sz - MAX_PRE_FRAMES)) {
- buf = pop(ctx, &ctx->read_idx);
- ctx->sz--;
- }
- return buf;
-}
-
-
-struct lookahead_entry *vp10_lookahead_peek(struct lookahead_ctx *ctx,
- int index) {
- struct lookahead_entry *buf = NULL;
-
- if (index >= 0) {
- // Forward peek
- if (index < (int)ctx->sz) {
- index += ctx->read_idx;
- if (index >= (int)ctx->max_sz)
- index -= ctx->max_sz;
- buf = ctx->buf + index;
- }
- } else if (index < 0) {
- // Backward peek
- if (-index <= MAX_PRE_FRAMES) {
- index += ctx->read_idx;
- if (index < 0)
- index += ctx->max_sz;
- buf = ctx->buf + index;
- }
- }
-
- return buf;
-}
-
-unsigned int vp10_lookahead_depth(struct lookahead_ctx *ctx) {
- return ctx->sz;
-}
+++ /dev/null
-/*
- * Copyright (c) 2011 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_LOOKAHEAD_H_
-#define VP10_ENCODER_LOOKAHEAD_H_
-
-#include "vpx_scale/yv12config.h"
-#include "vpx/vpx_integer.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define MAX_LAG_BUFFERS 25
-
-struct lookahead_entry {
- YV12_BUFFER_CONFIG img;
- int64_t ts_start;
- int64_t ts_end;
- unsigned int flags;
-};
-
-// The max of past frames we want to keep in the queue.
-#define MAX_PRE_FRAMES 1
-
-struct lookahead_ctx {
- unsigned int max_sz; /* Absolute size of the queue */
- unsigned int sz; /* Number of buffers currently in the queue */
- unsigned int read_idx; /* Read index */
- unsigned int write_idx; /* Write index */
- struct lookahead_entry *buf; /* Buffer list */
-};
-
-/**\brief Initializes the lookahead stage
- *
- * The lookahead stage is a queue of frame buffers on which some analysis
- * may be done when buffers are enqueued.
- */
-struct lookahead_ctx *vp10_lookahead_init(unsigned int width,
- unsigned int height,
- unsigned int subsampling_x,
- unsigned int subsampling_y,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- unsigned int depth);
-
-
-/**\brief Destroys the lookahead stage
- */
-void vp10_lookahead_destroy(struct lookahead_ctx *ctx);
-
-
-/**\brief Enqueue a source buffer
- *
- * This function will copy the source image into a new framebuffer with
- * the expected stride/border.
- *
- * If active_map is non-NULL and there is only one frame in the queue, then copy
- * only active macroblocks.
- *
- * \param[in] ctx Pointer to the lookahead context
- * \param[in] src Pointer to the image to enqueue
- * \param[in] ts_start Timestamp for the start of this frame
- * \param[in] ts_end Timestamp for the end of this frame
- * \param[in] flags Flags set on this frame
- * \param[in] active_map Map that specifies which macroblock is active
- */
-int vp10_lookahead_push(struct lookahead_ctx *ctx, YV12_BUFFER_CONFIG *src,
- int64_t ts_start, int64_t ts_end,
-#if CONFIG_VP9_HIGHBITDEPTH
- int use_highbitdepth,
-#endif
- unsigned int flags);
-
-
-/**\brief Get the next source buffer to encode
- *
- *
- * \param[in] ctx Pointer to the lookahead context
- * \param[in] drain Flag indicating the buffer should be drained
- * (return a buffer regardless of the current queue depth)
- *
- * \retval NULL, if drain set and queue is empty
- * \retval NULL, if drain not set and queue not of the configured depth
- */
-struct lookahead_entry *vp10_lookahead_pop(struct lookahead_ctx *ctx,
- int drain);
-
-
-/**\brief Get a future source buffer to encode
- *
- * \param[in] ctx Pointer to the lookahead context
- * \param[in] index Index of the frame to be returned, 0 == next frame
- *
- * \retval NULL, if no buffer exists at the specified index
- */
-struct lookahead_entry *vp10_lookahead_peek(struct lookahead_ctx *ctx,
- int index);
-
-
-/**\brief Get the number of frames currently in the lookahead queue
- *
- * \param[in] ctx Pointer to the lookahead context
- */
-unsigned int vp10_lookahead_depth(struct lookahead_ctx *ctx);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_LOOKAHEAD_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/system_state.h"
-#include "vp10/encoder/segmentation.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/common/blockd.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/reconintra.h"
-
-
-static unsigned int do_16x16_motion_iteration(VP10_COMP *cpi,
- const MV *ref_mv,
- MV *dst_mv,
- int mb_row,
- int mb_col) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- const MV_SPEED_FEATURES *const mv_sf = &cpi->sf.mv;
- const vp9_variance_fn_ptr_t v_fn_ptr = cpi->fn_ptr[BLOCK_16X16];
-
- const int tmp_col_min = x->mv_col_min;
- const int tmp_col_max = x->mv_col_max;
- const int tmp_row_min = x->mv_row_min;
- const int tmp_row_max = x->mv_row_max;
- MV ref_full;
- int cost_list[5];
-
- // Further step/diamond searches as necessary
- int step_param = mv_sf->reduce_first_step_size;
- step_param = VPXMIN(step_param, MAX_MVSEARCH_STEPS - 2);
-
- vp10_set_mv_search_range(x, ref_mv);
-
- ref_full.col = ref_mv->col >> 3;
- ref_full.row = ref_mv->row >> 3;
-
- /*cpi->sf.search_method == HEX*/
- vp10_hex_search(x, &ref_full, step_param, x->errorperbit, 0,
- cond_cost_list(cpi, cost_list),
- &v_fn_ptr, 0, ref_mv, dst_mv);
-
- // Try sub-pixel MC
- // if (bestsme > error_thresh && bestsme < INT_MAX)
- {
- int distortion;
- unsigned int sse;
- cpi->find_fractional_mv_step(
- x, dst_mv, ref_mv, cpi->common.allow_high_precision_mv, x->errorperbit,
- &v_fn_ptr, 0, mv_sf->subpel_iters_per_step,
- cond_cost_list(cpi, cost_list),
- NULL, NULL,
- &distortion, &sse, NULL, 0, 0);
- }
-
- xd->mi[0]->mbmi.mode = NEWMV;
- xd->mi[0]->mbmi.mv[0].as_mv = *dst_mv;
-
- vp10_build_inter_predictors_sby(xd, mb_row, mb_col, BLOCK_16X16);
-
- /* restore UMV window */
- x->mv_col_min = tmp_col_min;
- x->mv_col_max = tmp_col_max;
- x->mv_row_min = tmp_row_min;
- x->mv_row_max = tmp_row_max;
-
- return vpx_sad16x16(x->plane[0].src.buf, x->plane[0].src.stride,
- xd->plane[0].dst.buf, xd->plane[0].dst.stride);
-}
-
-static int do_16x16_motion_search(VP10_COMP *cpi, const MV *ref_mv,
- int_mv *dst_mv, int mb_row, int mb_col) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- unsigned int err, tmp_err;
- MV tmp_mv;
-
- // Try zero MV first
- // FIXME should really use something like near/nearest MV and/or MV prediction
- err = vpx_sad16x16(x->plane[0].src.buf, x->plane[0].src.stride,
- xd->plane[0].pre[0].buf, xd->plane[0].pre[0].stride);
- dst_mv->as_int = 0;
-
- // Test last reference frame using the previous best mv as the
- // starting point (best reference) for the search
- tmp_err = do_16x16_motion_iteration(cpi, ref_mv, &tmp_mv, mb_row, mb_col);
- if (tmp_err < err) {
- err = tmp_err;
- dst_mv->as_mv = tmp_mv;
- }
-
- // If the current best reference mv is not centered on 0,0 then do a 0,0
- // based search as well.
- if (ref_mv->row != 0 || ref_mv->col != 0) {
- unsigned int tmp_err;
- MV zero_ref_mv = {0, 0}, tmp_mv;
-
- tmp_err = do_16x16_motion_iteration(cpi, &zero_ref_mv, &tmp_mv,
- mb_row, mb_col);
- if (tmp_err < err) {
- dst_mv->as_mv = tmp_mv;
- err = tmp_err;
- }
- }
-
- return err;
-}
-
-static int do_16x16_zerozero_search(VP10_COMP *cpi, int_mv *dst_mv) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- unsigned int err;
-
- // Try zero MV first
- // FIXME should really use something like near/nearest MV and/or MV prediction
- err = vpx_sad16x16(x->plane[0].src.buf, x->plane[0].src.stride,
- xd->plane[0].pre[0].buf, xd->plane[0].pre[0].stride);
-
- dst_mv->as_int = 0;
-
- return err;
-}
-static int find_best_16x16_intra(VP10_COMP *cpi, PREDICTION_MODE *pbest_mode) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- PREDICTION_MODE best_mode = -1, mode;
- unsigned int best_err = INT_MAX;
-
- // calculate SATD for each intra prediction mode;
- // we're intentionally not doing 4x4, we just want a rough estimate
- for (mode = DC_PRED; mode <= TM_PRED; mode++) {
- unsigned int err;
-
- xd->mi[0]->mbmi.mode = mode;
- vp10_predict_intra_block(xd, 2, 2, TX_16X16, mode,
- x->plane[0].src.buf, x->plane[0].src.stride,
- xd->plane[0].dst.buf, xd->plane[0].dst.stride,
- 0, 0, 0);
- err = vpx_sad16x16(x->plane[0].src.buf, x->plane[0].src.stride,
- xd->plane[0].dst.buf, xd->plane[0].dst.stride);
-
- // find best
- if (err < best_err) {
- best_err = err;
- best_mode = mode;
- }
- }
-
- if (pbest_mode)
- *pbest_mode = best_mode;
-
- return best_err;
-}
-
-static void update_mbgraph_mb_stats
-(
- VP10_COMP *cpi,
- MBGRAPH_MB_STATS *stats,
- YV12_BUFFER_CONFIG *buf,
- int mb_y_offset,
- YV12_BUFFER_CONFIG *golden_ref,
- const MV *prev_golden_ref_mv,
- YV12_BUFFER_CONFIG *alt_ref,
- int mb_row,
- int mb_col
-) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- int intra_error;
- VP10_COMMON *cm = &cpi->common;
-
- // FIXME in practice we're completely ignoring chroma here
- x->plane[0].src.buf = buf->y_buffer + mb_y_offset;
- x->plane[0].src.stride = buf->y_stride;
-
- xd->plane[0].dst.buf = get_frame_new_buffer(cm)->y_buffer + mb_y_offset;
- xd->plane[0].dst.stride = get_frame_new_buffer(cm)->y_stride;
-
- // do intra 16x16 prediction
- intra_error = find_best_16x16_intra(cpi,
- &stats->ref[INTRA_FRAME].m.mode);
- if (intra_error <= 0)
- intra_error = 1;
- stats->ref[INTRA_FRAME].err = intra_error;
-
- // Golden frame MV search, if it exists and is different than last frame
- if (golden_ref) {
- int g_motion_error;
- xd->plane[0].pre[0].buf = golden_ref->y_buffer + mb_y_offset;
- xd->plane[0].pre[0].stride = golden_ref->y_stride;
- g_motion_error = do_16x16_motion_search(cpi,
- prev_golden_ref_mv,
- &stats->ref[GOLDEN_FRAME].m.mv,
- mb_row, mb_col);
- stats->ref[GOLDEN_FRAME].err = g_motion_error;
- } else {
- stats->ref[GOLDEN_FRAME].err = INT_MAX;
- stats->ref[GOLDEN_FRAME].m.mv.as_int = 0;
- }
-
- // Do an Alt-ref frame MV search, if it exists and is different than
- // last/golden frame.
- if (alt_ref) {
- int a_motion_error;
- xd->plane[0].pre[0].buf = alt_ref->y_buffer + mb_y_offset;
- xd->plane[0].pre[0].stride = alt_ref->y_stride;
- a_motion_error = do_16x16_zerozero_search(cpi,
- &stats->ref[ALTREF_FRAME].m.mv);
-
- stats->ref[ALTREF_FRAME].err = a_motion_error;
- } else {
- stats->ref[ALTREF_FRAME].err = INT_MAX;
- stats->ref[ALTREF_FRAME].m.mv.as_int = 0;
- }
-}
-
-static void update_mbgraph_frame_stats(VP10_COMP *cpi,
- MBGRAPH_FRAME_STATS *stats,
- YV12_BUFFER_CONFIG *buf,
- YV12_BUFFER_CONFIG *golden_ref,
- YV12_BUFFER_CONFIG *alt_ref) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- VP10_COMMON *const cm = &cpi->common;
-
- int mb_col, mb_row, offset = 0;
- int mb_y_offset = 0, arf_y_offset = 0, gld_y_offset = 0;
- MV gld_top_mv = {0, 0};
- MODE_INFO mi_local;
-
- vp10_zero(mi_local);
- // Set up limit values for motion vectors to prevent them extending outside
- // the UMV borders.
- x->mv_row_min = -BORDER_MV_PIXELS_B16;
- x->mv_row_max = (cm->mb_rows - 1) * 8 + BORDER_MV_PIXELS_B16;
- xd->up_available = 0;
- xd->plane[0].dst.stride = buf->y_stride;
- xd->plane[0].pre[0].stride = buf->y_stride;
- xd->plane[1].dst.stride = buf->uv_stride;
- xd->mi[0] = &mi_local;
- mi_local.mbmi.sb_type = BLOCK_16X16;
- mi_local.mbmi.ref_frame[0] = LAST_FRAME;
- mi_local.mbmi.ref_frame[1] = NONE;
-
- for (mb_row = 0; mb_row < cm->mb_rows; mb_row++) {
- MV gld_left_mv = gld_top_mv;
- int mb_y_in_offset = mb_y_offset;
- int arf_y_in_offset = arf_y_offset;
- int gld_y_in_offset = gld_y_offset;
-
- // Set up limit values for motion vectors to prevent them extending outside
- // the UMV borders.
- x->mv_col_min = -BORDER_MV_PIXELS_B16;
- x->mv_col_max = (cm->mb_cols - 1) * 8 + BORDER_MV_PIXELS_B16;
- xd->left_available = 0;
-
- for (mb_col = 0; mb_col < cm->mb_cols; mb_col++) {
- MBGRAPH_MB_STATS *mb_stats = &stats->mb_stats[offset + mb_col];
-
- update_mbgraph_mb_stats(cpi, mb_stats, buf, mb_y_in_offset,
- golden_ref, &gld_left_mv, alt_ref,
- mb_row, mb_col);
- gld_left_mv = mb_stats->ref[GOLDEN_FRAME].m.mv.as_mv;
- if (mb_col == 0) {
- gld_top_mv = gld_left_mv;
- }
- xd->left_available = 1;
- mb_y_in_offset += 16;
- gld_y_in_offset += 16;
- arf_y_in_offset += 16;
- x->mv_col_min -= 16;
- x->mv_col_max -= 16;
- }
- xd->up_available = 1;
- mb_y_offset += buf->y_stride * 16;
- gld_y_offset += golden_ref->y_stride * 16;
- if (alt_ref)
- arf_y_offset += alt_ref->y_stride * 16;
- x->mv_row_min -= 16;
- x->mv_row_max -= 16;
- offset += cm->mb_cols;
- }
-}
-
-// void separate_arf_mbs_byzz
-static void separate_arf_mbs(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- int mb_col, mb_row, offset, i;
- int mi_row, mi_col;
- int ncnt[4] = { 0 };
- int n_frames = cpi->mbgraph_n_frames;
-
- int *arf_not_zz;
-
- CHECK_MEM_ERROR(cm, arf_not_zz,
- vpx_calloc(cm->mb_rows * cm->mb_cols * sizeof(*arf_not_zz),
- 1));
-
- // We are not interested in results beyond the alt ref itself.
- if (n_frames > cpi->rc.frames_till_gf_update_due)
- n_frames = cpi->rc.frames_till_gf_update_due;
-
- // defer cost to reference frames
- for (i = n_frames - 1; i >= 0; i--) {
- MBGRAPH_FRAME_STATS *frame_stats = &cpi->mbgraph_stats[i];
-
- for (offset = 0, mb_row = 0; mb_row < cm->mb_rows;
- offset += cm->mb_cols, mb_row++) {
- for (mb_col = 0; mb_col < cm->mb_cols; mb_col++) {
- MBGRAPH_MB_STATS *mb_stats = &frame_stats->mb_stats[offset + mb_col];
-
- int altref_err = mb_stats->ref[ALTREF_FRAME].err;
- int intra_err = mb_stats->ref[INTRA_FRAME ].err;
- int golden_err = mb_stats->ref[GOLDEN_FRAME].err;
-
- // Test for altref vs intra and gf and that its mv was 0,0.
- if (altref_err > 1000 ||
- altref_err > intra_err ||
- altref_err > golden_err) {
- arf_not_zz[offset + mb_col]++;
- }
- }
- }
- }
-
- // arf_not_zz is indexed by MB, but this loop is indexed by MI to avoid out
- // of bound access in segmentation_map
- for (mi_row = 0; mi_row < cm->mi_rows; mi_row++) {
- for (mi_col = 0; mi_col < cm->mi_cols; mi_col++) {
- // If any of the blocks in the sequence failed then the MB
- // goes in segment 0
- if (arf_not_zz[mi_row / 2 * cm->mb_cols + mi_col / 2]) {
- ncnt[0]++;
- cpi->segmentation_map[mi_row * cm->mi_cols + mi_col] = 0;
- } else {
- cpi->segmentation_map[mi_row * cm->mi_cols + mi_col] = 1;
- ncnt[1]++;
- }
- }
- }
-
- // Only bother with segmentation if over 10% of the MBs in static segment
- // if ( ncnt[1] && (ncnt[0] / ncnt[1] < 10) )
- if (1) {
- // Note % of blocks that are marked as static
- if (cm->MBs)
- cpi->static_mb_pct = (ncnt[1] * 100) / (cm->mi_rows * cm->mi_cols);
-
- // This error case should not be reachable as this function should
- // never be called with the common data structure uninitialized.
- else
- cpi->static_mb_pct = 0;
-
- vp10_enable_segmentation(&cm->seg);
- } else {
- cpi->static_mb_pct = 0;
- vp10_disable_segmentation(&cm->seg);
- }
-
- // Free localy allocated storage
- vpx_free(arf_not_zz);
-}
-
-void vp10_update_mbgraph_stats(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- int i, n_frames = vp10_lookahead_depth(cpi->lookahead);
- YV12_BUFFER_CONFIG *golden_ref = get_ref_frame_buffer(cpi, GOLDEN_FRAME);
-
- assert(golden_ref != NULL);
-
- // we need to look ahead beyond where the ARF transitions into
- // being a GF - so exit if we don't look ahead beyond that
- if (n_frames <= cpi->rc.frames_till_gf_update_due)
- return;
-
- if (n_frames > MAX_LAG_BUFFERS)
- n_frames = MAX_LAG_BUFFERS;
-
- cpi->mbgraph_n_frames = n_frames;
- for (i = 0; i < n_frames; i++) {
- MBGRAPH_FRAME_STATS *frame_stats = &cpi->mbgraph_stats[i];
- memset(frame_stats->mb_stats, 0,
- cm->mb_rows * cm->mb_cols * sizeof(*cpi->mbgraph_stats[i].mb_stats));
- }
-
- // do motion search to find contribution of each reference to data
- // later on in this GF group
- // FIXME really, the GF/last MC search should be done forward, and
- // the ARF MC search backwards, to get optimal results for MV caching
- for (i = 0; i < n_frames; i++) {
- MBGRAPH_FRAME_STATS *frame_stats = &cpi->mbgraph_stats[i];
- struct lookahead_entry *q_cur = vp10_lookahead_peek(cpi->lookahead, i);
-
- assert(q_cur != NULL);
-
- update_mbgraph_frame_stats(cpi, frame_stats, &q_cur->img,
- golden_ref, cpi->Source);
- }
-
- vpx_clear_system_state();
-
- separate_arf_mbs(cpi);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_MBGRAPH_H_
-#define VP10_ENCODER_MBGRAPH_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct {
- struct {
- int err;
- union {
- int_mv mv;
- PREDICTION_MODE mode;
- } m;
- } ref[MAX_REF_FRAMES];
-} MBGRAPH_MB_STATS;
-
-typedef struct {
- MBGRAPH_MB_STATS *mb_stats;
-} MBGRAPH_FRAME_STATS;
-
-struct VP10_COMP;
-
-void vp10_update_mbgraph_stats(struct VP10_COMP *cpi);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_MBGRAPH_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-#include <math.h>
-#include <stdio.h>
-
-#include "./vpx_config.h"
-#include "./vpx_dsp_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/common.h"
-#include "vp10/common/reconinter.h"
-
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/mcomp.h"
-
-// #define NEW_DIAMOND_SEARCH
-
-static INLINE const uint8_t *get_buf_from_mv(const struct buf_2d *buf,
- const MV *mv) {
- return &buf->buf[mv->row * buf->stride + mv->col];
-}
-
-void vp10_set_mv_search_range(MACROBLOCK *x, const MV *mv) {
- int col_min = (mv->col >> 3) - MAX_FULL_PEL_VAL + (mv->col & 7 ? 1 : 0);
- int row_min = (mv->row >> 3) - MAX_FULL_PEL_VAL + (mv->row & 7 ? 1 : 0);
- int col_max = (mv->col >> 3) + MAX_FULL_PEL_VAL;
- int row_max = (mv->row >> 3) + MAX_FULL_PEL_VAL;
-
- col_min = VPXMAX(col_min, (MV_LOW >> 3) + 1);
- row_min = VPXMAX(row_min, (MV_LOW >> 3) + 1);
- col_max = VPXMIN(col_max, (MV_UPP >> 3) - 1);
- row_max = VPXMIN(row_max, (MV_UPP >> 3) - 1);
-
- // Get intersection of UMV window and valid MV window to reduce # of checks
- // in diamond search.
- if (x->mv_col_min < col_min)
- x->mv_col_min = col_min;
- if (x->mv_col_max > col_max)
- x->mv_col_max = col_max;
- if (x->mv_row_min < row_min)
- x->mv_row_min = row_min;
- if (x->mv_row_max > row_max)
- x->mv_row_max = row_max;
-}
-
-int vp10_init_search_range(int size) {
- int sr = 0;
- // Minimum search size no matter what the passed in value.
- size = VPXMAX(16, size);
-
- while ((size << sr) < MAX_FULL_PEL_VAL)
- sr++;
-
- sr = VPXMIN(sr, MAX_MVSEARCH_STEPS - 2);
- return sr;
-}
-
-static INLINE int mv_cost(const MV *mv,
- const int *joint_cost, int *const comp_cost[2]) {
- return joint_cost[vp10_get_mv_joint(mv)] +
- comp_cost[0][mv->row] + comp_cost[1][mv->col];
-}
-
-int vp10_mv_bit_cost(const MV *mv, const MV *ref,
- const int *mvjcost, int *mvcost[2], int weight) {
- const MV diff = { mv->row - ref->row,
- mv->col - ref->col };
- return ROUND_POWER_OF_TWO(mv_cost(&diff, mvjcost, mvcost) * weight, 7);
-}
-
-static int mv_err_cost(const MV *mv, const MV *ref,
- const int *mvjcost, int *mvcost[2],
- int error_per_bit) {
- if (mvcost) {
- const MV diff = { mv->row - ref->row,
- mv->col - ref->col };
- return ROUND_POWER_OF_TWO(mv_cost(&diff, mvjcost, mvcost) *
- error_per_bit, 13);
- }
- return 0;
-}
-
-static int mvsad_err_cost(const MACROBLOCK *x, const MV *mv, const MV *ref,
- int error_per_bit) {
- const MV diff = { mv->row - ref->row,
- mv->col - ref->col };
- return ROUND_POWER_OF_TWO(mv_cost(&diff, x->nmvjointsadcost,
- x->nmvsadcost) * error_per_bit, 8);
-}
-
-void vp10_init_dsmotion_compensation(search_site_config *cfg, int stride) {
- int len, ss_count = 1;
-
- cfg->ss[0].mv.col = cfg->ss[0].mv.row = 0;
- cfg->ss[0].offset = 0;
-
- for (len = MAX_FIRST_STEP; len > 0; len /= 2) {
- // Generate offsets for 4 search sites per step.
- const MV ss_mvs[] = {{-len, 0}, {len, 0}, {0, -len}, {0, len}};
- int i;
- for (i = 0; i < 4; ++i) {
- search_site *const ss = &cfg->ss[ss_count++];
- ss->mv = ss_mvs[i];
- ss->offset = ss->mv.row * stride + ss->mv.col;
- }
- }
-
- cfg->ss_count = ss_count;
- cfg->searches_per_step = 4;
-}
-
-void vp10_init3smotion_compensation(search_site_config *cfg, int stride) {
- int len, ss_count = 1;
-
- cfg->ss[0].mv.col = cfg->ss[0].mv.row = 0;
- cfg->ss[0].offset = 0;
-
- for (len = MAX_FIRST_STEP; len > 0; len /= 2) {
- // Generate offsets for 8 search sites per step.
- const MV ss_mvs[8] = {
- {-len, 0 }, {len, 0 }, { 0, -len}, {0, len},
- {-len, -len}, {-len, len}, {len, -len}, {len, len}
- };
- int i;
- for (i = 0; i < 8; ++i) {
- search_site *const ss = &cfg->ss[ss_count++];
- ss->mv = ss_mvs[i];
- ss->offset = ss->mv.row * stride + ss->mv.col;
- }
- }
-
- cfg->ss_count = ss_count;
- cfg->searches_per_step = 8;
-}
-
-/*
- * To avoid the penalty for crossing cache-line read, preload the reference
- * area in a small buffer, which is aligned to make sure there won't be crossing
- * cache-line read while reading from this buffer. This reduced the cpu
- * cycles spent on reading ref data in sub-pixel filter functions.
- * TODO: Currently, since sub-pixel search range here is -3 ~ 3, copy 22 rows x
- * 32 cols area that is enough for 16x16 macroblock. Later, for SPLITMV, we
- * could reduce the area.
- */
-
-/* estimated cost of a motion vector (r,c) */
-#define MVC(r, c) \
- (mvcost ? \
- ((mvjcost[((r) != rr) * 2 + ((c) != rc)] + \
- mvcost[0][((r) - rr)] + mvcost[1][((c) - rc)]) * \
- error_per_bit + 4096) >> 13 : 0)
-
-
-// convert motion vector component to offset for sv[a]f calc
-static INLINE int sp(int x) {
- return x & 7;
-}
-
-static INLINE const uint8_t *pre(const uint8_t *buf, int stride, int r, int c) {
- return &buf[(r >> 3) * stride + (c >> 3)];
-}
-
-/* checks if (r, c) has better score than previous best */
-#define CHECK_BETTER(v, r, c) \
- if (c >= minc && c <= maxc && r >= minr && r <= maxr) { \
- if (second_pred == NULL) \
- thismse = vfp->svf(pre(y, y_stride, r, c), y_stride, sp(c), sp(r), z, \
- src_stride, &sse); \
- else \
- thismse = vfp->svaf(pre(y, y_stride, r, c), y_stride, sp(c), sp(r), \
- z, src_stride, &sse, second_pred); \
- if ((v = MVC(r, c) + thismse) < besterr) { \
- besterr = v; \
- br = r; \
- bc = c; \
- *distortion = thismse; \
- *sse1 = sse; \
- } \
- } else { \
- v = INT_MAX; \
- }
-
-#define FIRST_LEVEL_CHECKS \
- { \
- unsigned int left, right, up, down, diag; \
- CHECK_BETTER(left, tr, tc - hstep); \
- CHECK_BETTER(right, tr, tc + hstep); \
- CHECK_BETTER(up, tr - hstep, tc); \
- CHECK_BETTER(down, tr + hstep, tc); \
- whichdir = (left < right ? 0 : 1) + \
- (up < down ? 0 : 2); \
- switch (whichdir) { \
- case 0: \
- CHECK_BETTER(diag, tr - hstep, tc - hstep); \
- break; \
- case 1: \
- CHECK_BETTER(diag, tr - hstep, tc + hstep); \
- break; \
- case 2: \
- CHECK_BETTER(diag, tr + hstep, tc - hstep); \
- break; \
- case 3: \
- CHECK_BETTER(diag, tr + hstep, tc + hstep); \
- break; \
- } \
- }
-
-#define SECOND_LEVEL_CHECKS \
- { \
- int kr, kc; \
- unsigned int second; \
- if (tr != br && tc != bc) { \
- kr = br - tr; \
- kc = bc - tc; \
- CHECK_BETTER(second, tr + kr, tc + 2 * kc); \
- CHECK_BETTER(second, tr + 2 * kr, tc + kc); \
- } else if (tr == br && tc != bc) { \
- kc = bc - tc; \
- CHECK_BETTER(second, tr + hstep, tc + 2 * kc); \
- CHECK_BETTER(second, tr - hstep, tc + 2 * kc); \
- switch (whichdir) { \
- case 0: \
- case 1: \
- CHECK_BETTER(second, tr + hstep, tc + kc); \
- break; \
- case 2: \
- case 3: \
- CHECK_BETTER(second, tr - hstep, tc + kc); \
- break; \
- } \
- } else if (tr != br && tc == bc) { \
- kr = br - tr; \
- CHECK_BETTER(second, tr + 2 * kr, tc + hstep); \
- CHECK_BETTER(second, tr + 2 * kr, tc - hstep); \
- switch (whichdir) { \
- case 0: \
- case 2: \
- CHECK_BETTER(second, tr + kr, tc + hstep); \
- break; \
- case 1: \
- case 3: \
- CHECK_BETTER(second, tr + kr, tc - hstep); \
- break; \
- } \
- } \
- }
-
-// TODO(yunqingwang): SECOND_LEVEL_CHECKS_BEST was a rewrote of
-// SECOND_LEVEL_CHECKS, and SECOND_LEVEL_CHECKS should be rewritten
-// later in the same way.
-#define SECOND_LEVEL_CHECKS_BEST \
- { \
- unsigned int second; \
- int br0 = br; \
- int bc0 = bc; \
- assert(tr == br || tc == bc); \
- if (tr == br && tc != bc) { \
- kc = bc - tc; \
- } else if (tr != br && tc == bc) { \
- kr = br - tr; \
- } \
- CHECK_BETTER(second, br0 + kr, bc0); \
- CHECK_BETTER(second, br0, bc0 + kc); \
- if (br0 != br || bc0 != bc) { \
- CHECK_BETTER(second, br0 + kr, bc0 + kc); \
- } \
- }
-
-#define SETUP_SUBPEL_SEARCH \
- const uint8_t *const z = x->plane[0].src.buf; \
- const int src_stride = x->plane[0].src.stride; \
- const MACROBLOCKD *xd = &x->e_mbd; \
- unsigned int besterr = INT_MAX; \
- unsigned int sse; \
- unsigned int whichdir; \
- int thismse; \
- const unsigned int halfiters = iters_per_step; \
- const unsigned int quarteriters = iters_per_step; \
- const unsigned int eighthiters = iters_per_step; \
- const int y_stride = xd->plane[0].pre[0].stride; \
- const int offset = bestmv->row * y_stride + bestmv->col; \
- const uint8_t *const y = xd->plane[0].pre[0].buf; \
- \
- int rr = ref_mv->row; \
- int rc = ref_mv->col; \
- int br = bestmv->row * 8; \
- int bc = bestmv->col * 8; \
- int hstep = 4; \
- const int minc = VPXMAX(x->mv_col_min * 8, ref_mv->col - MV_MAX); \
- const int maxc = VPXMIN(x->mv_col_max * 8, ref_mv->col + MV_MAX); \
- const int minr = VPXMAX(x->mv_row_min * 8, ref_mv->row - MV_MAX); \
- const int maxr = VPXMIN(x->mv_row_max * 8, ref_mv->row + MV_MAX); \
- int tr = br; \
- int tc = bc; \
- \
- bestmv->row *= 8; \
- bestmv->col *= 8;
-
-static unsigned int setup_center_error(const MACROBLOCKD *xd,
- const MV *bestmv,
- const MV *ref_mv,
- int error_per_bit,
- const vp9_variance_fn_ptr_t *vfp,
- const uint8_t *const src,
- const int src_stride,
- const uint8_t *const y,
- int y_stride,
- const uint8_t *second_pred,
- int w, int h, int offset,
- int *mvjcost, int *mvcost[2],
- unsigned int *sse1,
- int *distortion) {
- unsigned int besterr;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (second_pred != NULL) {
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- DECLARE_ALIGNED(16, uint16_t, comp_pred16[64 * 64]);
- vpx_highbd_comp_avg_pred(comp_pred16, second_pred, w, h, y + offset,
- y_stride);
- besterr = vfp->vf(CONVERT_TO_BYTEPTR(comp_pred16), w, src, src_stride,
- sse1);
- } else {
- DECLARE_ALIGNED(16, uint8_t, comp_pred[64 * 64]);
- vpx_comp_avg_pred(comp_pred, second_pred, w, h, y + offset, y_stride);
- besterr = vfp->vf(comp_pred, w, src, src_stride, sse1);
- }
- } else {
- besterr = vfp->vf(y + offset, y_stride, src, src_stride, sse1);
- }
- *distortion = besterr;
- besterr += mv_err_cost(bestmv, ref_mv, mvjcost, mvcost, error_per_bit);
-#else
- (void) xd;
- if (second_pred != NULL) {
- DECLARE_ALIGNED(16, uint8_t, comp_pred[64 * 64]);
- vpx_comp_avg_pred(comp_pred, second_pred, w, h, y + offset, y_stride);
- besterr = vfp->vf(comp_pred, w, src, src_stride, sse1);
- } else {
- besterr = vfp->vf(y + offset, y_stride, src, src_stride, sse1);
- }
- *distortion = besterr;
- besterr += mv_err_cost(bestmv, ref_mv, mvjcost, mvcost, error_per_bit);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- return besterr;
-}
-
-static INLINE int divide_and_round(const int n, const int d) {
- return ((n < 0) ^ (d < 0)) ? ((n - d / 2) / d) : ((n + d / 2) / d);
-}
-
-static INLINE int is_cost_list_wellbehaved(int *cost_list) {
- return cost_list[0] < cost_list[1] &&
- cost_list[0] < cost_list[2] &&
- cost_list[0] < cost_list[3] &&
- cost_list[0] < cost_list[4];
-}
-
-// Returns surface minima estimate at given precision in 1/2^n bits.
-// Assume a model for the cost surface: S = A(x - x0)^2 + B(y - y0)^2 + C
-// For a given set of costs S0, S1, S2, S3, S4 at points
-// (y, x) = (0, 0), (0, -1), (1, 0), (0, 1) and (-1, 0) respectively,
-// the solution for the location of the minima (x0, y0) is given by:
-// x0 = 1/2 (S1 - S3)/(S1 + S3 - 2*S0),
-// y0 = 1/2 (S4 - S2)/(S4 + S2 - 2*S0).
-// The code below is an integerized version of that.
-static void get_cost_surf_min(int *cost_list, int *ir, int *ic,
- int bits) {
- *ic = divide_and_round((cost_list[1] - cost_list[3]) * (1 << (bits - 1)),
- (cost_list[1] - 2 * cost_list[0] + cost_list[3]));
- *ir = divide_and_round((cost_list[4] - cost_list[2]) * (1 << (bits - 1)),
- (cost_list[4] - 2 * cost_list[0] + cost_list[2]));
-}
-
-int vp10_find_best_sub_pixel_tree_pruned_evenmore(
- const MACROBLOCK *x,
- MV *bestmv, const MV *ref_mv,
- int allow_hp,
- int error_per_bit,
- const vp9_variance_fn_ptr_t *vfp,
- int forced_stop,
- int iters_per_step,
- int *cost_list,
- int *mvjcost, int *mvcost[2],
- int *distortion,
- unsigned int *sse1,
- const uint8_t *second_pred,
- int w, int h) {
- SETUP_SUBPEL_SEARCH;
- besterr = setup_center_error(xd, bestmv, ref_mv, error_per_bit, vfp,
- z, src_stride, y, y_stride, second_pred,
- w, h, offset, mvjcost, mvcost,
- sse1, distortion);
- (void) halfiters;
- (void) quarteriters;
- (void) eighthiters;
- (void) whichdir;
- (void) allow_hp;
- (void) forced_stop;
- (void) hstep;
-
- if (cost_list &&
- cost_list[0] != INT_MAX && cost_list[1] != INT_MAX &&
- cost_list[2] != INT_MAX && cost_list[3] != INT_MAX &&
- cost_list[4] != INT_MAX &&
- is_cost_list_wellbehaved(cost_list)) {
- int ir, ic;
- unsigned int minpt;
- get_cost_surf_min(cost_list, &ir, &ic, 2);
- if (ir != 0 || ic != 0) {
- CHECK_BETTER(minpt, tr + 2 * ir, tc + 2 * ic);
- }
- } else {
- FIRST_LEVEL_CHECKS;
- if (halfiters > 1) {
- SECOND_LEVEL_CHECKS;
- }
-
- tr = br;
- tc = bc;
-
- // Each subsequent iteration checks at least one point in common with
- // the last iteration could be 2 ( if diag selected) 1/4 pel
- // Note forced_stop: 0 - full, 1 - qtr only, 2 - half only
- if (forced_stop != 2) {
- hstep >>= 1;
- FIRST_LEVEL_CHECKS;
- if (quarteriters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- }
- }
-
- tr = br;
- tc = bc;
-
- if (allow_hp && vp10_use_mv_hp(ref_mv) && forced_stop == 0) {
- hstep >>= 1;
- FIRST_LEVEL_CHECKS;
- if (eighthiters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- }
-
- bestmv->row = br;
- bestmv->col = bc;
-
- if ((abs(bestmv->col - ref_mv->col) > (MAX_FULL_PEL_VAL << 3)) ||
- (abs(bestmv->row - ref_mv->row) > (MAX_FULL_PEL_VAL << 3)))
- return INT_MAX;
-
- return besterr;
-}
-
-int vp10_find_best_sub_pixel_tree_pruned_more(const MACROBLOCK *x,
- MV *bestmv, const MV *ref_mv,
- int allow_hp,
- int error_per_bit,
- const vp9_variance_fn_ptr_t *vfp,
- int forced_stop,
- int iters_per_step,
- int *cost_list,
- int *mvjcost, int *mvcost[2],
- int *distortion,
- unsigned int *sse1,
- const uint8_t *second_pred,
- int w, int h) {
- SETUP_SUBPEL_SEARCH;
- besterr = setup_center_error(xd, bestmv, ref_mv, error_per_bit, vfp,
- z, src_stride, y, y_stride, second_pred,
- w, h, offset, mvjcost, mvcost,
- sse1, distortion);
- if (cost_list &&
- cost_list[0] != INT_MAX && cost_list[1] != INT_MAX &&
- cost_list[2] != INT_MAX && cost_list[3] != INT_MAX &&
- cost_list[4] != INT_MAX &&
- is_cost_list_wellbehaved(cost_list)) {
- unsigned int minpt;
- int ir, ic;
- get_cost_surf_min(cost_list, &ir, &ic, 1);
- if (ir != 0 || ic != 0) {
- CHECK_BETTER(minpt, tr + ir * hstep, tc + ic * hstep);
- }
- } else {
- FIRST_LEVEL_CHECKS;
- if (halfiters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- }
-
- // Each subsequent iteration checks at least one point in common with
- // the last iteration could be 2 ( if diag selected) 1/4 pel
-
- // Note forced_stop: 0 - full, 1 - qtr only, 2 - half only
- if (forced_stop != 2) {
- tr = br;
- tc = bc;
- hstep >>= 1;
- FIRST_LEVEL_CHECKS;
- if (quarteriters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- }
-
- if (allow_hp && vp10_use_mv_hp(ref_mv) && forced_stop == 0) {
- tr = br;
- tc = bc;
- hstep >>= 1;
- FIRST_LEVEL_CHECKS;
- if (eighthiters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- }
- // These lines insure static analysis doesn't warn that
- // tr and tc aren't used after the above point.
- (void) tr;
- (void) tc;
-
- bestmv->row = br;
- bestmv->col = bc;
-
- if ((abs(bestmv->col - ref_mv->col) > (MAX_FULL_PEL_VAL << 3)) ||
- (abs(bestmv->row - ref_mv->row) > (MAX_FULL_PEL_VAL << 3)))
- return INT_MAX;
-
- return besterr;
-}
-
-int vp10_find_best_sub_pixel_tree_pruned(const MACROBLOCK *x,
- MV *bestmv, const MV *ref_mv,
- int allow_hp,
- int error_per_bit,
- const vp9_variance_fn_ptr_t *vfp,
- int forced_stop,
- int iters_per_step,
- int *cost_list,
- int *mvjcost, int *mvcost[2],
- int *distortion,
- unsigned int *sse1,
- const uint8_t *second_pred,
- int w, int h) {
- SETUP_SUBPEL_SEARCH;
- besterr = setup_center_error(xd, bestmv, ref_mv, error_per_bit, vfp,
- z, src_stride, y, y_stride, second_pred,
- w, h, offset, mvjcost, mvcost,
- sse1, distortion);
- if (cost_list &&
- cost_list[0] != INT_MAX && cost_list[1] != INT_MAX &&
- cost_list[2] != INT_MAX && cost_list[3] != INT_MAX &&
- cost_list[4] != INT_MAX) {
- unsigned int left, right, up, down, diag;
- whichdir = (cost_list[1] < cost_list[3] ? 0 : 1) +
- (cost_list[2] < cost_list[4] ? 0 : 2);
- switch (whichdir) {
- case 0:
- CHECK_BETTER(left, tr, tc - hstep);
- CHECK_BETTER(down, tr + hstep, tc);
- CHECK_BETTER(diag, tr + hstep, tc - hstep);
- break;
- case 1:
- CHECK_BETTER(right, tr, tc + hstep);
- CHECK_BETTER(down, tr + hstep, tc);
- CHECK_BETTER(diag, tr + hstep, tc + hstep);
- break;
- case 2:
- CHECK_BETTER(left, tr, tc - hstep);
- CHECK_BETTER(up, tr - hstep, tc);
- CHECK_BETTER(diag, tr - hstep, tc - hstep);
- break;
- case 3:
- CHECK_BETTER(right, tr, tc + hstep);
- CHECK_BETTER(up, tr - hstep, tc);
- CHECK_BETTER(diag, tr - hstep, tc + hstep);
- break;
- }
- } else {
- FIRST_LEVEL_CHECKS;
- if (halfiters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- }
-
- tr = br;
- tc = bc;
-
- // Each subsequent iteration checks at least one point in common with
- // the last iteration could be 2 ( if diag selected) 1/4 pel
-
- // Note forced_stop: 0 - full, 1 - qtr only, 2 - half only
- if (forced_stop != 2) {
- hstep >>= 1;
- FIRST_LEVEL_CHECKS;
- if (quarteriters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- tr = br;
- tc = bc;
- }
-
- if (allow_hp && vp10_use_mv_hp(ref_mv) && forced_stop == 0) {
- hstep >>= 1;
- FIRST_LEVEL_CHECKS;
- if (eighthiters > 1) {
- SECOND_LEVEL_CHECKS;
- }
- tr = br;
- tc = bc;
- }
- // These lines insure static analysis doesn't warn that
- // tr and tc aren't used after the above point.
- (void) tr;
- (void) tc;
-
- bestmv->row = br;
- bestmv->col = bc;
-
- if ((abs(bestmv->col - ref_mv->col) > (MAX_FULL_PEL_VAL << 3)) ||
- (abs(bestmv->row - ref_mv->row) > (MAX_FULL_PEL_VAL << 3)))
- return INT_MAX;
-
- return besterr;
-}
-
-static const MV search_step_table[12] = {
- // left, right, up, down
- {0, -4}, {0, 4}, {-4, 0}, {4, 0},
- {0, -2}, {0, 2}, {-2, 0}, {2, 0},
- {0, -1}, {0, 1}, {-1, 0}, {1, 0}
-};
-
-int vp10_find_best_sub_pixel_tree(const MACROBLOCK *x,
- MV *bestmv, const MV *ref_mv,
- int allow_hp,
- int error_per_bit,
- const vp9_variance_fn_ptr_t *vfp,
- int forced_stop,
- int iters_per_step,
- int *cost_list,
- int *mvjcost, int *mvcost[2],
- int *distortion,
- unsigned int *sse1,
- const uint8_t *second_pred,
- int w, int h) {
- const uint8_t *const z = x->plane[0].src.buf;
- const uint8_t *const src_address = z;
- const int src_stride = x->plane[0].src.stride;
- const MACROBLOCKD *xd = &x->e_mbd;
- unsigned int besterr = INT_MAX;
- unsigned int sse;
- int thismse;
- const int y_stride = xd->plane[0].pre[0].stride;
- const int offset = bestmv->row * y_stride + bestmv->col;
- const uint8_t *const y = xd->plane[0].pre[0].buf;
-
- int rr = ref_mv->row;
- int rc = ref_mv->col;
- int br = bestmv->row * 8;
- int bc = bestmv->col * 8;
- int hstep = 4;
- int iter, round = 3 - forced_stop;
- const int minc = VPXMAX(x->mv_col_min * 8, ref_mv->col - MV_MAX);
- const int maxc = VPXMIN(x->mv_col_max * 8, ref_mv->col + MV_MAX);
- const int minr = VPXMAX(x->mv_row_min * 8, ref_mv->row - MV_MAX);
- const int maxr = VPXMIN(x->mv_row_max * 8, ref_mv->row + MV_MAX);
- int tr = br;
- int tc = bc;
- const MV *search_step = search_step_table;
- int idx, best_idx = -1;
- unsigned int cost_array[5];
- int kr, kc;
-
- if (!(allow_hp && vp10_use_mv_hp(ref_mv)))
- if (round == 3)
- round = 2;
-
- bestmv->row *= 8;
- bestmv->col *= 8;
-
- besterr = setup_center_error(xd, bestmv, ref_mv, error_per_bit, vfp,
- z, src_stride, y, y_stride, second_pred,
- w, h, offset, mvjcost, mvcost,
- sse1, distortion);
-
- (void) cost_list; // to silence compiler warning
-
- for (iter = 0; iter < round; ++iter) {
- // Check vertical and horizontal sub-pixel positions.
- for (idx = 0; idx < 4; ++idx) {
- tr = br + search_step[idx].row;
- tc = bc + search_step[idx].col;
- if (tc >= minc && tc <= maxc && tr >= minr && tr <= maxr) {
- const uint8_t *const pre_address = y + (tr >> 3) * y_stride + (tc >> 3);
- MV this_mv;
- this_mv.row = tr;
- this_mv.col = tc;
- if (second_pred == NULL)
- thismse = vfp->svf(pre_address, y_stride, sp(tc), sp(tr),
- src_address, src_stride, &sse);
- else
- thismse = vfp->svaf(pre_address, y_stride, sp(tc), sp(tr),
- src_address, src_stride, &sse, second_pred);
- cost_array[idx] = thismse +
- mv_err_cost(&this_mv, ref_mv, mvjcost, mvcost, error_per_bit);
-
- if (cost_array[idx] < besterr) {
- best_idx = idx;
- besterr = cost_array[idx];
- *distortion = thismse;
- *sse1 = sse;
- }
- } else {
- cost_array[idx] = INT_MAX;
- }
- }
-
- // Check diagonal sub-pixel position
- kc = (cost_array[0] <= cost_array[1] ? -hstep : hstep);
- kr = (cost_array[2] <= cost_array[3] ? -hstep : hstep);
-
- tc = bc + kc;
- tr = br + kr;
- if (tc >= minc && tc <= maxc && tr >= minr && tr <= maxr) {
- const uint8_t *const pre_address = y + (tr >> 3) * y_stride + (tc >> 3);
- MV this_mv = {tr, tc};
- if (second_pred == NULL)
- thismse = vfp->svf(pre_address, y_stride, sp(tc), sp(tr),
- src_address, src_stride, &sse);
- else
- thismse = vfp->svaf(pre_address, y_stride, sp(tc), sp(tr),
- src_address, src_stride, &sse, second_pred);
- cost_array[4] = thismse +
- mv_err_cost(&this_mv, ref_mv, mvjcost, mvcost, error_per_bit);
-
- if (cost_array[4] < besterr) {
- best_idx = 4;
- besterr = cost_array[4];
- *distortion = thismse;
- *sse1 = sse;
- }
- } else {
- cost_array[idx] = INT_MAX;
- }
-
- if (best_idx < 4 && best_idx >= 0) {
- br += search_step[best_idx].row;
- bc += search_step[best_idx].col;
- } else if (best_idx == 4) {
- br = tr;
- bc = tc;
- }
-
- if (iters_per_step > 1 && best_idx != -1)
- SECOND_LEVEL_CHECKS_BEST;
-
- tr = br;
- tc = bc;
-
- search_step += 4;
- hstep >>= 1;
- best_idx = -1;
- }
-
- // Each subsequent iteration checks at least one point in common with
- // the last iteration could be 2 ( if diag selected) 1/4 pel
-
- // These lines insure static analysis doesn't warn that
- // tr and tc aren't used after the above point.
- (void) tr;
- (void) tc;
-
- bestmv->row = br;
- bestmv->col = bc;
-
- if ((abs(bestmv->col - ref_mv->col) > (MAX_FULL_PEL_VAL << 3)) ||
- (abs(bestmv->row - ref_mv->row) > (MAX_FULL_PEL_VAL << 3)))
- return INT_MAX;
-
- return besterr;
-}
-
-#undef MVC
-#undef PRE
-#undef CHECK_BETTER
-
-static INLINE int check_bounds(const MACROBLOCK *x, int row, int col,
- int range) {
- return ((row - range) >= x->mv_row_min) &
- ((row + range) <= x->mv_row_max) &
- ((col - range) >= x->mv_col_min) &
- ((col + range) <= x->mv_col_max);
-}
-
-static INLINE int is_mv_in(const MACROBLOCK *x, const MV *mv) {
- return (mv->col >= x->mv_col_min) && (mv->col <= x->mv_col_max) &&
- (mv->row >= x->mv_row_min) && (mv->row <= x->mv_row_max);
-}
-
-#define CHECK_BETTER \
- {\
- if (thissad < bestsad) {\
- if (use_mvcost) \
- thissad += mvsad_err_cost(x, &this_mv, &fcenter_mv, sad_per_bit);\
- if (thissad < bestsad) {\
- bestsad = thissad;\
- best_site = i;\
- }\
- }\
- }
-
-#define MAX_PATTERN_SCALES 11
-#define MAX_PATTERN_CANDIDATES 8 // max number of canddiates per scale
-#define PATTERN_CANDIDATES_REF 3 // number of refinement candidates
-
-// Calculate and return a sad+mvcost list around an integer best pel.
-static INLINE void calc_int_cost_list(const MACROBLOCK *x,
- const MV *ref_mv,
- int sadpb,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *best_mv,
- int *cost_list) {
- static const MV neighbors[4] = {{0, -1}, {1, 0}, {0, 1}, {-1, 0}};
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &x->e_mbd.plane[0].pre[0];
- const MV fcenter_mv = {ref_mv->row >> 3, ref_mv->col >> 3};
- int br = best_mv->row;
- int bc = best_mv->col;
- MV this_mv;
- int i;
- unsigned int sse;
-
- this_mv.row = br;
- this_mv.col = bc;
- cost_list[0] = fn_ptr->vf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride, &sse) +
- mvsad_err_cost(x, &this_mv, &fcenter_mv, sadpb);
- if (check_bounds(x, br, bc, 1)) {
- for (i = 0; i < 4; i++) {
- const MV this_mv = {br + neighbors[i].row,
- bc + neighbors[i].col};
- cost_list[i + 1] = fn_ptr->vf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride, &sse) +
- // mvsad_err_cost(x, &this_mv, &fcenter_mv, sadpb);
- mv_err_cost(&this_mv, &fcenter_mv, x->nmvjointcost, x->mvcost,
- x->errorperbit);
- }
- } else {
- for (i = 0; i < 4; i++) {
- const MV this_mv = {br + neighbors[i].row,
- bc + neighbors[i].col};
- if (!is_mv_in(x, &this_mv))
- cost_list[i + 1] = INT_MAX;
- else
- cost_list[i + 1] = fn_ptr->vf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride, &sse) +
- // mvsad_err_cost(x, &this_mv, &fcenter_mv, sadpb);
- mv_err_cost(&this_mv, &fcenter_mv, x->nmvjointcost, x->mvcost,
- x->errorperbit);
- }
- }
-}
-
-// Generic pattern search function that searches over multiple scales.
-// Each scale can have a different number of candidates and shape of
-// candidates as indicated in the num_candidates and candidates arrays
-// passed into this function
-//
-static int vp10_pattern_search(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv,
- const int num_candidates[MAX_PATTERN_SCALES],
- const MV candidates[MAX_PATTERN_SCALES]
- [MAX_PATTERN_CANDIDATES]) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- static const int search_param_to_steps[MAX_MVSEARCH_STEPS] = {
- 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0,
- };
- int i, s, t;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- int br, bc;
- int bestsad = INT_MAX;
- int thissad;
- int k = -1;
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- int best_init_s = search_param_to_steps[search_param];
- // adjust ref_mv to make sure it is within MV range
- clamp_mv(ref_mv, x->mv_col_min, x->mv_col_max, x->mv_row_min, x->mv_row_max);
- br = ref_mv->row;
- bc = ref_mv->col;
-
- // Work out the start point for the search
- bestsad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, sad_per_bit);
-
- // Search all possible scales upto the search param around the center point
- // pick the scale of the point that is best as the starting scale of
- // further steps around it.
- if (do_init_search) {
- s = best_init_s;
- best_init_s = -1;
- for (t = 0; t <= s; ++t) {
- int best_site = -1;
- if (check_bounds(x, br, bc, 1 << t)) {
- for (i = 0; i < num_candidates[t]; i++) {
- const MV this_mv = {br + candidates[t][i].row,
- bc + candidates[t][i].col};
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < num_candidates[t]; i++) {
- const MV this_mv = {br + candidates[t][i].row,
- bc + candidates[t][i].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
- if (best_site == -1) {
- continue;
- } else {
- best_init_s = t;
- k = best_site;
- }
- }
- if (best_init_s != -1) {
- br += candidates[best_init_s][k].row;
- bc += candidates[best_init_s][k].col;
- }
- }
-
- // If the center point is still the best, just skip this and move to
- // the refinement step.
- if (best_init_s != -1) {
- int best_site = -1;
- s = best_init_s;
-
- do {
- // No need to search all 6 points the 1st time if initial search was used
- if (!do_init_search || s != best_init_s) {
- if (check_bounds(x, br, bc, 1 << s)) {
- for (i = 0; i < num_candidates[s]; i++) {
- const MV this_mv = {br + candidates[s][i].row,
- bc + candidates[s][i].col};
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < num_candidates[s]; i++) {
- const MV this_mv = {br + candidates[s][i].row,
- bc + candidates[s][i].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
-
- if (best_site == -1) {
- continue;
- } else {
- br += candidates[s][best_site].row;
- bc += candidates[s][best_site].col;
- k = best_site;
- }
- }
-
- do {
- int next_chkpts_indices[PATTERN_CANDIDATES_REF];
- best_site = -1;
- next_chkpts_indices[0] = (k == 0) ? num_candidates[s] - 1 : k - 1;
- next_chkpts_indices[1] = k;
- next_chkpts_indices[2] = (k == num_candidates[s] - 1) ? 0 : k + 1;
-
- if (check_bounds(x, br, bc, 1 << s)) {
- for (i = 0; i < PATTERN_CANDIDATES_REF; i++) {
- const MV this_mv = {br + candidates[s][next_chkpts_indices[i]].row,
- bc + candidates[s][next_chkpts_indices[i]].col};
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < PATTERN_CANDIDATES_REF; i++) {
- const MV this_mv = {br + candidates[s][next_chkpts_indices[i]].row,
- bc + candidates[s][next_chkpts_indices[i]].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
-
- if (best_site != -1) {
- k = next_chkpts_indices[best_site];
- br += candidates[s][k].row;
- bc += candidates[s][k].col;
- }
- } while (best_site != -1);
- } while (s--);
- }
-
- // Returns the one-away integer pel sad values around the best as follows:
- // cost_list[0]: cost at the best integer pel
- // cost_list[1]: cost at delta {0, -1} (left) from the best integer pel
- // cost_list[2]: cost at delta { 1, 0} (bottom) from the best integer pel
- // cost_list[3]: cost at delta { 0, 1} (right) from the best integer pel
- // cost_list[4]: cost at delta {-1, 0} (top) from the best integer pel
- if (cost_list) {
- const MV best_mv = { br, bc };
- calc_int_cost_list(x, &fcenter_mv, sad_per_bit, vfp, &best_mv, cost_list);
- }
- best_mv->row = br;
- best_mv->col = bc;
- return bestsad;
-}
-
-// A specialized function where the smallest scale search candidates
-// are 4 1-away neighbors, and cost_list is non-null
-// TODO(debargha): Merge this function with the one above. Also remove
-// use_mvcost option since it is always 1, to save unnecessary branches.
-static int vp10_pattern_search_sad(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv,
- const int num_candidates[MAX_PATTERN_SCALES],
- const MV candidates[MAX_PATTERN_SCALES]
- [MAX_PATTERN_CANDIDATES]) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- static const int search_param_to_steps[MAX_MVSEARCH_STEPS] = {
- 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0,
- };
- int i, s, t;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- int br, bc;
- int bestsad = INT_MAX;
- int thissad;
- int k = -1;
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- int best_init_s = search_param_to_steps[search_param];
- // adjust ref_mv to make sure it is within MV range
- clamp_mv(ref_mv, x->mv_col_min, x->mv_col_max, x->mv_row_min, x->mv_row_max);
- br = ref_mv->row;
- bc = ref_mv->col;
- if (cost_list != NULL) {
- cost_list[0] = cost_list[1] = cost_list[2] = cost_list[3] = cost_list[4] =
- INT_MAX;
- }
-
- // Work out the start point for the search
- bestsad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, sad_per_bit);
-
- // Search all possible scales upto the search param around the center point
- // pick the scale of the point that is best as the starting scale of
- // further steps around it.
- if (do_init_search) {
- s = best_init_s;
- best_init_s = -1;
- for (t = 0; t <= s; ++t) {
- int best_site = -1;
- if (check_bounds(x, br, bc, 1 << t)) {
- for (i = 0; i < num_candidates[t]; i++) {
- const MV this_mv = {br + candidates[t][i].row,
- bc + candidates[t][i].col};
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < num_candidates[t]; i++) {
- const MV this_mv = {br + candidates[t][i].row,
- bc + candidates[t][i].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
- if (best_site == -1) {
- continue;
- } else {
- best_init_s = t;
- k = best_site;
- }
- }
- if (best_init_s != -1) {
- br += candidates[best_init_s][k].row;
- bc += candidates[best_init_s][k].col;
- }
- }
-
- // If the center point is still the best, just skip this and move to
- // the refinement step.
- if (best_init_s != -1) {
- int do_sad = (num_candidates[0] == 4 && cost_list != NULL);
- int best_site = -1;
- s = best_init_s;
-
- for (; s >= do_sad; s--) {
- if (!do_init_search || s != best_init_s) {
- if (check_bounds(x, br, bc, 1 << s)) {
- for (i = 0; i < num_candidates[s]; i++) {
- const MV this_mv = {br + candidates[s][i].row,
- bc + candidates[s][i].col};
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < num_candidates[s]; i++) {
- const MV this_mv = {br + candidates[s][i].row,
- bc + candidates[s][i].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
-
- if (best_site == -1) {
- continue;
- } else {
- br += candidates[s][best_site].row;
- bc += candidates[s][best_site].col;
- k = best_site;
- }
- }
-
- do {
- int next_chkpts_indices[PATTERN_CANDIDATES_REF];
- best_site = -1;
- next_chkpts_indices[0] = (k == 0) ? num_candidates[s] - 1 : k - 1;
- next_chkpts_indices[1] = k;
- next_chkpts_indices[2] = (k == num_candidates[s] - 1) ? 0 : k + 1;
-
- if (check_bounds(x, br, bc, 1 << s)) {
- for (i = 0; i < PATTERN_CANDIDATES_REF; i++) {
- const MV this_mv = {br + candidates[s][next_chkpts_indices[i]].row,
- bc + candidates[s][next_chkpts_indices[i]].col};
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < PATTERN_CANDIDATES_REF; i++) {
- const MV this_mv = {br + candidates[s][next_chkpts_indices[i]].row,
- bc + candidates[s][next_chkpts_indices[i]].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
-
- if (best_site != -1) {
- k = next_chkpts_indices[best_site];
- br += candidates[s][k].row;
- bc += candidates[s][k].col;
- }
- } while (best_site != -1);
- }
-
- // Note: If we enter the if below, then cost_list must be non-NULL.
- if (s == 0) {
- cost_list[0] = bestsad;
- if (!do_init_search || s != best_init_s) {
- if (check_bounds(x, br, bc, 1 << s)) {
- for (i = 0; i < num_candidates[s]; i++) {
- const MV this_mv = {br + candidates[s][i].row,
- bc + candidates[s][i].col};
- cost_list[i + 1] =
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < num_candidates[s]; i++) {
- const MV this_mv = {br + candidates[s][i].row,
- bc + candidates[s][i].col};
- if (!is_mv_in(x, &this_mv))
- continue;
- cost_list[i + 1] =
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
-
- if (best_site != -1) {
- br += candidates[s][best_site].row;
- bc += candidates[s][best_site].col;
- k = best_site;
- }
- }
- while (best_site != -1) {
- int next_chkpts_indices[PATTERN_CANDIDATES_REF];
- best_site = -1;
- next_chkpts_indices[0] = (k == 0) ? num_candidates[s] - 1 : k - 1;
- next_chkpts_indices[1] = k;
- next_chkpts_indices[2] = (k == num_candidates[s] - 1) ? 0 : k + 1;
- cost_list[1] = cost_list[2] = cost_list[3] = cost_list[4] = INT_MAX;
- cost_list[((k + 2) % 4) + 1] = cost_list[0];
- cost_list[0] = bestsad;
-
- if (check_bounds(x, br, bc, 1 << s)) {
- for (i = 0; i < PATTERN_CANDIDATES_REF; i++) {
- const MV this_mv = {br + candidates[s][next_chkpts_indices[i]].row,
- bc + candidates[s][next_chkpts_indices[i]].col};
- cost_list[next_chkpts_indices[i] + 1] =
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- } else {
- for (i = 0; i < PATTERN_CANDIDATES_REF; i++) {
- const MV this_mv = {br + candidates[s][next_chkpts_indices[i]].row,
- bc + candidates[s][next_chkpts_indices[i]].col};
- if (!is_mv_in(x, &this_mv)) {
- cost_list[next_chkpts_indices[i] + 1] = INT_MAX;
- continue;
- }
- cost_list[next_chkpts_indices[i] + 1] =
- thissad = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- CHECK_BETTER
- }
- }
-
- if (best_site != -1) {
- k = next_chkpts_indices[best_site];
- br += candidates[s][k].row;
- bc += candidates[s][k].col;
- }
- }
- }
- }
-
- // Returns the one-away integer pel sad values around the best as follows:
- // cost_list[0]: sad at the best integer pel
- // cost_list[1]: sad at delta {0, -1} (left) from the best integer pel
- // cost_list[2]: sad at delta { 1, 0} (bottom) from the best integer pel
- // cost_list[3]: sad at delta { 0, 1} (right) from the best integer pel
- // cost_list[4]: sad at delta {-1, 0} (top) from the best integer pel
- if (cost_list) {
- static const MV neighbors[4] = {{0, -1}, {1, 0}, {0, 1}, {-1, 0}};
- if (cost_list[0] == INT_MAX) {
- cost_list[0] = bestsad;
- if (check_bounds(x, br, bc, 1)) {
- for (i = 0; i < 4; i++) {
- const MV this_mv = { br + neighbors[i].row,
- bc + neighbors[i].col };
- cost_list[i + 1] = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- }
- } else {
- for (i = 0; i < 4; i++) {
- const MV this_mv = {br + neighbors[i].row,
- bc + neighbors[i].col};
- if (!is_mv_in(x, &this_mv))
- cost_list[i + 1] = INT_MAX;
- else
- cost_list[i + 1] = vfp->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &this_mv),
- in_what->stride);
- }
- }
- } else {
- if (use_mvcost) {
- for (i = 0; i < 4; i++) {
- const MV this_mv = {br + neighbors[i].row,
- bc + neighbors[i].col};
- if (cost_list[i + 1] != INT_MAX) {
- cost_list[i + 1] +=
- mvsad_err_cost(x, &this_mv, &fcenter_mv, sad_per_bit);
- }
- }
- }
- }
- }
- best_mv->row = br;
- best_mv->col = bc;
- return bestsad;
-}
-
-int vp10_get_mvpred_var(const MACROBLOCK *x,
- const MV *best_mv, const MV *center_mv,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const MV mv = {best_mv->row * 8, best_mv->col * 8};
- unsigned int unused;
-
- return vfp->vf(what->buf, what->stride,
- get_buf_from_mv(in_what, best_mv), in_what->stride, &unused) +
- (use_mvcost ? mv_err_cost(&mv, center_mv, x->nmvjointcost,
- x->mvcost, x->errorperbit) : 0);
-}
-
-int vp10_get_mvpred_av_var(const MACROBLOCK *x,
- const MV *best_mv, const MV *center_mv,
- const uint8_t *second_pred,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const MV mv = {best_mv->row * 8, best_mv->col * 8};
- unsigned int unused;
-
- return vfp->svaf(get_buf_from_mv(in_what, best_mv), in_what->stride, 0, 0,
- what->buf, what->stride, &unused, second_pred) +
- (use_mvcost ? mv_err_cost(&mv, center_mv, x->nmvjointcost,
- x->mvcost, x->errorperbit) : 0);
-}
-
-int vp10_hex_search(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv, MV *best_mv) {
- // First scale has 8-closest points, the rest have 6 points in hex shape
- // at increasing scales
- static const int hex_num_candidates[MAX_PATTERN_SCALES] = {
- 8, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6
- };
- // Note that the largest candidate step at each scale is 2^scale
- static const MV hex_candidates[MAX_PATTERN_SCALES][MAX_PATTERN_CANDIDATES] = {
- {{-1, -1}, {0, -1}, {1, -1}, {1, 0}, {1, 1}, { 0, 1}, { -1, 1}, {-1, 0}},
- {{-1, -2}, {1, -2}, {2, 0}, {1, 2}, { -1, 2}, { -2, 0}},
- {{-2, -4}, {2, -4}, {4, 0}, {2, 4}, { -2, 4}, { -4, 0}},
- {{-4, -8}, {4, -8}, {8, 0}, {4, 8}, { -4, 8}, { -8, 0}},
- {{-8, -16}, {8, -16}, {16, 0}, {8, 16}, { -8, 16}, { -16, 0}},
- {{-16, -32}, {16, -32}, {32, 0}, {16, 32}, { -16, 32}, { -32, 0}},
- {{-32, -64}, {32, -64}, {64, 0}, {32, 64}, { -32, 64}, { -64, 0}},
- {{-64, -128}, {64, -128}, {128, 0}, {64, 128}, { -64, 128}, { -128, 0}},
- {{-128, -256}, {128, -256}, {256, 0}, {128, 256}, { -128, 256}, { -256, 0}},
- {{-256, -512}, {256, -512}, {512, 0}, {256, 512}, { -256, 512}, { -512, 0}},
- {{-512, -1024}, {512, -1024}, {1024, 0}, {512, 1024}, { -512, 1024},
- { -1024, 0}},
- };
- return vp10_pattern_search(x, ref_mv, search_param, sad_per_bit,
- do_init_search, cost_list, vfp, use_mvcost,
- center_mv, best_mv,
- hex_num_candidates, hex_candidates);
-}
-
-int vp10_bigdia_search(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv) {
- // First scale has 4-closest points, the rest have 8 points in diamond
- // shape at increasing scales
- static const int bigdia_num_candidates[MAX_PATTERN_SCALES] = {
- 4, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- };
- // Note that the largest candidate step at each scale is 2^scale
- static const MV bigdia_candidates[MAX_PATTERN_SCALES]
- [MAX_PATTERN_CANDIDATES] = {
- {{0, -1}, {1, 0}, { 0, 1}, {-1, 0}},
- {{-1, -1}, {0, -2}, {1, -1}, {2, 0}, {1, 1}, {0, 2}, {-1, 1}, {-2, 0}},
- {{-2, -2}, {0, -4}, {2, -2}, {4, 0}, {2, 2}, {0, 4}, {-2, 2}, {-4, 0}},
- {{-4, -4}, {0, -8}, {4, -4}, {8, 0}, {4, 4}, {0, 8}, {-4, 4}, {-8, 0}},
- {{-8, -8}, {0, -16}, {8, -8}, {16, 0}, {8, 8}, {0, 16}, {-8, 8}, {-16, 0}},
- {{-16, -16}, {0, -32}, {16, -16}, {32, 0}, {16, 16}, {0, 32},
- {-16, 16}, {-32, 0}},
- {{-32, -32}, {0, -64}, {32, -32}, {64, 0}, {32, 32}, {0, 64},
- {-32, 32}, {-64, 0}},
- {{-64, -64}, {0, -128}, {64, -64}, {128, 0}, {64, 64}, {0, 128},
- {-64, 64}, {-128, 0}},
- {{-128, -128}, {0, -256}, {128, -128}, {256, 0}, {128, 128}, {0, 256},
- {-128, 128}, {-256, 0}},
- {{-256, -256}, {0, -512}, {256, -256}, {512, 0}, {256, 256}, {0, 512},
- {-256, 256}, {-512, 0}},
- {{-512, -512}, {0, -1024}, {512, -512}, {1024, 0}, {512, 512}, {0, 1024},
- {-512, 512}, {-1024, 0}},
- };
- return vp10_pattern_search_sad(x, ref_mv, search_param, sad_per_bit,
- do_init_search, cost_list, vfp, use_mvcost,
- center_mv, best_mv,
- bigdia_num_candidates, bigdia_candidates);
-}
-
-int vp10_square_search(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv) {
- // All scales have 8 closest points in square shape
- static const int square_num_candidates[MAX_PATTERN_SCALES] = {
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- };
- // Note that the largest candidate step at each scale is 2^scale
- static const MV square_candidates[MAX_PATTERN_SCALES]
- [MAX_PATTERN_CANDIDATES] = {
- {{-1, -1}, {0, -1}, {1, -1}, {1, 0}, {1, 1}, {0, 1}, {-1, 1}, {-1, 0}},
- {{-2, -2}, {0, -2}, {2, -2}, {2, 0}, {2, 2}, {0, 2}, {-2, 2}, {-2, 0}},
- {{-4, -4}, {0, -4}, {4, -4}, {4, 0}, {4, 4}, {0, 4}, {-4, 4}, {-4, 0}},
- {{-8, -8}, {0, -8}, {8, -8}, {8, 0}, {8, 8}, {0, 8}, {-8, 8}, {-8, 0}},
- {{-16, -16}, {0, -16}, {16, -16}, {16, 0}, {16, 16}, {0, 16},
- {-16, 16}, {-16, 0}},
- {{-32, -32}, {0, -32}, {32, -32}, {32, 0}, {32, 32}, {0, 32},
- {-32, 32}, {-32, 0}},
- {{-64, -64}, {0, -64}, {64, -64}, {64, 0}, {64, 64}, {0, 64},
- {-64, 64}, {-64, 0}},
- {{-128, -128}, {0, -128}, {128, -128}, {128, 0}, {128, 128}, {0, 128},
- {-128, 128}, {-128, 0}},
- {{-256, -256}, {0, -256}, {256, -256}, {256, 0}, {256, 256}, {0, 256},
- {-256, 256}, {-256, 0}},
- {{-512, -512}, {0, -512}, {512, -512}, {512, 0}, {512, 512}, {0, 512},
- {-512, 512}, {-512, 0}},
- {{-1024, -1024}, {0, -1024}, {1024, -1024}, {1024, 0}, {1024, 1024},
- {0, 1024}, {-1024, 1024}, {-1024, 0}},
- };
- return vp10_pattern_search(x, ref_mv, search_param, sad_per_bit,
- do_init_search, cost_list, vfp, use_mvcost,
- center_mv, best_mv,
- square_num_candidates, square_candidates);
-}
-
-int vp10_fast_hex_search(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search, // must be zero for fast_hex
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv) {
- return vp10_hex_search(
- x, ref_mv, VPXMAX(MAX_MVSEARCH_STEPS - 2, search_param), sad_per_bit,
- do_init_search, cost_list, vfp, use_mvcost, center_mv, best_mv);
-}
-
-int vp10_fast_dia_search(const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int sad_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv) {
- return vp10_bigdia_search(
- x, ref_mv, VPXMAX(MAX_MVSEARCH_STEPS - 2, search_param), sad_per_bit,
- do_init_search, cost_list, vfp, use_mvcost, center_mv, best_mv);
-}
-
-#undef CHECK_BETTER
-
-int vp10_full_range_search_c(const MACROBLOCK *x,
- const search_site_config *cfg,
- MV *ref_mv, MV *best_mv,
- int search_param, int sad_per_bit, int *num00,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const int range = 64;
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- unsigned int best_sad = INT_MAX;
- int r, c, i;
- int start_col, end_col, start_row, end_row;
-
- // The cfg and search_param parameters are not used in this search variant
- (void)cfg;
- (void)search_param;
-
- clamp_mv(ref_mv, x->mv_col_min, x->mv_col_max, x->mv_row_min, x->mv_row_max);
- *best_mv = *ref_mv;
- *num00 = 11;
- best_sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, sad_per_bit);
- start_row = VPXMAX(-range, x->mv_row_min - ref_mv->row);
- start_col = VPXMAX(-range, x->mv_col_min - ref_mv->col);
- end_row = VPXMIN(range, x->mv_row_max - ref_mv->row);
- end_col = VPXMIN(range, x->mv_col_max - ref_mv->col);
-
- for (r = start_row; r <= end_row; ++r) {
- for (c = start_col; c <= end_col; c += 4) {
- if (c + 3 <= end_col) {
- unsigned int sads[4];
- const uint8_t *addrs[4];
- for (i = 0; i < 4; ++i) {
- const MV mv = {ref_mv->row + r, ref_mv->col + c + i};
- addrs[i] = get_buf_from_mv(in_what, &mv);
- }
-
- fn_ptr->sdx4df(what->buf, what->stride, addrs, in_what->stride, sads);
-
- for (i = 0; i < 4; ++i) {
- if (sads[i] < best_sad) {
- const MV mv = {ref_mv->row + r, ref_mv->col + c + i};
- const unsigned int sad = sads[i] +
- mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- }
- } else {
- for (i = 0; i < end_col - c; ++i) {
- const MV mv = {ref_mv->row + r, ref_mv->col + c + i};
- unsigned int sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &mv), in_what->stride);
- if (sad < best_sad) {
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- }
- }
- }
- }
-
- return best_sad;
-}
-
-int vp10_diamond_search_sad_c(const MACROBLOCK *x,
- const search_site_config *cfg,
- MV *ref_mv, MV *best_mv, int search_param,
- int sad_per_bit, int *num00,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv) {
- int i, j, step;
-
- const MACROBLOCKD *const xd = &x->e_mbd;
- uint8_t *what = x->plane[0].src.buf;
- const int what_stride = x->plane[0].src.stride;
- const uint8_t *in_what;
- const int in_what_stride = xd->plane[0].pre[0].stride;
- const uint8_t *best_address;
-
- unsigned int bestsad = INT_MAX;
- int best_site = 0;
- int last_site = 0;
-
- int ref_row;
- int ref_col;
-
- // search_param determines the length of the initial step and hence the number
- // of iterations.
- // 0 = initial step (MAX_FIRST_STEP) pel
- // 1 = (MAX_FIRST_STEP/2) pel,
- // 2 = (MAX_FIRST_STEP/4) pel...
- const search_site *ss = &cfg->ss[search_param * cfg->searches_per_step];
- const int tot_steps = (cfg->ss_count / cfg->searches_per_step) - search_param;
-
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- clamp_mv(ref_mv, x->mv_col_min, x->mv_col_max, x->mv_row_min, x->mv_row_max);
- ref_row = ref_mv->row;
- ref_col = ref_mv->col;
- *num00 = 0;
- best_mv->row = ref_row;
- best_mv->col = ref_col;
-
- // Work out the start point for the search
- in_what = xd->plane[0].pre[0].buf + ref_row * in_what_stride + ref_col;
- best_address = in_what;
-
- // Check the starting position
- bestsad = fn_ptr->sdf(what, what_stride, in_what, in_what_stride)
- + mvsad_err_cost(x, best_mv, &fcenter_mv, sad_per_bit);
-
- i = 1;
-
- for (step = 0; step < tot_steps; step++) {
- int all_in = 1, t;
-
- // All_in is true if every one of the points we are checking are within
- // the bounds of the image.
- all_in &= ((best_mv->row + ss[i].mv.row) > x->mv_row_min);
- all_in &= ((best_mv->row + ss[i + 1].mv.row) < x->mv_row_max);
- all_in &= ((best_mv->col + ss[i + 2].mv.col) > x->mv_col_min);
- all_in &= ((best_mv->col + ss[i + 3].mv.col) < x->mv_col_max);
-
- // If all the pixels are within the bounds we don't check whether the
- // search point is valid in this loop, otherwise we check each point
- // for validity..
- if (all_in) {
- unsigned int sad_array[4];
-
- for (j = 0; j < cfg->searches_per_step; j += 4) {
- unsigned char const *block_offset[4];
-
- for (t = 0; t < 4; t++)
- block_offset[t] = ss[i + t].offset + best_address;
-
- fn_ptr->sdx4df(what, what_stride, block_offset, in_what_stride,
- sad_array);
-
- for (t = 0; t < 4; t++, i++) {
- if (sad_array[t] < bestsad) {
- const MV this_mv = {best_mv->row + ss[i].mv.row,
- best_mv->col + ss[i].mv.col};
- sad_array[t] += mvsad_err_cost(x, &this_mv, &fcenter_mv,
- sad_per_bit);
- if (sad_array[t] < bestsad) {
- bestsad = sad_array[t];
- best_site = i;
- }
- }
- }
- }
- } else {
- for (j = 0; j < cfg->searches_per_step; j++) {
- // Trap illegal vectors
- const MV this_mv = {best_mv->row + ss[i].mv.row,
- best_mv->col + ss[i].mv.col};
-
- if (is_mv_in(x, &this_mv)) {
- const uint8_t *const check_here = ss[i].offset + best_address;
- unsigned int thissad = fn_ptr->sdf(what, what_stride, check_here,
- in_what_stride);
-
- if (thissad < bestsad) {
- thissad += mvsad_err_cost(x, &this_mv, &fcenter_mv, sad_per_bit);
- if (thissad < bestsad) {
- bestsad = thissad;
- best_site = i;
- }
- }
- }
- i++;
- }
- }
- if (best_site != last_site) {
- best_mv->row += ss[best_site].mv.row;
- best_mv->col += ss[best_site].mv.col;
- best_address += ss[best_site].offset;
- last_site = best_site;
-#if defined(NEW_DIAMOND_SEARCH)
- while (1) {
- const MV this_mv = {best_mv->row + ss[best_site].mv.row,
- best_mv->col + ss[best_site].mv.col};
- if (is_mv_in(x, &this_mv)) {
- const uint8_t *const check_here = ss[best_site].offset + best_address;
- unsigned int thissad = fn_ptr->sdf(what, what_stride, check_here,
- in_what_stride);
- if (thissad < bestsad) {
- thissad += mvsad_err_cost(x, &this_mv, &fcenter_mv, sad_per_bit);
- if (thissad < bestsad) {
- bestsad = thissad;
- best_mv->row += ss[best_site].mv.row;
- best_mv->col += ss[best_site].mv.col;
- best_address += ss[best_site].offset;
- continue;
- }
- }
- }
- break;
- }
-#endif
- } else if (best_address == in_what) {
- (*num00)++;
- }
- }
- return bestsad;
-}
-
-static int vector_match(int16_t *ref, int16_t *src, int bwl) {
- int best_sad = INT_MAX;
- int this_sad;
- int d;
- int center, offset = 0;
- int bw = 4 << bwl; // redundant variable, to be changed in the experiments.
- for (d = 0; d <= bw; d += 16) {
- this_sad = vp10_vector_var(&ref[d], src, bwl);
- if (this_sad < best_sad) {
- best_sad = this_sad;
- offset = d;
- }
- }
- center = offset;
-
- for (d = -8; d <= 8; d += 16) {
- int this_pos = offset + d;
- // check limit
- if (this_pos < 0 || this_pos > bw)
- continue;
- this_sad = vp10_vector_var(&ref[this_pos], src, bwl);
- if (this_sad < best_sad) {
- best_sad = this_sad;
- center = this_pos;
- }
- }
- offset = center;
-
- for (d = -4; d <= 4; d += 8) {
- int this_pos = offset + d;
- // check limit
- if (this_pos < 0 || this_pos > bw)
- continue;
- this_sad = vp10_vector_var(&ref[this_pos], src, bwl);
- if (this_sad < best_sad) {
- best_sad = this_sad;
- center = this_pos;
- }
- }
- offset = center;
-
- for (d = -2; d <= 2; d += 4) {
- int this_pos = offset + d;
- // check limit
- if (this_pos < 0 || this_pos > bw)
- continue;
- this_sad = vp10_vector_var(&ref[this_pos], src, bwl);
- if (this_sad < best_sad) {
- best_sad = this_sad;
- center = this_pos;
- }
- }
- offset = center;
-
- for (d = -1; d <= 1; d += 2) {
- int this_pos = offset + d;
- // check limit
- if (this_pos < 0 || this_pos > bw)
- continue;
- this_sad = vp10_vector_var(&ref[this_pos], src, bwl);
- if (this_sad < best_sad) {
- best_sad = this_sad;
- center = this_pos;
- }
- }
-
- return (center - (bw >> 1));
-}
-
-static const MV search_pos[4] = {
- {-1, 0}, {0, -1}, {0, 1}, {1, 0},
-};
-
-unsigned int vp10_int_pro_motion_estimation(const VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bsize,
- int mi_row, int mi_col) {
- MACROBLOCKD *xd = &x->e_mbd;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- struct buf_2d backup_yv12[MAX_MB_PLANE] = {{0, 0}};
- DECLARE_ALIGNED(16, int16_t, hbuf[128]);
- DECLARE_ALIGNED(16, int16_t, vbuf[128]);
- DECLARE_ALIGNED(16, int16_t, src_hbuf[64]);
- DECLARE_ALIGNED(16, int16_t, src_vbuf[64]);
- int idx;
- const int bw = 4 << b_width_log2_lookup[bsize];
- const int bh = 4 << b_height_log2_lookup[bsize];
- const int search_width = bw << 1;
- const int search_height = bh << 1;
- const int src_stride = x->plane[0].src.stride;
- const int ref_stride = xd->plane[0].pre[0].stride;
- uint8_t const *ref_buf, *src_buf;
- MV *tmp_mv = &xd->mi[0]->mbmi.mv[0].as_mv;
- unsigned int best_sad, tmp_sad, this_sad[4];
- MV this_mv;
- const int norm_factor = 3 + (bw >> 5);
- const YV12_BUFFER_CONFIG *scaled_ref_frame =
- vp10_get_scaled_ref_frame(cpi, mbmi->ref_frame[0]);
-
- if (scaled_ref_frame) {
- int i;
- // Swap out the reference frame for a version that's been scaled to
- // match the resolution of the current frame, allowing the existing
- // motion search code to be used without additional modifications.
- for (i = 0; i < MAX_MB_PLANE; i++)
- backup_yv12[i] = xd->plane[i].pre[0];
- vp10_setup_pre_planes(xd, 0, scaled_ref_frame, mi_row, mi_col, NULL);
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- {
- unsigned int this_sad;
- tmp_mv->row = 0;
- tmp_mv->col = 0;
- this_sad = cpi->fn_ptr[bsize].sdf(x->plane[0].src.buf, src_stride,
- xd->plane[0].pre[0].buf, ref_stride);
-
- if (scaled_ref_frame) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++)
- xd->plane[i].pre[0] = backup_yv12[i];
- }
- return this_sad;
- }
-#endif
-
- // Set up prediction 1-D reference set
- ref_buf = xd->plane[0].pre[0].buf - (bw >> 1);
- for (idx = 0; idx < search_width; idx += 16) {
- vp10_int_pro_row(&hbuf[idx], ref_buf, ref_stride, bh);
- ref_buf += 16;
- }
-
- ref_buf = xd->plane[0].pre[0].buf - (bh >> 1) * ref_stride;
- for (idx = 0; idx < search_height; ++idx) {
- vbuf[idx] = vp10_int_pro_col(ref_buf, bw) >> norm_factor;
- ref_buf += ref_stride;
- }
-
- // Set up src 1-D reference set
- for (idx = 0; idx < bw; idx += 16) {
- src_buf = x->plane[0].src.buf + idx;
- vp10_int_pro_row(&src_hbuf[idx], src_buf, src_stride, bh);
- }
-
- src_buf = x->plane[0].src.buf;
- for (idx = 0; idx < bh; ++idx) {
- src_vbuf[idx] = vp10_int_pro_col(src_buf, bw) >> norm_factor;
- src_buf += src_stride;
- }
-
- // Find the best match per 1-D search
- tmp_mv->col = vector_match(hbuf, src_hbuf, b_width_log2_lookup[bsize]);
- tmp_mv->row = vector_match(vbuf, src_vbuf, b_height_log2_lookup[bsize]);
-
- this_mv = *tmp_mv;
- src_buf = x->plane[0].src.buf;
- ref_buf = xd->plane[0].pre[0].buf + this_mv.row * ref_stride + this_mv.col;
- best_sad = cpi->fn_ptr[bsize].sdf(src_buf, src_stride, ref_buf, ref_stride);
-
- {
- const uint8_t * const pos[4] = {
- ref_buf - ref_stride,
- ref_buf - 1,
- ref_buf + 1,
- ref_buf + ref_stride,
- };
-
- cpi->fn_ptr[bsize].sdx4df(src_buf, src_stride, pos, ref_stride, this_sad);
- }
-
- for (idx = 0; idx < 4; ++idx) {
- if (this_sad[idx] < best_sad) {
- best_sad = this_sad[idx];
- tmp_mv->row = search_pos[idx].row + this_mv.row;
- tmp_mv->col = search_pos[idx].col + this_mv.col;
- }
- }
-
- if (this_sad[0] < this_sad[3])
- this_mv.row -= 1;
- else
- this_mv.row += 1;
-
- if (this_sad[1] < this_sad[2])
- this_mv.col -= 1;
- else
- this_mv.col += 1;
-
- ref_buf = xd->plane[0].pre[0].buf + this_mv.row * ref_stride + this_mv.col;
-
- tmp_sad = cpi->fn_ptr[bsize].sdf(src_buf, src_stride,
- ref_buf, ref_stride);
- if (best_sad > tmp_sad) {
- *tmp_mv = this_mv;
- best_sad = tmp_sad;
- }
-
- tmp_mv->row *= 8;
- tmp_mv->col *= 8;
-
- if (scaled_ref_frame) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++)
- xd->plane[i].pre[0] = backup_yv12[i];
- }
-
- return best_sad;
-}
-
-/* do_refine: If last step (1-away) of n-step search doesn't pick the center
- point as the best match, we will do a final 1-away diamond
- refining search */
-int vp10_full_pixel_diamond(const VP10_COMP *cpi, MACROBLOCK *x,
- MV *mvp_full, int step_param,
- int sadpb, int further_steps, int do_refine,
- int *cost_list,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *ref_mv, MV *dst_mv) {
- MV temp_mv;
- int thissme, n, num00 = 0;
- int bestsme = cpi->diamond_search_sad(x, &cpi->ss_cfg, mvp_full, &temp_mv,
- step_param, sadpb, &n,
- fn_ptr, ref_mv);
- if (bestsme < INT_MAX)
- bestsme = vp10_get_mvpred_var(x, &temp_mv, ref_mv, fn_ptr, 1);
- *dst_mv = temp_mv;
-
- // If there won't be more n-step search, check to see if refining search is
- // needed.
- if (n > further_steps)
- do_refine = 0;
-
- while (n < further_steps) {
- ++n;
-
- if (num00) {
- num00--;
- } else {
- thissme = cpi->diamond_search_sad(x, &cpi->ss_cfg, mvp_full, &temp_mv,
- step_param + n, sadpb, &num00,
- fn_ptr, ref_mv);
- if (thissme < INT_MAX)
- thissme = vp10_get_mvpred_var(x, &temp_mv, ref_mv, fn_ptr, 1);
-
- // check to see if refining search is needed.
- if (num00 > further_steps - n)
- do_refine = 0;
-
- if (thissme < bestsme) {
- bestsme = thissme;
- *dst_mv = temp_mv;
- }
- }
- }
-
- // final 1-away diamond refining search
- if (do_refine) {
- const int search_range = 8;
- MV best_mv = *dst_mv;
- thissme = vp10_refining_search_sad(x, &best_mv, sadpb, search_range,
- fn_ptr, ref_mv);
- if (thissme < INT_MAX)
- thissme = vp10_get_mvpred_var(x, &best_mv, ref_mv, fn_ptr, 1);
- if (thissme < bestsme) {
- bestsme = thissme;
- *dst_mv = best_mv;
- }
- }
-
- // Return cost list.
- if (cost_list) {
- calc_int_cost_list(x, ref_mv, sadpb, fn_ptr, dst_mv, cost_list);
- }
- return bestsme;
-}
-
-int vp10_full_search_sad_c(const MACROBLOCK *x, const MV *ref_mv,
- int sad_per_bit, int distance,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv, MV *best_mv) {
- int r, c;
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const int row_min = VPXMAX(ref_mv->row - distance, x->mv_row_min);
- const int row_max = VPXMIN(ref_mv->row + distance, x->mv_row_max);
- const int col_min = VPXMAX(ref_mv->col - distance, x->mv_col_min);
- const int col_max = VPXMIN(ref_mv->col + distance, x->mv_col_max);
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- int best_sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, sad_per_bit);
- *best_mv = *ref_mv;
-
- for (r = row_min; r < row_max; ++r) {
- for (c = col_min; c < col_max; ++c) {
- const MV mv = {r, c};
- const int sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &mv), in_what->stride) +
- mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- }
- return best_sad;
-}
-
-int vp10_full_search_sadx3(const MACROBLOCK *x, const MV *ref_mv,
- int sad_per_bit, int distance,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv, MV *best_mv) {
- int r;
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const int row_min = VPXMAX(ref_mv->row - distance, x->mv_row_min);
- const int row_max = VPXMIN(ref_mv->row + distance, x->mv_row_max);
- const int col_min = VPXMAX(ref_mv->col - distance, x->mv_col_min);
- const int col_max = VPXMIN(ref_mv->col + distance, x->mv_col_max);
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- unsigned int best_sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, sad_per_bit);
- *best_mv = *ref_mv;
-
- for (r = row_min; r < row_max; ++r) {
- int c = col_min;
- const uint8_t *check_here = &in_what->buf[r * in_what->stride + c];
-
- if (fn_ptr->sdx3f != NULL) {
- while ((c + 2) < col_max) {
- int i;
- DECLARE_ALIGNED(16, uint32_t, sads[3]);
-
- fn_ptr->sdx3f(what->buf, what->stride, check_here, in_what->stride,
- sads);
-
- for (i = 0; i < 3; ++i) {
- unsigned int sad = sads[i];
- if (sad < best_sad) {
- const MV mv = {r, c};
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- ++check_here;
- ++c;
- }
- }
- }
-
- while (c < col_max) {
- unsigned int sad = fn_ptr->sdf(what->buf, what->stride,
- check_here, in_what->stride);
- if (sad < best_sad) {
- const MV mv = {r, c};
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- ++check_here;
- ++c;
- }
- }
-
- return best_sad;
-}
-
-int vp10_full_search_sadx8(const MACROBLOCK *x, const MV *ref_mv,
- int sad_per_bit, int distance,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv, MV *best_mv) {
- int r;
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const int row_min = VPXMAX(ref_mv->row - distance, x->mv_row_min);
- const int row_max = VPXMIN(ref_mv->row + distance, x->mv_row_max);
- const int col_min = VPXMAX(ref_mv->col - distance, x->mv_col_min);
- const int col_max = VPXMIN(ref_mv->col + distance, x->mv_col_max);
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- unsigned int best_sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, sad_per_bit);
- *best_mv = *ref_mv;
-
- for (r = row_min; r < row_max; ++r) {
- int c = col_min;
- const uint8_t *check_here = &in_what->buf[r * in_what->stride + c];
-
- if (fn_ptr->sdx8f != NULL) {
- while ((c + 7) < col_max) {
- int i;
- DECLARE_ALIGNED(16, uint32_t, sads[8]);
-
- fn_ptr->sdx8f(what->buf, what->stride, check_here, in_what->stride,
- sads);
-
- for (i = 0; i < 8; ++i) {
- unsigned int sad = sads[i];
- if (sad < best_sad) {
- const MV mv = {r, c};
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- ++check_here;
- ++c;
- }
- }
- }
-
- if (fn_ptr->sdx3f != NULL) {
- while ((c + 2) < col_max) {
- int i;
- DECLARE_ALIGNED(16, uint32_t, sads[3]);
-
- fn_ptr->sdx3f(what->buf, what->stride, check_here, in_what->stride,
- sads);
-
- for (i = 0; i < 3; ++i) {
- unsigned int sad = sads[i];
- if (sad < best_sad) {
- const MV mv = {r, c};
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- ++check_here;
- ++c;
- }
- }
- }
-
- while (c < col_max) {
- unsigned int sad = fn_ptr->sdf(what->buf, what->stride,
- check_here, in_what->stride);
- if (sad < best_sad) {
- const MV mv = {r, c};
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, sad_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- *best_mv = mv;
- }
- }
- ++check_here;
- ++c;
- }
- }
-
- return best_sad;
-}
-
-int vp10_refining_search_sad(const MACROBLOCK *x,
- MV *ref_mv, int error_per_bit,
- int search_range,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv) {
- const MACROBLOCKD *const xd = &x->e_mbd;
- const MV neighbors[4] = {{ -1, 0}, {0, -1}, {0, 1}, {1, 0}};
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- const uint8_t *best_address = get_buf_from_mv(in_what, ref_mv);
- unsigned int best_sad = fn_ptr->sdf(what->buf, what->stride, best_address,
- in_what->stride) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, error_per_bit);
- int i, j;
-
- for (i = 0; i < search_range; i++) {
- int best_site = -1;
- const int all_in = ((ref_mv->row - 1) > x->mv_row_min) &
- ((ref_mv->row + 1) < x->mv_row_max) &
- ((ref_mv->col - 1) > x->mv_col_min) &
- ((ref_mv->col + 1) < x->mv_col_max);
-
- if (all_in) {
- unsigned int sads[4];
- const uint8_t *const positions[4] = {
- best_address - in_what->stride,
- best_address - 1,
- best_address + 1,
- best_address + in_what->stride
- };
-
- fn_ptr->sdx4df(what->buf, what->stride, positions, in_what->stride, sads);
-
- for (j = 0; j < 4; ++j) {
- if (sads[j] < best_sad) {
- const MV mv = {ref_mv->row + neighbors[j].row,
- ref_mv->col + neighbors[j].col};
- sads[j] += mvsad_err_cost(x, &mv, &fcenter_mv, error_per_bit);
- if (sads[j] < best_sad) {
- best_sad = sads[j];
- best_site = j;
- }
- }
- }
- } else {
- for (j = 0; j < 4; ++j) {
- const MV mv = {ref_mv->row + neighbors[j].row,
- ref_mv->col + neighbors[j].col};
-
- if (is_mv_in(x, &mv)) {
- unsigned int sad = fn_ptr->sdf(what->buf, what->stride,
- get_buf_from_mv(in_what, &mv),
- in_what->stride);
- if (sad < best_sad) {
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, error_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- best_site = j;
- }
- }
- }
- }
- }
-
- if (best_site == -1) {
- break;
- } else {
- ref_mv->row += neighbors[best_site].row;
- ref_mv->col += neighbors[best_site].col;
- best_address = get_buf_from_mv(in_what, ref_mv);
- }
- }
-
- return best_sad;
-}
-
-// This function is called when we do joint motion search in comp_inter_inter
-// mode.
-int vp10_refining_search_8p_c(const MACROBLOCK *x,
- MV *ref_mv, int error_per_bit,
- int search_range,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv,
- const uint8_t *second_pred) {
- const MV neighbors[8] = {{-1, 0}, {0, -1}, {0, 1}, {1, 0},
- {-1, -1}, {1, -1}, {-1, 1}, {1, 1}};
- const MACROBLOCKD *const xd = &x->e_mbd;
- const struct buf_2d *const what = &x->plane[0].src;
- const struct buf_2d *const in_what = &xd->plane[0].pre[0];
- const MV fcenter_mv = {center_mv->row >> 3, center_mv->col >> 3};
- unsigned int best_sad = fn_ptr->sdaf(what->buf, what->stride,
- get_buf_from_mv(in_what, ref_mv), in_what->stride, second_pred) +
- mvsad_err_cost(x, ref_mv, &fcenter_mv, error_per_bit);
- int i, j;
-
- for (i = 0; i < search_range; ++i) {
- int best_site = -1;
-
- for (j = 0; j < 8; ++j) {
- const MV mv = {ref_mv->row + neighbors[j].row,
- ref_mv->col + neighbors[j].col};
-
- if (is_mv_in(x, &mv)) {
- unsigned int sad = fn_ptr->sdaf(what->buf, what->stride,
- get_buf_from_mv(in_what, &mv), in_what->stride, second_pred);
- if (sad < best_sad) {
- sad += mvsad_err_cost(x, &mv, &fcenter_mv, error_per_bit);
- if (sad < best_sad) {
- best_sad = sad;
- best_site = j;
- }
- }
- }
- }
-
- if (best_site == -1) {
- break;
- } else {
- ref_mv->row += neighbors[best_site].row;
- ref_mv->col += neighbors[best_site].col;
- }
- }
- return best_sad;
-}
-
-int vp10_full_pixel_search(VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bsize, MV *mvp_full,
- int step_param, int error_per_bit,
- int *cost_list,
- const MV *ref_mv, MV *tmp_mv,
- int var_max, int rd) {
- const SPEED_FEATURES *const sf = &cpi->sf;
- const SEARCH_METHODS method = sf->mv.search_method;
- vp9_variance_fn_ptr_t *fn_ptr = &cpi->fn_ptr[bsize];
- int var = 0;
- if (cost_list) {
- cost_list[0] = INT_MAX;
- cost_list[1] = INT_MAX;
- cost_list[2] = INT_MAX;
- cost_list[3] = INT_MAX;
- cost_list[4] = INT_MAX;
- }
-
- switch (method) {
- case FAST_DIAMOND:
- var = vp10_fast_dia_search(x, mvp_full, step_param, error_per_bit, 0,
- cost_list, fn_ptr, 1, ref_mv, tmp_mv);
- break;
- case FAST_HEX:
- var = vp10_fast_hex_search(x, mvp_full, step_param, error_per_bit, 0,
- cost_list, fn_ptr, 1, ref_mv, tmp_mv);
- break;
- case HEX:
- var = vp10_hex_search(x, mvp_full, step_param, error_per_bit, 1,
- cost_list, fn_ptr, 1, ref_mv, tmp_mv);
- break;
- case SQUARE:
- var = vp10_square_search(x, mvp_full, step_param, error_per_bit, 1,
- cost_list, fn_ptr, 1, ref_mv, tmp_mv);
- break;
- case BIGDIA:
- var = vp10_bigdia_search(x, mvp_full, step_param, error_per_bit, 1,
- cost_list, fn_ptr, 1, ref_mv, tmp_mv);
- break;
- case NSTEP:
- var = vp10_full_pixel_diamond(cpi, x, mvp_full, step_param, error_per_bit,
- MAX_MVSEARCH_STEPS - 1 - step_param,
- 1, cost_list, fn_ptr, ref_mv, tmp_mv);
- break;
- default:
- assert(0 && "Invalid search method.");
- }
-
- if (method != NSTEP && rd && var < var_max)
- var = vp10_get_mvpred_var(x, tmp_mv, ref_mv, fn_ptr, 1);
-
- return var;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_MCOMP_H_
-#define VP10_ENCODER_MCOMP_H_
-
-#include "vp10/encoder/block.h"
-#include "vpx_dsp/variance.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// The maximum number of steps in a step search given the largest
-// allowed initial step
-#define MAX_MVSEARCH_STEPS 11
-// Max full pel mv specified in the unit of full pixel
-// Enable the use of motion vector in range [-1023, 1023].
-#define MAX_FULL_PEL_VAL ((1 << (MAX_MVSEARCH_STEPS - 1)) - 1)
-// Maximum size of the first step in full pel units
-#define MAX_FIRST_STEP (1 << (MAX_MVSEARCH_STEPS-1))
-// Allowed motion vector pixel distance outside image border
-// for Block_16x16
-#define BORDER_MV_PIXELS_B16 (16 + VP9_INTERP_EXTEND)
-
-// motion search site
-typedef struct search_site {
- MV mv;
- int offset;
-} search_site;
-
-typedef struct search_site_config {
- search_site ss[8 * MAX_MVSEARCH_STEPS + 1];
- int ss_count;
- int searches_per_step;
-} search_site_config;
-
-void vp10_init_dsmotion_compensation(search_site_config *cfg, int stride);
-void vp10_init3smotion_compensation(search_site_config *cfg, int stride);
-
-void vp10_set_mv_search_range(MACROBLOCK *x, const MV *mv);
-int vp10_mv_bit_cost(const MV *mv, const MV *ref,
- const int *mvjcost, int *mvcost[2], int weight);
-
-// Utility to compute variance + MV rate cost for a given MV
-int vp10_get_mvpred_var(const MACROBLOCK *x,
- const MV *best_mv, const MV *center_mv,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost);
-int vp10_get_mvpred_av_var(const MACROBLOCK *x,
- const MV *best_mv, const MV *center_mv,
- const uint8_t *second_pred,
- const vp9_variance_fn_ptr_t *vfp,
- int use_mvcost);
-
-struct VP10_COMP;
-struct SPEED_FEATURES;
-
-int vp10_init_search_range(int size);
-
-int vp10_refining_search_sad(const struct macroblock *x,
- struct mv *ref_mv,
- int sad_per_bit, int distance,
- const struct vp9_variance_vtable *fn_ptr,
- const struct mv *center_mv);
-
-// Runs sequence of diamond searches in smaller steps for RD.
-int vp10_full_pixel_diamond(const struct VP10_COMP *cpi, MACROBLOCK *x,
- MV *mvp_full, int step_param,
- int sadpb, int further_steps, int do_refine,
- int *cost_list,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *ref_mv, MV *dst_mv);
-
-// Perform integral projection based motion estimation.
-unsigned int vp10_int_pro_motion_estimation(const struct VP10_COMP *cpi,
- MACROBLOCK *x,
- BLOCK_SIZE bsize,
- int mi_row, int mi_col);
-
-typedef int (integer_mv_pattern_search_fn) (
- const MACROBLOCK *x,
- MV *ref_mv,
- int search_param,
- int error_per_bit,
- int do_init_search,
- int *cost_list,
- const vp9_variance_fn_ptr_t *vf,
- int use_mvcost,
- const MV *center_mv,
- MV *best_mv);
-
-integer_mv_pattern_search_fn vp10_hex_search;
-integer_mv_pattern_search_fn vp10_bigdia_search;
-integer_mv_pattern_search_fn vp10_square_search;
-integer_mv_pattern_search_fn vp10_fast_hex_search;
-integer_mv_pattern_search_fn vp10_fast_dia_search;
-
-typedef int (fractional_mv_step_fp) (
- const MACROBLOCK *x,
- MV *bestmv, const MV *ref_mv,
- int allow_hp,
- int error_per_bit,
- const vp9_variance_fn_ptr_t *vfp,
- int forced_stop, // 0 - full, 1 - qtr only, 2 - half only
- int iters_per_step,
- int *cost_list,
- int *mvjcost, int *mvcost[2],
- int *distortion, unsigned int *sse1,
- const uint8_t *second_pred,
- int w, int h);
-
-extern fractional_mv_step_fp vp10_find_best_sub_pixel_tree;
-extern fractional_mv_step_fp vp10_find_best_sub_pixel_tree_pruned;
-extern fractional_mv_step_fp vp10_find_best_sub_pixel_tree_pruned_more;
-extern fractional_mv_step_fp vp10_find_best_sub_pixel_tree_pruned_evenmore;
-
-typedef int (*vp10_full_search_fn_t)(const MACROBLOCK *x,
- const MV *ref_mv, int sad_per_bit,
- int distance,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv, MV *best_mv);
-
-typedef int (*vp10_refining_search_fn_t)(const MACROBLOCK *x,
- MV *ref_mv, int sad_per_bit,
- int distance,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv);
-
-typedef int (*vp10_diamond_search_fn_t)(const MACROBLOCK *x,
- const search_site_config *cfg,
- MV *ref_mv, MV *best_mv,
- int search_param, int sad_per_bit,
- int *num00,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv);
-
-int vp10_refining_search_8p_c(const MACROBLOCK *x,
- MV *ref_mv, int error_per_bit,
- int search_range,
- const vp9_variance_fn_ptr_t *fn_ptr,
- const MV *center_mv, const uint8_t *second_pred);
-
-struct VP10_COMP;
-
-int vp10_full_pixel_search(struct VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bsize, MV *mvp_full,
- int step_param, int error_per_bit,
- int *cost_list,
- const MV *ref_mv, MV *tmp_mv,
- int var_max, int rd);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_MCOMP_H_
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vp10_rtcd.h"
-#include "vpx_dsp/mips/macros_msa.h"
-
-uint32_t vp10_avg_8x8_msa(const uint8_t *src, int32_t src_stride) {
- uint32_t sum_out;
- v16u8 src0, src1, src2, src3, src4, src5, src6, src7;
- v8u16 sum0, sum1, sum2, sum3, sum4, sum5, sum6, sum7;
- v4u32 sum = { 0 };
-
- LD_UB8(src, src_stride, src0, src1, src2, src3, src4, src5, src6, src7);
- HADD_UB4_UH(src0, src1, src2, src3, sum0, sum1, sum2, sum3);
- HADD_UB4_UH(src4, src5, src6, src7, sum4, sum5, sum6, sum7);
- ADD4(sum0, sum1, sum2, sum3, sum4, sum5, sum6, sum7, sum0, sum2, sum4, sum6);
- ADD2(sum0, sum2, sum4, sum6, sum0, sum4);
- sum0 += sum4;
-
- sum = __msa_hadd_u_w(sum0, sum0);
- sum0 = (v8u16)__msa_pckev_h((v8i16)sum, (v8i16)sum);
- sum = __msa_hadd_u_w(sum0, sum0);
- sum = (v4u32)__msa_srari_w((v4i32)sum, 6);
- sum_out = __msa_copy_u_w((v4i32)sum, 0);
-
- return sum_out;
-}
-
-uint32_t vp10_avg_4x4_msa(const uint8_t *src, int32_t src_stride) {
- uint32_t sum_out;
- uint32_t src0, src1, src2, src3;
- v16u8 vec = { 0 };
- v8u16 sum0;
- v4u32 sum1;
- v2u64 sum2;
-
- LW4(src, src_stride, src0, src1, src2, src3);
- INSERT_W4_UB(src0, src1, src2, src3, vec);
-
- sum0 = __msa_hadd_u_h(vec, vec);
- sum1 = __msa_hadd_u_w(sum0, sum0);
- sum0 = (v8u16)__msa_pckev_h((v8i16)sum1, (v8i16)sum1);
- sum1 = __msa_hadd_u_w(sum0, sum0);
- sum2 = __msa_hadd_u_d(sum1, sum1);
- sum1 = (v4u32)__msa_srari_w((v4i32)sum2, 4);
- sum_out = __msa_copy_u_w((v4i32)sum1, 0);
-
- return sum_out;
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vp10_rtcd.h"
-#include "vpx_dsp/mips/macros_msa.h"
-
-#define BLOCK_ERROR_BLOCKSIZE_MSA(BSize) \
-static int64_t block_error_##BSize##size_msa(const int16_t *coeff_ptr, \
- const int16_t *dq_coeff_ptr, \
- int64_t *ssz) { \
- int64_t err = 0; \
- uint32_t loop_cnt; \
- v8i16 coeff, dq_coeff, coeff_r_h, coeff_l_h; \
- v4i32 diff_r, diff_l, coeff_r_w, coeff_l_w; \
- v2i64 sq_coeff_r, sq_coeff_l; \
- v2i64 err0, err_dup0, err1, err_dup1; \
- \
- coeff = LD_SH(coeff_ptr); \
- dq_coeff = LD_SH(dq_coeff_ptr); \
- UNPCK_SH_SW(coeff, coeff_r_w, coeff_l_w); \
- ILVRL_H2_SH(coeff, dq_coeff, coeff_r_h, coeff_l_h); \
- HSUB_UH2_SW(coeff_r_h, coeff_l_h, diff_r, diff_l); \
- DOTP_SW2_SD(coeff_r_w, coeff_l_w, coeff_r_w, coeff_l_w, \
- sq_coeff_r, sq_coeff_l); \
- DOTP_SW2_SD(diff_r, diff_l, diff_r, diff_l, err0, err1); \
- \
- coeff = LD_SH(coeff_ptr + 8); \
- dq_coeff = LD_SH(dq_coeff_ptr + 8); \
- UNPCK_SH_SW(coeff, coeff_r_w, coeff_l_w); \
- ILVRL_H2_SH(coeff, dq_coeff, coeff_r_h, coeff_l_h); \
- HSUB_UH2_SW(coeff_r_h, coeff_l_h, diff_r, diff_l); \
- DPADD_SD2_SD(coeff_r_w, coeff_l_w, sq_coeff_r, sq_coeff_l); \
- DPADD_SD2_SD(diff_r, diff_l, err0, err1); \
- \
- coeff_ptr += 16; \
- dq_coeff_ptr += 16; \
- \
- for (loop_cnt = ((BSize >> 4) - 1); loop_cnt--;) { \
- coeff = LD_SH(coeff_ptr); \
- dq_coeff = LD_SH(dq_coeff_ptr); \
- UNPCK_SH_SW(coeff, coeff_r_w, coeff_l_w); \
- ILVRL_H2_SH(coeff, dq_coeff, coeff_r_h, coeff_l_h); \
- HSUB_UH2_SW(coeff_r_h, coeff_l_h, diff_r, diff_l); \
- DPADD_SD2_SD(coeff_r_w, coeff_l_w, sq_coeff_r, sq_coeff_l); \
- DPADD_SD2_SD(diff_r, diff_l, err0, err1); \
- \
- coeff = LD_SH(coeff_ptr + 8); \
- dq_coeff = LD_SH(dq_coeff_ptr + 8); \
- UNPCK_SH_SW(coeff, coeff_r_w, coeff_l_w); \
- ILVRL_H2_SH(coeff, dq_coeff, coeff_r_h, coeff_l_h); \
- HSUB_UH2_SW(coeff_r_h, coeff_l_h, diff_r, diff_l); \
- DPADD_SD2_SD(coeff_r_w, coeff_l_w, sq_coeff_r, sq_coeff_l); \
- DPADD_SD2_SD(diff_r, diff_l, err0, err1); \
- \
- coeff_ptr += 16; \
- dq_coeff_ptr += 16; \
- } \
- \
- err_dup0 = __msa_splati_d(sq_coeff_r, 1); \
- err_dup1 = __msa_splati_d(sq_coeff_l, 1); \
- sq_coeff_r += err_dup0; \
- sq_coeff_l += err_dup1; \
- *ssz = __msa_copy_s_d(sq_coeff_r, 0); \
- *ssz += __msa_copy_s_d(sq_coeff_l, 0); \
- \
- err_dup0 = __msa_splati_d(err0, 1); \
- err_dup1 = __msa_splati_d(err1, 1); \
- err0 += err_dup0; \
- err1 += err_dup1; \
- err = __msa_copy_s_d(err0, 0); \
- err += __msa_copy_s_d(err1, 0); \
- \
- return err; \
-}
-
-BLOCK_ERROR_BLOCKSIZE_MSA(16);
-BLOCK_ERROR_BLOCKSIZE_MSA(64);
-BLOCK_ERROR_BLOCKSIZE_MSA(256);
-BLOCK_ERROR_BLOCKSIZE_MSA(1024);
-
-int64_t vp10_block_error_msa(const tran_low_t *coeff_ptr,
- const tran_low_t *dq_coeff_ptr,
- intptr_t blk_size, int64_t *ssz) {
- int64_t err;
- const int16_t *coeff = (const int16_t *)coeff_ptr;
- const int16_t *dq_coeff = (const int16_t *)dq_coeff_ptr;
-
- switch (blk_size) {
- case 16:
- err = block_error_16size_msa(coeff, dq_coeff, ssz);
- break;
- case 64:
- err = block_error_64size_msa(coeff, dq_coeff, ssz);
- break;
- case 256:
- err = block_error_256size_msa(coeff, dq_coeff, ssz);
- break;
- case 1024:
- err = block_error_1024size_msa(coeff, dq_coeff, ssz);
- break;
- default:
- err = vp10_block_error_c(coeff_ptr, dq_coeff_ptr, blk_size, ssz);
- break;
- }
-
- return err;
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/enums.h"
-#include "vp10/encoder/mips/msa/fdct_msa.h"
-#include "vpx_dsp/mips/fwd_txfm_msa.h"
-
-static void fadst16_cols_step1_msa(const int16_t *input, int32_t stride,
- const int32_t *const0, int16_t *int_buf) {
- v8i16 r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15;
- v8i16 tp0, tp1, tp2, tp3, g0, g1, g2, g3, g8, g9, g10, g11, h0, h1, h2, h3;
- v4i32 k0, k1, k2, k3;
-
- /* load input data */
- r0 = LD_SH(input);
- r15 = LD_SH(input + 15 * stride);
- r7 = LD_SH(input + 7 * stride);
- r8 = LD_SH(input + 8 * stride);
- SLLI_4V(r0, r15, r7, r8, 2);
-
- /* stage 1 */
- LD_SW2(const0, 4, k0, k1);
- LD_SW2(const0 + 8, 4, k2, k3);
- MADD_BF(r15, r0, r7, r8, k0, k1, k2, k3, g0, g1, g2, g3);
-
- r3 = LD_SH(input + 3 * stride);
- r4 = LD_SH(input + 4 * stride);
- r11 = LD_SH(input + 11 * stride);
- r12 = LD_SH(input + 12 * stride);
- SLLI_4V(r3, r4, r11, r12, 2);
-
- LD_SW2(const0 + 4 * 4, 4, k0, k1);
- LD_SW2(const0 + 4 * 6, 4, k2, k3);
- MADD_BF(r11, r4, r3, r12, k0, k1, k2, k3, g8, g9, g10, g11);
-
- /* stage 2 */
- BUTTERFLY_4(g0, g2, g10, g8, tp0, tp2, tp3, tp1);
- ST_SH2(tp0, tp2, int_buf, 8);
- ST_SH2(tp1, tp3, int_buf + 4 * 8, 8);
-
- LD_SW2(const0 + 4 * 8, 4, k0, k1);
- k2 = LD_SW(const0 + 4 * 10);
- MADD_BF(g1, g3, g9, g11, k0, k1, k2, k0, h0, h1, h2, h3);
-
- ST_SH2(h0, h1, int_buf + 8 * 8, 8);
- ST_SH2(h3, h2, int_buf + 12 * 8, 8);
-
- r9 = LD_SH(input + 9 * stride);
- r6 = LD_SH(input + 6 * stride);
- r1 = LD_SH(input + stride);
- r14 = LD_SH(input + 14 * stride);
- SLLI_4V(r9, r6, r1, r14, 2);
-
- LD_SW2(const0 + 4 * 11, 4, k0, k1);
- LD_SW2(const0 + 4 * 13, 4, k2, k3);
- MADD_BF(r9, r6, r1, r14, k0, k1, k2, k3, g0, g1, g2, g3);
-
- ST_SH2(g1, g3, int_buf + 3 * 8, 4 * 8);
-
- r13 = LD_SH(input + 13 * stride);
- r2 = LD_SH(input + 2 * stride);
- r5 = LD_SH(input + 5 * stride);
- r10 = LD_SH(input + 10 * stride);
- SLLI_4V(r13, r2, r5, r10, 2);
-
- LD_SW2(const0 + 4 * 15, 4, k0, k1);
- LD_SW2(const0 + 4 * 17, 4, k2, k3);
- MADD_BF(r13, r2, r5, r10, k0, k1, k2, k3, h0, h1, h2, h3);
-
- ST_SH2(h1, h3, int_buf + 11 * 8, 4 * 8);
-
- BUTTERFLY_4(h0, h2, g2, g0, tp0, tp1, tp2, tp3);
- ST_SH4(tp0, tp1, tp2, tp3, int_buf + 2 * 8, 4 * 8);
-}
-
-static void fadst16_cols_step2_msa(int16_t *int_buf, const int32_t *const0,
- int16_t *out) {
- int16_t *out_ptr = out + 128;
- v8i16 tp0, tp1, tp2, tp3, g5, g7, g13, g15;
- v8i16 h0, h1, h2, h3, h4, h5, h6, h7, h10, h11;
- v8i16 out0, out1, out2, out3, out4, out5, out6, out7;
- v8i16 out8, out9, out10, out11, out12, out13, out14, out15;
- v4i32 k0, k1, k2, k3;
-
- LD_SH2(int_buf + 3 * 8, 4 * 8, g13, g15);
- LD_SH2(int_buf + 11 * 8, 4 * 8, g5, g7);
- LD_SW2(const0 + 4 * 19, 4, k0, k1);
- k2 = LD_SW(const0 + 4 * 21);
- MADD_BF(g7, g5, g15, g13, k0, k1, k2, k0, h4, h5, h6, h7);
-
- tp0 = LD_SH(int_buf + 4 * 8);
- tp1 = LD_SH(int_buf + 5 * 8);
- tp3 = LD_SH(int_buf + 10 * 8);
- tp2 = LD_SH(int_buf + 14 * 8);
- LD_SW2(const0 + 4 * 22, 4, k0, k1);
- k2 = LD_SW(const0 + 4 * 24);
- MADD_BF(tp0, tp1, tp2, tp3, k0, k1, k2, k0, out4, out6, out5, out7);
- out4 = -out4;
- ST_SH(out4, (out + 3 * 16));
- ST_SH(out5, (out_ptr + 4 * 16));
-
- h1 = LD_SH(int_buf + 9 * 8);
- h3 = LD_SH(int_buf + 12 * 8);
- MADD_BF(h1, h3, h5, h7, k0, k1, k2, k0, out12, out14, out13, out15);
- out13 = -out13;
- ST_SH(out12, (out + 2 * 16));
- ST_SH(out13, (out_ptr + 5 * 16));
-
- tp0 = LD_SH(int_buf);
- tp1 = LD_SH(int_buf + 8);
- tp2 = LD_SH(int_buf + 2 * 8);
- tp3 = LD_SH(int_buf + 6 * 8);
-
- BUTTERFLY_4(tp0, tp1, tp3, tp2, out0, out1, h11, h10);
- out1 = -out1;
- ST_SH(out0, (out));
- ST_SH(out1, (out_ptr + 7 * 16));
-
- h0 = LD_SH(int_buf + 8 * 8);
- h2 = LD_SH(int_buf + 13 * 8);
-
- BUTTERFLY_4(h0, h2, h6, h4, out8, out9, out11, out10);
- out8 = -out8;
- ST_SH(out8, (out + 16));
- ST_SH(out9, (out_ptr + 6 * 16));
-
- /* stage 4 */
- LD_SW2(const0 + 4 * 25, 4, k0, k1);
- LD_SW2(const0 + 4 * 27, 4, k2, k3);
- MADD_SHORT(h10, h11, k1, k2, out2, out3);
- ST_SH(out2, (out + 7 * 16));
- ST_SH(out3, (out_ptr));
-
- MADD_SHORT(out6, out7, k0, k3, out6, out7);
- ST_SH(out6, (out + 4 * 16));
- ST_SH(out7, (out_ptr + 3 * 16));
-
- MADD_SHORT(out10, out11, k0, k3, out10, out11);
- ST_SH(out10, (out + 6 * 16));
- ST_SH(out11, (out_ptr + 16));
-
- MADD_SHORT(out14, out15, k1, k2, out14, out15);
- ST_SH(out14, (out + 5 * 16));
- ST_SH(out15, (out_ptr + 2 * 16));
-}
-
-static void fadst16_transpose_postproc_msa(int16_t *input, int16_t *out) {
- v8i16 r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15;
- v8i16 l0, l1, l2, l3, l4, l5, l6, l7, l8, l9, l10, l11, l12, l13, l14, l15;
-
- /* load input data */
- LD_SH8(input, 16, l0, l1, l2, l3, l4, l5, l6, l7);
- TRANSPOSE8x8_SH_SH(l0, l1, l2, l3, l4, l5, l6, l7,
- r0, r1, r2, r3, r4, r5, r6, r7);
- FDCT_POSTPROC_2V_NEG_H(r0, r1);
- FDCT_POSTPROC_2V_NEG_H(r2, r3);
- FDCT_POSTPROC_2V_NEG_H(r4, r5);
- FDCT_POSTPROC_2V_NEG_H(r6, r7);
- ST_SH8(r0, r1, r2, r3, r4, r5, r6, r7, out, 8);
- out += 64;
-
- LD_SH8(input + 8, 16, l8, l9, l10, l11, l12, l13, l14, l15);
- TRANSPOSE8x8_SH_SH(l8, l9, l10, l11, l12, l13, l14, l15,
- r8, r9, r10, r11, r12, r13, r14, r15);
- FDCT_POSTPROC_2V_NEG_H(r8, r9);
- FDCT_POSTPROC_2V_NEG_H(r10, r11);
- FDCT_POSTPROC_2V_NEG_H(r12, r13);
- FDCT_POSTPROC_2V_NEG_H(r14, r15);
- ST_SH8(r8, r9, r10, r11, r12, r13, r14, r15, out, 8);
- out += 64;
-
- /* load input data */
- input += 128;
- LD_SH8(input, 16, l0, l1, l2, l3, l4, l5, l6, l7);
- TRANSPOSE8x8_SH_SH(l0, l1, l2, l3, l4, l5, l6, l7,
- r0, r1, r2, r3, r4, r5, r6, r7);
- FDCT_POSTPROC_2V_NEG_H(r0, r1);
- FDCT_POSTPROC_2V_NEG_H(r2, r3);
- FDCT_POSTPROC_2V_NEG_H(r4, r5);
- FDCT_POSTPROC_2V_NEG_H(r6, r7);
- ST_SH8(r0, r1, r2, r3, r4, r5, r6, r7, out, 8);
- out += 64;
-
- LD_SH8(input + 8, 16, l8, l9, l10, l11, l12, l13, l14, l15);
- TRANSPOSE8x8_SH_SH(l8, l9, l10, l11, l12, l13, l14, l15,
- r8, r9, r10, r11, r12, r13, r14, r15);
- FDCT_POSTPROC_2V_NEG_H(r8, r9);
- FDCT_POSTPROC_2V_NEG_H(r10, r11);
- FDCT_POSTPROC_2V_NEG_H(r12, r13);
- FDCT_POSTPROC_2V_NEG_H(r14, r15);
- ST_SH8(r8, r9, r10, r11, r12, r13, r14, r15, out, 8);
-}
-
-static void fadst16_rows_step1_msa(int16_t *input, const int32_t *const0,
- int16_t *int_buf) {
- v8i16 r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15;
- v8i16 tp0, tp1, tp2, tp3, g0, g1, g2, g3, g8, g9, g10, g11, h0, h1, h2, h3;
- v4i32 k0, k1, k2, k3;
-
- /* load input data */
- r0 = LD_SH(input);
- r7 = LD_SH(input + 7 * 8);
- r8 = LD_SH(input + 8 * 8);
- r15 = LD_SH(input + 15 * 8);
-
- /* stage 1 */
- LD_SW2(const0, 4, k0, k1);
- LD_SW2(const0 + 4 * 2, 4, k2, k3);
- MADD_BF(r15, r0, r7, r8, k0, k1, k2, k3, g0, g1, g2, g3);
-
- r3 = LD_SH(input + 3 * 8);
- r4 = LD_SH(input + 4 * 8);
- r11 = LD_SH(input + 11 * 8);
- r12 = LD_SH(input + 12 * 8);
-
- LD_SW2(const0 + 4 * 4, 4, k0, k1);
- LD_SW2(const0 + 4 * 6, 4, k2, k3);
- MADD_BF(r11, r4, r3, r12, k0, k1, k2, k3, g8, g9, g10, g11);
-
- /* stage 2 */
- BUTTERFLY_4(g0, g2, g10, g8, tp0, tp2, tp3, tp1);
- ST_SH2(tp0, tp1, int_buf, 4 * 8);
- ST_SH2(tp2, tp3, int_buf + 8, 4 * 8);
-
- LD_SW2(const0 + 4 * 8, 4, k0, k1);
- k2 = LD_SW(const0 + 4 * 10);
- MADD_BF(g1, g3, g9, g11, k0, k1, k2, k0, h0, h1, h2, h3);
- ST_SH2(h0, h3, int_buf + 8 * 8, 4 * 8);
- ST_SH2(h1, h2, int_buf + 9 * 8, 4 * 8);
-
- r1 = LD_SH(input + 8);
- r6 = LD_SH(input + 6 * 8);
- r9 = LD_SH(input + 9 * 8);
- r14 = LD_SH(input + 14 * 8);
-
- LD_SW2(const0 + 4 * 11, 4, k0, k1);
- LD_SW2(const0 + 4 * 13, 4, k2, k3);
- MADD_BF(r9, r6, r1, r14, k0, k1, k2, k3, g0, g1, g2, g3);
- ST_SH2(g1, g3, int_buf + 3 * 8, 4 * 8);
-
- r2 = LD_SH(input + 2 * 8);
- r5 = LD_SH(input + 5 * 8);
- r10 = LD_SH(input + 10 * 8);
- r13 = LD_SH(input + 13 * 8);
-
- LD_SW2(const0 + 4 * 15, 4, k0, k1);
- LD_SW2(const0 + 4 * 17, 4, k2, k3);
- MADD_BF(r13, r2, r5, r10, k0, k1, k2, k3, h0, h1, h2, h3);
- ST_SH2(h1, h3, int_buf + 11 * 8, 4 * 8);
- BUTTERFLY_4(h0, h2, g2, g0, tp0, tp1, tp2, tp3);
- ST_SH4(tp0, tp1, tp2, tp3, int_buf + 2 * 8, 4 * 8);
-}
-
-static void fadst16_rows_step2_msa(int16_t *int_buf, const int32_t *const0,
- int16_t *out) {
- int16_t *out_ptr = out + 8;
- v8i16 tp0, tp1, tp2, tp3, g5, g7, g13, g15;
- v8i16 h0, h1, h2, h3, h4, h5, h6, h7, h10, h11;
- v8i16 out0, out1, out2, out3, out4, out5, out6, out7;
- v8i16 out8, out9, out10, out11, out12, out13, out14, out15;
- v4i32 k0, k1, k2, k3;
-
- g13 = LD_SH(int_buf + 3 * 8);
- g15 = LD_SH(int_buf + 7 * 8);
- g5 = LD_SH(int_buf + 11 * 8);
- g7 = LD_SH(int_buf + 15 * 8);
-
- LD_SW2(const0 + 4 * 19, 4, k0, k1);
- k2 = LD_SW(const0 + 4 * 21);
- MADD_BF(g7, g5, g15, g13, k0, k1, k2, k0, h4, h5, h6, h7);
-
- tp0 = LD_SH(int_buf + 4 * 8);
- tp1 = LD_SH(int_buf + 5 * 8);
- tp3 = LD_SH(int_buf + 10 * 8);
- tp2 = LD_SH(int_buf + 14 * 8);
-
- LD_SW2(const0 + 4 * 22, 4, k0, k1);
- k2 = LD_SW(const0 + 4 * 24);
- MADD_BF(tp0, tp1, tp2, tp3, k0, k1, k2, k0, out4, out6, out5, out7);
- out4 = -out4;
- ST_SH(out4, (out + 3 * 16));
- ST_SH(out5, (out_ptr + 4 * 16));
-
- h1 = LD_SH(int_buf + 9 * 8);
- h3 = LD_SH(int_buf + 12 * 8);
- MADD_BF(h1, h3, h5, h7, k0, k1, k2, k0, out12, out14, out13, out15);
- out13 = -out13;
- ST_SH(out12, (out + 2 * 16));
- ST_SH(out13, (out_ptr + 5 * 16));
-
- tp0 = LD_SH(int_buf);
- tp1 = LD_SH(int_buf + 8);
- tp2 = LD_SH(int_buf + 2 * 8);
- tp3 = LD_SH(int_buf + 6 * 8);
-
- BUTTERFLY_4(tp0, tp1, tp3, tp2, out0, out1, h11, h10);
- out1 = -out1;
- ST_SH(out0, (out));
- ST_SH(out1, (out_ptr + 7 * 16));
-
- h0 = LD_SH(int_buf + 8 * 8);
- h2 = LD_SH(int_buf + 13 * 8);
- BUTTERFLY_4(h0, h2, h6, h4, out8, out9, out11, out10);
- out8 = -out8;
- ST_SH(out8, (out + 16));
- ST_SH(out9, (out_ptr + 6 * 16));
-
- /* stage 4 */
- LD_SW2(const0 + 4 * 25, 4, k0, k1);
- LD_SW2(const0 + 4 * 27, 4, k2, k3);
- MADD_SHORT(h10, h11, k1, k2, out2, out3);
- ST_SH(out2, (out + 7 * 16));
- ST_SH(out3, (out_ptr));
-
- MADD_SHORT(out6, out7, k0, k3, out6, out7);
- ST_SH(out6, (out + 4 * 16));
- ST_SH(out7, (out_ptr + 3 * 16));
-
- MADD_SHORT(out10, out11, k0, k3, out10, out11);
- ST_SH(out10, (out + 6 * 16));
- ST_SH(out11, (out_ptr + 16));
-
- MADD_SHORT(out14, out15, k1, k2, out14, out15);
- ST_SH(out14, (out + 5 * 16));
- ST_SH(out15, (out_ptr + 2 * 16));
-}
-
-static void fadst16_transpose_msa(int16_t *input, int16_t *out) {
- v8i16 r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15;
- v8i16 l0, l1, l2, l3, l4, l5, l6, l7, l8, l9, l10, l11, l12, l13, l14, l15;
-
- /* load input data */
- LD_SH16(input, 8, l0, l8, l1, l9, l2, l10, l3, l11,
- l4, l12, l5, l13, l6, l14, l7, l15);
- TRANSPOSE8x8_SH_SH(l0, l1, l2, l3, l4, l5, l6, l7,
- r0, r1, r2, r3, r4, r5, r6, r7);
- TRANSPOSE8x8_SH_SH(l8, l9, l10, l11, l12, l13, l14, l15,
- r8, r9, r10, r11, r12, r13, r14, r15);
- ST_SH8(r0, r8, r1, r9, r2, r10, r3, r11, out, 8);
- ST_SH8(r4, r12, r5, r13, r6, r14, r7, r15, (out + 64), 8);
- out += 16 * 8;
-
- /* load input data */
- input += 128;
- LD_SH16(input, 8, l0, l8, l1, l9, l2, l10, l3, l11,
- l4, l12, l5, l13, l6, l14, l7, l15);
- TRANSPOSE8x8_SH_SH(l0, l1, l2, l3, l4, l5, l6, l7,
- r0, r1, r2, r3, r4, r5, r6, r7);
- TRANSPOSE8x8_SH_SH(l8, l9, l10, l11, l12, l13, l14, l15,
- r8, r9, r10, r11, r12, r13, r14, r15);
- ST_SH8(r0, r8, r1, r9, r2, r10, r3, r11, out, 8);
- ST_SH8(r4, r12, r5, r13, r6, r14, r7, r15, (out + 64), 8);
-}
-
-static void postproc_fdct16x8_1d_row(int16_t *intermediate, int16_t *output) {
- int16_t *temp = intermediate;
- int16_t *out = output;
- v8i16 tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7;
- v8i16 in0, in1, in2, in3, in4, in5, in6, in7, in8, in9, in10, in11;
- v8i16 in12, in13, in14, in15;
-
- LD_SH8(temp, 16, in0, in1, in2, in3, in4, in5, in6, in7);
- temp = intermediate + 8;
- LD_SH8(temp, 16, in8, in9, in10, in11, in12, in13, in14, in15);
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- TRANSPOSE8x8_SH_SH(in8, in9, in10, in11, in12, in13, in14, in15,
- in8, in9, in10, in11, in12, in13, in14, in15);
- FDCT_POSTPROC_2V_NEG_H(in0, in1);
- FDCT_POSTPROC_2V_NEG_H(in2, in3);
- FDCT_POSTPROC_2V_NEG_H(in4, in5);
- FDCT_POSTPROC_2V_NEG_H(in6, in7);
- FDCT_POSTPROC_2V_NEG_H(in8, in9);
- FDCT_POSTPROC_2V_NEG_H(in10, in11);
- FDCT_POSTPROC_2V_NEG_H(in12, in13);
- FDCT_POSTPROC_2V_NEG_H(in14, in15);
- BUTTERFLY_16(in0, in1, in2, in3, in4, in5, in6, in7,
- in8, in9, in10, in11, in12, in13, in14, in15,
- tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7,
- in8, in9, in10, in11, in12, in13, in14, in15);
- temp = intermediate;
- ST_SH8(in8, in9, in10, in11, in12, in13, in14, in15, temp, 16);
- FDCT8x16_EVEN(tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7,
- tmp0, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7);
- temp = intermediate;
- LD_SH8(temp, 16, in8, in9, in10, in11, in12, in13, in14, in15);
- FDCT8x16_ODD(in8, in9, in10, in11, in12, in13, in14, in15,
- in0, in1, in2, in3, in4, in5, in6, in7);
- TRANSPOSE8x8_SH_SH(tmp0, in0, tmp1, in1, tmp2, in2, tmp3, in3,
- tmp0, in0, tmp1, in1, tmp2, in2, tmp3, in3);
- ST_SH8(tmp0, in0, tmp1, in1, tmp2, in2, tmp3, in3, out, 16);
- TRANSPOSE8x8_SH_SH(tmp4, in4, tmp5, in5, tmp6, in6, tmp7, in7,
- tmp4, in4, tmp5, in5, tmp6, in6, tmp7, in7);
- out = output + 8;
- ST_SH8(tmp4, in4, tmp5, in5, tmp6, in6, tmp7, in7, out, 16);
-}
-
-void vp10_fht16x16_msa(const int16_t *input, int16_t *output,
- int32_t stride, int32_t tx_type) {
- DECLARE_ALIGNED(32, int16_t, tmp[256]);
- DECLARE_ALIGNED(32, int16_t, trans_buf[256]);
- DECLARE_ALIGNED(32, int16_t, tmp_buf[128]);
- int32_t i;
- int16_t *ptmpbuf = &tmp_buf[0];
- int16_t *trans = &trans_buf[0];
- const int32_t const_arr[29 * 4] = {
- 52707308, 52707308, 52707308, 52707308,
- -1072430300, -1072430300, -1072430300, -1072430300,
- 795618043, 795618043, 795618043, 795618043,
- -721080468, -721080468, -721080468, -721080468,
- 459094491, 459094491, 459094491, 459094491,
- -970646691, -970646691, -970646691, -970646691,
- 1010963856, 1010963856, 1010963856, 1010963856,
- -361743294, -361743294, -361743294, -361743294,
- 209469125, 209469125, 209469125, 209469125,
- -1053094788, -1053094788, -1053094788, -1053094788,
- 1053160324, 1053160324, 1053160324, 1053160324,
- 639644520, 639644520, 639644520, 639644520,
- -862444000, -862444000, -862444000, -862444000,
- 1062144356, 1062144356, 1062144356, 1062144356,
- -157532337, -157532337, -157532337, -157532337,
- 260914709, 260914709, 260914709, 260914709,
- -1041559667, -1041559667, -1041559667, -1041559667,
- 920985831, 920985831, 920985831, 920985831,
- -551995675, -551995675, -551995675, -551995675,
- 596522295, 596522295, 596522295, 596522295,
- 892853362, 892853362, 892853362, 892853362,
- -892787826, -892787826, -892787826, -892787826,
- 410925857, 410925857, 410925857, 410925857,
- -992012162, -992012162, -992012162, -992012162,
- 992077698, 992077698, 992077698, 992077698,
- 759246145, 759246145, 759246145, 759246145,
- -759180609, -759180609, -759180609, -759180609,
- -759222975, -759222975, -759222975, -759222975,
- 759288511, 759288511, 759288511, 759288511 };
-
- switch (tx_type) {
- case DCT_DCT:
- /* column transform */
- for (i = 0; i < 2; ++i) {
- fdct8x16_1d_column(input + 8 * i, tmp + 8 * i, stride);
- }
-
- /* row transform */
- for (i = 0; i < 2; ++i) {
- fdct16x8_1d_row(tmp + (128 * i), output + (128 * i));
- }
- break;
- case ADST_DCT:
- /* column transform */
- for (i = 0; i < 2; ++i) {
- fadst16_cols_step1_msa(input + (i << 3), stride, const_arr, ptmpbuf);
- fadst16_cols_step2_msa(ptmpbuf, const_arr, tmp + (i << 3));
- }
-
- /* row transform */
- for (i = 0; i < 2; ++i) {
- postproc_fdct16x8_1d_row(tmp + (128 * i), output + (128 * i));
- }
- break;
- case DCT_ADST:
- /* column transform */
- for (i = 0; i < 2; ++i) {
- fdct8x16_1d_column(input + 8 * i, tmp + 8 * i, stride);
- }
-
- fadst16_transpose_postproc_msa(tmp, trans);
-
- /* row transform */
- for (i = 0; i < 2; ++i) {
- fadst16_rows_step1_msa(trans + (i << 7), const_arr, ptmpbuf);
- fadst16_rows_step2_msa(ptmpbuf, const_arr, tmp + (i << 7));
- }
-
- fadst16_transpose_msa(tmp, output);
- break;
- case ADST_ADST:
- /* column transform */
- for (i = 0; i < 2; ++i) {
- fadst16_cols_step1_msa(input + (i << 3), stride, const_arr, ptmpbuf);
- fadst16_cols_step2_msa(ptmpbuf, const_arr, tmp + (i << 3));
- }
-
- fadst16_transpose_postproc_msa(tmp, trans);
-
- /* row transform */
- for (i = 0; i < 2; ++i) {
- fadst16_rows_step1_msa(trans + (i << 7), const_arr, ptmpbuf);
- fadst16_rows_step2_msa(ptmpbuf, const_arr, tmp + (i << 7));
- }
-
- fadst16_transpose_msa(tmp, output);
- break;
- default:
- assert(0);
- break;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/enums.h"
-#include "vp10/encoder/mips/msa/fdct_msa.h"
-
-void vp10_fwht4x4_msa(const int16_t *input, int16_t *output,
- int32_t src_stride) {
- v8i16 in0, in1, in2, in3, in4;
-
- LD_SH4(input, src_stride, in0, in1, in2, in3);
-
- in0 += in1;
- in3 -= in2;
- in4 = (in0 - in3) >> 1;
- SUB2(in4, in1, in4, in2, in1, in2);
- in0 -= in2;
- in3 += in1;
-
- TRANSPOSE4x4_SH_SH(in0, in2, in3, in1, in0, in2, in3, in1);
-
- in0 += in2;
- in1 -= in3;
- in4 = (in0 - in1) >> 1;
- SUB2(in4, in2, in4, in3, in2, in3);
- in0 -= in3;
- in1 += in2;
-
- SLLI_4V(in0, in1, in2, in3, 2);
-
- TRANSPOSE4x4_SH_SH(in0, in3, in1, in2, in0, in3, in1, in2);
-
- ST4x2_UB(in0, output, 4);
- ST4x2_UB(in3, output + 4, 4);
- ST4x2_UB(in1, output + 8, 4);
- ST4x2_UB(in2, output + 12, 4);
-}
-
-void vp10_fht4x4_msa(const int16_t *input, int16_t *output, int32_t stride,
- int32_t tx_type) {
- v8i16 in0, in1, in2, in3;
-
- LD_SH4(input, stride, in0, in1, in2, in3);
-
- /* fdct4 pre-process */
- {
- v8i16 temp, mask;
- v16i8 zero = { 0 };
- v16i8 one = __msa_ldi_b(1);
-
- mask = (v8i16)__msa_sldi_b(zero, one, 15);
- SLLI_4V(in0, in1, in2, in3, 4);
- temp = __msa_ceqi_h(in0, 0);
- temp = (v8i16)__msa_xori_b((v16u8)temp, 255);
- temp = mask & temp;
- in0 += temp;
- }
-
- switch (tx_type) {
- case DCT_DCT:
- VP9_FDCT4(in0, in1, in2, in3, in0, in1, in2, in3);
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_FDCT4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- case ADST_DCT:
- VP9_FADST4(in0, in1, in2, in3, in0, in1, in2, in3);
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_FDCT4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- case DCT_ADST:
- VP9_FDCT4(in0, in1, in2, in3, in0, in1, in2, in3);
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_FADST4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- case ADST_ADST:
- VP9_FADST4(in0, in1, in2, in3, in0, in1, in2, in3);
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- VP9_FADST4(in0, in1, in2, in3, in0, in1, in2, in3);
- break;
- default:
- assert(0);
- break;
- }
-
- TRANSPOSE4x4_SH_SH(in0, in1, in2, in3, in0, in1, in2, in3);
- ADD4(in0, 1, in1, 1, in2, 1, in3, 1, in0, in1, in2, in3);
- SRA_4V(in0, in1, in2, in3, 2);
- PCKEV_D2_SH(in1, in0, in3, in2, in0, in2);
- ST_SH2(in0, in2, output, 8);
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-
-#include "vp10/common/enums.h"
-#include "vp10/encoder/mips/msa/fdct_msa.h"
-
-void vp10_fht8x8_msa(const int16_t *input, int16_t *output, int32_t stride,
- int32_t tx_type) {
- v8i16 in0, in1, in2, in3, in4, in5, in6, in7;
-
- LD_SH8(input, stride, in0, in1, in2, in3, in4, in5, in6, in7);
- SLLI_4V(in0, in1, in2, in3, 2);
- SLLI_4V(in4, in5, in6, in7, 2);
-
- switch (tx_type) {
- case DCT_DCT:
- VP9_FDCT8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_FDCT8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- case ADST_DCT:
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_FDCT8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- case DCT_ADST:
- VP9_FDCT8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- case ADST_ADST:
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- break;
- default:
- assert(0);
- break;
- }
-
- TRANSPOSE8x8_SH_SH(in0, in1, in2, in3, in4, in5, in6, in7,
- in0, in1, in2, in3, in4, in5, in6, in7);
- SRLI_AVE_S_4V_H(in0, in1, in2, in3, in4, in5, in6, in7);
- ST_SH8(in0, in1, in2, in3, in4, in5, in6, in7, output, 8);
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP9_ENCODER_MIPS_MSA_VP9_FDCT_MSA_H_
-#define VP9_ENCODER_MIPS_MSA_VP9_FDCT_MSA_H_
-
-#include "vpx_dsp/mips/fwd_txfm_msa.h"
-#include "vpx_dsp/mips/txfm_macros_msa.h"
-#include "vpx_ports/mem.h"
-
-#define VP9_ADST8(in0, in1, in2, in3, in4, in5, in6, in7, \
- out0, out1, out2, out3, out4, out5, out6, out7) { \
- v8i16 cnst0_m, cnst1_m, cnst2_m, cnst3_m, cnst4_m; \
- v8i16 vec0_m, vec1_m, vec2_m, vec3_m, s0_m, s1_m; \
- v8i16 coeff0_m = { cospi_2_64, cospi_6_64, cospi_10_64, cospi_14_64, \
- cospi_18_64, cospi_22_64, cospi_26_64, cospi_30_64 }; \
- v8i16 coeff1_m = { cospi_8_64, -cospi_8_64, cospi_16_64, -cospi_16_64, \
- cospi_24_64, -cospi_24_64, 0, 0 }; \
- \
- SPLATI_H2_SH(coeff0_m, 0, 7, cnst0_m, cnst1_m); \
- cnst2_m = -cnst0_m; \
- ILVEV_H2_SH(cnst0_m, cnst1_m, cnst1_m, cnst2_m, cnst0_m, cnst1_m); \
- SPLATI_H2_SH(coeff0_m, 4, 3, cnst2_m, cnst3_m); \
- cnst4_m = -cnst2_m; \
- ILVEV_H2_SH(cnst2_m, cnst3_m, cnst3_m, cnst4_m, cnst2_m, cnst3_m); \
- \
- ILVRL_H2_SH(in0, in7, vec1_m, vec0_m); \
- ILVRL_H2_SH(in4, in3, vec3_m, vec2_m); \
- DOT_ADD_SUB_SRARI_PCK(vec0_m, vec1_m, vec2_m, vec3_m, cnst0_m, \
- cnst1_m, cnst2_m, cnst3_m, in7, in0, \
- in4, in3); \
- \
- SPLATI_H2_SH(coeff0_m, 2, 5, cnst0_m, cnst1_m); \
- cnst2_m = -cnst0_m; \
- ILVEV_H2_SH(cnst0_m, cnst1_m, cnst1_m, cnst2_m, cnst0_m, cnst1_m); \
- SPLATI_H2_SH(coeff0_m, 6, 1, cnst2_m, cnst3_m); \
- cnst4_m = -cnst2_m; \
- ILVEV_H2_SH(cnst2_m, cnst3_m, cnst3_m, cnst4_m, cnst2_m, cnst3_m); \
- \
- ILVRL_H2_SH(in2, in5, vec1_m, vec0_m); \
- ILVRL_H2_SH(in6, in1, vec3_m, vec2_m); \
- \
- DOT_ADD_SUB_SRARI_PCK(vec0_m, vec1_m, vec2_m, vec3_m, cnst0_m, \
- cnst1_m, cnst2_m, cnst3_m, in5, in2, \
- in6, in1); \
- BUTTERFLY_4(in7, in0, in2, in5, s1_m, s0_m, in2, in5); \
- out7 = -s0_m; \
- out0 = s1_m; \
- \
- SPLATI_H4_SH(coeff1_m, 0, 4, 1, 5, cnst0_m, cnst1_m, cnst2_m, cnst3_m); \
- \
- ILVEV_H2_SH(cnst3_m, cnst0_m, cnst1_m, cnst2_m, cnst3_m, cnst2_m); \
- cnst0_m = __msa_ilvev_h(cnst1_m, cnst0_m); \
- cnst1_m = cnst0_m; \
- \
- ILVRL_H2_SH(in4, in3, vec1_m, vec0_m); \
- ILVRL_H2_SH(in6, in1, vec3_m, vec2_m); \
- DOT_ADD_SUB_SRARI_PCK(vec0_m, vec1_m, vec2_m, vec3_m, cnst0_m, \
- cnst2_m, cnst3_m, cnst1_m, out1, out6, \
- s0_m, s1_m); \
- \
- SPLATI_H2_SH(coeff1_m, 2, 3, cnst0_m, cnst1_m); \
- cnst1_m = __msa_ilvev_h(cnst1_m, cnst0_m); \
- \
- ILVRL_H2_SH(in2, in5, vec1_m, vec0_m); \
- ILVRL_H2_SH(s0_m, s1_m, vec3_m, vec2_m); \
- out3 = DOT_SHIFT_RIGHT_PCK_H(vec0_m, vec1_m, cnst0_m); \
- out4 = DOT_SHIFT_RIGHT_PCK_H(vec0_m, vec1_m, cnst1_m); \
- out2 = DOT_SHIFT_RIGHT_PCK_H(vec2_m, vec3_m, cnst0_m); \
- out5 = DOT_SHIFT_RIGHT_PCK_H(vec2_m, vec3_m, cnst1_m); \
- \
- out1 = -out1; \
- out3 = -out3; \
- out5 = -out5; \
-}
-
-#define VP9_FADST4(in0, in1, in2, in3, out0, out1, out2, out3) { \
- v4i32 s0_m, s1_m, s2_m, s3_m, constant_m; \
- v4i32 in0_r_m, in1_r_m, in2_r_m, in3_r_m; \
- \
- UNPCK_R_SH_SW(in0, in0_r_m); \
- UNPCK_R_SH_SW(in1, in1_r_m); \
- UNPCK_R_SH_SW(in2, in2_r_m); \
- UNPCK_R_SH_SW(in3, in3_r_m); \
- \
- constant_m = __msa_fill_w(sinpi_4_9); \
- MUL2(in0_r_m, constant_m, in3_r_m, constant_m, s1_m, s0_m); \
- \
- constant_m = __msa_fill_w(sinpi_1_9); \
- s0_m += in0_r_m * constant_m; \
- s1_m -= in1_r_m * constant_m; \
- \
- constant_m = __msa_fill_w(sinpi_2_9); \
- s0_m += in1_r_m * constant_m; \
- s1_m += in3_r_m * constant_m; \
- \
- s2_m = in0_r_m + in1_r_m - in3_r_m; \
- \
- constant_m = __msa_fill_w(sinpi_3_9); \
- MUL2(in2_r_m, constant_m, s2_m, constant_m, s3_m, in1_r_m); \
- \
- in0_r_m = s0_m + s3_m; \
- s2_m = s1_m - s3_m; \
- s3_m = s1_m - s0_m + s3_m; \
- \
- SRARI_W4_SW(in0_r_m, in1_r_m, s2_m, s3_m, DCT_CONST_BITS); \
- PCKEV_H4_SH(in0_r_m, in0_r_m, in1_r_m, in1_r_m, s2_m, s2_m, \
- s3_m, s3_m, out0, out1, out2, out3); \
-}
-#endif /* VP9_ENCODER_MIPS_MSA_VP9_FDCT_MSA_H_ */
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "./vp10_rtcd.h"
-#include "vpx_dsp/mips/macros_msa.h"
-
-static void temporal_filter_apply_8size_msa(uint8_t *frm1_ptr,
- uint32_t stride,
- uint8_t *frm2_ptr,
- int32_t filt_sth,
- int32_t filt_wgt,
- uint32_t *acc,
- uint16_t *cnt) {
- uint32_t row;
- uint64_t f0, f1, f2, f3;
- v16i8 frm2, frm1 = { 0 };
- v16i8 frm4, frm3 = { 0 };
- v16u8 frm_r, frm_l;
- v8i16 frm2_r, frm2_l;
- v8i16 diff0, diff1, mod0_h, mod1_h;
- v4i32 cnst3, cnst16, filt_wt, strength;
- v4i32 mod0_w, mod1_w, mod2_w, mod3_w;
- v4i32 diff0_r, diff0_l, diff1_r, diff1_l;
- v4i32 frm2_rr, frm2_rl, frm2_lr, frm2_ll;
- v4i32 acc0, acc1, acc2, acc3;
- v8i16 cnt0, cnt1;
-
- filt_wt = __msa_fill_w(filt_wgt);
- strength = __msa_fill_w(filt_sth);
- cnst3 = __msa_ldi_w(3);
- cnst16 = __msa_ldi_w(16);
-
- for (row = 2; row--;) {
- LD4(frm1_ptr, stride, f0, f1, f2, f3);
- frm1_ptr += (4 * stride);
-
- LD_SB2(frm2_ptr, 16, frm2, frm4);
- frm2_ptr += 32;
-
- LD_SW2(acc, 4, acc0, acc1);
- LD_SW2(acc + 8, 4, acc2, acc3);
- LD_SH2(cnt, 8, cnt0, cnt1);
-
- INSERT_D2_SB(f0, f1, frm1);
- INSERT_D2_SB(f2, f3, frm3);
- ILVRL_B2_UB(frm1, frm2, frm_r, frm_l);
- HSUB_UB2_SH(frm_r, frm_l, diff0, diff1);
- UNPCK_SH_SW(diff0, diff0_r, diff0_l);
- UNPCK_SH_SW(diff1, diff1_r, diff1_l);
- MUL4(diff0_r, diff0_r, diff0_l, diff0_l, diff1_r, diff1_r, diff1_l,
- diff1_l, mod0_w, mod1_w, mod2_w, mod3_w);
- MUL4(mod0_w, cnst3, mod1_w, cnst3, mod2_w, cnst3, mod3_w, cnst3,
- mod0_w, mod1_w, mod2_w, mod3_w);
- SRAR_W4_SW(mod0_w, mod1_w, mod2_w, mod3_w, strength);
-
- diff0_r = (mod0_w < cnst16);
- diff0_l = (mod1_w < cnst16);
- diff1_r = (mod2_w < cnst16);
- diff1_l = (mod3_w < cnst16);
-
- SUB4(cnst16, mod0_w, cnst16, mod1_w, cnst16, mod2_w, cnst16, mod3_w,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- mod0_w = diff0_r & mod0_w;
- mod1_w = diff0_l & mod1_w;
- mod2_w = diff1_r & mod2_w;
- mod3_w = diff1_l & mod3_w;
-
- MUL4(mod0_w, filt_wt, mod1_w, filt_wt, mod2_w, filt_wt, mod3_w, filt_wt,
- mod0_w, mod1_w, mod2_w, mod3_w);
- PCKEV_H2_SH(mod1_w, mod0_w, mod3_w, mod2_w, mod0_h, mod1_h);
- ADD2(mod0_h, cnt0, mod1_h, cnt1, mod0_h, mod1_h);
- ST_SH2(mod0_h, mod1_h, cnt, 8);
- cnt += 16;
-
- UNPCK_UB_SH(frm2, frm2_r, frm2_l);
- UNPCK_SH_SW(frm2_r, frm2_rr, frm2_rl);
- UNPCK_SH_SW(frm2_l, frm2_lr, frm2_ll);
- MUL4(mod0_w, frm2_rr, mod1_w, frm2_rl, mod2_w, frm2_lr, mod3_w, frm2_ll,
- mod0_w, mod1_w, mod2_w, mod3_w);
- ADD4(mod0_w, acc0, mod1_w, acc1, mod2_w, acc2, mod3_w, acc3,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- ST_SW2(mod0_w, mod1_w, acc, 4);
- acc += 8;
- ST_SW2(mod2_w, mod3_w, acc, 4);
- acc += 8;
-
- LD_SW2(acc, 4, acc0, acc1);
- LD_SW2(acc + 8, 4, acc2, acc3);
- LD_SH2(cnt, 8, cnt0, cnt1);
-
- ILVRL_B2_UB(frm3, frm4, frm_r, frm_l);
- HSUB_UB2_SH(frm_r, frm_l, diff0, diff1);
- UNPCK_SH_SW(diff0, diff0_r, diff0_l);
- UNPCK_SH_SW(diff1, diff1_r, diff1_l);
- MUL4(diff0_r, diff0_r, diff0_l, diff0_l, diff1_r, diff1_r, diff1_l,
- diff1_l, mod0_w, mod1_w, mod2_w, mod3_w);
- MUL4(mod0_w, cnst3, mod1_w, cnst3, mod2_w, cnst3, mod3_w, cnst3,
- mod0_w, mod1_w, mod2_w, mod3_w);
- SRAR_W4_SW(mod0_w, mod1_w, mod2_w, mod3_w, strength);
-
- diff0_r = (mod0_w < cnst16);
- diff0_l = (mod1_w < cnst16);
- diff1_r = (mod2_w < cnst16);
- diff1_l = (mod3_w < cnst16);
-
- SUB4(cnst16, mod0_w, cnst16, mod1_w, cnst16, mod2_w, cnst16, mod3_w,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- mod0_w = diff0_r & mod0_w;
- mod1_w = diff0_l & mod1_w;
- mod2_w = diff1_r & mod2_w;
- mod3_w = diff1_l & mod3_w;
-
- MUL4(mod0_w, filt_wt, mod1_w, filt_wt, mod2_w, filt_wt, mod3_w, filt_wt,
- mod0_w, mod1_w, mod2_w, mod3_w);
- PCKEV_H2_SH(mod1_w, mod0_w, mod3_w, mod2_w, mod0_h, mod1_h);
- ADD2(mod0_h, cnt0, mod1_h, cnt1, mod0_h, mod1_h);
- ST_SH2(mod0_h, mod1_h, cnt, 8);
- cnt += 16;
- UNPCK_UB_SH(frm4, frm2_r, frm2_l);
- UNPCK_SH_SW(frm2_r, frm2_rr, frm2_rl);
- UNPCK_SH_SW(frm2_l, frm2_lr, frm2_ll);
- MUL4(mod0_w, frm2_rr, mod1_w, frm2_rl, mod2_w, frm2_lr, mod3_w, frm2_ll,
- mod0_w, mod1_w, mod2_w, mod3_w);
- ADD4(mod0_w, acc0, mod1_w, acc1, mod2_w, acc2, mod3_w, acc3,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- ST_SW2(mod0_w, mod1_w, acc, 4);
- acc += 8;
- ST_SW2(mod2_w, mod3_w, acc, 4);
- acc += 8;
- }
-}
-
-static void temporal_filter_apply_16size_msa(uint8_t *frm1_ptr,
- uint32_t stride,
- uint8_t *frm2_ptr,
- int32_t filt_sth,
- int32_t filt_wgt,
- uint32_t *acc,
- uint16_t *cnt) {
- uint32_t row;
- v16i8 frm1, frm2, frm3, frm4;
- v16u8 frm_r, frm_l;
- v16i8 zero = { 0 };
- v8u16 frm2_r, frm2_l;
- v8i16 diff0, diff1, mod0_h, mod1_h;
- v4i32 cnst3, cnst16, filt_wt, strength;
- v4i32 mod0_w, mod1_w, mod2_w, mod3_w;
- v4i32 diff0_r, diff0_l, diff1_r, diff1_l;
- v4i32 frm2_rr, frm2_rl, frm2_lr, frm2_ll;
- v4i32 acc0, acc1, acc2, acc3;
- v8i16 cnt0, cnt1;
-
- filt_wt = __msa_fill_w(filt_wgt);
- strength = __msa_fill_w(filt_sth);
- cnst3 = __msa_ldi_w(3);
- cnst16 = __msa_ldi_w(16);
-
- for (row = 8; row--;) {
- LD_SB2(frm1_ptr, stride, frm1, frm3);
- frm1_ptr += stride;
-
- LD_SB2(frm2_ptr, 16, frm2, frm4);
- frm2_ptr += 16;
-
- LD_SW2(acc, 4, acc0, acc1);
- LD_SW2(acc, 4, acc2, acc3);
- LD_SH2(cnt, 8, cnt0, cnt1);
-
- ILVRL_B2_UB(frm1, frm2, frm_r, frm_l);
- HSUB_UB2_SH(frm_r, frm_l, diff0, diff1);
- UNPCK_SH_SW(diff0, diff0_r, diff0_l);
- UNPCK_SH_SW(diff1, diff1_r, diff1_l);
- MUL4(diff0_r, diff0_r, diff0_l, diff0_l, diff1_r, diff1_r, diff1_l, diff1_l,
- mod0_w, mod1_w, mod2_w, mod3_w);
- MUL4(mod0_w, cnst3, mod1_w, cnst3, mod2_w, cnst3, mod3_w, cnst3,
- mod0_w, mod1_w, mod2_w, mod3_w);
- SRAR_W4_SW(mod0_w, mod1_w, mod2_w, mod3_w, strength);
-
- diff0_r = (mod0_w < cnst16);
- diff0_l = (mod1_w < cnst16);
- diff1_r = (mod2_w < cnst16);
- diff1_l = (mod3_w < cnst16);
-
- SUB4(cnst16, mod0_w, cnst16, mod1_w, cnst16, mod2_w, cnst16, mod3_w,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- mod0_w = diff0_r & mod0_w;
- mod1_w = diff0_l & mod1_w;
- mod2_w = diff1_r & mod2_w;
- mod3_w = diff1_l & mod3_w;
-
- MUL4(mod0_w, filt_wt, mod1_w, filt_wt, mod2_w, filt_wt, mod3_w, filt_wt,
- mod0_w, mod1_w, mod2_w, mod3_w);
- PCKEV_H2_SH(mod1_w, mod0_w, mod3_w, mod2_w, mod0_h, mod1_h);
- ADD2(mod0_h, cnt0, mod1_h, cnt1, mod0_h, mod1_h);
- ST_SH2(mod0_h, mod1_h, cnt, 8);
- cnt += 16;
-
- ILVRL_B2_UH(zero, frm2, frm2_r, frm2_l);
- UNPCK_SH_SW(frm2_r, frm2_rr, frm2_rl);
- UNPCK_SH_SW(frm2_l, frm2_lr, frm2_ll);
- MUL4(mod0_w, frm2_rr, mod1_w, frm2_rl, mod2_w, frm2_lr, mod3_w, frm2_ll,
- mod0_w, mod1_w, mod2_w, mod3_w);
- ADD4(mod0_w, acc0, mod1_w, acc1, mod2_w, acc2, mod3_w, acc3,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- ST_SW2(mod0_w, mod1_w, acc, 4);
- acc += 8;
- ST_SW2(mod2_w, mod3_w, acc, 4);
- acc += 8;
-
- LD_SW2(acc, 4, acc0, acc1);
- LD_SW2(acc + 8, 4, acc2, acc3);
- LD_SH2(cnt, 8, cnt0, cnt1);
-
- ILVRL_B2_UB(frm3, frm4, frm_r, frm_l);
- HSUB_UB2_SH(frm_r, frm_l, diff0, diff1);
- UNPCK_SH_SW(diff0, diff0_r, diff0_l);
- UNPCK_SH_SW(diff1, diff1_r, diff1_l);
- MUL4(diff0_r, diff0_r, diff0_l, diff0_l, diff1_r, diff1_r, diff1_l, diff1_l,
- mod0_w, mod1_w, mod2_w, mod3_w);
- MUL4(mod0_w, cnst3, mod1_w, cnst3, mod2_w, cnst3, mod3_w, cnst3,
- mod0_w, mod1_w, mod2_w, mod3_w);
- SRAR_W4_SW(mod0_w, mod1_w, mod2_w, mod3_w, strength);
-
- diff0_r = (mod0_w < cnst16);
- diff0_l = (mod1_w < cnst16);
- diff1_r = (mod2_w < cnst16);
- diff1_l = (mod3_w < cnst16);
-
- SUB4(cnst16, mod0_w, cnst16, mod1_w, cnst16, mod2_w, cnst16, mod3_w,
- mod0_w, mod1_w, mod2_w, mod3_w);
-
- mod0_w = diff0_r & mod0_w;
- mod1_w = diff0_l & mod1_w;
- mod2_w = diff1_r & mod2_w;
- mod3_w = diff1_l & mod3_w;
-
- MUL4(mod0_w, filt_wt, mod1_w, filt_wt, mod2_w, filt_wt, mod3_w, filt_wt,
- mod0_w, mod1_w, mod2_w, mod3_w);
- PCKEV_H2_SH(mod1_w, mod0_w, mod3_w, mod2_w, mod0_h, mod1_h);
- ADD2(mod0_h, cnt0, mod1_h, cnt1, mod0_h, mod1_h);
- ST_SH2(mod0_h, mod1_h, cnt, 8);
- cnt += 16;
-
- ILVRL_B2_UH(zero, frm4, frm2_r, frm2_l);
- UNPCK_SH_SW(frm2_r, frm2_rr, frm2_rl);
- UNPCK_SH_SW(frm2_l, frm2_lr, frm2_ll);
- MUL4(mod0_w, frm2_rr, mod1_w, frm2_rl, mod2_w, frm2_lr, mod3_w, frm2_ll,
- mod0_w, mod1_w, mod2_w, mod3_w);
- ADD4(mod0_w, acc0, mod1_w, acc1, mod2_w, acc2, mod3_w, acc3,
- mod0_w, mod1_w, mod2_w, mod3_w);
- ST_SW2(mod0_w, mod1_w, acc, 4);
- acc += 8;
- ST_SW2(mod2_w, mod3_w, acc, 4);
- acc += 8;
-
- frm1_ptr += stride;
- frm2_ptr += 16;
- }
-}
-
-void vp10_temporal_filter_apply_msa(uint8_t *frame1_ptr, uint32_t stride,
- uint8_t *frame2_ptr, uint32_t blk_w,
- uint32_t blk_h, int32_t strength,
- int32_t filt_wgt, uint32_t *accu,
- uint16_t *cnt) {
- if (8 == (blk_w * blk_h)) {
- temporal_filter_apply_8size_msa(frame1_ptr, stride, frame2_ptr,
- strength, filt_wgt, accu, cnt);
- } else if (16 == (blk_w * blk_h)) {
- temporal_filter_apply_16size_msa(frame1_ptr, stride, frame2_ptr,
- strength, filt_wgt, accu, cnt);
- } else {
- vp10_temporal_filter_apply_c(frame1_ptr, stride, frame2_ptr, blk_w, blk_h,
- strength, filt_wgt, accu, cnt);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-#include "vp10/encoder/palette.h"
-
-static double calc_dist(const double *p1, const double *p2, int dim) {
- double dist = 0;
- int i = 0;
-
- for (i = 0; i < dim; ++i) {
- dist = dist + (p1[i] - round(p2[i])) * (p1[i] - round(p2[i]));
- }
- return dist;
-}
-
-void vp10_calc_indices(const double *data, const double *centroids,
- uint8_t *indices, int n, int k, int dim) {
- int i, j;
- double min_dist, this_dist;
-
- for (i = 0; i < n; ++i) {
- min_dist = calc_dist(data + i * dim, centroids, dim);
- indices[i] = 0;
- for (j = 1; j < k; ++j) {
- this_dist = calc_dist(data + i * dim, centroids + j * dim, dim);
- if (this_dist < min_dist) {
- min_dist = this_dist;
- indices[i] = j;
- }
- }
- }
-}
-
-// Generate a random number in the range [0, 32768).
-static unsigned int lcg_rand16(unsigned int *state) {
- *state = *state * 1103515245 + 12345;
- return *state / 65536 % 32768;
-}
-
-static void calc_centroids(const double *data, double *centroids,
- const uint8_t *indices, int n, int k, int dim) {
- int i, j, index;
- int count[PALETTE_MAX_SIZE];
- unsigned int rand_state = data[0];
-
- assert(n <= 32768);
-
- memset(count, 0, sizeof(count[0]) * k);
- memset(centroids, 0, sizeof(centroids[0]) * k * dim);
-
- for (i = 0; i < n; ++i) {
- index = indices[i];
- assert(index < k);
- ++count[index];
- for (j = 0; j < dim; ++j) {
- centroids[index * dim + j] += data[i * dim + j];
- }
- }
-
- for (i = 0; i < k; ++i) {
- if (count[i] == 0) {
- memcpy(centroids + i * dim, data + (lcg_rand16(&rand_state) % n) * dim,
- sizeof(centroids[0]) * dim);
- } else {
- const double norm = 1.0 / count[i];
- for (j = 0; j < dim; ++j)
- centroids[i * dim + j] *= norm;
- }
- }
-}
-
-static double calc_total_dist(const double *data, const double *centroids,
- const uint8_t *indices, int n, int k, int dim) {
- double dist = 0;
- int i;
- (void) k;
-
- for (i = 0; i < n; ++i)
- dist += calc_dist(data + i * dim, centroids + indices[i] * dim, dim);
-
- return dist;
-}
-
-int vp10_k_means(const double *data, double *centroids, uint8_t *indices,
- uint8_t *pre_indices, int n, int k, int dim, int max_itr) {
- int i = 0;
- double pre_dist, this_dist;
- double pre_centroids[PALETTE_MAX_SIZE];
-
- vp10_calc_indices(data, centroids, indices, n, k, dim);
- pre_dist = calc_total_dist(data, centroids, indices, n, k, dim);
- memcpy(pre_centroids, centroids, sizeof(pre_centroids[0]) * k * dim);
- memcpy(pre_indices, indices, sizeof(pre_indices[0]) * n);
- while (i < max_itr) {
- calc_centroids(data, centroids, indices, n, k, dim);
- vp10_calc_indices(data, centroids, indices, n, k, dim);
- this_dist = calc_total_dist(data, centroids, indices, n, k, dim);
-
- if (this_dist > pre_dist) {
- memcpy(centroids, pre_centroids, sizeof(pre_centroids[0]) * k * dim);
- memcpy(indices, pre_indices, sizeof(pre_indices[0]) * n);
- break;
- }
- if (!memcmp(centroids, pre_centroids, sizeof(pre_centroids[0]) * k * dim))
- break;
-
- memcpy(pre_centroids, centroids, sizeof(pre_centroids[0]) * k * dim);
- memcpy(pre_indices, indices, sizeof(pre_indices[0]) * n);
- pre_dist = this_dist;
- ++i;
- }
-
- return i;
-}
-
-void vp10_insertion_sort(double *data, int n) {
- int i, j, k;
- double val;
-
- if (n <= 1)
- return;
-
- for (i = 1; i < n; ++i) {
- val = data[i];
- j = 0;
- while (val > data[j] && j < i)
- ++j;
-
- if (j == i)
- continue;
-
- for (k = i; k > j; --k)
- data[k] = data[k - 1];
- data[j] = val;
- }
-}
-
-int vp10_count_colors(const uint8_t *src, int stride, int rows, int cols) {
- int n = 0, r, c, i, val_count[256];
- uint8_t val;
- memset(val_count, 0, sizeof(val_count));
-
- for (r = 0; r < rows; ++r) {
- for (c = 0; c < cols; ++c) {
- val = src[r * stride + c];
- ++val_count[val];
- }
- }
-
- for (i = 0; i < 256; ++i) {
- if (val_count[i]) {
- ++n;
- }
- }
-
- return n;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-int vp10_count_colors_highbd(const uint8_t *src8, int stride, int rows,
- int cols, int bit_depth) {
- int n = 0, r, c, i;
- uint16_t val;
- uint16_t *src = CONVERT_TO_SHORTPTR(src8);
- int val_count[1 << 12];
-
- assert(bit_depth <= 12);
- memset(val_count, 0, (1 << 12) * sizeof(val_count[0]));
- for (r = 0; r < rows; ++r) {
- for (c = 0; c < cols; ++c) {
- val = src[r * stride + c];
- ++val_count[val];
- }
- }
-
- for (i = 0; i < (1 << bit_depth); ++i) {
- if (val_count[i]) {
- ++n;
- }
- }
-
- return n;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_PALETTE_H_
-#define VP10_ENCODER_PALETTE_H_
-
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_insertion_sort(double *data, int n);
-void vp10_calc_indices(const double *data, const double *centroids,
- uint8_t *indices, int n, int k, int dim);
-int vp10_k_means(const double *data, double *centroids, uint8_t *indices,
- uint8_t *pre_indices, int n, int k, int dim, int max_itr);
-int vp10_count_colors(const uint8_t *src, int stride, int rows, int cols);
-#if CONFIG_VP9_HIGHBITDEPTH
-int vp10_count_colors_highbd(const uint8_t *src8, int stride, int rows,
- int cols, int bit_depth);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif /* VP10_ENCODER_PALETTE_H_ */
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <limits.h>
-
-#include "./vpx_scale_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/loopfilter.h"
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/quant_common.h"
-
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/picklpf.h"
-#include "vp10/encoder/quantize.h"
-
-static int get_max_filter_level(const VP10_COMP *cpi) {
- if (cpi->oxcf.pass == 2) {
- return cpi->twopass.section_intra_rating > 8 ? MAX_LOOP_FILTER * 3 / 4
- : MAX_LOOP_FILTER;
- } else {
- return MAX_LOOP_FILTER;
- }
-}
-
-
-static int64_t try_filter_frame(const YV12_BUFFER_CONFIG *sd,
- VP10_COMP *const cpi,
- int filt_level, int partial_frame) {
- VP10_COMMON *const cm = &cpi->common;
- int64_t filt_err;
-
- if (cpi->num_workers > 1)
- vp10_loop_filter_frame_mt(cm->frame_to_show, cm, cpi->td.mb.e_mbd.plane,
- filt_level, 1, partial_frame,
- cpi->workers, cpi->num_workers, &cpi->lf_row_sync);
- else
- vp10_loop_filter_frame(cm->frame_to_show, cm, &cpi->td.mb.e_mbd, filt_level,
- 1, partial_frame);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cm->use_highbitdepth) {
- filt_err = vp10_highbd_get_y_sse(sd, cm->frame_to_show);
- } else {
- filt_err = vp10_get_y_sse(sd, cm->frame_to_show);
- }
-#else
- filt_err = vp10_get_y_sse(sd, cm->frame_to_show);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Re-instate the unfiltered frame
- vpx_yv12_copy_y(&cpi->last_frame_uf, cm->frame_to_show);
-
- return filt_err;
-}
-
-static int search_filter_level(const YV12_BUFFER_CONFIG *sd, VP10_COMP *cpi,
- int partial_frame) {
- const VP10_COMMON *const cm = &cpi->common;
- const struct loopfilter *const lf = &cm->lf;
- const int min_filter_level = 0;
- const int max_filter_level = get_max_filter_level(cpi);
- int filt_direction = 0;
- int64_t best_err;
- int filt_best;
-
- // Start the search at the previous frame filter level unless it is now out of
- // range.
- int filt_mid = clamp(lf->filter_level, min_filter_level, max_filter_level);
- int filter_step = filt_mid < 16 ? 4 : filt_mid / 4;
- // Sum squared error at each filter level
- int64_t ss_err[MAX_LOOP_FILTER + 1];
-
- // Set each entry to -1
- memset(ss_err, 0xFF, sizeof(ss_err));
-
- // Make a copy of the unfiltered / processed recon buffer
- vpx_yv12_copy_y(cm->frame_to_show, &cpi->last_frame_uf);
-
- best_err = try_filter_frame(sd, cpi, filt_mid, partial_frame);
- filt_best = filt_mid;
- ss_err[filt_mid] = best_err;
-
- while (filter_step > 0) {
- const int filt_high = VPXMIN(filt_mid + filter_step, max_filter_level);
- const int filt_low = VPXMAX(filt_mid - filter_step, min_filter_level);
-
- // Bias against raising loop filter in favor of lowering it.
- int64_t bias = (best_err >> (15 - (filt_mid / 8))) * filter_step;
-
- if ((cpi->oxcf.pass == 2) && (cpi->twopass.section_intra_rating < 20))
- bias = (bias * cpi->twopass.section_intra_rating) / 20;
-
- // yx, bias less for large block size
- if (cm->tx_mode != ONLY_4X4)
- bias >>= 1;
-
- if (filt_direction <= 0 && filt_low != filt_mid) {
- // Get Low filter error score
- if (ss_err[filt_low] < 0) {
- ss_err[filt_low] = try_filter_frame(sd, cpi, filt_low, partial_frame);
- }
- // If value is close to the best so far then bias towards a lower loop
- // filter value.
- if ((ss_err[filt_low] - bias) < best_err) {
- // Was it actually better than the previous best?
- if (ss_err[filt_low] < best_err)
- best_err = ss_err[filt_low];
-
- filt_best = filt_low;
- }
- }
-
- // Now look at filt_high
- if (filt_direction >= 0 && filt_high != filt_mid) {
- if (ss_err[filt_high] < 0) {
- ss_err[filt_high] = try_filter_frame(sd, cpi, filt_high, partial_frame);
- }
- // Was it better than the previous best?
- if (ss_err[filt_high] < (best_err - bias)) {
- best_err = ss_err[filt_high];
- filt_best = filt_high;
- }
- }
-
- // Half the step distance if the best filter value was the same as last time
- if (filt_best == filt_mid) {
- filter_step /= 2;
- filt_direction = 0;
- } else {
- filt_direction = (filt_best < filt_mid) ? -1 : 1;
- filt_mid = filt_best;
- }
- }
-
- return filt_best;
-}
-
-void vp10_pick_filter_level(const YV12_BUFFER_CONFIG *sd, VP10_COMP *cpi,
- LPF_PICK_METHOD method) {
- VP10_COMMON *const cm = &cpi->common;
- struct loopfilter *const lf = &cm->lf;
-
- lf->sharpness_level = cm->frame_type == KEY_FRAME ? 0
- : cpi->oxcf.sharpness;
-
- if (method == LPF_PICK_MINIMAL_LPF && lf->filter_level) {
- lf->filter_level = 0;
- } else if (method >= LPF_PICK_FROM_Q) {
- const int min_filter_level = 0;
- const int max_filter_level = get_max_filter_level(cpi);
- const int q = vp10_ac_quant(cm->base_qindex, 0, cm->bit_depth);
- // These values were determined by linear fitting the result of the
- // searched level, filt_guess = q * 0.316206 + 3.87252
-#if CONFIG_VP9_HIGHBITDEPTH
- int filt_guess;
- switch (cm->bit_depth) {
- case VPX_BITS_8:
- filt_guess = ROUND_POWER_OF_TWO(q * 20723 + 1015158, 18);
- break;
- case VPX_BITS_10:
- filt_guess = ROUND_POWER_OF_TWO(q * 20723 + 4060632, 20);
- break;
- case VPX_BITS_12:
- filt_guess = ROUND_POWER_OF_TWO(q * 20723 + 16242526, 22);
- break;
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 "
- "or VPX_BITS_12");
- return;
- }
-#else
- int filt_guess = ROUND_POWER_OF_TWO(q * 20723 + 1015158, 18);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- if (cm->frame_type == KEY_FRAME)
- filt_guess -= 4;
- lf->filter_level = clamp(filt_guess, min_filter_level, max_filter_level);
- } else {
- lf->filter_level = search_filter_level(sd, cpi,
- method == LPF_PICK_FROM_SUBIMAGE);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_PICKLPF_H_
-#define VP10_ENCODER_PICKLPF_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#include "vp10/encoder/encoder.h"
-
-struct yv12_buffer_config;
-struct VP10_COMP;
-
-void vp10_pick_filter_level(const struct yv12_buffer_config *sd,
- struct VP10_COMP *cpi, LPF_PICK_METHOD method);
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_PICKLPF_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-#include "./vpx_dsp_rtcd.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-
-#include "vp10/common/quant_common.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/rd.h"
-
-void vp10_quantize_fp_c(const tran_low_t *coeff_ptr, intptr_t n_coeffs,
- int skip_block,
- const int16_t *zbin_ptr, const int16_t *round_ptr,
- const int16_t *quant_ptr, const int16_t *quant_shift_ptr,
- tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr,
- const int16_t *dequant_ptr,
- uint16_t *eob_ptr,
- const int16_t *scan, const int16_t *iscan) {
- int i, eob = -1;
- // TODO(jingning) Decide the need of these arguments after the
- // quantization process is completed.
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)iscan;
-
- memset(qcoeff_ptr, 0, n_coeffs * sizeof(*qcoeff_ptr));
- memset(dqcoeff_ptr, 0, n_coeffs * sizeof(*dqcoeff_ptr));
-
- if (!skip_block) {
- // Quantization pass: All coefficients with index >= zero_flag are
- // skippable. Note: zero_flag can be zero.
- for (i = 0; i < n_coeffs; i++) {
- const int rc = scan[i];
- const int coeff = coeff_ptr[rc];
- const int coeff_sign = (coeff >> 31);
- const int abs_coeff = (coeff ^ coeff_sign) - coeff_sign;
-
- int tmp = clamp(abs_coeff + round_ptr[rc != 0], INT16_MIN, INT16_MAX);
- tmp = (tmp * quant_ptr[rc != 0]) >> 16;
-
- qcoeff_ptr[rc] = (tmp ^ coeff_sign) - coeff_sign;
- dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0];
-
- if (tmp)
- eob = i;
- }
- }
- *eob_ptr = eob + 1;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_quantize_fp_c(const tran_low_t *coeff_ptr,
- intptr_t count,
- int skip_block,
- const int16_t *zbin_ptr,
- const int16_t *round_ptr,
- const int16_t *quant_ptr,
- const int16_t *quant_shift_ptr,
- tran_low_t *qcoeff_ptr,
- tran_low_t *dqcoeff_ptr,
- const int16_t *dequant_ptr,
- uint16_t *eob_ptr,
- const int16_t *scan,
- const int16_t *iscan) {
- int i;
- int eob = -1;
- // TODO(jingning) Decide the need of these arguments after the
- // quantization process is completed.
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)iscan;
-
- memset(qcoeff_ptr, 0, count * sizeof(*qcoeff_ptr));
- memset(dqcoeff_ptr, 0, count * sizeof(*dqcoeff_ptr));
-
- if (!skip_block) {
- // Quantization pass: All coefficients with index >= zero_flag are
- // skippable. Note: zero_flag can be zero.
- for (i = 0; i < count; i++) {
- const int rc = scan[i];
- const int coeff = coeff_ptr[rc];
- const int coeff_sign = (coeff >> 31);
- const int abs_coeff = (coeff ^ coeff_sign) - coeff_sign;
- const int64_t tmp = abs_coeff + round_ptr[rc != 0];
- const uint32_t abs_qcoeff = (uint32_t)((tmp * quant_ptr[rc != 0]) >> 16);
- qcoeff_ptr[rc] = (tran_low_t)((abs_qcoeff ^ coeff_sign) - coeff_sign);
- dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0];
- if (abs_qcoeff)
- eob = i;
- }
- }
- *eob_ptr = eob + 1;
-}
-#endif
-
-// TODO(jingning) Refactor this file and combine functions with similar
-// operations.
-void vp10_quantize_fp_32x32_c(const tran_low_t *coeff_ptr, intptr_t n_coeffs,
- int skip_block,
- const int16_t *zbin_ptr, const int16_t *round_ptr,
- const int16_t *quant_ptr,
- const int16_t *quant_shift_ptr,
- tran_low_t *qcoeff_ptr, tran_low_t *dqcoeff_ptr,
- const int16_t *dequant_ptr,
- uint16_t *eob_ptr,
- const int16_t *scan, const int16_t *iscan) {
- int i, eob = -1;
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)iscan;
-
- memset(qcoeff_ptr, 0, n_coeffs * sizeof(*qcoeff_ptr));
- memset(dqcoeff_ptr, 0, n_coeffs * sizeof(*dqcoeff_ptr));
-
- if (!skip_block) {
- for (i = 0; i < n_coeffs; i++) {
- const int rc = scan[i];
- const int coeff = coeff_ptr[rc];
- const int coeff_sign = (coeff >> 31);
- int tmp = 0;
- int abs_coeff = (coeff ^ coeff_sign) - coeff_sign;
-
- if (abs_coeff >= (dequant_ptr[rc != 0] >> 2)) {
- abs_coeff += ROUND_POWER_OF_TWO(round_ptr[rc != 0], 1);
- abs_coeff = clamp(abs_coeff, INT16_MIN, INT16_MAX);
- tmp = (abs_coeff * quant_ptr[rc != 0]) >> 15;
- qcoeff_ptr[rc] = (tmp ^ coeff_sign) - coeff_sign;
- dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0] / 2;
- }
-
- if (tmp)
- eob = i;
- }
- }
- *eob_ptr = eob + 1;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_quantize_fp_32x32_c(const tran_low_t *coeff_ptr,
- intptr_t n_coeffs, int skip_block,
- const int16_t *zbin_ptr,
- const int16_t *round_ptr,
- const int16_t *quant_ptr,
- const int16_t *quant_shift_ptr,
- tran_low_t *qcoeff_ptr,
- tran_low_t *dqcoeff_ptr,
- const int16_t *dequant_ptr,
- uint16_t *eob_ptr,
- const int16_t *scan, const int16_t *iscan) {
- int i, eob = -1;
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)iscan;
-
- memset(qcoeff_ptr, 0, n_coeffs * sizeof(*qcoeff_ptr));
- memset(dqcoeff_ptr, 0, n_coeffs * sizeof(*dqcoeff_ptr));
-
- if (!skip_block) {
- for (i = 0; i < n_coeffs; i++) {
- uint32_t abs_qcoeff = 0;
- const int rc = scan[i];
- const int coeff = coeff_ptr[rc];
- const int coeff_sign = (coeff >> 31);
- const int abs_coeff = (coeff ^ coeff_sign) - coeff_sign;
-
- if (abs_coeff >= (dequant_ptr[rc != 0] >> 2)) {
- const int64_t tmp = abs_coeff
- + ROUND_POWER_OF_TWO(round_ptr[rc != 0], 1);
- abs_qcoeff = (uint32_t) ((tmp * quant_ptr[rc != 0]) >> 15);
- qcoeff_ptr[rc] = (tran_low_t)((abs_qcoeff ^ coeff_sign) - coeff_sign);
- dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0] / 2;
- }
-
- if (abs_qcoeff)
- eob = i;
- }
- }
- *eob_ptr = eob + 1;
-}
-#endif
-
-void vp10_regular_quantize_b_4x4(MACROBLOCK *x, int plane, int block,
- const int16_t *scan, const int16_t *iscan) {
- MACROBLOCKD *const xd = &x->e_mbd;
- struct macroblock_plane *p = &x->plane[plane];
- struct macroblockd_plane *pd = &xd->plane[plane];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- vpx_highbd_quantize_b(BLOCK_OFFSET(p->coeff, block),
- 16, x->skip_block,
- p->zbin, p->round, p->quant, p->quant_shift,
- BLOCK_OFFSET(p->qcoeff, block),
- BLOCK_OFFSET(pd->dqcoeff, block),
- pd->dequant, &p->eobs[block],
- scan, iscan);
- return;
- }
-#endif
- vpx_quantize_b(BLOCK_OFFSET(p->coeff, block),
- 16, x->skip_block,
- p->zbin, p->round, p->quant, p->quant_shift,
- BLOCK_OFFSET(p->qcoeff, block),
- BLOCK_OFFSET(pd->dqcoeff, block),
- pd->dequant, &p->eobs[block], scan, iscan);
-}
-
-static void invert_quant(int16_t *quant, int16_t *shift, int d) {
- unsigned t;
- int l;
- t = d;
- for (l = 0; t > 1; l++)
- t >>= 1;
- t = 1 + (1 << (16 + l)) / d;
- *quant = (int16_t)(t - (1 << 16));
- *shift = 1 << (16 - l);
-}
-
-static int get_qzbin_factor(int q, vpx_bit_depth_t bit_depth) {
- const int quant = vp10_dc_quant(q, 0, bit_depth);
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (bit_depth) {
- case VPX_BITS_8:
- return q == 0 ? 64 : (quant < 148 ? 84 : 80);
- case VPX_BITS_10:
- return q == 0 ? 64 : (quant < 592 ? 84 : 80);
- case VPX_BITS_12:
- return q == 0 ? 64 : (quant < 2368 ? 84 : 80);
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1;
- }
-#else
- (void) bit_depth;
- return q == 0 ? 64 : (quant < 148 ? 84 : 80);
-#endif
-}
-
-void vp10_init_quantizer(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- QUANTS *const quants = &cpi->quants;
- int i, q, quant;
-
- for (q = 0; q < QINDEX_RANGE; q++) {
- const int qzbin_factor = get_qzbin_factor(q, cm->bit_depth);
- const int qrounding_factor = q == 0 ? 64 : 48;
-
- for (i = 0; i < 2; ++i) {
- int qrounding_factor_fp = i == 0 ? 48 : 42;
- if (q == 0)
- qrounding_factor_fp = 64;
-
- // y
- quant = i == 0 ? vp10_dc_quant(q, cm->y_dc_delta_q, cm->bit_depth)
- : vp10_ac_quant(q, 0, cm->bit_depth);
- invert_quant(&quants->y_quant[q][i], &quants->y_quant_shift[q][i], quant);
- quants->y_quant_fp[q][i] = (1 << 16) / quant;
- quants->y_round_fp[q][i] = (qrounding_factor_fp * quant) >> 7;
- quants->y_zbin[q][i] = ROUND_POWER_OF_TWO(qzbin_factor * quant, 7);
- quants->y_round[q][i] = (qrounding_factor * quant) >> 7;
- cpi->y_dequant[q][i] = quant;
-
- // uv
- quant = i == 0 ? vp10_dc_quant(q, cm->uv_dc_delta_q, cm->bit_depth)
- : vp10_ac_quant(q, cm->uv_ac_delta_q, cm->bit_depth);
- invert_quant(&quants->uv_quant[q][i],
- &quants->uv_quant_shift[q][i], quant);
- quants->uv_quant_fp[q][i] = (1 << 16) / quant;
- quants->uv_round_fp[q][i] = (qrounding_factor_fp * quant) >> 7;
- quants->uv_zbin[q][i] = ROUND_POWER_OF_TWO(qzbin_factor * quant, 7);
- quants->uv_round[q][i] = (qrounding_factor * quant) >> 7;
- cpi->uv_dequant[q][i] = quant;
- }
-
- for (i = 2; i < 8; i++) {
- quants->y_quant[q][i] = quants->y_quant[q][1];
- quants->y_quant_fp[q][i] = quants->y_quant_fp[q][1];
- quants->y_round_fp[q][i] = quants->y_round_fp[q][1];
- quants->y_quant_shift[q][i] = quants->y_quant_shift[q][1];
- quants->y_zbin[q][i] = quants->y_zbin[q][1];
- quants->y_round[q][i] = quants->y_round[q][1];
- cpi->y_dequant[q][i] = cpi->y_dequant[q][1];
-
- quants->uv_quant[q][i] = quants->uv_quant[q][1];
- quants->uv_quant_fp[q][i] = quants->uv_quant_fp[q][1];
- quants->uv_round_fp[q][i] = quants->uv_round_fp[q][1];
- quants->uv_quant_shift[q][i] = quants->uv_quant_shift[q][1];
- quants->uv_zbin[q][i] = quants->uv_zbin[q][1];
- quants->uv_round[q][i] = quants->uv_round[q][1];
- cpi->uv_dequant[q][i] = cpi->uv_dequant[q][1];
- }
- }
-}
-
-void vp10_init_plane_quantizers(VP10_COMP *cpi, MACROBLOCK *x) {
- const VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- QUANTS *const quants = &cpi->quants;
- const int segment_id = xd->mi[0]->mbmi.segment_id;
- const int qindex = vp10_get_qindex(&cm->seg, segment_id, cm->base_qindex);
- const int rdmult = vp10_compute_rd_mult(cpi, qindex + cm->y_dc_delta_q);
- int i;
-
- // Y
- x->plane[0].quant = quants->y_quant[qindex];
- x->plane[0].quant_fp = quants->y_quant_fp[qindex];
- x->plane[0].round_fp = quants->y_round_fp[qindex];
- x->plane[0].quant_shift = quants->y_quant_shift[qindex];
- x->plane[0].zbin = quants->y_zbin[qindex];
- x->plane[0].round = quants->y_round[qindex];
- xd->plane[0].dequant = cpi->y_dequant[qindex];
-
- x->plane[0].quant_thred[0] = x->plane[0].zbin[0] * x->plane[0].zbin[0];
- x->plane[0].quant_thred[1] = x->plane[0].zbin[1] * x->plane[0].zbin[1];
-
- // UV
- for (i = 1; i < 3; i++) {
- x->plane[i].quant = quants->uv_quant[qindex];
- x->plane[i].quant_fp = quants->uv_quant_fp[qindex];
- x->plane[i].round_fp = quants->uv_round_fp[qindex];
- x->plane[i].quant_shift = quants->uv_quant_shift[qindex];
- x->plane[i].zbin = quants->uv_zbin[qindex];
- x->plane[i].round = quants->uv_round[qindex];
- xd->plane[i].dequant = cpi->uv_dequant[qindex];
-
- x->plane[i].quant_thred[0] = x->plane[i].zbin[0] * x->plane[i].zbin[0];
- x->plane[i].quant_thred[1] = x->plane[i].zbin[1] * x->plane[i].zbin[1];
- }
-
- x->skip_block = segfeature_active(&cm->seg, segment_id, SEG_LVL_SKIP);
- x->q_index = qindex;
-
- x->errorperbit = rdmult >> 6;
- x->errorperbit += (x->errorperbit == 0);
-
- vp10_initialize_me_consts(cpi, x, x->q_index);
-}
-
-void vp10_frame_init_quantizer(VP10_COMP *cpi) {
- vp10_init_plane_quantizers(cpi, &cpi->td.mb);
-}
-
-void vp10_set_quantizer(VP10_COMMON *cm, int q) {
- // quantizer has to be reinitialized with vp10_init_quantizer() if any
- // delta_q changes.
- cm->base_qindex = q;
- cm->y_dc_delta_q = 0;
- cm->uv_dc_delta_q = 0;
- cm->uv_ac_delta_q = 0;
-}
-
-// Table that converts 0-63 Q-range values passed in outside to the Qindex
-// range used internally.
-static const int quantizer_to_qindex[] = {
- 0, 4, 8, 12, 16, 20, 24, 28,
- 32, 36, 40, 44, 48, 52, 56, 60,
- 64, 68, 72, 76, 80, 84, 88, 92,
- 96, 100, 104, 108, 112, 116, 120, 124,
- 128, 132, 136, 140, 144, 148, 152, 156,
- 160, 164, 168, 172, 176, 180, 184, 188,
- 192, 196, 200, 204, 208, 212, 216, 220,
- 224, 228, 232, 236, 240, 244, 249, 255,
-};
-
-int vp10_quantizer_to_qindex(int quantizer) {
- return quantizer_to_qindex[quantizer];
-}
-
-int vp10_qindex_to_quantizer(int qindex) {
- int quantizer;
-
- for (quantizer = 0; quantizer < 64; ++quantizer)
- if (quantizer_to_qindex[quantizer] >= qindex)
- return quantizer;
-
- return 63;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_QUANTIZE_H_
-#define VP10_ENCODER_QUANTIZE_H_
-
-#include "./vpx_config.h"
-#include "vp10/encoder/block.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-typedef struct {
- DECLARE_ALIGNED(16, int16_t, y_quant[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, y_quant_shift[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, y_zbin[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, y_round[QINDEX_RANGE][8]);
-
- // TODO(jingning): in progress of re-working the quantization. will decide
- // if we want to deprecate the current use of y_quant.
- DECLARE_ALIGNED(16, int16_t, y_quant_fp[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, uv_quant_fp[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, y_round_fp[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, uv_round_fp[QINDEX_RANGE][8]);
-
- DECLARE_ALIGNED(16, int16_t, uv_quant[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, uv_quant_shift[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, uv_zbin[QINDEX_RANGE][8]);
- DECLARE_ALIGNED(16, int16_t, uv_round[QINDEX_RANGE][8]);
-} QUANTS;
-
-void vp10_regular_quantize_b_4x4(MACROBLOCK *x, int plane, int block,
- const int16_t *scan, const int16_t *iscan);
-
-struct VP10_COMP;
-struct VP10Common;
-
-void vp10_frame_init_quantizer(struct VP10_COMP *cpi);
-
-void vp10_init_plane_quantizers(struct VP10_COMP *cpi, MACROBLOCK *x);
-
-void vp10_init_quantizer(struct VP10_COMP *cpi);
-
-void vp10_set_quantizer(struct VP10Common *cm, int q);
-
-int vp10_quantizer_to_qindex(int quantizer);
-
-int vp10_qindex_to_quantizer(int qindex);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_QUANTIZE_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <limits.h>
-#include <math.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/encoder/aq_cyclicrefresh.h"
-#include "vp10/common/common.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/ratectrl.h"
-
-// Max rate target for 1080P and below encodes under normal circumstances
-// (1920 * 1080 / (16 * 16)) * MAX_MB_RATE bits per MB
-#define MAX_MB_RATE 250
-#define MAXRATE_1080P 2025000
-
-#define DEFAULT_KF_BOOST 2000
-#define DEFAULT_GF_BOOST 2000
-
-#define LIMIT_QRANGE_FOR_ALTREF_AND_KEY 1
-
-#define MIN_BPB_FACTOR 0.005
-#define MAX_BPB_FACTOR 50
-
-#define FRAME_OVERHEAD_BITS 200
-
-#if CONFIG_VP9_HIGHBITDEPTH
-#define ASSIGN_MINQ_TABLE(bit_depth, name) \
- do { \
- switch (bit_depth) { \
- case VPX_BITS_8: \
- name = name##_8; \
- break; \
- case VPX_BITS_10: \
- name = name##_10; \
- break; \
- case VPX_BITS_12: \
- name = name##_12; \
- break; \
- default: \
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10" \
- " or VPX_BITS_12"); \
- name = NULL; \
- } \
- } while (0)
-#else
-#define ASSIGN_MINQ_TABLE(bit_depth, name) \
- do { \
- (void) bit_depth; \
- name = name##_8; \
- } while (0)
-#endif
-
-// Tables relating active max Q to active min Q
-static int kf_low_motion_minq_8[QINDEX_RANGE];
-static int kf_high_motion_minq_8[QINDEX_RANGE];
-static int arfgf_low_motion_minq_8[QINDEX_RANGE];
-static int arfgf_high_motion_minq_8[QINDEX_RANGE];
-static int inter_minq_8[QINDEX_RANGE];
-static int rtc_minq_8[QINDEX_RANGE];
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static int kf_low_motion_minq_10[QINDEX_RANGE];
-static int kf_high_motion_minq_10[QINDEX_RANGE];
-static int arfgf_low_motion_minq_10[QINDEX_RANGE];
-static int arfgf_high_motion_minq_10[QINDEX_RANGE];
-static int inter_minq_10[QINDEX_RANGE];
-static int rtc_minq_10[QINDEX_RANGE];
-static int kf_low_motion_minq_12[QINDEX_RANGE];
-static int kf_high_motion_minq_12[QINDEX_RANGE];
-static int arfgf_low_motion_minq_12[QINDEX_RANGE];
-static int arfgf_high_motion_minq_12[QINDEX_RANGE];
-static int inter_minq_12[QINDEX_RANGE];
-static int rtc_minq_12[QINDEX_RANGE];
-#endif
-
-static int gf_high = 2000;
-static int gf_low = 400;
-static int kf_high = 5000;
-static int kf_low = 400;
-
-// Functions to compute the active minq lookup table entries based on a
-// formulaic approach to facilitate easier adjustment of the Q tables.
-// The formulae were derived from computing a 3rd order polynomial best
-// fit to the original data (after plotting real maxq vs minq (not q index))
-static int get_minq_index(double maxq, double x3, double x2, double x1,
- vpx_bit_depth_t bit_depth) {
- int i;
- const double minqtarget = VPXMIN(((x3 * maxq + x2) * maxq + x1) * maxq, maxq);
-
- // Special case handling to deal with the step from q2.0
- // down to lossless mode represented by q 1.0.
- if (minqtarget <= 2.0)
- return 0;
-
- for (i = 0; i < QINDEX_RANGE; i++) {
- if (minqtarget <= vp10_convert_qindex_to_q(i, bit_depth))
- return i;
- }
-
- return QINDEX_RANGE - 1;
-}
-
-static void init_minq_luts(int *kf_low_m, int *kf_high_m,
- int *arfgf_low, int *arfgf_high,
- int *inter, int *rtc, vpx_bit_depth_t bit_depth) {
- int i;
- for (i = 0; i < QINDEX_RANGE; i++) {
- const double maxq = vp10_convert_qindex_to_q(i, bit_depth);
- kf_low_m[i] = get_minq_index(maxq, 0.000001, -0.0004, 0.150, bit_depth);
- kf_high_m[i] = get_minq_index(maxq, 0.0000021, -0.00125, 0.55, bit_depth);
- arfgf_low[i] = get_minq_index(maxq, 0.0000015, -0.0009, 0.30, bit_depth);
- arfgf_high[i] = get_minq_index(maxq, 0.0000021, -0.00125, 0.55, bit_depth);
- inter[i] = get_minq_index(maxq, 0.00000271, -0.00113, 0.90, bit_depth);
- rtc[i] = get_minq_index(maxq, 0.00000271, -0.00113, 0.70, bit_depth);
- }
-}
-
-void vp10_rc_init_minq_luts(void) {
- init_minq_luts(kf_low_motion_minq_8, kf_high_motion_minq_8,
- arfgf_low_motion_minq_8, arfgf_high_motion_minq_8,
- inter_minq_8, rtc_minq_8, VPX_BITS_8);
-#if CONFIG_VP9_HIGHBITDEPTH
- init_minq_luts(kf_low_motion_minq_10, kf_high_motion_minq_10,
- arfgf_low_motion_minq_10, arfgf_high_motion_minq_10,
- inter_minq_10, rtc_minq_10, VPX_BITS_10);
- init_minq_luts(kf_low_motion_minq_12, kf_high_motion_minq_12,
- arfgf_low_motion_minq_12, arfgf_high_motion_minq_12,
- inter_minq_12, rtc_minq_12, VPX_BITS_12);
-#endif
-}
-
-// These functions use formulaic calculations to make playing with the
-// quantizer tables easier. If necessary they can be replaced by lookup
-// tables if and when things settle down in the experimental bitstream
-double vp10_convert_qindex_to_q(int qindex, vpx_bit_depth_t bit_depth) {
- // Convert the index to a real Q value (scaled down to match old Q values)
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (bit_depth) {
- case VPX_BITS_8:
- return vp10_ac_quant(qindex, 0, bit_depth) / 4.0;
- case VPX_BITS_10:
- return vp10_ac_quant(qindex, 0, bit_depth) / 16.0;
- case VPX_BITS_12:
- return vp10_ac_quant(qindex, 0, bit_depth) / 64.0;
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1.0;
- }
-#else
- return vp10_ac_quant(qindex, 0, bit_depth) / 4.0;
-#endif
-}
-
-int vp10_rc_bits_per_mb(FRAME_TYPE frame_type, int qindex,
- double correction_factor,
- vpx_bit_depth_t bit_depth) {
- const double q = vp10_convert_qindex_to_q(qindex, bit_depth);
- int enumerator = frame_type == KEY_FRAME ? 2700000 : 1800000;
-
- assert(correction_factor <= MAX_BPB_FACTOR &&
- correction_factor >= MIN_BPB_FACTOR);
-
- // q based adjustment to baseline enumerator
- enumerator += (int)(enumerator * q) >> 12;
- return (int)(enumerator * correction_factor / q);
-}
-
-int vp10_estimate_bits_at_q(FRAME_TYPE frame_type, int q, int mbs,
- double correction_factor,
- vpx_bit_depth_t bit_depth) {
- const int bpm = (int)(vp10_rc_bits_per_mb(frame_type, q, correction_factor,
- bit_depth));
- return VPXMAX(FRAME_OVERHEAD_BITS,
- (int)((uint64_t)bpm * mbs) >> BPER_MB_NORMBITS);
-}
-
-int vp10_rc_clamp_pframe_target_size(const VP10_COMP *const cpi, int target) {
- const RATE_CONTROL *rc = &cpi->rc;
- const VP10EncoderConfig *oxcf = &cpi->oxcf;
- const int min_frame_target = VPXMAX(rc->min_frame_bandwidth,
- rc->avg_frame_bandwidth >> 5);
- if (target < min_frame_target)
- target = min_frame_target;
- if (cpi->refresh_golden_frame && rc->is_src_frame_alt_ref) {
- // If there is an active ARF at this location use the minimum
- // bits on this frame even if it is a constructed arf.
- // The active maximum quantizer insures that an appropriate
- // number of bits will be spent if needed for constructed ARFs.
- target = min_frame_target;
- }
- // Clip the frame target to the maximum allowed value.
- if (target > rc->max_frame_bandwidth)
- target = rc->max_frame_bandwidth;
- if (oxcf->rc_max_inter_bitrate_pct) {
- const int max_rate = rc->avg_frame_bandwidth *
- oxcf->rc_max_inter_bitrate_pct / 100;
- target = VPXMIN(target, max_rate);
- }
- return target;
-}
-
-int vp10_rc_clamp_iframe_target_size(const VP10_COMP *const cpi, int target) {
- const RATE_CONTROL *rc = &cpi->rc;
- const VP10EncoderConfig *oxcf = &cpi->oxcf;
- if (oxcf->rc_max_intra_bitrate_pct) {
- const int max_rate = rc->avg_frame_bandwidth *
- oxcf->rc_max_intra_bitrate_pct / 100;
- target = VPXMIN(target, max_rate);
- }
- if (target > rc->max_frame_bandwidth)
- target = rc->max_frame_bandwidth;
- return target;
-}
-
-// Update the buffer level: leaky bucket model.
-static void update_buffer_level(VP10_COMP *cpi, int encoded_frame_size) {
- const VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
-
- // Non-viewable frames are a special case and are treated as pure overhead.
- if (!cm->show_frame) {
- rc->bits_off_target -= encoded_frame_size;
- } else {
- rc->bits_off_target += rc->avg_frame_bandwidth - encoded_frame_size;
- }
-
- // Clip the buffer level to the maximum specified buffer size.
- rc->bits_off_target = VPXMIN(rc->bits_off_target, rc->maximum_buffer_size);
- rc->buffer_level = rc->bits_off_target;
-}
-
-int vp10_rc_get_default_min_gf_interval(
- int width, int height, double framerate) {
- // Assume we do not need any constraint lower than 4K 20 fps
- static const double factor_safe = 3840 * 2160 * 20.0;
- const double factor = width * height * framerate;
- const int default_interval =
- clamp((int)(framerate * 0.125), MIN_GF_INTERVAL, MAX_GF_INTERVAL);
-
- if (factor <= factor_safe)
- return default_interval;
- else
- return VPXMAX(default_interval,
- (int)(MIN_GF_INTERVAL * factor / factor_safe + 0.5));
- // Note this logic makes:
- // 4K24: 5
- // 4K30: 6
- // 4K60: 12
-}
-
-int vp10_rc_get_default_max_gf_interval(double framerate, int min_gf_interval) {
- int interval = VPXMIN(MAX_GF_INTERVAL, (int)(framerate * 0.75));
- interval += (interval & 0x01); // Round to even value
- return VPXMAX(interval, min_gf_interval);
-}
-
-void vp10_rc_init(const VP10EncoderConfig *oxcf, int pass, RATE_CONTROL *rc) {
- int i;
-
- if (pass == 0 && oxcf->rc_mode == VPX_CBR) {
- rc->avg_frame_qindex[KEY_FRAME] = oxcf->worst_allowed_q;
- rc->avg_frame_qindex[INTER_FRAME] = oxcf->worst_allowed_q;
- } else {
- rc->avg_frame_qindex[KEY_FRAME] = (oxcf->worst_allowed_q +
- oxcf->best_allowed_q) / 2;
- rc->avg_frame_qindex[INTER_FRAME] = (oxcf->worst_allowed_q +
- oxcf->best_allowed_q) / 2;
- }
-
- rc->last_q[KEY_FRAME] = oxcf->best_allowed_q;
- rc->last_q[INTER_FRAME] = oxcf->worst_allowed_q;
-
- rc->buffer_level = rc->starting_buffer_level;
- rc->bits_off_target = rc->starting_buffer_level;
-
- rc->rolling_target_bits = rc->avg_frame_bandwidth;
- rc->rolling_actual_bits = rc->avg_frame_bandwidth;
- rc->long_rolling_target_bits = rc->avg_frame_bandwidth;
- rc->long_rolling_actual_bits = rc->avg_frame_bandwidth;
-
- rc->total_actual_bits = 0;
- rc->total_target_bits = 0;
- rc->total_target_vs_actual = 0;
-
- rc->frames_since_key = 8; // Sensible default for first frame.
- rc->this_key_frame_forced = 0;
- rc->next_key_frame_forced = 0;
- rc->source_alt_ref_pending = 0;
- rc->source_alt_ref_active = 0;
-
- rc->frames_till_gf_update_due = 0;
- rc->ni_av_qi = oxcf->worst_allowed_q;
- rc->ni_tot_qi = 0;
- rc->ni_frames = 0;
-
- rc->tot_q = 0.0;
- rc->avg_q = vp10_convert_qindex_to_q(oxcf->worst_allowed_q, oxcf->bit_depth);
-
- for (i = 0; i < RATE_FACTOR_LEVELS; ++i) {
- rc->rate_correction_factors[i] = 1.0;
- }
-
- rc->min_gf_interval = oxcf->min_gf_interval;
- rc->max_gf_interval = oxcf->max_gf_interval;
- if (rc->min_gf_interval == 0)
- rc->min_gf_interval = vp10_rc_get_default_min_gf_interval(
- oxcf->width, oxcf->height, oxcf->init_framerate);
- if (rc->max_gf_interval == 0)
- rc->max_gf_interval = vp10_rc_get_default_max_gf_interval(
- oxcf->init_framerate, rc->min_gf_interval);
- rc->baseline_gf_interval = (rc->min_gf_interval + rc->max_gf_interval) / 2;
-}
-
-int vp10_rc_drop_frame(VP10_COMP *cpi) {
- const VP10EncoderConfig *oxcf = &cpi->oxcf;
- RATE_CONTROL *const rc = &cpi->rc;
-
- if (!oxcf->drop_frames_water_mark) {
- return 0;
- } else {
- if (rc->buffer_level < 0) {
- // Always drop if buffer is below 0.
- return 1;
- } else {
- // If buffer is below drop_mark, for now just drop every other frame
- // (starting with the next frame) until it increases back over drop_mark.
- int drop_mark = (int)(oxcf->drop_frames_water_mark *
- rc->optimal_buffer_level / 100);
- if ((rc->buffer_level > drop_mark) &&
- (rc->decimation_factor > 0)) {
- --rc->decimation_factor;
- } else if (rc->buffer_level <= drop_mark &&
- rc->decimation_factor == 0) {
- rc->decimation_factor = 1;
- }
- if (rc->decimation_factor > 0) {
- if (rc->decimation_count > 0) {
- --rc->decimation_count;
- return 1;
- } else {
- rc->decimation_count = rc->decimation_factor;
- return 0;
- }
- } else {
- rc->decimation_count = 0;
- return 0;
- }
- }
- }
-}
-
-static double get_rate_correction_factor(const VP10_COMP *cpi) {
- const RATE_CONTROL *const rc = &cpi->rc;
- double rcf;
-
- if (cpi->common.frame_type == KEY_FRAME) {
- rcf = rc->rate_correction_factors[KF_STD];
- } else if (cpi->oxcf.pass == 2) {
- RATE_FACTOR_LEVEL rf_lvl =
- cpi->twopass.gf_group.rf_level[cpi->twopass.gf_group.index];
- rcf = rc->rate_correction_factors[rf_lvl];
- } else {
- if ((cpi->refresh_alt_ref_frame || cpi->refresh_golden_frame) &&
- !rc->is_src_frame_alt_ref &&
- (cpi->oxcf.rc_mode != VPX_CBR || cpi->oxcf.gf_cbr_boost_pct > 20))
- rcf = rc->rate_correction_factors[GF_ARF_STD];
- else
- rcf = rc->rate_correction_factors[INTER_NORMAL];
- }
- rcf *= rcf_mult[rc->frame_size_selector];
- return fclamp(rcf, MIN_BPB_FACTOR, MAX_BPB_FACTOR);
-}
-
-static void set_rate_correction_factor(VP10_COMP *cpi, double factor) {
- RATE_CONTROL *const rc = &cpi->rc;
-
- // Normalize RCF to account for the size-dependent scaling factor.
- factor /= rcf_mult[cpi->rc.frame_size_selector];
-
- factor = fclamp(factor, MIN_BPB_FACTOR, MAX_BPB_FACTOR);
-
- if (cpi->common.frame_type == KEY_FRAME) {
- rc->rate_correction_factors[KF_STD] = factor;
- } else if (cpi->oxcf.pass == 2) {
- RATE_FACTOR_LEVEL rf_lvl =
- cpi->twopass.gf_group.rf_level[cpi->twopass.gf_group.index];
- rc->rate_correction_factors[rf_lvl] = factor;
- } else {
- if ((cpi->refresh_alt_ref_frame || cpi->refresh_golden_frame) &&
- !rc->is_src_frame_alt_ref &&
- (cpi->oxcf.rc_mode != VPX_CBR || cpi->oxcf.gf_cbr_boost_pct > 20))
- rc->rate_correction_factors[GF_ARF_STD] = factor;
- else
- rc->rate_correction_factors[INTER_NORMAL] = factor;
- }
-}
-
-void vp10_rc_update_rate_correction_factors(VP10_COMP *cpi) {
- const VP10_COMMON *const cm = &cpi->common;
- int correction_factor = 100;
- double rate_correction_factor = get_rate_correction_factor(cpi);
- double adjustment_limit;
-
- int projected_size_based_on_q = 0;
-
- // Do not update the rate factors for arf overlay frames.
- if (cpi->rc.is_src_frame_alt_ref)
- return;
-
- // Clear down mmx registers to allow floating point in what follows
- vpx_clear_system_state();
-
- // Work out how big we would have expected the frame to be at this Q given
- // the current correction factor.
- // Stay in double to avoid int overflow when values are large
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ && cpi->common.seg.enabled) {
- projected_size_based_on_q =
- vp10_cyclic_refresh_estimate_bits_at_q(cpi, rate_correction_factor);
- } else {
- projected_size_based_on_q = vp10_estimate_bits_at_q(cpi->common.frame_type,
- cm->base_qindex,
- cm->MBs,
- rate_correction_factor,
- cm->bit_depth);
- }
- // Work out a size correction factor.
- if (projected_size_based_on_q > FRAME_OVERHEAD_BITS)
- correction_factor = (int)((100 * (int64_t)cpi->rc.projected_frame_size) /
- projected_size_based_on_q);
-
- // More heavily damped adjustment used if we have been oscillating either side
- // of target.
- adjustment_limit = 0.25 +
- 0.5 * VPXMIN(1, fabs(log10(0.01 * correction_factor)));
-
- cpi->rc.q_2_frame = cpi->rc.q_1_frame;
- cpi->rc.q_1_frame = cm->base_qindex;
- cpi->rc.rc_2_frame = cpi->rc.rc_1_frame;
- if (correction_factor > 110)
- cpi->rc.rc_1_frame = -1;
- else if (correction_factor < 90)
- cpi->rc.rc_1_frame = 1;
- else
- cpi->rc.rc_1_frame = 0;
-
- if (correction_factor > 102) {
- // We are not already at the worst allowable quality
- correction_factor = (int)(100 + ((correction_factor - 100) *
- adjustment_limit));
- rate_correction_factor = (rate_correction_factor * correction_factor) / 100;
- // Keep rate_correction_factor within limits
- if (rate_correction_factor > MAX_BPB_FACTOR)
- rate_correction_factor = MAX_BPB_FACTOR;
- } else if (correction_factor < 99) {
- // We are not already at the best allowable quality
- correction_factor = (int)(100 - ((100 - correction_factor) *
- adjustment_limit));
- rate_correction_factor = (rate_correction_factor * correction_factor) / 100;
-
- // Keep rate_correction_factor within limits
- if (rate_correction_factor < MIN_BPB_FACTOR)
- rate_correction_factor = MIN_BPB_FACTOR;
- }
-
- set_rate_correction_factor(cpi, rate_correction_factor);
-}
-
-
-int vp10_rc_regulate_q(const VP10_COMP *cpi, int target_bits_per_frame,
- int active_best_quality, int active_worst_quality) {
- const VP10_COMMON *const cm = &cpi->common;
- int q = active_worst_quality;
- int last_error = INT_MAX;
- int i, target_bits_per_mb, bits_per_mb_at_this_q;
- const double correction_factor = get_rate_correction_factor(cpi);
-
- // Calculate required scaling factor based on target frame size and size of
- // frame produced using previous Q.
- target_bits_per_mb =
- ((uint64_t)target_bits_per_frame << BPER_MB_NORMBITS) / cm->MBs;
-
- i = active_best_quality;
-
- do {
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ && cm->seg.enabled) {
- bits_per_mb_at_this_q =
- (int)vp10_cyclic_refresh_rc_bits_per_mb(cpi, i, correction_factor);
- } else {
- bits_per_mb_at_this_q = (int)vp10_rc_bits_per_mb(cm->frame_type, i,
- correction_factor,
- cm->bit_depth);
- }
-
- if (bits_per_mb_at_this_q <= target_bits_per_mb) {
- if ((target_bits_per_mb - bits_per_mb_at_this_q) <= last_error)
- q = i;
- else
- q = i - 1;
-
- break;
- } else {
- last_error = bits_per_mb_at_this_q - target_bits_per_mb;
- }
- } while (++i <= active_worst_quality);
-
- // In CBR mode, this makes sure q is between oscillating Qs to prevent
- // resonance.
- if (cpi->oxcf.rc_mode == VPX_CBR &&
- (cpi->rc.rc_1_frame * cpi->rc.rc_2_frame == -1) &&
- cpi->rc.q_1_frame != cpi->rc.q_2_frame) {
- q = clamp(q, VPXMIN(cpi->rc.q_1_frame, cpi->rc.q_2_frame),
- VPXMAX(cpi->rc.q_1_frame, cpi->rc.q_2_frame));
- }
- return q;
-}
-
-static int get_active_quality(int q, int gfu_boost, int low, int high,
- int *low_motion_minq, int *high_motion_minq) {
- if (gfu_boost > high) {
- return low_motion_minq[q];
- } else if (gfu_boost < low) {
- return high_motion_minq[q];
- } else {
- const int gap = high - low;
- const int offset = high - gfu_boost;
- const int qdiff = high_motion_minq[q] - low_motion_minq[q];
- const int adjustment = ((offset * qdiff) + (gap >> 1)) / gap;
- return low_motion_minq[q] + adjustment;
- }
-}
-
-static int get_kf_active_quality(const RATE_CONTROL *const rc, int q,
- vpx_bit_depth_t bit_depth) {
- int *kf_low_motion_minq;
- int *kf_high_motion_minq;
- ASSIGN_MINQ_TABLE(bit_depth, kf_low_motion_minq);
- ASSIGN_MINQ_TABLE(bit_depth, kf_high_motion_minq);
- return get_active_quality(q, rc->kf_boost, kf_low, kf_high,
- kf_low_motion_minq, kf_high_motion_minq);
-}
-
-static int get_gf_active_quality(const RATE_CONTROL *const rc, int q,
- vpx_bit_depth_t bit_depth) {
- int *arfgf_low_motion_minq;
- int *arfgf_high_motion_minq;
- ASSIGN_MINQ_TABLE(bit_depth, arfgf_low_motion_minq);
- ASSIGN_MINQ_TABLE(bit_depth, arfgf_high_motion_minq);
- return get_active_quality(q, rc->gfu_boost, gf_low, gf_high,
- arfgf_low_motion_minq, arfgf_high_motion_minq);
-}
-
-static int calc_active_worst_quality_one_pass_vbr(const VP10_COMP *cpi) {
- const RATE_CONTROL *const rc = &cpi->rc;
- const unsigned int curr_frame = cpi->common.current_video_frame;
- int active_worst_quality;
-
- if (cpi->common.frame_type == KEY_FRAME) {
- active_worst_quality = curr_frame == 0 ? rc->worst_quality
- : rc->last_q[KEY_FRAME] * 2;
- } else {
- if (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) {
- active_worst_quality = curr_frame == 1 ? rc->last_q[KEY_FRAME] * 5 / 4
- : rc->last_q[INTER_FRAME];
- } else {
- active_worst_quality = curr_frame == 1 ? rc->last_q[KEY_FRAME] * 2
- : rc->last_q[INTER_FRAME] * 2;
- }
- }
- return VPXMIN(active_worst_quality, rc->worst_quality);
-}
-
-// Adjust active_worst_quality level based on buffer level.
-static int calc_active_worst_quality_one_pass_cbr(const VP10_COMP *cpi) {
- // Adjust active_worst_quality: If buffer is above the optimal/target level,
- // bring active_worst_quality down depending on fullness of buffer.
- // If buffer is below the optimal level, let the active_worst_quality go from
- // ambient Q (at buffer = optimal level) to worst_quality level
- // (at buffer = critical level).
- const VP10_COMMON *const cm = &cpi->common;
- const RATE_CONTROL *rc = &cpi->rc;
- // Buffer level below which we push active_worst to worst_quality.
- int64_t critical_level = rc->optimal_buffer_level >> 3;
- int64_t buff_lvl_step = 0;
- int adjustment = 0;
- int active_worst_quality;
- int ambient_qp;
- if (cm->frame_type == KEY_FRAME)
- return rc->worst_quality;
- // For ambient_qp we use minimum of avg_frame_qindex[KEY_FRAME/INTER_FRAME]
- // for the first few frames following key frame. These are both initialized
- // to worst_quality and updated with (3/4, 1/4) average in postencode_update.
- // So for first few frames following key, the qp of that key frame is weighted
- // into the active_worst_quality setting.
- ambient_qp = (cm->current_video_frame < 5) ?
- VPXMIN(rc->avg_frame_qindex[INTER_FRAME],
- rc->avg_frame_qindex[KEY_FRAME]) :
- rc->avg_frame_qindex[INTER_FRAME];
- active_worst_quality = VPXMIN(rc->worst_quality, ambient_qp * 5 / 4);
- if (rc->buffer_level > rc->optimal_buffer_level) {
- // Adjust down.
- // Maximum limit for down adjustment, ~30%.
- int max_adjustment_down = active_worst_quality / 3;
- if (max_adjustment_down) {
- buff_lvl_step = ((rc->maximum_buffer_size -
- rc->optimal_buffer_level) / max_adjustment_down);
- if (buff_lvl_step)
- adjustment = (int)((rc->buffer_level - rc->optimal_buffer_level) /
- buff_lvl_step);
- active_worst_quality -= adjustment;
- }
- } else if (rc->buffer_level > critical_level) {
- // Adjust up from ambient Q.
- if (critical_level) {
- buff_lvl_step = (rc->optimal_buffer_level - critical_level);
- if (buff_lvl_step) {
- adjustment = (int)((rc->worst_quality - ambient_qp) *
- (rc->optimal_buffer_level - rc->buffer_level) /
- buff_lvl_step);
- }
- active_worst_quality = ambient_qp + adjustment;
- }
- } else {
- // Set to worst_quality if buffer is below critical level.
- active_worst_quality = rc->worst_quality;
- }
- return active_worst_quality;
-}
-
-static int rc_pick_q_and_bounds_one_pass_cbr(const VP10_COMP *cpi,
- int *bottom_index,
- int *top_index) {
- const VP10_COMMON *const cm = &cpi->common;
- const RATE_CONTROL *const rc = &cpi->rc;
- int active_best_quality;
- int active_worst_quality = calc_active_worst_quality_one_pass_cbr(cpi);
- int q;
- int *rtc_minq;
- ASSIGN_MINQ_TABLE(cm->bit_depth, rtc_minq);
-
- if (frame_is_intra_only(cm)) {
- active_best_quality = rc->best_quality;
- // Handle the special case for key frames forced when we have reached
- // the maximum key frame interval. Here force the Q to a range
- // based on the ambient Q to reduce the risk of popping.
- if (rc->this_key_frame_forced) {
- int qindex = rc->last_boosted_qindex;
- double last_boosted_q = vp10_convert_qindex_to_q(qindex, cm->bit_depth);
- int delta_qindex = vp10_compute_qdelta(rc, last_boosted_q,
- (last_boosted_q * 0.75),
- cm->bit_depth);
- active_best_quality = VPXMAX(qindex + delta_qindex, rc->best_quality);
- } else if (cm->current_video_frame > 0) {
- // not first frame of one pass and kf_boost is set
- double q_adj_factor = 1.0;
- double q_val;
-
- active_best_quality =
- get_kf_active_quality(rc, rc->avg_frame_qindex[KEY_FRAME],
- cm->bit_depth);
-
- // Allow somewhat lower kf minq with small image formats.
- if ((cm->width * cm->height) <= (352 * 288)) {
- q_adj_factor -= 0.25;
- }
-
- // Convert the adjustment factor to a qindex delta
- // on active_best_quality.
- q_val = vp10_convert_qindex_to_q(active_best_quality, cm->bit_depth);
- active_best_quality += vp10_compute_qdelta(rc, q_val,
- q_val * q_adj_factor,
- cm->bit_depth);
- }
- } else if (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) {
- // Use the lower of active_worst_quality and recent
- // average Q as basis for GF/ARF best Q limit unless last frame was
- // a key frame.
- if (rc->frames_since_key > 1 &&
- rc->avg_frame_qindex[INTER_FRAME] < active_worst_quality) {
- q = rc->avg_frame_qindex[INTER_FRAME];
- } else {
- q = active_worst_quality;
- }
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
- } else {
- // Use the lower of active_worst_quality and recent/average Q.
- if (cm->current_video_frame > 1) {
- if (rc->avg_frame_qindex[INTER_FRAME] < active_worst_quality)
- active_best_quality = rtc_minq[rc->avg_frame_qindex[INTER_FRAME]];
- else
- active_best_quality = rtc_minq[active_worst_quality];
- } else {
- if (rc->avg_frame_qindex[KEY_FRAME] < active_worst_quality)
- active_best_quality = rtc_minq[rc->avg_frame_qindex[KEY_FRAME]];
- else
- active_best_quality = rtc_minq[active_worst_quality];
- }
- }
-
- // Clip the active best and worst quality values to limits
- active_best_quality = clamp(active_best_quality,
- rc->best_quality, rc->worst_quality);
- active_worst_quality = clamp(active_worst_quality,
- active_best_quality, rc->worst_quality);
-
- *top_index = active_worst_quality;
- *bottom_index = active_best_quality;
-
-#if LIMIT_QRANGE_FOR_ALTREF_AND_KEY
- // Limit Q range for the adaptive loop.
- if (cm->frame_type == KEY_FRAME &&
- !rc->this_key_frame_forced &&
- !(cm->current_video_frame == 0)) {
- int qdelta = 0;
- vpx_clear_system_state();
- qdelta = vp10_compute_qdelta_by_rate(&cpi->rc, cm->frame_type,
- active_worst_quality, 2.0,
- cm->bit_depth);
- *top_index = active_worst_quality + qdelta;
- *top_index = (*top_index > *bottom_index) ? *top_index : *bottom_index;
- }
-#endif
-
- // Special case code to try and match quality with forced key frames
- if (cm->frame_type == KEY_FRAME && rc->this_key_frame_forced) {
- q = rc->last_boosted_qindex;
- } else {
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- active_best_quality, active_worst_quality);
- if (q > *top_index) {
- // Special case when we are targeting the max allowed rate
- if (rc->this_frame_target >= rc->max_frame_bandwidth)
- *top_index = q;
- else
- q = *top_index;
- }
- }
- assert(*top_index <= rc->worst_quality &&
- *top_index >= rc->best_quality);
- assert(*bottom_index <= rc->worst_quality &&
- *bottom_index >= rc->best_quality);
- assert(q <= rc->worst_quality && q >= rc->best_quality);
- return q;
-}
-
-static int get_active_cq_level(const RATE_CONTROL *rc,
- const VP10EncoderConfig *const oxcf) {
- static const double cq_adjust_threshold = 0.1;
- int active_cq_level = oxcf->cq_level;
- if (oxcf->rc_mode == VPX_CQ &&
- rc->total_target_bits > 0) {
- const double x = (double)rc->total_actual_bits / rc->total_target_bits;
- if (x < cq_adjust_threshold) {
- active_cq_level = (int)(active_cq_level * x / cq_adjust_threshold);
- }
- }
- return active_cq_level;
-}
-
-static int rc_pick_q_and_bounds_one_pass_vbr(const VP10_COMP *cpi,
- int *bottom_index,
- int *top_index) {
- const VP10_COMMON *const cm = &cpi->common;
- const RATE_CONTROL *const rc = &cpi->rc;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- const int cq_level = get_active_cq_level(rc, oxcf);
- int active_best_quality;
- int active_worst_quality = calc_active_worst_quality_one_pass_vbr(cpi);
- int q;
- int *inter_minq;
- ASSIGN_MINQ_TABLE(cm->bit_depth, inter_minq);
-
- if (frame_is_intra_only(cm)) {
-
- // Handle the special case for key frames forced when we have reached
- // the maximum key frame interval. Here force the Q to a range
- // based on the ambient Q to reduce the risk of popping.
- if (rc->this_key_frame_forced) {
- int qindex = rc->last_boosted_qindex;
- double last_boosted_q = vp10_convert_qindex_to_q(qindex, cm->bit_depth);
- int delta_qindex = vp10_compute_qdelta(rc, last_boosted_q,
- last_boosted_q * 0.75,
- cm->bit_depth);
- active_best_quality = VPXMAX(qindex + delta_qindex, rc->best_quality);
- } else {
- // not first frame of one pass and kf_boost is set
- double q_adj_factor = 1.0;
- double q_val;
-
- active_best_quality =
- get_kf_active_quality(rc, rc->avg_frame_qindex[KEY_FRAME],
- cm->bit_depth);
-
- // Allow somewhat lower kf minq with small image formats.
- if ((cm->width * cm->height) <= (352 * 288)) {
- q_adj_factor -= 0.25;
- }
-
- // Convert the adjustment factor to a qindex delta
- // on active_best_quality.
- q_val = vp10_convert_qindex_to_q(active_best_quality, cm->bit_depth);
- active_best_quality += vp10_compute_qdelta(rc, q_val,
- q_val * q_adj_factor,
- cm->bit_depth);
- }
- } else if (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) {
- // Use the lower of active_worst_quality and recent
- // average Q as basis for GF/ARF best Q limit unless last frame was
- // a key frame.
- if (rc->frames_since_key > 1 &&
- rc->avg_frame_qindex[INTER_FRAME] < active_worst_quality) {
- q = rc->avg_frame_qindex[INTER_FRAME];
- } else {
- q = rc->avg_frame_qindex[KEY_FRAME];
- }
- // For constrained quality dont allow Q less than the cq level
- if (oxcf->rc_mode == VPX_CQ) {
- if (q < cq_level)
- q = cq_level;
-
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
-
- // Constrained quality use slightly lower active best.
- active_best_quality = active_best_quality * 15 / 16;
-
- } else if (oxcf->rc_mode == VPX_Q) {
- if (!cpi->refresh_alt_ref_frame) {
- active_best_quality = cq_level;
- } else {
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
- }
- } else {
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
- }
- } else {
- if (oxcf->rc_mode == VPX_Q) {
- active_best_quality = cq_level;
- } else {
- // Use the lower of active_worst_quality and recent/average Q.
- if (cm->current_video_frame > 1)
- active_best_quality = inter_minq[rc->avg_frame_qindex[INTER_FRAME]];
- else
- active_best_quality = inter_minq[rc->avg_frame_qindex[KEY_FRAME]];
- // For the constrained quality mode we don't want
- // q to fall below the cq level.
- if ((oxcf->rc_mode == VPX_CQ) &&
- (active_best_quality < cq_level)) {
- active_best_quality = cq_level;
- }
- }
- }
-
- // Clip the active best and worst quality values to limits
- active_best_quality = clamp(active_best_quality,
- rc->best_quality, rc->worst_quality);
- active_worst_quality = clamp(active_worst_quality,
- active_best_quality, rc->worst_quality);
-
- *top_index = active_worst_quality;
- *bottom_index = active_best_quality;
-
-#if LIMIT_QRANGE_FOR_ALTREF_AND_KEY
- {
- int qdelta = 0;
- vpx_clear_system_state();
-
- // Limit Q range for the adaptive loop.
- if (cm->frame_type == KEY_FRAME &&
- !rc->this_key_frame_forced &&
- !(cm->current_video_frame == 0)) {
- qdelta = vp10_compute_qdelta_by_rate(&cpi->rc, cm->frame_type,
- active_worst_quality, 2.0,
- cm->bit_depth);
- } else if (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) {
- qdelta = vp10_compute_qdelta_by_rate(&cpi->rc, cm->frame_type,
- active_worst_quality, 1.75,
- cm->bit_depth);
- }
- *top_index = active_worst_quality + qdelta;
- *top_index = (*top_index > *bottom_index) ? *top_index : *bottom_index;
- }
-#endif
-
- if (oxcf->rc_mode == VPX_Q) {
- q = active_best_quality;
- // Special case code to try and match quality with forced key frames
- } else if ((cm->frame_type == KEY_FRAME) && rc->this_key_frame_forced) {
- q = rc->last_boosted_qindex;
- } else {
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- active_best_quality, active_worst_quality);
- if (q > *top_index) {
- // Special case when we are targeting the max allowed rate
- if (rc->this_frame_target >= rc->max_frame_bandwidth)
- *top_index = q;
- else
- q = *top_index;
- }
- }
-
- assert(*top_index <= rc->worst_quality &&
- *top_index >= rc->best_quality);
- assert(*bottom_index <= rc->worst_quality &&
- *bottom_index >= rc->best_quality);
- assert(q <= rc->worst_quality && q >= rc->best_quality);
- return q;
-}
-
-int vp10_frame_type_qdelta(const VP10_COMP *cpi, int rf_level, int q) {
- static const double rate_factor_deltas[RATE_FACTOR_LEVELS] = {
- 1.00, // INTER_NORMAL
- 1.00, // INTER_HIGH
- 1.50, // GF_ARF_LOW
- 1.75, // GF_ARF_STD
- 2.00, // KF_STD
- };
- static const FRAME_TYPE frame_type[RATE_FACTOR_LEVELS] =
- {INTER_FRAME, INTER_FRAME, INTER_FRAME, INTER_FRAME, KEY_FRAME};
- const VP10_COMMON *const cm = &cpi->common;
- int qdelta = vp10_compute_qdelta_by_rate(&cpi->rc, frame_type[rf_level],
- q, rate_factor_deltas[rf_level],
- cm->bit_depth);
- return qdelta;
-}
-
-#define STATIC_MOTION_THRESH 95
-static int rc_pick_q_and_bounds_two_pass(const VP10_COMP *cpi,
- int *bottom_index,
- int *top_index) {
- const VP10_COMMON *const cm = &cpi->common;
- const RATE_CONTROL *const rc = &cpi->rc;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- const GF_GROUP *gf_group = &cpi->twopass.gf_group;
- const int cq_level = get_active_cq_level(rc, oxcf);
- int active_best_quality;
- int active_worst_quality = cpi->twopass.active_worst_quality;
- int q;
- int *inter_minq;
- ASSIGN_MINQ_TABLE(cm->bit_depth, inter_minq);
-
- if (frame_is_intra_only(cm)) {
- // Handle the special case for key frames forced when we have reached
- // the maximum key frame interval. Here force the Q to a range
- // based on the ambient Q to reduce the risk of popping.
- if (rc->this_key_frame_forced) {
- double last_boosted_q;
- int delta_qindex;
- int qindex;
-
- if (cpi->twopass.last_kfgroup_zeromotion_pct >= STATIC_MOTION_THRESH) {
- qindex = VPXMIN(rc->last_kf_qindex, rc->last_boosted_qindex);
- active_best_quality = qindex;
- last_boosted_q = vp10_convert_qindex_to_q(qindex, cm->bit_depth);
- delta_qindex = vp10_compute_qdelta(rc, last_boosted_q,
- last_boosted_q * 1.25,
- cm->bit_depth);
- active_worst_quality =
- VPXMIN(qindex + delta_qindex, active_worst_quality);
- } else {
- qindex = rc->last_boosted_qindex;
- last_boosted_q = vp10_convert_qindex_to_q(qindex, cm->bit_depth);
- delta_qindex = vp10_compute_qdelta(rc, last_boosted_q,
- last_boosted_q * 0.75,
- cm->bit_depth);
- active_best_quality = VPXMAX(qindex + delta_qindex, rc->best_quality);
- }
- } else {
- // Not forced keyframe.
- double q_adj_factor = 1.0;
- double q_val;
- // Baseline value derived from cpi->active_worst_quality and kf boost.
- active_best_quality = get_kf_active_quality(rc, active_worst_quality,
- cm->bit_depth);
-
- // Allow somewhat lower kf minq with small image formats.
- if ((cm->width * cm->height) <= (352 * 288)) {
- q_adj_factor -= 0.25;
- }
-
- // Make a further adjustment based on the kf zero motion measure.
- q_adj_factor += 0.05 - (0.001 * (double)cpi->twopass.kf_zeromotion_pct);
-
- // Convert the adjustment factor to a qindex delta
- // on active_best_quality.
- q_val = vp10_convert_qindex_to_q(active_best_quality, cm->bit_depth);
- active_best_quality += vp10_compute_qdelta(rc, q_val,
- q_val * q_adj_factor,
- cm->bit_depth);
- }
- } else if (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) {
- // Use the lower of active_worst_quality and recent
- // average Q as basis for GF/ARF best Q limit unless last frame was
- // a key frame.
- if (rc->frames_since_key > 1 &&
- rc->avg_frame_qindex[INTER_FRAME] < active_worst_quality) {
- q = rc->avg_frame_qindex[INTER_FRAME];
- } else {
- q = active_worst_quality;
- }
- // For constrained quality dont allow Q less than the cq level
- if (oxcf->rc_mode == VPX_CQ) {
- if (q < cq_level)
- q = cq_level;
-
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
-
- // Constrained quality use slightly lower active best.
- active_best_quality = active_best_quality * 15 / 16;
-
- } else if (oxcf->rc_mode == VPX_Q) {
- if (!cpi->refresh_alt_ref_frame) {
- active_best_quality = cq_level;
- } else {
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
-
- // Modify best quality for second level arfs. For mode VPX_Q this
- // becomes the baseline frame q.
- if (gf_group->rf_level[gf_group->index] == GF_ARF_LOW)
- active_best_quality = (active_best_quality + cq_level + 1) / 2;
- }
- } else {
- active_best_quality = get_gf_active_quality(rc, q, cm->bit_depth);
- }
- } else {
- if (oxcf->rc_mode == VPX_Q) {
- active_best_quality = cq_level;
- } else {
- active_best_quality = inter_minq[active_worst_quality];
-
- // For the constrained quality mode we don't want
- // q to fall below the cq level.
- if ((oxcf->rc_mode == VPX_CQ) &&
- (active_best_quality < cq_level)) {
- active_best_quality = cq_level;
- }
- }
- }
-
- // Extension to max or min Q if undershoot or overshoot is outside
- // the permitted range.
- if ((cpi->oxcf.rc_mode != VPX_Q) &&
- (cpi->twopass.gf_zeromotion_pct < VLOW_MOTION_THRESHOLD)) {
- if (frame_is_intra_only(cm) ||
- (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame))) {
- active_best_quality -=
- (cpi->twopass.extend_minq + cpi->twopass.extend_minq_fast);
- active_worst_quality += (cpi->twopass.extend_maxq / 2);
- } else {
- active_best_quality -=
- (cpi->twopass.extend_minq + cpi->twopass.extend_minq_fast) / 2;
- active_worst_quality += cpi->twopass.extend_maxq;
- }
- }
-
-#if LIMIT_QRANGE_FOR_ALTREF_AND_KEY
- vpx_clear_system_state();
- // Static forced key frames Q restrictions dealt with elsewhere.
- if (!(frame_is_intra_only(cm)) ||
- !rc->this_key_frame_forced ||
- (cpi->twopass.last_kfgroup_zeromotion_pct < STATIC_MOTION_THRESH)) {
- int qdelta = vp10_frame_type_qdelta(cpi, gf_group->rf_level[gf_group->index],
- active_worst_quality);
- active_worst_quality = VPXMAX(active_worst_quality + qdelta,
- active_best_quality);
- }
-#endif
-
- // Modify active_best_quality for downscaled normal frames.
- if (rc->frame_size_selector != UNSCALED && !frame_is_kf_gf_arf(cpi)) {
- int qdelta = vp10_compute_qdelta_by_rate(rc, cm->frame_type,
- active_best_quality, 2.0,
- cm->bit_depth);
- active_best_quality =
- VPXMAX(active_best_quality + qdelta, rc->best_quality);
- }
-
- active_best_quality = clamp(active_best_quality,
- rc->best_quality, rc->worst_quality);
- active_worst_quality = clamp(active_worst_quality,
- active_best_quality, rc->worst_quality);
-
- if (oxcf->rc_mode == VPX_Q) {
- q = active_best_quality;
- // Special case code to try and match quality with forced key frames.
- } else if (frame_is_intra_only(cm) && rc->this_key_frame_forced) {
- // If static since last kf use better of last boosted and last kf q.
- if (cpi->twopass.last_kfgroup_zeromotion_pct >= STATIC_MOTION_THRESH) {
- q = VPXMIN(rc->last_kf_qindex, rc->last_boosted_qindex);
- } else {
- q = rc->last_boosted_qindex;
- }
- } else {
- q = vp10_rc_regulate_q(cpi, rc->this_frame_target,
- active_best_quality, active_worst_quality);
- if (q > active_worst_quality) {
- // Special case when we are targeting the max allowed rate.
- if (rc->this_frame_target >= rc->max_frame_bandwidth)
- active_worst_quality = q;
- else
- q = active_worst_quality;
- }
- }
- clamp(q, active_best_quality, active_worst_quality);
-
- *top_index = active_worst_quality;
- *bottom_index = active_best_quality;
-
- assert(*top_index <= rc->worst_quality &&
- *top_index >= rc->best_quality);
- assert(*bottom_index <= rc->worst_quality &&
- *bottom_index >= rc->best_quality);
- assert(q <= rc->worst_quality && q >= rc->best_quality);
- return q;
-}
-
-int vp10_rc_pick_q_and_bounds(const VP10_COMP *cpi,
- int *bottom_index, int *top_index) {
- int q;
- if (cpi->oxcf.pass == 0) {
- if (cpi->oxcf.rc_mode == VPX_CBR)
- q = rc_pick_q_and_bounds_one_pass_cbr(cpi, bottom_index, top_index);
- else
- q = rc_pick_q_and_bounds_one_pass_vbr(cpi, bottom_index, top_index);
- } else {
- q = rc_pick_q_and_bounds_two_pass(cpi, bottom_index, top_index);
- }
-
- return q;
-}
-
-void vp10_rc_compute_frame_size_bounds(const VP10_COMP *cpi,
- int frame_target,
- int *frame_under_shoot_limit,
- int *frame_over_shoot_limit) {
- if (cpi->oxcf.rc_mode == VPX_Q) {
- *frame_under_shoot_limit = 0;
- *frame_over_shoot_limit = INT_MAX;
- } else {
- // For very small rate targets where the fractional adjustment
- // may be tiny make sure there is at least a minimum range.
- const int tolerance = (cpi->sf.recode_tolerance * frame_target) / 100;
- *frame_under_shoot_limit = VPXMAX(frame_target - tolerance - 200, 0);
- *frame_over_shoot_limit = VPXMIN(frame_target + tolerance + 200,
- cpi->rc.max_frame_bandwidth);
- }
-}
-
-void vp10_rc_set_frame_target(VP10_COMP *cpi, int target) {
- const VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
-
- rc->this_frame_target = target;
-
- // Modify frame size target when down-scaling.
- if (cpi->oxcf.resize_mode == RESIZE_DYNAMIC &&
- rc->frame_size_selector != UNSCALED)
- rc->this_frame_target = (int)(rc->this_frame_target
- * rate_thresh_mult[rc->frame_size_selector]);
-
- // Target rate per SB64 (including partial SB64s.
- rc->sb64_target_rate = ((int64_t)rc->this_frame_target * 64 * 64) /
- (cm->width * cm->height);
-}
-
-static void update_alt_ref_frame_stats(VP10_COMP *cpi) {
- // this frame refreshes means next frames don't unless specified by user
- RATE_CONTROL *const rc = &cpi->rc;
- rc->frames_since_golden = 0;
-
- // Mark the alt ref as done (setting to 0 means no further alt refs pending).
- rc->source_alt_ref_pending = 0;
-
- // Set the alternate reference frame active flag
- rc->source_alt_ref_active = 1;
-}
-
-static void update_golden_frame_stats(VP10_COMP *cpi) {
- RATE_CONTROL *const rc = &cpi->rc;
-
- // Update the Golden frame usage counts.
- if (cpi->refresh_golden_frame) {
- // this frame refreshes means next frames don't unless specified by user
- rc->frames_since_golden = 0;
-
- // If we are not using alt ref in the up and coming group clear the arf
- // active flag.
- if (!rc->source_alt_ref_pending) {
- rc->source_alt_ref_active = 0;
- }
-
- // Decrement count down till next gf
- if (rc->frames_till_gf_update_due > 0)
- rc->frames_till_gf_update_due--;
-
- } else if (!cpi->refresh_alt_ref_frame) {
- // Decrement count down till next gf
- if (rc->frames_till_gf_update_due > 0)
- rc->frames_till_gf_update_due--;
-
- rc->frames_since_golden++;
- }
-}
-
-void vp10_rc_postencode_update(VP10_COMP *cpi, uint64_t bytes_used) {
- const VP10_COMMON *const cm = &cpi->common;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- RATE_CONTROL *const rc = &cpi->rc;
- const int qindex = cm->base_qindex;
-
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ && cm->seg.enabled) {
- vp10_cyclic_refresh_postencode(cpi);
- }
-
- // Update rate control heuristics
- rc->projected_frame_size = (int)(bytes_used << 3);
-
- // Post encode loop adjustment of Q prediction.
- vp10_rc_update_rate_correction_factors(cpi);
-
- // Keep a record of last Q and ambient average Q.
- if (cm->frame_type == KEY_FRAME) {
- rc->last_q[KEY_FRAME] = qindex;
- rc->avg_frame_qindex[KEY_FRAME] =
- ROUND_POWER_OF_TWO(3 * rc->avg_frame_qindex[KEY_FRAME] + qindex, 2);
- } else {
- if (rc->is_src_frame_alt_ref ||
- !(cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) {
- rc->last_q[INTER_FRAME] = qindex;
- rc->avg_frame_qindex[INTER_FRAME] =
- ROUND_POWER_OF_TWO(3 * rc->avg_frame_qindex[INTER_FRAME] + qindex, 2);
- rc->ni_frames++;
- rc->tot_q += vp10_convert_qindex_to_q(qindex, cm->bit_depth);
- rc->avg_q = rc->tot_q / rc->ni_frames;
- // Calculate the average Q for normal inter frames (not key or GFU
- // frames).
- rc->ni_tot_qi += qindex;
- rc->ni_av_qi = rc->ni_tot_qi / rc->ni_frames;
- }
- }
-
- // Keep record of last boosted (KF/KF/ARF) Q value.
- // If the current frame is coded at a lower Q then we also update it.
- // If all mbs in this group are skipped only update if the Q value is
- // better than that already stored.
- // This is used to help set quality in forced key frames to reduce popping
- if ((qindex < rc->last_boosted_qindex) ||
- (cm->frame_type == KEY_FRAME) ||
- (!rc->constrained_gf_group &&
- (cpi->refresh_alt_ref_frame ||
- (cpi->refresh_golden_frame && !rc->is_src_frame_alt_ref)))) {
- rc->last_boosted_qindex = qindex;
- }
- if (cm->frame_type == KEY_FRAME)
- rc->last_kf_qindex = qindex;
-
- update_buffer_level(cpi, rc->projected_frame_size);
-
- // Rolling monitors of whether we are over or underspending used to help
- // regulate min and Max Q in two pass.
- if (cm->frame_type != KEY_FRAME) {
- rc->rolling_target_bits = ROUND_POWER_OF_TWO(
- rc->rolling_target_bits * 3 + rc->this_frame_target, 2);
- rc->rolling_actual_bits = ROUND_POWER_OF_TWO(
- rc->rolling_actual_bits * 3 + rc->projected_frame_size, 2);
- rc->long_rolling_target_bits = ROUND_POWER_OF_TWO(
- rc->long_rolling_target_bits * 31 + rc->this_frame_target, 5);
- rc->long_rolling_actual_bits = ROUND_POWER_OF_TWO(
- rc->long_rolling_actual_bits * 31 + rc->projected_frame_size, 5);
- }
-
- // Actual bits spent
- rc->total_actual_bits += rc->projected_frame_size;
- rc->total_target_bits += cm->show_frame ? rc->avg_frame_bandwidth : 0;
-
- rc->total_target_vs_actual = rc->total_actual_bits - rc->total_target_bits;
-
- if (is_altref_enabled(cpi) && cpi->refresh_alt_ref_frame &&
- (cm->frame_type != KEY_FRAME))
- // Update the alternate reference frame stats as appropriate.
- update_alt_ref_frame_stats(cpi);
- else
- // Update the Golden frame stats as appropriate.
- update_golden_frame_stats(cpi);
-
- if (cm->frame_type == KEY_FRAME)
- rc->frames_since_key = 0;
- if (cm->show_frame) {
- rc->frames_since_key++;
- rc->frames_to_key--;
- }
-
- // Trigger the resizing of the next frame if it is scaled.
- if (oxcf->pass != 0) {
- cpi->resize_pending =
- rc->next_frame_size_selector != rc->frame_size_selector;
- rc->frame_size_selector = rc->next_frame_size_selector;
- }
-}
-
-void vp10_rc_postencode_update_drop_frame(VP10_COMP *cpi) {
- // Update buffer level with zero size, update frame counters, and return.
- update_buffer_level(cpi, 0);
- cpi->rc.frames_since_key++;
- cpi->rc.frames_to_key--;
- cpi->rc.rc_2_frame = 0;
- cpi->rc.rc_1_frame = 0;
-}
-
-// Use this macro to turn on/off use of alt-refs in one-pass mode.
-#define USE_ALTREF_FOR_ONE_PASS 1
-
-static int calc_pframe_target_size_one_pass_vbr(const VP10_COMP *const cpi) {
- static const int af_ratio = 10;
- const RATE_CONTROL *const rc = &cpi->rc;
- int target;
-#if USE_ALTREF_FOR_ONE_PASS
- target = (!rc->is_src_frame_alt_ref &&
- (cpi->refresh_golden_frame || cpi->refresh_alt_ref_frame)) ?
- (rc->avg_frame_bandwidth * rc->baseline_gf_interval * af_ratio) /
- (rc->baseline_gf_interval + af_ratio - 1) :
- (rc->avg_frame_bandwidth * rc->baseline_gf_interval) /
- (rc->baseline_gf_interval + af_ratio - 1);
-#else
- target = rc->avg_frame_bandwidth;
-#endif
- return vp10_rc_clamp_pframe_target_size(cpi, target);
-}
-
-static int calc_iframe_target_size_one_pass_vbr(const VP10_COMP *const cpi) {
- static const int kf_ratio = 25;
- const RATE_CONTROL *rc = &cpi->rc;
- const int target = rc->avg_frame_bandwidth * kf_ratio;
- return vp10_rc_clamp_iframe_target_size(cpi, target);
-}
-
-void vp10_rc_get_one_pass_vbr_params(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- int target;
- // TODO(yaowu): replace the "auto_key && 0" below with proper decision logic.
- if (!cpi->refresh_alt_ref_frame &&
- (cm->current_video_frame == 0 ||
- (cpi->frame_flags & FRAMEFLAGS_KEY) ||
- rc->frames_to_key == 0 ||
- (cpi->oxcf.auto_key && 0))) {
- cm->frame_type = KEY_FRAME;
- rc->this_key_frame_forced = cm->current_video_frame != 0 &&
- rc->frames_to_key == 0;
- rc->frames_to_key = cpi->oxcf.key_freq;
- rc->kf_boost = DEFAULT_KF_BOOST;
- rc->source_alt_ref_active = 0;
- } else {
- cm->frame_type = INTER_FRAME;
- }
- if (rc->frames_till_gf_update_due == 0) {
- rc->baseline_gf_interval = (rc->min_gf_interval + rc->max_gf_interval) / 2;
- rc->frames_till_gf_update_due = rc->baseline_gf_interval;
- // NOTE: frames_till_gf_update_due must be <= frames_to_key.
- if (rc->frames_till_gf_update_due > rc->frames_to_key) {
- rc->frames_till_gf_update_due = rc->frames_to_key;
- rc->constrained_gf_group = 1;
- } else {
- rc->constrained_gf_group = 0;
- }
- cpi->refresh_golden_frame = 1;
- rc->source_alt_ref_pending = USE_ALTREF_FOR_ONE_PASS;
- rc->gfu_boost = DEFAULT_GF_BOOST;
- }
- if (cm->frame_type == KEY_FRAME)
- target = calc_iframe_target_size_one_pass_vbr(cpi);
- else
- target = calc_pframe_target_size_one_pass_vbr(cpi);
- vp10_rc_set_frame_target(cpi, target);
-}
-
-static int calc_pframe_target_size_one_pass_cbr(const VP10_COMP *cpi) {
- const VP10EncoderConfig *oxcf = &cpi->oxcf;
- const RATE_CONTROL *rc = &cpi->rc;
- const int64_t diff = rc->optimal_buffer_level - rc->buffer_level;
- const int64_t one_pct_bits = 1 + rc->optimal_buffer_level / 100;
- int min_frame_target =
- VPXMAX(rc->avg_frame_bandwidth >> 4, FRAME_OVERHEAD_BITS);
- int target;
-
- if (oxcf->gf_cbr_boost_pct) {
- const int af_ratio_pct = oxcf->gf_cbr_boost_pct + 100;
- target = cpi->refresh_golden_frame ?
- (rc->avg_frame_bandwidth * rc->baseline_gf_interval * af_ratio_pct) /
- (rc->baseline_gf_interval * 100 + af_ratio_pct - 100) :
- (rc->avg_frame_bandwidth * rc->baseline_gf_interval * 100) /
- (rc->baseline_gf_interval * 100 + af_ratio_pct - 100);
- } else {
- target = rc->avg_frame_bandwidth;
- }
-
- if (diff > 0) {
- // Lower the target bandwidth for this frame.
- const int pct_low = (int)VPXMIN(diff / one_pct_bits, oxcf->under_shoot_pct);
- target -= (target * pct_low) / 200;
- } else if (diff < 0) {
- // Increase the target bandwidth for this frame.
- const int pct_high =
- (int)VPXMIN(-diff / one_pct_bits, oxcf->over_shoot_pct);
- target += (target * pct_high) / 200;
- }
- if (oxcf->rc_max_inter_bitrate_pct) {
- const int max_rate = rc->avg_frame_bandwidth *
- oxcf->rc_max_inter_bitrate_pct / 100;
- target = VPXMIN(target, max_rate);
- }
- return VPXMAX(min_frame_target, target);
-}
-
-static int calc_iframe_target_size_one_pass_cbr(const VP10_COMP *cpi) {
- const RATE_CONTROL *rc = &cpi->rc;
- int target;
- if (cpi->common.current_video_frame == 0) {
- target = ((rc->starting_buffer_level / 2) > INT_MAX)
- ? INT_MAX : (int)(rc->starting_buffer_level / 2);
- } else {
- int kf_boost = 32;
- double framerate = cpi->framerate;
-
- kf_boost = VPXMAX(kf_boost, (int)(2 * framerate - 16));
- if (rc->frames_since_key < framerate / 2) {
- kf_boost = (int)(kf_boost * rc->frames_since_key /
- (framerate / 2));
- }
- target = ((16 + kf_boost) * rc->avg_frame_bandwidth) >> 4;
- }
- return vp10_rc_clamp_iframe_target_size(cpi, target);
-}
-
-void vp10_rc_get_one_pass_cbr_params(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- int target;
- // TODO(yaowu): replace the "auto_key && 0" below with proper decision logic.
- if ((cm->current_video_frame == 0 ||
- (cpi->frame_flags & FRAMEFLAGS_KEY) ||
- rc->frames_to_key == 0 ||
- (cpi->oxcf.auto_key && 0))) {
- cm->frame_type = KEY_FRAME;
- rc->this_key_frame_forced = cm->current_video_frame != 0 &&
- rc->frames_to_key == 0;
- rc->frames_to_key = cpi->oxcf.key_freq;
- rc->kf_boost = DEFAULT_KF_BOOST;
- rc->source_alt_ref_active = 0;
- } else {
- cm->frame_type = INTER_FRAME;
- }
- if (rc->frames_till_gf_update_due == 0) {
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ)
- vp10_cyclic_refresh_set_golden_update(cpi);
- else
- rc->baseline_gf_interval =
- (rc->min_gf_interval + rc->max_gf_interval) / 2;
- rc->frames_till_gf_update_due = rc->baseline_gf_interval;
- // NOTE: frames_till_gf_update_due must be <= frames_to_key.
- if (rc->frames_till_gf_update_due > rc->frames_to_key)
- rc->frames_till_gf_update_due = rc->frames_to_key;
- cpi->refresh_golden_frame = 1;
- rc->gfu_boost = DEFAULT_GF_BOOST;
- }
-
- // Any update/change of global cyclic refresh parameters (amount/delta-qp)
- // should be done here, before the frame qp is selected.
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ)
- vp10_cyclic_refresh_update_parameters(cpi);
-
- if (cm->frame_type == KEY_FRAME)
- target = calc_iframe_target_size_one_pass_cbr(cpi);
- else
- target = calc_pframe_target_size_one_pass_cbr(cpi);
-
- vp10_rc_set_frame_target(cpi, target);
- if (cpi->oxcf.resize_mode == RESIZE_DYNAMIC)
- cpi->resize_pending = vp10_resize_one_pass_cbr(cpi);
- else
- cpi->resize_pending = 0;
-}
-
-int vp10_compute_qdelta(const RATE_CONTROL *rc, double qstart, double qtarget,
- vpx_bit_depth_t bit_depth) {
- int start_index = rc->worst_quality;
- int target_index = rc->worst_quality;
- int i;
-
- // Convert the average q value to an index.
- for (i = rc->best_quality; i < rc->worst_quality; ++i) {
- start_index = i;
- if (vp10_convert_qindex_to_q(i, bit_depth) >= qstart)
- break;
- }
-
- // Convert the q target to an index
- for (i = rc->best_quality; i < rc->worst_quality; ++i) {
- target_index = i;
- if (vp10_convert_qindex_to_q(i, bit_depth) >= qtarget)
- break;
- }
-
- return target_index - start_index;
-}
-
-int vp10_compute_qdelta_by_rate(const RATE_CONTROL *rc, FRAME_TYPE frame_type,
- int qindex, double rate_target_ratio,
- vpx_bit_depth_t bit_depth) {
- int target_index = rc->worst_quality;
- int i;
-
- // Look up the current projected bits per block for the base index
- const int base_bits_per_mb = vp10_rc_bits_per_mb(frame_type, qindex, 1.0,
- bit_depth);
-
- // Find the target bits per mb based on the base value and given ratio.
- const int target_bits_per_mb = (int)(rate_target_ratio * base_bits_per_mb);
-
- // Convert the q target to an index
- for (i = rc->best_quality; i < rc->worst_quality; ++i) {
- if (vp10_rc_bits_per_mb(frame_type, i, 1.0, bit_depth) <=
- target_bits_per_mb) {
- target_index = i;
- break;
- }
- }
- return target_index - qindex;
-}
-
-void vp10_rc_set_gf_interval_range(const VP10_COMP *const cpi,
- RATE_CONTROL *const rc) {
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
-
- // Set Maximum gf/arf interval
- rc->max_gf_interval = oxcf->max_gf_interval;
- rc->min_gf_interval = oxcf->min_gf_interval;
- if (rc->min_gf_interval == 0)
- rc->min_gf_interval = vp10_rc_get_default_min_gf_interval(
- oxcf->width, oxcf->height, cpi->framerate);
- if (rc->max_gf_interval == 0)
- rc->max_gf_interval = vp10_rc_get_default_max_gf_interval(
- cpi->framerate, rc->min_gf_interval);
-
- // Extended interval for genuinely static scenes
- rc->static_scene_max_gf_interval = MAX_LAG_BUFFERS * 2;
-
- if (is_altref_enabled(cpi)) {
- if (rc->static_scene_max_gf_interval > oxcf->lag_in_frames - 1)
- rc->static_scene_max_gf_interval = oxcf->lag_in_frames - 1;
- }
-
- if (rc->max_gf_interval > rc->static_scene_max_gf_interval)
- rc->max_gf_interval = rc->static_scene_max_gf_interval;
-
- // Clamp min to max
- rc->min_gf_interval = VPXMIN(rc->min_gf_interval, rc->max_gf_interval);
-}
-
-void vp10_rc_update_framerate(VP10_COMP *cpi) {
- const VP10_COMMON *const cm = &cpi->common;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- RATE_CONTROL *const rc = &cpi->rc;
- int vbr_max_bits;
-
- rc->avg_frame_bandwidth = (int)(oxcf->target_bandwidth / cpi->framerate);
- rc->min_frame_bandwidth = (int)(rc->avg_frame_bandwidth *
- oxcf->two_pass_vbrmin_section / 100);
-
- rc->min_frame_bandwidth =
- VPXMAX(rc->min_frame_bandwidth, FRAME_OVERHEAD_BITS);
-
- // A maximum bitrate for a frame is defined.
- // The baseline for this aligns with HW implementations that
- // can support decode of 1080P content up to a bitrate of MAX_MB_RATE bits
- // per 16x16 MB (averaged over a frame). However this limit is extended if
- // a very high rate is given on the command line or the the rate cannnot
- // be acheived because of a user specificed max q (e.g. when the user
- // specifies lossless encode.
- vbr_max_bits = (int)(((int64_t)rc->avg_frame_bandwidth *
- oxcf->two_pass_vbrmax_section) / 100);
- rc->max_frame_bandwidth =
- VPXMAX(VPXMAX((cm->MBs * MAX_MB_RATE), MAXRATE_1080P), vbr_max_bits);
-
- vp10_rc_set_gf_interval_range(cpi, rc);
-}
-
-#define VBR_PCT_ADJUSTMENT_LIMIT 50
-// For VBR...adjustment to the frame target based on error from previous frames
-static void vbr_rate_correction(VP10_COMP *cpi, int *this_frame_target) {
- RATE_CONTROL *const rc = &cpi->rc;
- int64_t vbr_bits_off_target = rc->vbr_bits_off_target;
- int max_delta;
- double position_factor = 1.0;
-
- // How far through the clip are we.
- // This number is used to damp the per frame rate correction.
- // Range 0 - 1.0
- if (cpi->twopass.total_stats.count) {
- position_factor = sqrt((double)cpi->common.current_video_frame /
- cpi->twopass.total_stats.count);
- }
- max_delta = (int)(position_factor *
- ((*this_frame_target * VBR_PCT_ADJUSTMENT_LIMIT) / 100));
-
- // vbr_bits_off_target > 0 means we have extra bits to spend
- if (vbr_bits_off_target > 0) {
- *this_frame_target +=
- (vbr_bits_off_target > max_delta) ? max_delta
- : (int)vbr_bits_off_target;
- } else {
- *this_frame_target -=
- (vbr_bits_off_target < -max_delta) ? max_delta
- : (int)-vbr_bits_off_target;
- }
-
- // Fast redistribution of bits arising from massive local undershoot.
- // Dont do it for kf,arf,gf or overlay frames.
- if (!frame_is_kf_gf_arf(cpi) && !rc->is_src_frame_alt_ref &&
- rc->vbr_bits_off_target_fast) {
- int one_frame_bits = VPXMAX(rc->avg_frame_bandwidth, *this_frame_target);
- int fast_extra_bits;
- fast_extra_bits = (int)VPXMIN(rc->vbr_bits_off_target_fast, one_frame_bits);
- fast_extra_bits = (int)VPXMIN(
- fast_extra_bits,
- VPXMAX(one_frame_bits / 8, rc->vbr_bits_off_target_fast / 8));
- *this_frame_target += (int)fast_extra_bits;
- rc->vbr_bits_off_target_fast -= fast_extra_bits;
- }
-}
-
-void vp10_set_target_rate(VP10_COMP *cpi) {
- RATE_CONTROL *const rc = &cpi->rc;
- int target_rate = rc->base_frame_target;
-
- // Correction to rate target based on prior over or under shoot.
- if (cpi->oxcf.rc_mode == VPX_VBR || cpi->oxcf.rc_mode == VPX_CQ)
- vbr_rate_correction(cpi, &target_rate);
- vp10_rc_set_frame_target(cpi, target_rate);
-}
-
-// Check if we should resize, based on average QP from past x frames.
-// Only allow for resize at most one scale down for now, scaling factor is 2.
-int vp10_resize_one_pass_cbr(VP10_COMP *cpi) {
- const VP10_COMMON *const cm = &cpi->common;
- RATE_CONTROL *const rc = &cpi->rc;
- int resize_now = 0;
- cpi->resize_scale_num = 1;
- cpi->resize_scale_den = 1;
- // Don't resize on key frame; reset the counters on key frame.
- if (cm->frame_type == KEY_FRAME) {
- cpi->resize_avg_qp = 0;
- cpi->resize_count = 0;
- return 0;
- }
- // Resize based on average buffer underflow and QP over some window.
- // Ignore samples close to key frame, since QP is usually high after key.
- if (cpi->rc.frames_since_key > 2 * cpi->framerate) {
- const int window = (int)(5 * cpi->framerate);
- cpi->resize_avg_qp += cm->base_qindex;
- if (cpi->rc.buffer_level < (int)(30 * rc->optimal_buffer_level / 100))
- ++cpi->resize_buffer_underflow;
- ++cpi->resize_count;
- // Check for resize action every "window" frames.
- if (cpi->resize_count >= window) {
- int avg_qp = cpi->resize_avg_qp / cpi->resize_count;
- // Resize down if buffer level has underflowed sufficent amount in past
- // window, and we are at original resolution.
- // Resize back up if average QP is low, and we are currently in a resized
- // down state.
- if (cpi->resize_state == 0 &&
- cpi->resize_buffer_underflow > (cpi->resize_count >> 2)) {
- resize_now = 1;
- cpi->resize_state = 1;
- } else if (cpi->resize_state == 1 &&
- avg_qp < 40 * cpi->rc.worst_quality / 100) {
- resize_now = -1;
- cpi->resize_state = 0;
- }
- // Reset for next window measurement.
- cpi->resize_avg_qp = 0;
- cpi->resize_count = 0;
- cpi->resize_buffer_underflow = 0;
- }
- }
- // If decision is to resize, reset some quantities, and check is we should
- // reduce rate correction factor,
- if (resize_now != 0) {
- int target_bits_per_frame;
- int active_worst_quality;
- int qindex;
- int tot_scale_change;
- // For now, resize is by 1/2 x 1/2.
- cpi->resize_scale_num = 1;
- cpi->resize_scale_den = 2;
- tot_scale_change = (cpi->resize_scale_den * cpi->resize_scale_den) /
- (cpi->resize_scale_num * cpi->resize_scale_num);
- // Reset buffer level to optimal, update target size.
- rc->buffer_level = rc->optimal_buffer_level;
- rc->bits_off_target = rc->optimal_buffer_level;
- rc->this_frame_target = calc_pframe_target_size_one_pass_cbr(cpi);
- // Reset cyclic refresh parameters.
- if (cpi->oxcf.aq_mode == CYCLIC_REFRESH_AQ && cm->seg.enabled)
- vp10_cyclic_refresh_reset_resize(cpi);
- // Get the projected qindex, based on the scaled target frame size (scaled
- // so target_bits_per_mb in vp10_rc_regulate_q will be correct target).
- target_bits_per_frame = (resize_now == 1) ?
- rc->this_frame_target * tot_scale_change :
- rc->this_frame_target / tot_scale_change;
- active_worst_quality = calc_active_worst_quality_one_pass_cbr(cpi);
- qindex = vp10_rc_regulate_q(cpi,
- target_bits_per_frame,
- rc->best_quality,
- active_worst_quality);
- // If resize is down, check if projected q index is close to worst_quality,
- // and if so, reduce the rate correction factor (since likely can afford
- // lower q for resized frame).
- if (resize_now == 1 &&
- qindex > 90 * cpi->rc.worst_quality / 100) {
- rc->rate_correction_factors[INTER_NORMAL] *= 0.85;
- }
- // If resize is back up, check if projected q index is too much above the
- // current base_qindex, and if so, reduce the rate correction factor
- // (since prefer to keep q for resized frame at least close to previous q).
- if (resize_now == -1 &&
- qindex > 130 * cm->base_qindex / 100) {
- rc->rate_correction_factors[INTER_NORMAL] *= 0.9;
- }
- }
- return resize_now;
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_RATECTRL_H_
-#define VP10_ENCODER_RATECTRL_H_
-
-#include "vpx/vpx_codec.h"
-#include "vpx/vpx_integer.h"
-
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// Bits Per MB at different Q (Multiplied by 512)
-#define BPER_MB_NORMBITS 9
-
-#define MIN_GF_INTERVAL 4
-#define MAX_GF_INTERVAL 16
-
-typedef enum {
- INTER_NORMAL = 0,
- INTER_HIGH = 1,
- GF_ARF_LOW = 2,
- GF_ARF_STD = 3,
- KF_STD = 4,
- RATE_FACTOR_LEVELS = 5
-} RATE_FACTOR_LEVEL;
-
-// Internal frame scaling level.
-typedef enum {
- UNSCALED = 0, // Frame is unscaled.
- SCALE_STEP1 = 1, // First-level down-scaling.
- FRAME_SCALE_STEPS
-} FRAME_SCALE_LEVEL;
-
-// Frame dimensions multiplier wrt the native frame size, in 1/16ths,
-// specified for the scale-up case.
-// e.g. 24 => 16/24 = 2/3 of native size. The restriction to 1/16th is
-// intended to match the capabilities of the normative scaling filters,
-// giving precedence to the up-scaling accuracy.
-static const int frame_scale_factor[FRAME_SCALE_STEPS] = {16, 24};
-
-// Multiplier of the target rate to be used as threshold for triggering scaling.
-static const double rate_thresh_mult[FRAME_SCALE_STEPS] = {1.0, 2.0};
-
-// Scale dependent Rate Correction Factor multipliers. Compensates for the
-// greater number of bits per pixel generated in down-scaled frames.
-static const double rcf_mult[FRAME_SCALE_STEPS] = {1.0, 2.0};
-
-typedef struct {
- // Rate targetting variables
- int base_frame_target; // A baseline frame target before adjustment
- // for previous under or over shoot.
- int this_frame_target; // Actual frame target after rc adjustment.
- int projected_frame_size;
- int sb64_target_rate;
- int last_q[FRAME_TYPES]; // Separate values for Intra/Inter
- int last_boosted_qindex; // Last boosted GF/KF/ARF q
- int last_kf_qindex; // Q index of the last key frame coded.
-
- int gfu_boost;
- int last_boost;
- int kf_boost;
-
- double rate_correction_factors[RATE_FACTOR_LEVELS];
-
- int frames_since_golden;
- int frames_till_gf_update_due;
- int min_gf_interval;
- int max_gf_interval;
- int static_scene_max_gf_interval;
- int baseline_gf_interval;
- int constrained_gf_group;
- int frames_to_key;
- int frames_since_key;
- int this_key_frame_forced;
- int next_key_frame_forced;
- int source_alt_ref_pending;
- int source_alt_ref_active;
- int is_src_frame_alt_ref;
-
- int avg_frame_bandwidth; // Average frame size target for clip
- int min_frame_bandwidth; // Minimum allocation used for any frame
- int max_frame_bandwidth; // Maximum burst rate allowed for a frame.
-
- int ni_av_qi;
- int ni_tot_qi;
- int ni_frames;
- int avg_frame_qindex[FRAME_TYPES];
- double tot_q;
- double avg_q;
-
- int64_t buffer_level;
- int64_t bits_off_target;
- int64_t vbr_bits_off_target;
- int64_t vbr_bits_off_target_fast;
-
- int decimation_factor;
- int decimation_count;
-
- int rolling_target_bits;
- int rolling_actual_bits;
-
- int long_rolling_target_bits;
- int long_rolling_actual_bits;
-
- int rate_error_estimate;
-
- int64_t total_actual_bits;
- int64_t total_target_bits;
- int64_t total_target_vs_actual;
-
- int worst_quality;
- int best_quality;
-
- int64_t starting_buffer_level;
- int64_t optimal_buffer_level;
- int64_t maximum_buffer_size;
-
- // rate control history for last frame(1) and the frame before(2).
- // -1: undershot
- // 1: overshoot
- // 0: not initialized.
- int rc_1_frame;
- int rc_2_frame;
- int q_1_frame;
- int q_2_frame;
-
- // Auto frame-scaling variables.
- FRAME_SCALE_LEVEL frame_size_selector;
- FRAME_SCALE_LEVEL next_frame_size_selector;
- int frame_width[FRAME_SCALE_STEPS];
- int frame_height[FRAME_SCALE_STEPS];
- int rf_level_maxq[RATE_FACTOR_LEVELS];
-} RATE_CONTROL;
-
-struct VP10_COMP;
-struct VP10EncoderConfig;
-
-void vp10_rc_init(const struct VP10EncoderConfig *oxcf, int pass,
- RATE_CONTROL *rc);
-
-int vp10_estimate_bits_at_q(FRAME_TYPE frame_kind, int q, int mbs,
- double correction_factor,
- vpx_bit_depth_t bit_depth);
-
-double vp10_convert_qindex_to_q(int qindex, vpx_bit_depth_t bit_depth);
-
-void vp10_rc_init_minq_luts(void);
-
-int vp10_rc_get_default_min_gf_interval(int width, int height, double framerate);
-// Note vp10_rc_get_default_max_gf_interval() requires the min_gf_interval to
-// be passed in to ensure that the max_gf_interval returned is at least as bis
-// as that.
-int vp10_rc_get_default_max_gf_interval(double framerate, int min_frame_rate);
-
-// Generally at the high level, the following flow is expected
-// to be enforced for rate control:
-// First call per frame, one of:
-// vp10_rc_get_one_pass_vbr_params()
-// vp10_rc_get_one_pass_cbr_params()
-// vp10_rc_get_first_pass_params()
-// vp10_rc_get_second_pass_params()
-// depending on the usage to set the rate control encode parameters desired.
-//
-// Then, call encode_frame_to_data_rate() to perform the
-// actual encode. This function will in turn call encode_frame()
-// one or more times, followed by one of:
-// vp10_rc_postencode_update()
-// vp10_rc_postencode_update_drop_frame()
-//
-// The majority of rate control parameters are only expected
-// to be set in the vp10_rc_get_..._params() functions and
-// updated during the vp10_rc_postencode_update...() functions.
-// The only exceptions are vp10_rc_drop_frame() and
-// vp10_rc_update_rate_correction_factors() functions.
-
-// Functions to set parameters for encoding before the actual
-// encode_frame_to_data_rate() function.
-void vp10_rc_get_one_pass_vbr_params(struct VP10_COMP *cpi);
-void vp10_rc_get_one_pass_cbr_params(struct VP10_COMP *cpi);
-
-// Post encode update of the rate control parameters based
-// on bytes used
-void vp10_rc_postencode_update(struct VP10_COMP *cpi, uint64_t bytes_used);
-// Post encode update of the rate control parameters for dropped frames
-void vp10_rc_postencode_update_drop_frame(struct VP10_COMP *cpi);
-
-// Updates rate correction factors
-// Changes only the rate correction factors in the rate control structure.
-void vp10_rc_update_rate_correction_factors(struct VP10_COMP *cpi);
-
-// Decide if we should drop this frame: For 1-pass CBR.
-// Changes only the decimation count in the rate control structure
-int vp10_rc_drop_frame(struct VP10_COMP *cpi);
-
-// Computes frame size bounds.
-void vp10_rc_compute_frame_size_bounds(const struct VP10_COMP *cpi,
- int this_frame_target,
- int *frame_under_shoot_limit,
- int *frame_over_shoot_limit);
-
-// Picks q and q bounds given the target for bits
-int vp10_rc_pick_q_and_bounds(const struct VP10_COMP *cpi,
- int *bottom_index,
- int *top_index);
-
-// Estimates q to achieve a target bits per frame
-int vp10_rc_regulate_q(const struct VP10_COMP *cpi, int target_bits_per_frame,
- int active_best_quality, int active_worst_quality);
-
-// Estimates bits per mb for a given qindex and correction factor.
-int vp10_rc_bits_per_mb(FRAME_TYPE frame_type, int qindex,
- double correction_factor, vpx_bit_depth_t bit_depth);
-
-// Clamping utilities for bitrate targets for iframes and pframes.
-int vp10_rc_clamp_iframe_target_size(const struct VP10_COMP *const cpi,
- int target);
-int vp10_rc_clamp_pframe_target_size(const struct VP10_COMP *const cpi,
- int target);
-// Utility to set frame_target into the RATE_CONTROL structure
-// This function is called only from the vp10_rc_get_..._params() functions.
-void vp10_rc_set_frame_target(struct VP10_COMP *cpi, int target);
-
-// Computes a q delta (in "q index" terms) to get from a starting q value
-// to a target q value
-int vp10_compute_qdelta(const RATE_CONTROL *rc, double qstart, double qtarget,
- vpx_bit_depth_t bit_depth);
-
-// Computes a q delta (in "q index" terms) to get from a starting q value
-// to a value that should equate to the given rate ratio.
-int vp10_compute_qdelta_by_rate(const RATE_CONTROL *rc, FRAME_TYPE frame_type,
- int qindex, double rate_target_ratio,
- vpx_bit_depth_t bit_depth);
-
-int vp10_frame_type_qdelta(const struct VP10_COMP *cpi, int rf_level, int q);
-
-void vp10_rc_update_framerate(struct VP10_COMP *cpi);
-
-void vp10_rc_set_gf_interval_range(const struct VP10_COMP *const cpi,
- RATE_CONTROL *const rc);
-
-void vp10_set_target_rate(struct VP10_COMP *cpi);
-
-int vp10_resize_one_pass_cbr(struct VP10_COMP *cpi);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_RATECTRL_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <math.h>
-#include <stdio.h>
-
-#include "./vp10_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/bitops.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/mvref_common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/reconintra.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/encodemb.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/tokenize.h"
-
-#define RD_THRESH_POW 1.25
-#define RD_MULT_EPB_RATIO 64
-
-// Factor to weigh the rate for switchable interp filters.
-#define SWITCHABLE_INTERP_RATE_FACTOR 1
-
-void vp10_rd_cost_reset(RD_COST *rd_cost) {
- rd_cost->rate = INT_MAX;
- rd_cost->dist = INT64_MAX;
- rd_cost->rdcost = INT64_MAX;
-}
-
-void vp10_rd_cost_init(RD_COST *rd_cost) {
- rd_cost->rate = 0;
- rd_cost->dist = 0;
- rd_cost->rdcost = 0;
-}
-
-// The baseline rd thresholds for breaking out of the rd loop for
-// certain modes are assumed to be based on 8x8 blocks.
-// This table is used to correct for block size.
-// The factors here are << 2 (2 = x0.5, 32 = x8 etc).
-static const uint8_t rd_thresh_block_size_factor[BLOCK_SIZES] = {
- 2, 3, 3, 4, 6, 6, 8, 12, 12, 16, 24, 24, 32
-};
-
-static void fill_mode_costs(VP10_COMP *cpi) {
- const FRAME_CONTEXT *const fc = cpi->common.fc;
- int i, j;
-
- for (i = 0; i < INTRA_MODES; ++i)
- for (j = 0; j < INTRA_MODES; ++j)
- vp10_cost_tokens(cpi->y_mode_costs[i][j], vp10_kf_y_mode_prob[i][j],
- vp10_intra_mode_tree);
-
- vp10_cost_tokens(cpi->mbmode_cost, fc->y_mode_prob[1], vp10_intra_mode_tree);
- vp10_cost_tokens(cpi->intra_uv_mode_cost,
- fc->uv_mode_prob[TM_PRED], vp10_intra_mode_tree);
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- vp10_cost_tokens(cpi->switchable_interp_costs[i],
- fc->switchable_interp_prob[i], vp10_switchable_interp_tree);
-
- for (i = 0; i < PALETTE_BLOCK_SIZES; ++i) {
- vp10_cost_tokens(cpi->palette_y_size_cost[i],
- vp10_default_palette_y_size_prob[i],
- vp10_palette_size_tree);
- vp10_cost_tokens(cpi->palette_uv_size_cost[i],
- vp10_default_palette_uv_size_prob[i],
- vp10_palette_size_tree);
- }
-
- for (i = 0; i < PALETTE_MAX_SIZE - 1; ++i)
- for (j = 0; j < PALETTE_COLOR_CONTEXTS; ++j) {
- vp10_cost_tokens(cpi->palette_y_color_cost[i][j],
- vp10_default_palette_y_color_prob[i][j],
- vp10_palette_color_tree[i]);
- vp10_cost_tokens(cpi->palette_uv_color_cost[i][j],
- vp10_default_palette_uv_color_prob[i][j],
- vp10_palette_color_tree[i]);
- }
-}
-
-static void fill_token_costs(vp10_coeff_cost *c,
- vp10_coeff_probs_model (*p)[PLANE_TYPES]) {
- int i, j, k, l;
- TX_SIZE t;
- for (t = TX_4X4; t <= TX_32X32; ++t)
- for (i = 0; i < PLANE_TYPES; ++i)
- for (j = 0; j < REF_TYPES; ++j)
- for (k = 0; k < COEF_BANDS; ++k)
- for (l = 0; l < BAND_COEFF_CONTEXTS(k); ++l) {
- vpx_prob probs[ENTROPY_NODES];
- vp10_model_to_full_probs(p[t][i][j][k][l], probs);
- vp10_cost_tokens((int *)c[t][i][j][k][0][l], probs,
- vp10_coef_tree);
- vp10_cost_tokens_skip((int *)c[t][i][j][k][1][l], probs,
- vp10_coef_tree);
- assert(c[t][i][j][k][0][l][EOB_TOKEN] ==
- c[t][i][j][k][1][l][EOB_TOKEN]);
- }
-}
-
-// Values are now correlated to quantizer.
-static int sad_per_bit16lut_8[QINDEX_RANGE];
-static int sad_per_bit4lut_8[QINDEX_RANGE];
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static int sad_per_bit16lut_10[QINDEX_RANGE];
-static int sad_per_bit4lut_10[QINDEX_RANGE];
-static int sad_per_bit16lut_12[QINDEX_RANGE];
-static int sad_per_bit4lut_12[QINDEX_RANGE];
-#endif
-
-static void init_me_luts_bd(int *bit16lut, int *bit4lut, int range,
- vpx_bit_depth_t bit_depth) {
- int i;
- // Initialize the sad lut tables using a formulaic calculation for now.
- // This is to make it easier to resolve the impact of experimental changes
- // to the quantizer tables.
- for (i = 0; i < range; i++) {
- const double q = vp10_convert_qindex_to_q(i, bit_depth);
- bit16lut[i] = (int)(0.0418 * q + 2.4107);
- bit4lut[i] = (int)(0.063 * q + 2.742);
- }
-}
-
-void vp10_init_me_luts(void) {
- init_me_luts_bd(sad_per_bit16lut_8, sad_per_bit4lut_8, QINDEX_RANGE,
- VPX_BITS_8);
-#if CONFIG_VP9_HIGHBITDEPTH
- init_me_luts_bd(sad_per_bit16lut_10, sad_per_bit4lut_10, QINDEX_RANGE,
- VPX_BITS_10);
- init_me_luts_bd(sad_per_bit16lut_12, sad_per_bit4lut_12, QINDEX_RANGE,
- VPX_BITS_12);
-#endif
-}
-
-static const int rd_boost_factor[16] = {
- 64, 32, 32, 32, 24, 16, 12, 12,
- 8, 8, 4, 4, 2, 2, 1, 0
-};
-static const int rd_frame_type_factor[FRAME_UPDATE_TYPES] = {
- 128, 144, 128, 128, 144
-};
-
-int vp10_compute_rd_mult(const VP10_COMP *cpi, int qindex) {
- const int64_t q = vp10_dc_quant(qindex, 0, cpi->common.bit_depth);
-#if CONFIG_VP9_HIGHBITDEPTH
- int64_t rdmult = 0;
- switch (cpi->common.bit_depth) {
- case VPX_BITS_8:
- rdmult = 88 * q * q / 24;
- break;
- case VPX_BITS_10:
- rdmult = ROUND_POWER_OF_TWO(88 * q * q / 24, 4);
- break;
- case VPX_BITS_12:
- rdmult = ROUND_POWER_OF_TWO(88 * q * q / 24, 8);
- break;
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1;
- }
-#else
- int64_t rdmult = 88 * q * q / 24;
-#endif // CONFIG_VP9_HIGHBITDEPTH
- if (cpi->oxcf.pass == 2 && (cpi->common.frame_type != KEY_FRAME)) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- const FRAME_UPDATE_TYPE frame_type = gf_group->update_type[gf_group->index];
- const int boost_index = VPXMIN(15, (cpi->rc.gfu_boost / 100));
-
- rdmult = (rdmult * rd_frame_type_factor[frame_type]) >> 7;
- rdmult += ((rdmult * rd_boost_factor[boost_index]) >> 7);
- }
- if (rdmult < 1)
- rdmult = 1;
- return (int)rdmult;
-}
-
-static int compute_rd_thresh_factor(int qindex, vpx_bit_depth_t bit_depth) {
- double q;
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (bit_depth) {
- case VPX_BITS_8:
- q = vp10_dc_quant(qindex, 0, VPX_BITS_8) / 4.0;
- break;
- case VPX_BITS_10:
- q = vp10_dc_quant(qindex, 0, VPX_BITS_10) / 16.0;
- break;
- case VPX_BITS_12:
- q = vp10_dc_quant(qindex, 0, VPX_BITS_12) / 64.0;
- break;
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1;
- }
-#else
- (void) bit_depth;
- q = vp10_dc_quant(qindex, 0, VPX_BITS_8) / 4.0;
-#endif // CONFIG_VP9_HIGHBITDEPTH
- // TODO(debargha): Adjust the function below.
- return VPXMAX((int)(pow(q, RD_THRESH_POW) * 5.12), 8);
-}
-
-void vp10_initialize_me_consts(VP10_COMP *cpi, MACROBLOCK *x, int qindex) {
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (cpi->common.bit_depth) {
- case VPX_BITS_8:
- x->sadperbit16 = sad_per_bit16lut_8[qindex];
- x->sadperbit4 = sad_per_bit4lut_8[qindex];
- break;
- case VPX_BITS_10:
- x->sadperbit16 = sad_per_bit16lut_10[qindex];
- x->sadperbit4 = sad_per_bit4lut_10[qindex];
- break;
- case VPX_BITS_12:
- x->sadperbit16 = sad_per_bit16lut_12[qindex];
- x->sadperbit4 = sad_per_bit4lut_12[qindex];
- break;
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- }
-#else
- (void)cpi;
- x->sadperbit16 = sad_per_bit16lut_8[qindex];
- x->sadperbit4 = sad_per_bit4lut_8[qindex];
-#endif // CONFIG_VP9_HIGHBITDEPTH
-}
-
-static void set_block_thresholds(const VP10_COMMON *cm, RD_OPT *rd) {
- int i, bsize, segment_id;
-
- for (segment_id = 0; segment_id < MAX_SEGMENTS; ++segment_id) {
- const int qindex =
- clamp(vp10_get_qindex(&cm->seg, segment_id, cm->base_qindex) +
- cm->y_dc_delta_q, 0, MAXQ);
- const int q = compute_rd_thresh_factor(qindex, cm->bit_depth);
-
- for (bsize = 0; bsize < BLOCK_SIZES; ++bsize) {
- // Threshold here seems unnecessarily harsh but fine given actual
- // range of values used for cpi->sf.thresh_mult[].
- const int t = q * rd_thresh_block_size_factor[bsize];
- const int thresh_max = INT_MAX / t;
-
- if (bsize >= BLOCK_8X8) {
- for (i = 0; i < MAX_MODES; ++i)
- rd->threshes[segment_id][bsize][i] =
- rd->thresh_mult[i] < thresh_max
- ? rd->thresh_mult[i] * t / 4
- : INT_MAX;
- } else {
- for (i = 0; i < MAX_REFS; ++i)
- rd->threshes[segment_id][bsize][i] =
- rd->thresh_mult_sub8x8[i] < thresh_max
- ? rd->thresh_mult_sub8x8[i] * t / 4
- : INT_MAX;
- }
- }
- }
-}
-
-void vp10_initialize_rd_consts(VP10_COMP *cpi) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCK *const x = &cpi->td.mb;
- RD_OPT *const rd = &cpi->rd;
- int i;
-
- vpx_clear_system_state();
-
- rd->RDDIV = RDDIV_BITS; // In bits (to multiply D by 128).
- rd->RDMULT = vp10_compute_rd_mult(cpi, cm->base_qindex + cm->y_dc_delta_q);
-
- x->errorperbit = rd->RDMULT / RD_MULT_EPB_RATIO;
- x->errorperbit += (x->errorperbit == 0);
-
- x->select_tx_size = (cpi->sf.tx_size_search_method == USE_LARGESTALL &&
- cm->frame_type != KEY_FRAME) ? 0 : 1;
-
- set_block_thresholds(cm, rd);
-
- fill_token_costs(x->token_costs, cm->fc->coef_probs);
-
- if (cpi->sf.partition_search_type != VAR_BASED_PARTITION ||
- cm->frame_type == KEY_FRAME) {
- for (i = 0; i < PARTITION_CONTEXTS; ++i)
- vp10_cost_tokens(cpi->partition_cost[i], cm->fc->partition_prob[i],
- vp10_partition_tree);
- }
-
- fill_mode_costs(cpi);
-
- if (!frame_is_intra_only(cm)) {
- vp10_build_nmv_cost_table(x->nmvjointcost,
- cm->allow_high_precision_mv ? x->nmvcost_hp
- : x->nmvcost,
- &cm->fc->nmvc, cm->allow_high_precision_mv);
-
- for (i = 0; i < INTER_MODE_CONTEXTS; ++i)
- vp10_cost_tokens((int *)cpi->inter_mode_cost[i],
- cm->fc->inter_mode_probs[i], vp10_inter_mode_tree);
- }
-}
-
-static void model_rd_norm(int xsq_q10, int *r_q10, int *d_q10) {
- // NOTE: The tables below must be of the same size.
-
- // The functions described below are sampled at the four most significant
- // bits of x^2 + 8 / 256.
-
- // Normalized rate:
- // This table models the rate for a Laplacian source with given variance
- // when quantized with a uniform quantizer with given stepsize. The
- // closed form expression is:
- // Rn(x) = H(sqrt(r)) + sqrt(r)*[1 + H(r)/(1 - r)],
- // where r = exp(-sqrt(2) * x) and x = qpstep / sqrt(variance),
- // and H(x) is the binary entropy function.
- static const int rate_tab_q10[] = {
- 65536, 6086, 5574, 5275, 5063, 4899, 4764, 4651,
- 4553, 4389, 4255, 4142, 4044, 3958, 3881, 3811,
- 3748, 3635, 3538, 3453, 3376, 3307, 3244, 3186,
- 3133, 3037, 2952, 2877, 2809, 2747, 2690, 2638,
- 2589, 2501, 2423, 2353, 2290, 2232, 2179, 2130,
- 2084, 2001, 1928, 1862, 1802, 1748, 1698, 1651,
- 1608, 1530, 1460, 1398, 1342, 1290, 1243, 1199,
- 1159, 1086, 1021, 963, 911, 864, 821, 781,
- 745, 680, 623, 574, 530, 490, 455, 424,
- 395, 345, 304, 269, 239, 213, 190, 171,
- 154, 126, 104, 87, 73, 61, 52, 44,
- 38, 28, 21, 16, 12, 10, 8, 6,
- 5, 3, 2, 1, 1, 1, 0, 0,
- };
- // Normalized distortion:
- // This table models the normalized distortion for a Laplacian source
- // with given variance when quantized with a uniform quantizer
- // with given stepsize. The closed form expression is:
- // Dn(x) = 1 - 1/sqrt(2) * x / sinh(x/sqrt(2))
- // where x = qpstep / sqrt(variance).
- // Note the actual distortion is Dn * variance.
- static const int dist_tab_q10[] = {
- 0, 0, 1, 1, 1, 2, 2, 2,
- 3, 3, 4, 5, 5, 6, 7, 7,
- 8, 9, 11, 12, 13, 15, 16, 17,
- 18, 21, 24, 26, 29, 31, 34, 36,
- 39, 44, 49, 54, 59, 64, 69, 73,
- 78, 88, 97, 106, 115, 124, 133, 142,
- 151, 167, 184, 200, 215, 231, 245, 260,
- 274, 301, 327, 351, 375, 397, 418, 439,
- 458, 495, 528, 559, 587, 613, 637, 659,
- 680, 717, 749, 777, 801, 823, 842, 859,
- 874, 899, 919, 936, 949, 960, 969, 977,
- 983, 994, 1001, 1006, 1010, 1013, 1015, 1017,
- 1018, 1020, 1022, 1022, 1023, 1023, 1023, 1024,
- };
- static const int xsq_iq_q10[] = {
- 0, 4, 8, 12, 16, 20, 24, 28,
- 32, 40, 48, 56, 64, 72, 80, 88,
- 96, 112, 128, 144, 160, 176, 192, 208,
- 224, 256, 288, 320, 352, 384, 416, 448,
- 480, 544, 608, 672, 736, 800, 864, 928,
- 992, 1120, 1248, 1376, 1504, 1632, 1760, 1888,
- 2016, 2272, 2528, 2784, 3040, 3296, 3552, 3808,
- 4064, 4576, 5088, 5600, 6112, 6624, 7136, 7648,
- 8160, 9184, 10208, 11232, 12256, 13280, 14304, 15328,
- 16352, 18400, 20448, 22496, 24544, 26592, 28640, 30688,
- 32736, 36832, 40928, 45024, 49120, 53216, 57312, 61408,
- 65504, 73696, 81888, 90080, 98272, 106464, 114656, 122848,
- 131040, 147424, 163808, 180192, 196576, 212960, 229344, 245728,
- };
- const int tmp = (xsq_q10 >> 2) + 8;
- const int k = get_msb(tmp) - 3;
- const int xq = (k << 3) + ((tmp >> k) & 0x7);
- const int one_q10 = 1 << 10;
- const int a_q10 = ((xsq_q10 - xsq_iq_q10[xq]) << 10) >> (2 + k);
- const int b_q10 = one_q10 - a_q10;
- *r_q10 = (rate_tab_q10[xq] * b_q10 + rate_tab_q10[xq + 1] * a_q10) >> 10;
- *d_q10 = (dist_tab_q10[xq] * b_q10 + dist_tab_q10[xq + 1] * a_q10) >> 10;
-}
-
-void vp10_model_rd_from_var_lapndz(unsigned int var, unsigned int n_log2,
- unsigned int qstep, int *rate,
- int64_t *dist) {
- // This function models the rate and distortion for a Laplacian
- // source with given variance when quantized with a uniform quantizer
- // with given stepsize. The closed form expressions are in:
- // Hang and Chen, "Source Model for transform video coder and its
- // application - Part I: Fundamental Theory", IEEE Trans. Circ.
- // Sys. for Video Tech., April 1997.
- if (var == 0) {
- *rate = 0;
- *dist = 0;
- } else {
- int d_q10, r_q10;
- static const uint32_t MAX_XSQ_Q10 = 245727;
- const uint64_t xsq_q10_64 =
- (((uint64_t)qstep * qstep << (n_log2 + 10)) + (var >> 1)) / var;
- const int xsq_q10 = (int)VPXMIN(xsq_q10_64, MAX_XSQ_Q10);
- model_rd_norm(xsq_q10, &r_q10, &d_q10);
- *rate = ((r_q10 << n_log2) + 2) >> 2;
- *dist = (var * (int64_t)d_q10 + 512) >> 10;
- }
-}
-
-void vp10_get_entropy_contexts(BLOCK_SIZE bsize, TX_SIZE tx_size,
- const struct macroblockd_plane *pd,
- ENTROPY_CONTEXT t_above[16],
- ENTROPY_CONTEXT t_left[16]) {
- const BLOCK_SIZE plane_bsize = get_plane_block_size(bsize, pd);
- const int num_4x4_w = num_4x4_blocks_wide_lookup[plane_bsize];
- const int num_4x4_h = num_4x4_blocks_high_lookup[plane_bsize];
- const ENTROPY_CONTEXT *const above = pd->above_context;
- const ENTROPY_CONTEXT *const left = pd->left_context;
-
- int i;
- switch (tx_size) {
- case TX_4X4:
- memcpy(t_above, above, sizeof(ENTROPY_CONTEXT) * num_4x4_w);
- memcpy(t_left, left, sizeof(ENTROPY_CONTEXT) * num_4x4_h);
- break;
- case TX_8X8:
- for (i = 0; i < num_4x4_w; i += 2)
- t_above[i] = !!*(const uint16_t *)&above[i];
- for (i = 0; i < num_4x4_h; i += 2)
- t_left[i] = !!*(const uint16_t *)&left[i];
- break;
- case TX_16X16:
- for (i = 0; i < num_4x4_w; i += 4)
- t_above[i] = !!*(const uint32_t *)&above[i];
- for (i = 0; i < num_4x4_h; i += 4)
- t_left[i] = !!*(const uint32_t *)&left[i];
- break;
- case TX_32X32:
- for (i = 0; i < num_4x4_w; i += 8)
- t_above[i] = !!*(const uint64_t *)&above[i];
- for (i = 0; i < num_4x4_h; i += 8)
- t_left[i] = !!*(const uint64_t *)&left[i];
- break;
- default:
- assert(0 && "Invalid transform size.");
- break;
- }
-}
-
-void vp10_mv_pred(VP10_COMP *cpi, MACROBLOCK *x,
- uint8_t *ref_y_buffer, int ref_y_stride,
- int ref_frame, BLOCK_SIZE block_size) {
- int i;
- int zero_seen = 0;
- int best_index = 0;
- int best_sad = INT_MAX;
- int this_sad = INT_MAX;
- int max_mv = 0;
- int near_same_nearest;
- uint8_t *src_y_ptr = x->plane[0].src.buf;
- uint8_t *ref_y_ptr;
- const int num_mv_refs = MAX_MV_REF_CANDIDATES +
- (cpi->sf.adaptive_motion_search &&
- block_size < x->max_partition_size);
-
- MV pred_mv[3];
- pred_mv[0] = x->mbmi_ext->ref_mvs[ref_frame][0].as_mv;
- pred_mv[1] = x->mbmi_ext->ref_mvs[ref_frame][1].as_mv;
- pred_mv[2] = x->pred_mv[ref_frame];
- assert(num_mv_refs <= (int)(sizeof(pred_mv) / sizeof(pred_mv[0])));
-
- near_same_nearest =
- x->mbmi_ext->ref_mvs[ref_frame][0].as_int ==
- x->mbmi_ext->ref_mvs[ref_frame][1].as_int;
- // Get the sad for each candidate reference mv.
- for (i = 0; i < num_mv_refs; ++i) {
- const MV *this_mv = &pred_mv[i];
- int fp_row, fp_col;
-
- if (i == 1 && near_same_nearest)
- continue;
- fp_row = (this_mv->row + 3 + (this_mv->row >= 0)) >> 3;
- fp_col = (this_mv->col + 3 + (this_mv->col >= 0)) >> 3;
- max_mv = VPXMAX(max_mv, VPXMAX(abs(this_mv->row), abs(this_mv->col)) >> 3);
-
- if (fp_row ==0 && fp_col == 0 && zero_seen)
- continue;
- zero_seen |= (fp_row ==0 && fp_col == 0);
-
- ref_y_ptr =&ref_y_buffer[ref_y_stride * fp_row + fp_col];
- // Find sad for current vector.
- this_sad = cpi->fn_ptr[block_size].sdf(src_y_ptr, x->plane[0].src.stride,
- ref_y_ptr, ref_y_stride);
- // Note if it is the best so far.
- if (this_sad < best_sad) {
- best_sad = this_sad;
- best_index = i;
- }
- }
-
- // Note the index of the mv that worked best in the reference list.
- x->mv_best_ref_index[ref_frame] = best_index;
- x->max_mv_context[ref_frame] = max_mv;
- x->pred_mv_sad[ref_frame] = best_sad;
-}
-
-void vp10_setup_pred_block(const MACROBLOCKD *xd,
- struct buf_2d dst[MAX_MB_PLANE],
- const YV12_BUFFER_CONFIG *src,
- int mi_row, int mi_col,
- const struct scale_factors *scale,
- const struct scale_factors *scale_uv) {
- int i;
-
- dst[0].buf = src->y_buffer;
- dst[0].stride = src->y_stride;
- dst[1].buf = src->u_buffer;
- dst[2].buf = src->v_buffer;
- dst[1].stride = dst[2].stride = src->uv_stride;
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- setup_pred_plane(dst + i, dst[i].buf, dst[i].stride, mi_row, mi_col,
- i ? scale_uv : scale,
- xd->plane[i].subsampling_x, xd->plane[i].subsampling_y);
- }
-}
-
-int vp10_raster_block_offset(BLOCK_SIZE plane_bsize,
- int raster_block, int stride) {
- const int bw = b_width_log2_lookup[plane_bsize];
- const int y = 4 * (raster_block >> bw);
- const int x = 4 * (raster_block & ((1 << bw) - 1));
- return y * stride + x;
-}
-
-int16_t* vp10_raster_block_offset_int16(BLOCK_SIZE plane_bsize,
- int raster_block, int16_t *base) {
- const int stride = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- return base + vp10_raster_block_offset(plane_bsize, raster_block, stride);
-}
-
-YV12_BUFFER_CONFIG *vp10_get_scaled_ref_frame(const VP10_COMP *cpi,
- int ref_frame) {
- const VP10_COMMON *const cm = &cpi->common;
- const int scaled_idx = cpi->scaled_ref_idx[ref_frame - 1];
- const int ref_idx = get_ref_frame_buf_idx(cpi, ref_frame);
- return
- (scaled_idx != ref_idx && scaled_idx != INVALID_IDX) ?
- &cm->buffer_pool->frame_bufs[scaled_idx].buf : NULL;
-}
-
-int vp10_get_switchable_rate(const VP10_COMP *cpi,
- const MACROBLOCKD *const xd) {
- const MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- const int ctx = vp10_get_pred_context_switchable_interp(xd);
- return SWITCHABLE_INTERP_RATE_FACTOR *
- cpi->switchable_interp_costs[ctx][mbmi->interp_filter];
-}
-
-void vp10_set_rd_speed_thresholds(VP10_COMP *cpi) {
- int i;
- RD_OPT *const rd = &cpi->rd;
- SPEED_FEATURES *const sf = &cpi->sf;
-
- // Set baseline threshold values.
- for (i = 0; i < MAX_MODES; ++i)
- rd->thresh_mult[i] = cpi->oxcf.mode == BEST ? -500 : 0;
-
- if (sf->adaptive_rd_thresh) {
- rd->thresh_mult[THR_NEARESTMV] = 300;
- rd->thresh_mult[THR_NEARESTG] = 300;
- rd->thresh_mult[THR_NEARESTA] = 300;
- } else {
- rd->thresh_mult[THR_NEARESTMV] = 0;
- rd->thresh_mult[THR_NEARESTG] = 0;
- rd->thresh_mult[THR_NEARESTA] = 0;
- }
-
- rd->thresh_mult[THR_DC] += 1000;
-
- rd->thresh_mult[THR_NEWMV] += 1000;
- rd->thresh_mult[THR_NEWA] += 1000;
- rd->thresh_mult[THR_NEWG] += 1000;
-
- rd->thresh_mult[THR_NEARMV] += 1000;
- rd->thresh_mult[THR_NEARA] += 1000;
- rd->thresh_mult[THR_COMP_NEARESTLA] += 1000;
- rd->thresh_mult[THR_COMP_NEARESTGA] += 1000;
-
- rd->thresh_mult[THR_TM] += 1000;
-
- rd->thresh_mult[THR_COMP_NEARLA] += 1500;
- rd->thresh_mult[THR_COMP_NEWLA] += 2000;
- rd->thresh_mult[THR_NEARG] += 1000;
- rd->thresh_mult[THR_COMP_NEARGA] += 1500;
- rd->thresh_mult[THR_COMP_NEWGA] += 2000;
-
- rd->thresh_mult[THR_ZEROMV] += 2000;
- rd->thresh_mult[THR_ZEROG] += 2000;
- rd->thresh_mult[THR_ZEROA] += 2000;
- rd->thresh_mult[THR_COMP_ZEROLA] += 2500;
- rd->thresh_mult[THR_COMP_ZEROGA] += 2500;
-
- rd->thresh_mult[THR_H_PRED] += 2000;
- rd->thresh_mult[THR_V_PRED] += 2000;
- rd->thresh_mult[THR_D45_PRED ] += 2500;
- rd->thresh_mult[THR_D135_PRED] += 2500;
- rd->thresh_mult[THR_D117_PRED] += 2500;
- rd->thresh_mult[THR_D153_PRED] += 2500;
- rd->thresh_mult[THR_D207_PRED] += 2500;
- rd->thresh_mult[THR_D63_PRED] += 2500;
-}
-
-void vp10_set_rd_speed_thresholds_sub8x8(VP10_COMP *cpi) {
- static const int thresh_mult[2][MAX_REFS] =
- {{2500, 2500, 2500, 4500, 4500, 2500},
- {2000, 2000, 2000, 4000, 4000, 2000}};
- RD_OPT *const rd = &cpi->rd;
- const int idx = cpi->oxcf.mode == BEST;
- memcpy(rd->thresh_mult_sub8x8, thresh_mult[idx], sizeof(thresh_mult[idx]));
-}
-
-void vp10_update_rd_thresh_fact(int (*factor_buf)[MAX_MODES], int rd_thresh,
- int bsize, int best_mode_index) {
- if (rd_thresh > 0) {
- const int top_mode = bsize < BLOCK_8X8 ? MAX_REFS : MAX_MODES;
- int mode;
- for (mode = 0; mode < top_mode; ++mode) {
- const BLOCK_SIZE min_size = VPXMAX(bsize - 1, BLOCK_4X4);
- const BLOCK_SIZE max_size = VPXMIN(bsize + 2, BLOCK_64X64);
- BLOCK_SIZE bs;
- for (bs = min_size; bs <= max_size; ++bs) {
- int *const fact = &factor_buf[bs][mode];
- if (mode == best_mode_index) {
- *fact -= (*fact >> 4);
- } else {
- *fact = VPXMIN(*fact + RD_THRESH_INC, rd_thresh * RD_THRESH_MAX_FACT);
- }
- }
- }
- }
-}
-
-int vp10_get_intra_cost_penalty(int qindex, int qdelta,
- vpx_bit_depth_t bit_depth) {
- const int q = vp10_dc_quant(qindex, qdelta, bit_depth);
-#if CONFIG_VP9_HIGHBITDEPTH
- switch (bit_depth) {
- case VPX_BITS_8:
- return 20 * q;
- case VPX_BITS_10:
- return 5 * q;
- case VPX_BITS_12:
- return ROUND_POWER_OF_TWO(5 * q, 2);
- default:
- assert(0 && "bit_depth should be VPX_BITS_8, VPX_BITS_10 or VPX_BITS_12");
- return -1;
- }
-#else
- return 20 * q;
-#endif // CONFIG_VP9_HIGHBITDEPTH
-}
-
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_RD_H_
-#define VP10_ENCODER_RD_H_
-
-#include <limits.h>
-
-#include "vp10/common/blockd.h"
-
-#include "vp10/encoder/block.h"
-#include "vp10/encoder/context_tree.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define RDDIV_BITS 7
-
-#define RDCOST(RM, DM, R, D) \
- (((128 + ((int64_t)R) * (RM)) >> 8) + (D << DM))
-#define QIDX_SKIP_THRESH 115
-
-#define MV_COST_WEIGHT 108
-#define MV_COST_WEIGHT_SUB 120
-
-#define INVALID_MV 0x80008000
-
-#define MAX_MODES 30
-#define MAX_REFS 6
-
-#define RD_THRESH_MAX_FACT 64
-#define RD_THRESH_INC 1
-
-// This enumerator type needs to be kept aligned with the mode order in
-// const MODE_DEFINITION vp10_mode_order[MAX_MODES] used in the rd code.
-typedef enum {
- THR_NEARESTMV,
- THR_NEARESTA,
- THR_NEARESTG,
-
- THR_DC,
-
- THR_NEWMV,
- THR_NEWA,
- THR_NEWG,
-
- THR_NEARMV,
- THR_NEARA,
- THR_NEARG,
-
- THR_ZEROMV,
- THR_ZEROG,
- THR_ZEROA,
-
- THR_COMP_NEARESTLA,
- THR_COMP_NEARESTGA,
-
- THR_TM,
-
- THR_COMP_NEARLA,
- THR_COMP_NEWLA,
- THR_COMP_NEARGA,
- THR_COMP_NEWGA,
-
- THR_COMP_ZEROLA,
- THR_COMP_ZEROGA,
-
- THR_H_PRED,
- THR_V_PRED,
- THR_D135_PRED,
- THR_D207_PRED,
- THR_D153_PRED,
- THR_D63_PRED,
- THR_D117_PRED,
- THR_D45_PRED,
-} THR_MODES;
-
-typedef enum {
- THR_LAST,
- THR_GOLD,
- THR_ALTR,
- THR_COMP_LA,
- THR_COMP_GA,
- THR_INTRA,
-} THR_MODES_SUB8X8;
-
-typedef struct RD_OPT {
- // Thresh_mult is used to set a threshold for the rd score. A higher value
- // means that we will accept the best mode so far more often. This number
- // is used in combination with the current block size, and thresh_freq_fact
- // to pick a threshold.
- int thresh_mult[MAX_MODES];
- int thresh_mult_sub8x8[MAX_REFS];
-
- int threshes[MAX_SEGMENTS][BLOCK_SIZES][MAX_MODES];
-
- int64_t prediction_type_threshes[MAX_REF_FRAMES][REFERENCE_MODES];
-
- int64_t filter_threshes[MAX_REF_FRAMES][SWITCHABLE_FILTER_CONTEXTS];
-
- int RDMULT;
- int RDDIV;
-} RD_OPT;
-
-typedef struct RD_COST {
- int rate;
- int64_t dist;
- int64_t rdcost;
-} RD_COST;
-
-// Reset the rate distortion cost values to maximum (invalid) value.
-void vp10_rd_cost_reset(RD_COST *rd_cost);
-// Initialize the rate distortion cost values to zero.
-void vp10_rd_cost_init(RD_COST *rd_cost);
-
-struct TileInfo;
-struct TileDataEnc;
-struct VP10_COMP;
-struct macroblock;
-
-int vp10_compute_rd_mult(const struct VP10_COMP *cpi, int qindex);
-
-void vp10_initialize_rd_consts(struct VP10_COMP *cpi);
-
-void vp10_initialize_me_consts(struct VP10_COMP *cpi,
- MACROBLOCK *x, int qindex);
-
-void vp10_model_rd_from_var_lapndz(unsigned int var, unsigned int n,
- unsigned int qstep, int *rate,
- int64_t *dist);
-
-int vp10_get_switchable_rate(const struct VP10_COMP *cpi,
- const MACROBLOCKD *const xd);
-
-int vp10_raster_block_offset(BLOCK_SIZE plane_bsize,
- int raster_block, int stride);
-
-int16_t* vp10_raster_block_offset_int16(BLOCK_SIZE plane_bsize,
- int raster_block, int16_t *base);
-
-YV12_BUFFER_CONFIG *vp10_get_scaled_ref_frame(const struct VP10_COMP *cpi,
- int ref_frame);
-
-void vp10_init_me_luts(void);
-
-void vp10_get_entropy_contexts(BLOCK_SIZE bsize, TX_SIZE tx_size,
- const struct macroblockd_plane *pd,
- ENTROPY_CONTEXT t_above[16],
- ENTROPY_CONTEXT t_left[16]);
-
-void vp10_set_rd_speed_thresholds(struct VP10_COMP *cpi);
-
-void vp10_set_rd_speed_thresholds_sub8x8(struct VP10_COMP *cpi);
-
-void vp10_update_rd_thresh_fact(int (*fact)[MAX_MODES], int rd_thresh,
- int bsize, int best_mode_index);
-
-static INLINE int rd_less_than_thresh(int64_t best_rd, int thresh,
- int thresh_fact) {
- return best_rd < ((int64_t)thresh * thresh_fact >> 5) || thresh == INT_MAX;
-}
-
-void vp10_mv_pred(struct VP10_COMP *cpi, MACROBLOCK *x,
- uint8_t *ref_y_buffer, int ref_y_stride,
- int ref_frame, BLOCK_SIZE block_size);
-
-void vp10_setup_pred_block(const MACROBLOCKD *xd,
- struct buf_2d dst[MAX_MB_PLANE],
- const YV12_BUFFER_CONFIG *src,
- int mi_row, int mi_col,
- const struct scale_factors *scale,
- const struct scale_factors *scale_uv);
-
-int vp10_get_intra_cost_penalty(int qindex, int qdelta,
- vpx_bit_depth_t bit_depth);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_RD_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <math.h>
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/system_state.h"
-
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#include "vp10/common/entropymode.h"
-#include "vp10/common/idct.h"
-#include "vp10/common/mvref_common.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/common/reconintra.h"
-#include "vp10/common/scan.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/encodemb.h"
-#include "vp10/encoder/encodemv.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/encoder/palette.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/rd.h"
-#include "vp10/encoder/rdopt.h"
-#include "vp10/encoder/aq_variance.h"
-
-#define LAST_FRAME_MODE_MASK ((1 << GOLDEN_FRAME) | (1 << ALTREF_FRAME) | \
- (1 << INTRA_FRAME))
-#define GOLDEN_FRAME_MODE_MASK ((1 << LAST_FRAME) | (1 << ALTREF_FRAME) | \
- (1 << INTRA_FRAME))
-#define ALT_REF_MODE_MASK ((1 << LAST_FRAME) | (1 << GOLDEN_FRAME) | \
- (1 << INTRA_FRAME))
-
-#define SECOND_REF_FRAME_MASK ((1 << ALTREF_FRAME) | 0x01)
-
-#define MIN_EARLY_TERM_INDEX 3
-#define NEW_MV_DISCOUNT_FACTOR 8
-
-typedef struct {
- PREDICTION_MODE mode;
- MV_REFERENCE_FRAME ref_frame[2];
-} MODE_DEFINITION;
-
-typedef struct {
- MV_REFERENCE_FRAME ref_frame[2];
-} REF_DEFINITION;
-
-struct rdcost_block_args {
- MACROBLOCK *x;
- ENTROPY_CONTEXT t_above[16];
- ENTROPY_CONTEXT t_left[16];
- int this_rate;
- int64_t this_dist;
- int64_t this_sse;
- int64_t this_rd;
- int64_t best_rd;
- int exit_early;
- int use_fast_coef_costing;
- const scan_order *so;
- uint8_t skippable;
-};
-
-#define LAST_NEW_MV_INDEX 6
-static const MODE_DEFINITION vp10_mode_order[MAX_MODES] = {
- {NEARESTMV, {LAST_FRAME, NONE}},
- {NEARESTMV, {ALTREF_FRAME, NONE}},
- {NEARESTMV, {GOLDEN_FRAME, NONE}},
-
- {DC_PRED, {INTRA_FRAME, NONE}},
-
- {NEWMV, {LAST_FRAME, NONE}},
- {NEWMV, {ALTREF_FRAME, NONE}},
- {NEWMV, {GOLDEN_FRAME, NONE}},
-
- {NEARMV, {LAST_FRAME, NONE}},
- {NEARMV, {ALTREF_FRAME, NONE}},
- {NEARMV, {GOLDEN_FRAME, NONE}},
-
- {ZEROMV, {LAST_FRAME, NONE}},
- {ZEROMV, {GOLDEN_FRAME, NONE}},
- {ZEROMV, {ALTREF_FRAME, NONE}},
-
- {NEARESTMV, {LAST_FRAME, ALTREF_FRAME}},
- {NEARESTMV, {GOLDEN_FRAME, ALTREF_FRAME}},
-
- {TM_PRED, {INTRA_FRAME, NONE}},
-
- {NEARMV, {LAST_FRAME, ALTREF_FRAME}},
- {NEWMV, {LAST_FRAME, ALTREF_FRAME}},
- {NEARMV, {GOLDEN_FRAME, ALTREF_FRAME}},
- {NEWMV, {GOLDEN_FRAME, ALTREF_FRAME}},
-
- {ZEROMV, {LAST_FRAME, ALTREF_FRAME}},
- {ZEROMV, {GOLDEN_FRAME, ALTREF_FRAME}},
-
- {H_PRED, {INTRA_FRAME, NONE}},
- {V_PRED, {INTRA_FRAME, NONE}},
- {D135_PRED, {INTRA_FRAME, NONE}},
- {D207_PRED, {INTRA_FRAME, NONE}},
- {D153_PRED, {INTRA_FRAME, NONE}},
- {D63_PRED, {INTRA_FRAME, NONE}},
- {D117_PRED, {INTRA_FRAME, NONE}},
- {D45_PRED, {INTRA_FRAME, NONE}},
-};
-
-static const REF_DEFINITION vp10_ref_order[MAX_REFS] = {
- {{LAST_FRAME, NONE}},
- {{GOLDEN_FRAME, NONE}},
- {{ALTREF_FRAME, NONE}},
- {{LAST_FRAME, ALTREF_FRAME}},
- {{GOLDEN_FRAME, ALTREF_FRAME}},
- {{INTRA_FRAME, NONE}},
-};
-
-static INLINE int write_uniform_cost(int n, int v) {
- int l = get_unsigned_bits(n), m = (1 << l) - n;
- if (l == 0)
- return 0;
- if (v < m)
- return (l - 1) * vp10_cost_bit(128, 0);
- else
- return l * vp10_cost_bit(128, 0);
-}
-
-static void swap_block_ptr(MACROBLOCK *x, PICK_MODE_CONTEXT *ctx,
- int m, int n, int min_plane, int max_plane) {
- int i;
-
- for (i = min_plane; i < max_plane; ++i) {
- struct macroblock_plane *const p = &x->plane[i];
- struct macroblockd_plane *const pd = &x->e_mbd.plane[i];
-
- p->coeff = ctx->coeff_pbuf[i][m];
- p->qcoeff = ctx->qcoeff_pbuf[i][m];
- pd->dqcoeff = ctx->dqcoeff_pbuf[i][m];
- p->eobs = ctx->eobs_pbuf[i][m];
-
- ctx->coeff_pbuf[i][m] = ctx->coeff_pbuf[i][n];
- ctx->qcoeff_pbuf[i][m] = ctx->qcoeff_pbuf[i][n];
- ctx->dqcoeff_pbuf[i][m] = ctx->dqcoeff_pbuf[i][n];
- ctx->eobs_pbuf[i][m] = ctx->eobs_pbuf[i][n];
-
- ctx->coeff_pbuf[i][n] = p->coeff;
- ctx->qcoeff_pbuf[i][n] = p->qcoeff;
- ctx->dqcoeff_pbuf[i][n] = pd->dqcoeff;
- ctx->eobs_pbuf[i][n] = p->eobs;
- }
-}
-
-static void model_rd_for_sb(VP10_COMP *cpi, BLOCK_SIZE bsize,
- MACROBLOCK *x, MACROBLOCKD *xd,
- int *out_rate_sum, int64_t *out_dist_sum,
- int *skip_txfm_sb, int64_t *skip_sse_sb) {
- // Note our transform coeffs are 8 times an orthogonal transform.
- // Hence quantizer step is also 8 times. To get effective quantizer
- // we need to divide by 8 before sending to modeling function.
- int i;
- int64_t rate_sum = 0;
- int64_t dist_sum = 0;
- const int ref = xd->mi[0]->mbmi.ref_frame[0];
- unsigned int sse;
- unsigned int var = 0;
- unsigned int sum_sse = 0;
- int64_t total_sse = 0;
- int skip_flag = 1;
- const int shift = 6;
- int rate;
- int64_t dist;
- const int dequant_shift =
-#if CONFIG_VP9_HIGHBITDEPTH
- (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) ?
- xd->bd - 5 :
-#endif // CONFIG_VP9_HIGHBITDEPTH
- 3;
-
- x->pred_sse[ref] = 0;
-
- for (i = 0; i < MAX_MB_PLANE; ++i) {
- struct macroblock_plane *const p = &x->plane[i];
- struct macroblockd_plane *const pd = &xd->plane[i];
- const BLOCK_SIZE bs = get_plane_block_size(bsize, pd);
- const TX_SIZE max_tx_size = max_txsize_lookup[bs];
- const BLOCK_SIZE unit_size = txsize_to_bsize[max_tx_size];
- const int64_t dc_thr = p->quant_thred[0] >> shift;
- const int64_t ac_thr = p->quant_thred[1] >> shift;
- // The low thresholds are used to measure if the prediction errors are
- // low enough so that we can skip the mode search.
- const int64_t low_dc_thr = VPXMIN(50, dc_thr >> 2);
- const int64_t low_ac_thr = VPXMIN(80, ac_thr >> 2);
- int bw = 1 << (b_width_log2_lookup[bs] - b_width_log2_lookup[unit_size]);
- int bh = 1 << (b_height_log2_lookup[bs] - b_width_log2_lookup[unit_size]);
- int idx, idy;
- int lw = b_width_log2_lookup[unit_size] + 2;
- int lh = b_height_log2_lookup[unit_size] + 2;
-
- sum_sse = 0;
-
- for (idy = 0; idy < bh; ++idy) {
- for (idx = 0; idx < bw; ++idx) {
- uint8_t *src = p->src.buf + (idy * p->src.stride << lh) + (idx << lw);
- uint8_t *dst = pd->dst.buf + (idy * pd->dst.stride << lh) + (idx << lh);
- int block_idx = (idy << 1) + idx;
- int low_err_skip = 0;
-
- var = cpi->fn_ptr[unit_size].vf(src, p->src.stride,
- dst, pd->dst.stride, &sse);
- x->bsse[(i << 2) + block_idx] = sse;
- sum_sse += sse;
-
- x->skip_txfm[(i << 2) + block_idx] = SKIP_TXFM_NONE;
- if (!x->select_tx_size) {
- // Check if all ac coefficients can be quantized to zero.
- if (var < ac_thr || var == 0) {
- x->skip_txfm[(i << 2) + block_idx] = SKIP_TXFM_AC_ONLY;
-
- // Check if dc coefficient can be quantized to zero.
- if (sse - var < dc_thr || sse == var) {
- x->skip_txfm[(i << 2) + block_idx] = SKIP_TXFM_AC_DC;
-
- if (!sse || (var < low_ac_thr && sse - var < low_dc_thr))
- low_err_skip = 1;
- }
- }
- }
-
- if (skip_flag && !low_err_skip)
- skip_flag = 0;
-
- if (i == 0)
- x->pred_sse[ref] += sse;
- }
- }
-
- total_sse += sum_sse;
-
- // Fast approximate the modelling function.
- if (cpi->sf.simple_model_rd_from_var) {
- int64_t rate;
- const int64_t square_error = sum_sse;
- int quantizer = (pd->dequant[1] >> dequant_shift);
-
- if (quantizer < 120)
- rate = (square_error * (280 - quantizer)) >> 8;
- else
- rate = 0;
- dist = (square_error * quantizer) >> 8;
- rate_sum += rate;
- dist_sum += dist;
- } else {
- vp10_model_rd_from_var_lapndz(sum_sse, num_pels_log2_lookup[bs],
- pd->dequant[1] >> dequant_shift,
- &rate, &dist);
- rate_sum += rate;
- dist_sum += dist;
- }
- }
-
- *skip_txfm_sb = skip_flag;
- *skip_sse_sb = total_sse << 4;
- *out_rate_sum = (int)rate_sum;
- *out_dist_sum = dist_sum << 4;
-}
-
-int64_t vp10_block_error_c(const tran_low_t *coeff, const tran_low_t *dqcoeff,
- intptr_t block_size, int64_t *ssz) {
- int i;
- int64_t error = 0, sqcoeff = 0;
-
- for (i = 0; i < block_size; i++) {
- const int diff = coeff[i] - dqcoeff[i];
- error += diff * diff;
- sqcoeff += coeff[i] * coeff[i];
- }
-
- *ssz = sqcoeff;
- return error;
-}
-
-int64_t vp10_block_error_fp_c(const int16_t *coeff, const int16_t *dqcoeff,
- int block_size) {
- int i;
- int64_t error = 0;
-
- for (i = 0; i < block_size; i++) {
- const int diff = coeff[i] - dqcoeff[i];
- error += diff * diff;
- }
-
- return error;
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-int64_t vp10_highbd_block_error_c(const tran_low_t *coeff,
- const tran_low_t *dqcoeff,
- intptr_t block_size,
- int64_t *ssz, int bd) {
- int i;
- int64_t error = 0, sqcoeff = 0;
- int shift = 2 * (bd - 8);
- int rounding = shift > 0 ? 1 << (shift - 1) : 0;
-
- for (i = 0; i < block_size; i++) {
- const int64_t diff = coeff[i] - dqcoeff[i];
- error += diff * diff;
- sqcoeff += (int64_t)coeff[i] * (int64_t)coeff[i];
- }
- assert(error >= 0 && sqcoeff >= 0);
- error = (error + rounding) >> shift;
- sqcoeff = (sqcoeff + rounding) >> shift;
-
- *ssz = sqcoeff;
- return error;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-/* The trailing '0' is a terminator which is used inside cost_coeffs() to
- * decide whether to include cost of a trailing EOB node or not (i.e. we
- * can skip this if the last coefficient in this transform block, e.g. the
- * 16th coefficient in a 4x4 block or the 64th coefficient in a 8x8 block,
- * were non-zero). */
-static const int16_t band_counts[TX_SIZES][8] = {
- { 1, 2, 3, 4, 3, 16 - 13, 0 },
- { 1, 2, 3, 4, 11, 64 - 21, 0 },
- { 1, 2, 3, 4, 11, 256 - 21, 0 },
- { 1, 2, 3, 4, 11, 1024 - 21, 0 },
-};
-static int cost_coeffs(MACROBLOCK *x,
- int plane, int block,
- ENTROPY_CONTEXT *A, ENTROPY_CONTEXT *L,
- TX_SIZE tx_size,
- const int16_t *scan, const int16_t *nb,
- int use_fast_coef_costing) {
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- const struct macroblock_plane *p = &x->plane[plane];
- const struct macroblockd_plane *pd = &xd->plane[plane];
- const PLANE_TYPE type = pd->plane_type;
- const int16_t *band_count = &band_counts[tx_size][1];
- const int eob = p->eobs[block];
- const tran_low_t *const qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- unsigned int (*token_costs)[2][COEFF_CONTEXTS][ENTROPY_TOKENS] =
- x->token_costs[tx_size][type][is_inter_block(mbmi)];
- uint8_t token_cache[32 * 32];
- int pt = combine_entropy_contexts(*A, *L);
- int c, cost;
-#if CONFIG_VP9_HIGHBITDEPTH
- const int16_t *cat6_high_cost = vp10_get_high_cost_table(xd->bd);
-#else
- const int16_t *cat6_high_cost = vp10_get_high_cost_table(8);
-#endif
-
- // Check for consistency of tx_size with mode info
- assert(type == PLANE_TYPE_Y ? mbmi->tx_size == tx_size
- : get_uv_tx_size(mbmi, pd) == tx_size);
-
- if (eob == 0) {
- // single eob token
- cost = token_costs[0][0][pt][EOB_TOKEN];
- c = 0;
- } else {
- int band_left = *band_count++;
-
- // dc token
- int v = qcoeff[0];
- int16_t prev_t;
- EXTRABIT e;
- vp10_get_token_extra(v, &prev_t, &e);
- cost = (*token_costs)[0][pt][prev_t] +
- vp10_get_cost(prev_t, e, cat6_high_cost);
-
- token_cache[0] = vp10_pt_energy_class[prev_t];
- ++token_costs;
-
- // ac tokens
- for (c = 1; c < eob; c++) {
- const int rc = scan[c];
- int16_t t;
-
- v = qcoeff[rc];
- vp10_get_token_extra(v, &t, &e);
- if (use_fast_coef_costing) {
- cost += (*token_costs)[!prev_t][!prev_t][t] +
- vp10_get_cost(t, e, cat6_high_cost);
- } else {
- pt = get_coef_context(nb, token_cache, c);
- cost += (*token_costs)[!prev_t][pt][t] +
- vp10_get_cost(t, e, cat6_high_cost);
- token_cache[rc] = vp10_pt_energy_class[t];
- }
- prev_t = t;
- if (!--band_left) {
- band_left = *band_count++;
- ++token_costs;
- }
- }
-
- // eob token
- if (band_left) {
- if (use_fast_coef_costing) {
- cost += (*token_costs)[0][!prev_t][EOB_TOKEN];
- } else {
- pt = get_coef_context(nb, token_cache, c);
- cost += (*token_costs)[0][pt][EOB_TOKEN];
- }
- }
- }
-
- // is eob first coefficient;
- *A = *L = (c > 0);
-
- return cost;
-}
-
-static void dist_block(MACROBLOCK *x, int plane, int block, TX_SIZE tx_size,
- int64_t *out_dist, int64_t *out_sse) {
- const int ss_txfrm_size = tx_size << 1;
- MACROBLOCKD* const xd = &x->e_mbd;
- const struct macroblock_plane *const p = &x->plane[plane];
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- int64_t this_sse;
- int shift = tx_size == TX_32X32 ? 0 : 2;
- tran_low_t *const coeff = BLOCK_OFFSET(p->coeff, block);
- tran_low_t *const dqcoeff = BLOCK_OFFSET(pd->dqcoeff, block);
-#if CONFIG_VP9_HIGHBITDEPTH
- const int bd = (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) ? xd->bd : 8;
- *out_dist = vp10_highbd_block_error(coeff, dqcoeff, 16 << ss_txfrm_size,
- &this_sse, bd) >> shift;
-#else
- *out_dist = vp10_block_error(coeff, dqcoeff, 16 << ss_txfrm_size,
- &this_sse) >> shift;
-#endif // CONFIG_VP9_HIGHBITDEPTH
- *out_sse = this_sse >> shift;
-}
-
-static int rate_block(int plane, int block, int blk_row, int blk_col,
- TX_SIZE tx_size, struct rdcost_block_args* args) {
- return cost_coeffs(args->x, plane, block, args->t_above + blk_col,
- args->t_left + blk_row, tx_size,
- args->so->scan, args->so->neighbors,
- args->use_fast_coef_costing);
-}
-
-static void block_rd_txfm(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg) {
- struct rdcost_block_args *args = arg;
- MACROBLOCK *const x = args->x;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- int64_t rd1, rd2, rd;
- int rate;
- int64_t dist;
- int64_t sse;
-
- if (args->exit_early)
- return;
-
- if (!is_inter_block(mbmi)) {
- struct encode_b_args arg = {x, NULL, &mbmi->skip};
- vp10_encode_block_intra(plane, block, blk_row, blk_col,
- plane_bsize, tx_size, &arg);
- dist_block(x, plane, block, tx_size, &dist, &sse);
- } else if (max_txsize_lookup[plane_bsize] == tx_size) {
- if (x->skip_txfm[(plane << 2) + (block >> (tx_size << 1))] ==
- SKIP_TXFM_NONE) {
- // full forward transform and quantization
- vp10_xform_quant(x, plane, block, blk_row, blk_col,
- plane_bsize, tx_size);
- dist_block(x, plane, block, tx_size, &dist, &sse);
- } else if (x->skip_txfm[(plane << 2) + (block >> (tx_size << 1))] ==
- SKIP_TXFM_AC_ONLY) {
- // compute DC coefficient
- tran_low_t *const coeff = BLOCK_OFFSET(x->plane[plane].coeff, block);
- tran_low_t *const dqcoeff = BLOCK_OFFSET(xd->plane[plane].dqcoeff, block);
- vp10_xform_quant_dc(x, plane, block, blk_row, blk_col,
- plane_bsize, tx_size);
- sse = x->bsse[(plane << 2) + (block >> (tx_size << 1))] << 4;
- dist = sse;
- if (x->plane[plane].eobs[block]) {
- const int64_t orig_sse = (int64_t)coeff[0] * coeff[0];
- const int64_t resd_sse = coeff[0] - dqcoeff[0];
- int64_t dc_correct = orig_sse - resd_sse * resd_sse;
-#if CONFIG_VP9_HIGHBITDEPTH
- dc_correct >>= ((xd->bd - 8) * 2);
-#endif
- if (tx_size != TX_32X32)
- dc_correct >>= 2;
-
- dist = VPXMAX(0, sse - dc_correct);
- }
- } else {
- // SKIP_TXFM_AC_DC
- // skip forward transform
- x->plane[plane].eobs[block] = 0;
- sse = x->bsse[(plane << 2) + (block >> (tx_size << 1))] << 4;
- dist = sse;
- }
- } else {
- // full forward transform and quantization
- vp10_xform_quant(x, plane, block, blk_row, blk_col, plane_bsize, tx_size);
- dist_block(x, plane, block, tx_size, &dist, &sse);
- }
-
- rd = RDCOST(x->rdmult, x->rddiv, 0, dist);
- if (args->this_rd + rd > args->best_rd) {
- args->exit_early = 1;
- return;
- }
-
- rate = rate_block(plane, block, blk_row, blk_col, tx_size, args);
- rd1 = RDCOST(x->rdmult, x->rddiv, rate, dist);
- rd2 = RDCOST(x->rdmult, x->rddiv, 0, sse);
-
- // TODO(jingning): temporarily enabled only for luma component
- rd = VPXMIN(rd1, rd2);
- if (plane == 0)
- x->zcoeff_blk[tx_size][block] = !x->plane[plane].eobs[block] ||
- (rd1 > rd2 && !xd->lossless[mbmi->segment_id]);
-
- args->this_rate += rate;
- args->this_dist += dist;
- args->this_sse += sse;
- args->this_rd += rd;
-
- if (args->this_rd > args->best_rd) {
- args->exit_early = 1;
- return;
- }
-
- args->skippable &= !x->plane[plane].eobs[block];
-}
-
-static void txfm_rd_in_plane(MACROBLOCK *x,
- int *rate, int64_t *distortion,
- int *skippable, int64_t *sse,
- int64_t ref_best_rd, int plane,
- BLOCK_SIZE bsize, TX_SIZE tx_size,
- int use_fast_coef_casting) {
- MACROBLOCKD *const xd = &x->e_mbd;
- const struct macroblockd_plane *const pd = &xd->plane[plane];
- TX_TYPE tx_type;
- struct rdcost_block_args args;
- vp10_zero(args);
- args.x = x;
- args.best_rd = ref_best_rd;
- args.use_fast_coef_costing = use_fast_coef_casting;
- args.skippable = 1;
-
- if (plane == 0)
- xd->mi[0]->mbmi.tx_size = tx_size;
-
- vp10_get_entropy_contexts(bsize, tx_size, pd, args.t_above, args.t_left);
-
- tx_type = get_tx_type(pd->plane_type, xd, 0);
- args.so = get_scan(tx_size, tx_type);
-
- vp10_foreach_transformed_block_in_plane(xd, bsize, plane,
- block_rd_txfm, &args);
- if (args.exit_early) {
- *rate = INT_MAX;
- *distortion = INT64_MAX;
- *sse = INT64_MAX;
- *skippable = 0;
- } else {
- *distortion = args.this_dist;
- *rate = args.this_rate;
- *sse = args.this_sse;
- *skippable = args.skippable;
- }
-}
-
-static void choose_largest_tx_size(VP10_COMP *cpi, MACROBLOCK *x,
- int *rate, int64_t *distortion,
- int *skip, int64_t *sse,
- int64_t ref_best_rd,
- BLOCK_SIZE bs) {
- const TX_SIZE max_tx_size = max_txsize_lookup[bs];
- VP10_COMMON *const cm = &cpi->common;
- const TX_SIZE largest_tx_size = tx_mode_to_biggest_tx_size[cm->tx_mode];
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
-
- mbmi->tx_size = VPXMIN(max_tx_size, largest_tx_size);
-
- txfm_rd_in_plane(x, rate, distortion, skip,
- sse, ref_best_rd, 0, bs,
- mbmi->tx_size, cpi->sf.use_fast_coef_costing);
-}
-
-static void choose_smallest_tx_size(VP10_COMP *cpi, MACROBLOCK *x,
- int *rate, int64_t *distortion,
- int *skip, int64_t *sse,
- int64_t ref_best_rd,
- BLOCK_SIZE bs) {
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
-
- mbmi->tx_size = TX_4X4;
-
- txfm_rd_in_plane(x, rate, distortion, skip,
- sse, ref_best_rd, 0, bs,
- mbmi->tx_size, cpi->sf.use_fast_coef_costing);
-}
-
-static void choose_tx_size_from_rd(VP10_COMP *cpi, MACROBLOCK *x,
- int *rate,
- int64_t *distortion,
- int *skip,
- int64_t *psse,
- int64_t ref_best_rd,
- BLOCK_SIZE bs) {
- const TX_SIZE max_tx_size = max_txsize_lookup[bs];
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- vpx_prob skip_prob = vp10_get_skip_prob(cm, xd);
- int r[TX_SIZES][2], s[TX_SIZES];
- int64_t d[TX_SIZES], sse[TX_SIZES];
- int64_t rd[TX_SIZES][2] = {{INT64_MAX, INT64_MAX},
- {INT64_MAX, INT64_MAX},
- {INT64_MAX, INT64_MAX},
- {INT64_MAX, INT64_MAX}};
- int n, m;
- int s0, s1;
- int64_t best_rd = INT64_MAX;
- TX_SIZE best_tx = max_tx_size;
- int start_tx, end_tx;
-
- const vpx_prob *tx_probs = get_tx_probs2(max_tx_size, xd, &cm->fc->tx_probs);
- assert(skip_prob > 0);
- s0 = vp10_cost_bit(skip_prob, 0);
- s1 = vp10_cost_bit(skip_prob, 1);
-
- if (cm->tx_mode == TX_MODE_SELECT) {
- start_tx = max_tx_size;
- end_tx = 0;
- } else {
- TX_SIZE chosen_tx_size = VPXMIN(max_tx_size,
- tx_mode_to_biggest_tx_size[cm->tx_mode]);
- start_tx = chosen_tx_size;
- end_tx = chosen_tx_size;
- }
-
- for (n = start_tx; n >= end_tx; n--) {
- int r_tx_size = 0;
- for (m = 0; m <= n - (n == (int) max_tx_size); m++) {
- if (m == n)
- r_tx_size += vp10_cost_zero(tx_probs[m]);
- else
- r_tx_size += vp10_cost_one(tx_probs[m]);
- }
- txfm_rd_in_plane(x, &r[n][0], &d[n], &s[n],
- &sse[n], ref_best_rd, 0, bs, n,
- cpi->sf.use_fast_coef_costing);
- r[n][1] = r[n][0];
- if (r[n][0] < INT_MAX) {
- r[n][1] += r_tx_size;
- }
- if (d[n] == INT64_MAX || r[n][0] == INT_MAX) {
- rd[n][0] = rd[n][1] = INT64_MAX;
- } else if (s[n]) {
- if (is_inter_block(mbmi)) {
- rd[n][0] = rd[n][1] = RDCOST(x->rdmult, x->rddiv, s1, sse[n]);
- r[n][1] -= r_tx_size;
- } else {
- rd[n][0] = RDCOST(x->rdmult, x->rddiv, s1, sse[n]);
- rd[n][1] = RDCOST(x->rdmult, x->rddiv, s1 + r_tx_size, sse[n]);
- }
- } else {
- rd[n][0] = RDCOST(x->rdmult, x->rddiv, r[n][0] + s0, d[n]);
- rd[n][1] = RDCOST(x->rdmult, x->rddiv, r[n][1] + s0, d[n]);
- }
-
- if (is_inter_block(mbmi) && !xd->lossless[mbmi->segment_id] &&
- !s[n] && sse[n] != INT64_MAX) {
- rd[n][0] = VPXMIN(rd[n][0], RDCOST(x->rdmult, x->rddiv, s1, sse[n]));
- rd[n][1] = VPXMIN(rd[n][1], RDCOST(x->rdmult, x->rddiv, s1, sse[n]));
- }
-
- // Early termination in transform size search.
- if (cpi->sf.tx_size_search_breakout &&
- (rd[n][1] == INT64_MAX ||
- (n < (int) max_tx_size && rd[n][1] > rd[n + 1][1]) ||
- s[n] == 1))
- break;
-
- if (rd[n][1] < best_rd) {
- best_tx = n;
- best_rd = rd[n][1];
- }
- }
- mbmi->tx_size = best_tx;
-
- *distortion = d[mbmi->tx_size];
- *rate = r[mbmi->tx_size][cm->tx_mode == TX_MODE_SELECT];
- *skip = s[mbmi->tx_size];
- *psse = sse[mbmi->tx_size];
-}
-
-static void super_block_yrd(VP10_COMP *cpi, MACROBLOCK *x, int *rate,
- int64_t *distortion, int *skip,
- int64_t *psse, BLOCK_SIZE bs,
- int64_t ref_best_rd) {
- MACROBLOCKD *xd = &x->e_mbd;
- int64_t sse;
- int64_t *ret_sse = psse ? psse : &sse;
-
- assert(bs == xd->mi[0]->mbmi.sb_type);
-
- if (CONFIG_MISC_FIXES && xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- choose_smallest_tx_size(cpi, x, rate, distortion, skip, ret_sse,
- ref_best_rd, bs);
- } else if (cpi->sf.tx_size_search_method == USE_LARGESTALL ||
- xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- choose_largest_tx_size(cpi, x, rate, distortion, skip, ret_sse, ref_best_rd,
- bs);
- } else {
- choose_tx_size_from_rd(cpi, x, rate, distortion, skip, ret_sse,
- ref_best_rd, bs);
- }
-}
-
-static int conditional_skipintra(PREDICTION_MODE mode,
- PREDICTION_MODE best_intra_mode) {
- if (mode == D117_PRED &&
- best_intra_mode != V_PRED &&
- best_intra_mode != D135_PRED)
- return 1;
- if (mode == D63_PRED &&
- best_intra_mode != V_PRED &&
- best_intra_mode != D45_PRED)
- return 1;
- if (mode == D207_PRED &&
- best_intra_mode != H_PRED &&
- best_intra_mode != D45_PRED)
- return 1;
- if (mode == D153_PRED &&
- best_intra_mode != H_PRED &&
- best_intra_mode != D135_PRED)
- return 1;
- return 0;
-}
-
-void rd_pick_palette_intra_sby(VP10_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bsize,
- int palette_ctx, int dc_mode_cost,
- PALETTE_MODE_INFO *palette_mode_info,
- uint8_t *best_palette_color_map,
- TX_SIZE *best_tx, PREDICTION_MODE *mode_selected,
- int64_t *best_rd) {
- MACROBLOCKD *const xd = &x->e_mbd;
- MODE_INFO *const mic = xd->mi[0];
- int rows = 4 * num_4x4_blocks_high_lookup[bsize];
- int cols = 4 * num_4x4_blocks_wide_lookup[bsize];
- int this_rate, this_rate_tokenonly, s;
- int64_t this_distortion, this_rd;
- int colors, n;
- int src_stride = x->plane[0].src.stride;
- uint8_t *src = x->plane[0].src.buf;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cpi->common.use_highbitdepth)
- colors = vp10_count_colors_highbd(src, src_stride, rows, cols,
- cpi->common.bit_depth);
- else
-#endif // CONFIG_VP9_HIGHBITDEPTH
- colors = vp10_count_colors(src, src_stride, rows, cols);
- palette_mode_info->palette_size[0] = 0;
-
- if (colors > 1 && colors <= 64 && cpi->common.allow_screen_content_tools) {
- int r, c, i, j, k;
- int max_itr = 50;
- int color_ctx, color_idx = 0;
- int color_order[PALETTE_MAX_SIZE];
- double *data = x->palette_buffer->kmeans_data_buf;
- uint8_t *indices = x->palette_buffer->kmeans_indices_buf;
- uint8_t *pre_indices = x->palette_buffer->kmeans_pre_indices_buf;
- double centroids[PALETTE_MAX_SIZE];
- uint8_t *color_map;
- double lb, ub, val;
- PALETTE_MODE_INFO *pmi = &mic->mbmi.palette_mode_info;
-#if CONFIG_VP9_HIGHBITDEPTH
- uint16_t *src16 = CONVERT_TO_SHORTPTR(src);
- if (cpi->common.use_highbitdepth)
- lb = ub = src16[0];
- else
-#endif // CONFIG_VP9_HIGHBITDEPTH
- lb = ub = src[0];
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cpi->common.use_highbitdepth) {
- for (r = 0; r < rows; ++r) {
- for (c = 0; c < cols; ++c) {
- val = src16[r * src_stride + c];
- data[r * cols + c] = val;
- if (val < lb)
- lb = val;
- else if (val > ub)
- ub = val;
- }
- }
- } else {
-#endif // CONFIG_VP9_HIGHBITDEPTH
- for (r = 0; r < rows; ++r) {
- for (c = 0; c < cols; ++c) {
- val = src[r * src_stride + c];
- data[r * cols + c] = val;
- if (val < lb)
- lb = val;
- else if (val > ub)
- ub = val;
- }
- }
-#if CONFIG_VP9_HIGHBITDEPTH
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- mic->mbmi.mode = DC_PRED;
-
- for (n = colors > PALETTE_MAX_SIZE ? PALETTE_MAX_SIZE : colors;
- n >= 2; --n) {
- for (i = 0; i < n; ++i)
- centroids[i] = lb + (2 * i + 1) * (ub - lb) / n / 2;
- vp10_k_means(data, centroids, indices, pre_indices, rows * cols,
- n, 1, max_itr);
- vp10_insertion_sort(centroids, n);
- for (i = 0; i < n; ++i)
- centroids[i] = round(centroids[i]);
- // remove duplicates
- i = 1;
- k = n;
- while (i < k) {
- if (centroids[i] == centroids[i - 1]) {
- j = i;
- while (j < k - 1) {
- centroids[j] = centroids[j + 1];
- ++j;
- }
- --k;
- } else {
- ++i;
- }
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (cpi->common.use_highbitdepth)
- for (i = 0; i < k; ++i)
- mic->mbmi.palette_mode_info.palette_colors[i] =
- clip_pixel_highbd(round(centroids[i]), cpi->common.bit_depth);
- else
-#endif // CONFIG_VP9_HIGHBITDEPTH
- for (i = 0; i < k; ++i)
- pmi->palette_colors[i] = clip_pixel((int)round(centroids[i]));
- pmi->palette_size[0] = k;
-
- vp10_calc_indices(data, centroids, indices, rows * cols, k, 1);
- for (r = 0; r < rows; ++r)
- for (c = 0; c < cols; ++c)
- xd->plane[0].color_index_map[r * cols + c] = indices[r * cols + c];
-
- super_block_yrd(cpi, x, &this_rate_tokenonly, &this_distortion,
- &s, NULL, bsize, *best_rd);
- if (this_rate_tokenonly == INT_MAX)
- continue;
-
- this_rate = this_rate_tokenonly + dc_mode_cost +
- cpi->common.bit_depth * k * vp10_cost_bit(128, 0) +
- cpi->palette_y_size_cost[bsize - BLOCK_8X8][k - 2];
- this_rate +=
- vp10_cost_bit(vp10_default_palette_y_mode_prob[bsize - BLOCK_8X8]
- [palette_ctx], 1);
- color_map = xd->plane[0].color_index_map;
- this_rate += write_uniform_cost(k, xd->plane[0].color_index_map[0]);
- for (i = 0; i < rows; ++i) {
- for (j = (i == 0 ? 1 : 0); j < cols; ++j) {
- color_ctx = vp10_get_palette_color_context(color_map, cols, i, j,
- k, color_order);
- for (r = 0; r < k; ++r)
- if (color_map[i * cols + j] == color_order[r]) {
- color_idx = r;
- break;
- }
- assert(color_idx < k);
- this_rate +=
- cpi->palette_y_color_cost[k - 2][color_ctx][color_idx];
- }
- }
- this_rd = RDCOST(x->rdmult, x->rddiv, this_rate, this_distortion);
-
- if (this_rd < *best_rd) {
- *best_rd = this_rd;
- *palette_mode_info = mic->mbmi.palette_mode_info;
- memcpy(best_palette_color_map, xd->plane[0].color_index_map,
- rows * cols * sizeof(xd->plane[0].color_index_map[0]));
- *mode_selected = DC_PRED;
- *best_tx = mic->mbmi.tx_size;
- }
- }
- }
-}
-
-static int64_t rd_pick_intra4x4block(VP10_COMP *cpi, MACROBLOCK *x,
- int row, int col,
- PREDICTION_MODE *best_mode,
- const int *bmode_costs,
- ENTROPY_CONTEXT *a, ENTROPY_CONTEXT *l,
- int *bestrate, int *bestratey,
- int64_t *bestdistortion,
- BLOCK_SIZE bsize, int64_t rd_thresh) {
- PREDICTION_MODE mode;
- MACROBLOCKD *const xd = &x->e_mbd;
- int64_t best_rd = rd_thresh;
- struct macroblock_plane *p = &x->plane[0];
- struct macroblockd_plane *pd = &xd->plane[0];
- const int src_stride = p->src.stride;
- const int dst_stride = pd->dst.stride;
- const uint8_t *src_init = &p->src.buf[row * 4 * src_stride + col * 4];
- uint8_t *dst_init = &pd->dst.buf[row * 4 * src_stride + col * 4];
- ENTROPY_CONTEXT ta[2], tempa[2];
- ENTROPY_CONTEXT tl[2], templ[2];
- const int num_4x4_blocks_wide = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_blocks_high = num_4x4_blocks_high_lookup[bsize];
- int idx, idy;
- uint8_t best_dst[8 * 8];
-#if CONFIG_VP9_HIGHBITDEPTH
- uint16_t best_dst16[8 * 8];
-#endif
-
- memcpy(ta, a, sizeof(ta));
- memcpy(tl, l, sizeof(tl));
- xd->mi[0]->mbmi.tx_size = TX_4X4;
- xd->mi[0]->mbmi.palette_mode_info.palette_size[0] = 0;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- for (mode = DC_PRED; mode <= TM_PRED; ++mode) {
- int64_t this_rd;
- int ratey = 0;
- int64_t distortion = 0;
- int rate = bmode_costs[mode];
-
- if (!(cpi->sf.intra_y_mode_mask[TX_4X4] & (1 << mode)))
- continue;
-
- // Only do the oblique modes if the best so far is
- // one of the neighboring directional modes
- if (cpi->sf.mode_search_skip_flags & FLAG_SKIP_INTRA_DIRMISMATCH) {
- if (conditional_skipintra(mode, *best_mode))
- continue;
- }
-
- memcpy(tempa, ta, sizeof(ta));
- memcpy(templ, tl, sizeof(tl));
-
- for (idy = 0; idy < num_4x4_blocks_high; ++idy) {
- for (idx = 0; idx < num_4x4_blocks_wide; ++idx) {
- const int block = (row + idy) * 2 + (col + idx);
- const uint8_t *const src = &src_init[idx * 4 + idy * 4 * src_stride];
- uint8_t *const dst = &dst_init[idx * 4 + idy * 4 * dst_stride];
- int16_t *const src_diff = vp10_raster_block_offset_int16(BLOCK_8X8,
- block,
- p->src_diff);
- tran_low_t *const coeff = BLOCK_OFFSET(x->plane[0].coeff, block);
- xd->mi[0]->bmi[block].as_mode = mode;
- vp10_predict_intra_block(xd, 1, 1, TX_4X4, mode, dst, dst_stride,
- dst, dst_stride,
- col + idx, row + idy, 0);
- vpx_highbd_subtract_block(4, 4, src_diff, 8, src, src_stride,
- dst, dst_stride, xd->bd);
- if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- TX_TYPE tx_type = get_tx_type(PLANE_TYPE_Y, xd, block);
- const scan_order *so = get_scan(TX_4X4, tx_type);
- vp10_highbd_fwd_txfm_4x4(src_diff, coeff, 8, DCT_DCT, 1);
- vp10_regular_quantize_b_4x4(x, 0, block, so->scan, so->iscan);
- ratey += cost_coeffs(x, 0, block, tempa + idx, templ + idy, TX_4X4,
- so->scan, so->neighbors,
- cpi->sf.use_fast_coef_costing);
- if (RDCOST(x->rdmult, x->rddiv, ratey, distortion) >= best_rd)
- goto next_highbd;
- vp10_highbd_inv_txfm_add_4x4(BLOCK_OFFSET(pd->dqcoeff, block),
- dst, dst_stride, p->eobs[block],
- xd->bd, DCT_DCT, 1);
- } else {
- int64_t unused;
- TX_TYPE tx_type = get_tx_type(PLANE_TYPE_Y, xd, block);
- const scan_order *so = get_scan(TX_4X4, tx_type);
- vp10_highbd_fwd_txfm_4x4(src_diff, coeff, 8, tx_type, 0);
- vp10_regular_quantize_b_4x4(x, 0, block, so->scan, so->iscan);
- ratey += cost_coeffs(x, 0, block, tempa + idx, templ + idy, TX_4X4,
- so->scan, so->neighbors,
- cpi->sf.use_fast_coef_costing);
- distortion += vp10_highbd_block_error(
- coeff, BLOCK_OFFSET(pd->dqcoeff, block),
- 16, &unused, xd->bd) >> 2;
- if (RDCOST(x->rdmult, x->rddiv, ratey, distortion) >= best_rd)
- goto next_highbd;
- vp10_highbd_inv_txfm_add_4x4(BLOCK_OFFSET(pd->dqcoeff, block),
- dst, dst_stride, p->eobs[block],
- xd->bd, tx_type, 0);
- }
- }
- }
-
- rate += ratey;
- this_rd = RDCOST(x->rdmult, x->rddiv, rate, distortion);
-
- if (this_rd < best_rd) {
- *bestrate = rate;
- *bestratey = ratey;
- *bestdistortion = distortion;
- best_rd = this_rd;
- *best_mode = mode;
- memcpy(a, tempa, sizeof(tempa));
- memcpy(l, templ, sizeof(templ));
- for (idy = 0; idy < num_4x4_blocks_high * 4; ++idy) {
- memcpy(best_dst16 + idy * 8,
- CONVERT_TO_SHORTPTR(dst_init + idy * dst_stride),
- num_4x4_blocks_wide * 4 * sizeof(uint16_t));
- }
- }
- next_highbd:
- {}
- }
- if (best_rd >= rd_thresh)
- return best_rd;
-
- for (idy = 0; idy < num_4x4_blocks_high * 4; ++idy) {
- memcpy(CONVERT_TO_SHORTPTR(dst_init + idy * dst_stride),
- best_dst16 + idy * 8,
- num_4x4_blocks_wide * 4 * sizeof(uint16_t));
- }
-
- return best_rd;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- for (mode = DC_PRED; mode <= TM_PRED; ++mode) {
- int64_t this_rd;
- int ratey = 0;
- int64_t distortion = 0;
- int rate = bmode_costs[mode];
-
- if (!(cpi->sf.intra_y_mode_mask[TX_4X4] & (1 << mode)))
- continue;
-
- // Only do the oblique modes if the best so far is
- // one of the neighboring directional modes
- if (cpi->sf.mode_search_skip_flags & FLAG_SKIP_INTRA_DIRMISMATCH) {
- if (conditional_skipintra(mode, *best_mode))
- continue;
- }
-
- memcpy(tempa, ta, sizeof(ta));
- memcpy(templ, tl, sizeof(tl));
-
- for (idy = 0; idy < num_4x4_blocks_high; ++idy) {
- for (idx = 0; idx < num_4x4_blocks_wide; ++idx) {
- const int block = (row + idy) * 2 + (col + idx);
- const uint8_t *const src = &src_init[idx * 4 + idy * 4 * src_stride];
- uint8_t *const dst = &dst_init[idx * 4 + idy * 4 * dst_stride];
- int16_t *const src_diff =
- vp10_raster_block_offset_int16(BLOCK_8X8, block, p->src_diff);
- tran_low_t *const coeff = BLOCK_OFFSET(x->plane[0].coeff, block);
- xd->mi[0]->bmi[block].as_mode = mode;
- vp10_predict_intra_block(xd, 1, 1, TX_4X4, mode, dst, dst_stride,
- dst, dst_stride, col + idx, row + idy, 0);
- vpx_subtract_block(4, 4, src_diff, 8, src, src_stride, dst, dst_stride);
-
- if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
- TX_TYPE tx_type = get_tx_type(PLANE_TYPE_Y, xd, block);
- const scan_order *so = get_scan(TX_4X4, tx_type);
- vp10_fwd_txfm_4x4(src_diff, coeff, 8, DCT_DCT, 1);
- vp10_regular_quantize_b_4x4(x, 0, block, so->scan, so->iscan);
- ratey += cost_coeffs(x, 0, block, tempa + idx, templ + idy, TX_4X4,
- so->scan, so->neighbors,
- cpi->sf.use_fast_coef_costing);
- if (RDCOST(x->rdmult, x->rddiv, ratey, distortion) >= best_rd)
- goto next;
- vp10_inv_txfm_add_4x4(BLOCK_OFFSET(pd->dqcoeff, block),
- dst, dst_stride, p->eobs[block], DCT_DCT, 1);
- } else {
- int64_t unused;
- TX_TYPE tx_type = get_tx_type(PLANE_TYPE_Y, xd, block);
- const scan_order *so = get_scan(TX_4X4, tx_type);
- vp10_fwd_txfm_4x4(src_diff, coeff, 8, tx_type, 0);
- vp10_regular_quantize_b_4x4(x, 0, block, so->scan, so->iscan);
- ratey += cost_coeffs(x, 0, block, tempa + idx, templ + idy, TX_4X4,
- so->scan, so->neighbors,
- cpi->sf.use_fast_coef_costing);
- distortion += vp10_block_error(coeff, BLOCK_OFFSET(pd->dqcoeff, block),
- 16, &unused) >> 2;
- if (RDCOST(x->rdmult, x->rddiv, ratey, distortion) >= best_rd)
- goto next;
- vp10_inv_txfm_add_4x4(BLOCK_OFFSET(pd->dqcoeff, block),
- dst, dst_stride, p->eobs[block], tx_type, 0);
- }
- }
- }
-
- rate += ratey;
- this_rd = RDCOST(x->rdmult, x->rddiv, rate, distortion);
-
- if (this_rd < best_rd) {
- *bestrate = rate;
- *bestratey = ratey;
- *bestdistortion = distortion;
- best_rd = this_rd;
- *best_mode = mode;
- memcpy(a, tempa, sizeof(tempa));
- memcpy(l, templ, sizeof(templ));
- for (idy = 0; idy < num_4x4_blocks_high * 4; ++idy)
- memcpy(best_dst + idy * 8, dst_init + idy * dst_stride,
- num_4x4_blocks_wide * 4);
- }
- next:
- {}
- }
-
- if (best_rd >= rd_thresh)
- return best_rd;
-
- for (idy = 0; idy < num_4x4_blocks_high * 4; ++idy)
- memcpy(dst_init + idy * dst_stride, best_dst + idy * 8,
- num_4x4_blocks_wide * 4);
-
- return best_rd;
-}
-
-static int64_t rd_pick_intra_sub_8x8_y_mode(VP10_COMP *cpi, MACROBLOCK *mb,
- int *rate, int *rate_y,
- int64_t *distortion,
- int64_t best_rd) {
- int i, j;
- const MACROBLOCKD *const xd = &mb->e_mbd;
- MODE_INFO *const mic = xd->mi[0];
- const MODE_INFO *above_mi = xd->above_mi;
- const MODE_INFO *left_mi = xd->left_mi;
- const BLOCK_SIZE bsize = xd->mi[0]->mbmi.sb_type;
- const int num_4x4_blocks_wide = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_blocks_high = num_4x4_blocks_high_lookup[bsize];
- int idx, idy;
- int cost = 0;
- int64_t total_distortion = 0;
- int tot_rate_y = 0;
- int64_t total_rd = 0;
- ENTROPY_CONTEXT t_above[4], t_left[4];
- const int *bmode_costs = cpi->mbmode_cost;
-
- memcpy(t_above, xd->plane[0].above_context, sizeof(t_above));
- memcpy(t_left, xd->plane[0].left_context, sizeof(t_left));
-
- // Pick modes for each sub-block (of size 4x4, 4x8, or 8x4) in an 8x8 block.
- for (idy = 0; idy < 2; idy += num_4x4_blocks_high) {
- for (idx = 0; idx < 2; idx += num_4x4_blocks_wide) {
- PREDICTION_MODE best_mode = DC_PRED;
- int r = INT_MAX, ry = INT_MAX;
- int64_t d = INT64_MAX, this_rd = INT64_MAX;
- i = idy * 2 + idx;
- if (cpi->common.frame_type == KEY_FRAME) {
- const PREDICTION_MODE A = vp10_above_block_mode(mic, above_mi, i);
- const PREDICTION_MODE L = vp10_left_block_mode(mic, left_mi, i);
-
- bmode_costs = cpi->y_mode_costs[A][L];
- }
-
- this_rd = rd_pick_intra4x4block(cpi, mb, idy, idx, &best_mode,
- bmode_costs, t_above + idx, t_left + idy,
- &r, &ry, &d, bsize, best_rd - total_rd);
- if (this_rd >= best_rd - total_rd)
- return INT64_MAX;
-
- total_rd += this_rd;
- cost += r;
- total_distortion += d;
- tot_rate_y += ry;
-
- mic->bmi[i].as_mode = best_mode;
- for (j = 1; j < num_4x4_blocks_high; ++j)
- mic->bmi[i + j * 2].as_mode = best_mode;
- for (j = 1; j < num_4x4_blocks_wide; ++j)
- mic->bmi[i + j].as_mode = best_mode;
-
- if (total_rd >= best_rd)
- return INT64_MAX;
- }
- }
-
- *rate = cost;
- *rate_y = tot_rate_y;
- *distortion = total_distortion;
- mic->mbmi.mode = mic->bmi[3].as_mode;
-
- return RDCOST(mb->rdmult, mb->rddiv, cost, total_distortion);
-}
-
-// This function is used only for intra_only frames
-static int64_t rd_pick_intra_sby_mode(VP10_COMP *cpi, MACROBLOCK *x,
- int *rate, int *rate_tokenonly,
- int64_t *distortion, int *skippable,
- BLOCK_SIZE bsize,
- int64_t best_rd) {
- PREDICTION_MODE mode;
- PREDICTION_MODE mode_selected = DC_PRED;
- MACROBLOCKD *const xd = &x->e_mbd;
- MODE_INFO *const mic = xd->mi[0];
- int this_rate, this_rate_tokenonly, s;
- int64_t this_distortion, this_rd;
- TX_SIZE best_tx = TX_4X4;
- int *bmode_costs;
- PALETTE_MODE_INFO palette_mode_info;
- uint8_t *best_palette_color_map = cpi->common.allow_screen_content_tools ?
- x->palette_buffer->best_palette_color_map : NULL;
- int rows = 4 * num_4x4_blocks_high_lookup[bsize];
- int cols = 4 * num_4x4_blocks_wide_lookup[bsize];
- int palette_ctx = 0;
- const MODE_INFO *above_mi = xd->above_mi;
- const MODE_INFO *left_mi = xd->left_mi;
- const PREDICTION_MODE A = vp10_above_block_mode(mic, above_mi, 0);
- const PREDICTION_MODE L = vp10_left_block_mode(mic, left_mi, 0);
- bmode_costs = cpi->y_mode_costs[A][L];
-
- memset(x->skip_txfm, SKIP_TXFM_NONE, sizeof(x->skip_txfm));
- palette_mode_info.palette_size[0] = 0;
- mic->mbmi.palette_mode_info.palette_size[0] = 0;
- if (above_mi)
- palette_ctx += (above_mi->mbmi.palette_mode_info.palette_size[0] > 0);
- if (left_mi)
- palette_ctx += (left_mi->mbmi.palette_mode_info.palette_size[0] > 0);
-
- /* Y Search for intra prediction mode */
- for (mode = DC_PRED; mode <= TM_PRED; mode++) {
- mic->mbmi.mode = mode;
-
- super_block_yrd(cpi, x, &this_rate_tokenonly, &this_distortion,
- &s, NULL, bsize, best_rd);
-
- if (this_rate_tokenonly == INT_MAX)
- continue;
-
- this_rate = this_rate_tokenonly + bmode_costs[mode];
- if (cpi->common.allow_screen_content_tools && mode == DC_PRED)
- this_rate +=
- vp10_cost_bit(vp10_default_palette_y_mode_prob[bsize - BLOCK_8X8]
- [palette_ctx], 0);
- this_rd = RDCOST(x->rdmult, x->rddiv, this_rate, this_distortion);
-
- if (this_rd < best_rd) {
- mode_selected = mode;
- best_rd = this_rd;
- best_tx = mic->mbmi.tx_size;
- *rate = this_rate;
- *rate_tokenonly = this_rate_tokenonly;
- *distortion = this_distortion;
- *skippable = s;
- }
- }
-
- if (cpi->common.allow_screen_content_tools)
- rd_pick_palette_intra_sby(cpi, x, bsize, palette_ctx, bmode_costs[DC_PRED],
- &palette_mode_info, best_palette_color_map,
- &best_tx, &mode_selected, &best_rd);
-
- mic->mbmi.mode = mode_selected;
- mic->mbmi.tx_size = best_tx;
- mic->mbmi.palette_mode_info.palette_size[0] =
- palette_mode_info.palette_size[0];
- if (palette_mode_info.palette_size[0] > 0) {
- memcpy(mic->mbmi.palette_mode_info.palette_colors,
- palette_mode_info.palette_colors,
- PALETTE_MAX_SIZE * sizeof(palette_mode_info.palette_colors[0]));
- memcpy(xd->plane[0].color_index_map, best_palette_color_map,
- rows * cols * sizeof(best_palette_color_map[0]));
- }
-
- return best_rd;
-}
-
-// Return value 0: early termination triggered, no valid rd cost available;
-// 1: rd cost values are valid.
-static int super_block_uvrd(const VP10_COMP *cpi, MACROBLOCK *x,
- int *rate, int64_t *distortion, int *skippable,
- int64_t *sse, BLOCK_SIZE bsize,
- int64_t ref_best_rd) {
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- const TX_SIZE uv_tx_size = get_uv_tx_size(mbmi, &xd->plane[1]);
- int plane;
- int pnrate = 0, pnskip = 1;
- int64_t pndist = 0, pnsse = 0;
- int is_cost_valid = 1;
-
- if (ref_best_rd < 0)
- is_cost_valid = 0;
-
- if (is_inter_block(mbmi) && is_cost_valid) {
- int plane;
- for (plane = 1; plane < MAX_MB_PLANE; ++plane)
- vp10_subtract_plane(x, bsize, plane);
- }
-
- *rate = 0;
- *distortion = 0;
- *sse = 0;
- *skippable = 1;
-
- for (plane = 1; plane < MAX_MB_PLANE; ++plane) {
- txfm_rd_in_plane(x, &pnrate, &pndist, &pnskip, &pnsse,
- ref_best_rd, plane, bsize, uv_tx_size,
- cpi->sf.use_fast_coef_costing);
- if (pnrate == INT_MAX) {
- is_cost_valid = 0;
- break;
- }
- *rate += pnrate;
- *distortion += pndist;
- *sse += pnsse;
- *skippable &= pnskip;
- }
-
- if (!is_cost_valid) {
- // reset cost value
- *rate = INT_MAX;
- *distortion = INT64_MAX;
- *sse = INT64_MAX;
- *skippable = 0;
- }
-
- return is_cost_valid;
-}
-
-static int64_t rd_pick_intra_sbuv_mode(VP10_COMP *cpi, MACROBLOCK *x,
- PICK_MODE_CONTEXT *ctx,
- int *rate, int *rate_tokenonly,
- int64_t *distortion, int *skippable,
- BLOCK_SIZE bsize, TX_SIZE max_tx_size) {
- MACROBLOCKD *xd = &x->e_mbd;
- PREDICTION_MODE mode;
- PREDICTION_MODE mode_selected = DC_PRED;
- int64_t best_rd = INT64_MAX, this_rd;
- int this_rate_tokenonly, this_rate, s;
- int64_t this_distortion, this_sse;
-
- memset(x->skip_txfm, SKIP_TXFM_NONE, sizeof(x->skip_txfm));
- xd->mi[0]->mbmi.palette_mode_info.palette_size[1] = 0;
- for (mode = DC_PRED; mode <= TM_PRED; ++mode) {
- if (!(cpi->sf.intra_uv_mode_mask[max_tx_size] & (1 << mode)))
- continue;
-
- xd->mi[0]->mbmi.uv_mode = mode;
-
- if (!super_block_uvrd(cpi, x, &this_rate_tokenonly,
- &this_distortion, &s, &this_sse, bsize, best_rd))
- continue;
- this_rate = this_rate_tokenonly + cpi->intra_uv_mode_cost[mode];
- this_rd = RDCOST(x->rdmult, x->rddiv, this_rate, this_distortion);
-
- if (this_rd < best_rd) {
- mode_selected = mode;
- best_rd = this_rd;
- *rate = this_rate;
- *rate_tokenonly = this_rate_tokenonly;
- *distortion = this_distortion;
- *skippable = s;
- if (!x->select_tx_size)
- swap_block_ptr(x, ctx, 2, 0, 1, MAX_MB_PLANE);
- }
- }
-
- xd->mi[0]->mbmi.uv_mode = mode_selected;
- return best_rd;
-}
-
-static int64_t rd_sbuv_dcpred(const VP10_COMP *cpi, MACROBLOCK *x,
- int *rate, int *rate_tokenonly,
- int64_t *distortion, int *skippable,
- BLOCK_SIZE bsize) {
- int64_t unused;
-
- x->e_mbd.mi[0]->mbmi.uv_mode = DC_PRED;
- memset(x->skip_txfm, SKIP_TXFM_NONE, sizeof(x->skip_txfm));
- super_block_uvrd(cpi, x, rate_tokenonly, distortion,
- skippable, &unused, bsize, INT64_MAX);
- *rate = *rate_tokenonly + cpi->intra_uv_mode_cost[DC_PRED];
- return RDCOST(x->rdmult, x->rddiv, *rate, *distortion);
-}
-
-static void choose_intra_uv_mode(VP10_COMP *cpi, MACROBLOCK *const x,
- PICK_MODE_CONTEXT *ctx,
- BLOCK_SIZE bsize, TX_SIZE max_tx_size,
- int *rate_uv, int *rate_uv_tokenonly,
- int64_t *dist_uv, int *skip_uv,
- PREDICTION_MODE *mode_uv) {
- // Use an estimated rd for uv_intra based on DC_PRED if the
- // appropriate speed flag is set.
- if (cpi->sf.use_uv_intra_rd_estimate) {
- rd_sbuv_dcpred(cpi, x, rate_uv, rate_uv_tokenonly, dist_uv,
- skip_uv, bsize < BLOCK_8X8 ? BLOCK_8X8 : bsize);
- // Else do a proper rd search for each possible transform size that may
- // be considered in the main rd loop.
- } else {
- rd_pick_intra_sbuv_mode(cpi, x, ctx,
- rate_uv, rate_uv_tokenonly, dist_uv, skip_uv,
- bsize < BLOCK_8X8 ? BLOCK_8X8 : bsize, max_tx_size);
- }
- *mode_uv = x->e_mbd.mi[0]->mbmi.uv_mode;
-}
-
-static int cost_mv_ref(const VP10_COMP *cpi, PREDICTION_MODE mode,
- int mode_context) {
- assert(is_inter_mode(mode));
- return cpi->inter_mode_cost[mode_context][INTER_OFFSET(mode)];
-}
-
-static int set_and_cost_bmi_mvs(VP10_COMP *cpi, MACROBLOCK *x, MACROBLOCKD *xd,
- int i,
- PREDICTION_MODE mode, int_mv this_mv[2],
- int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES],
- int_mv seg_mvs[MAX_REF_FRAMES],
- int_mv *best_ref_mv[2], const int *mvjcost,
- int *mvcost[2]) {
- MODE_INFO *const mic = xd->mi[0];
- const MB_MODE_INFO *const mbmi = &mic->mbmi;
- const MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
- int thismvcost = 0;
- int idx, idy;
- const int num_4x4_blocks_wide = num_4x4_blocks_wide_lookup[mbmi->sb_type];
- const int num_4x4_blocks_high = num_4x4_blocks_high_lookup[mbmi->sb_type];
- const int is_compound = has_second_ref(mbmi);
-
- switch (mode) {
- case NEWMV:
- this_mv[0].as_int = seg_mvs[mbmi->ref_frame[0]].as_int;
- thismvcost += vp10_mv_bit_cost(&this_mv[0].as_mv, &best_ref_mv[0]->as_mv,
- mvjcost, mvcost, MV_COST_WEIGHT_SUB);
- if (is_compound) {
- this_mv[1].as_int = seg_mvs[mbmi->ref_frame[1]].as_int;
- thismvcost += vp10_mv_bit_cost(&this_mv[1].as_mv, &best_ref_mv[1]->as_mv,
- mvjcost, mvcost, MV_COST_WEIGHT_SUB);
- }
- break;
- case NEARMV:
- case NEARESTMV:
- this_mv[0].as_int = frame_mv[mode][mbmi->ref_frame[0]].as_int;
- if (is_compound)
- this_mv[1].as_int = frame_mv[mode][mbmi->ref_frame[1]].as_int;
- break;
- case ZEROMV:
- this_mv[0].as_int = 0;
- if (is_compound)
- this_mv[1].as_int = 0;
- break;
- default:
- break;
- }
-
- mic->bmi[i].as_mv[0].as_int = this_mv[0].as_int;
- if (is_compound)
- mic->bmi[i].as_mv[1].as_int = this_mv[1].as_int;
-
- mic->bmi[i].as_mode = mode;
-
- for (idy = 0; idy < num_4x4_blocks_high; ++idy)
- for (idx = 0; idx < num_4x4_blocks_wide; ++idx)
- memmove(&mic->bmi[i + idy * 2 + idx], &mic->bmi[i], sizeof(mic->bmi[i]));
-
- return cost_mv_ref(cpi, mode, mbmi_ext->mode_context[mbmi->ref_frame[0]]) +
- thismvcost;
-}
-
-static int64_t encode_inter_mb_segment(VP10_COMP *cpi,
- MACROBLOCK *x,
- int64_t best_yrd,
- int i,
- int *labelyrate,
- int64_t *distortion, int64_t *sse,
- ENTROPY_CONTEXT *ta,
- ENTROPY_CONTEXT *tl,
- int ir, int ic,
- int mi_row, int mi_col) {
- int k;
- MACROBLOCKD *xd = &x->e_mbd;
- struct macroblockd_plane *const pd = &xd->plane[0];
- struct macroblock_plane *const p = &x->plane[0];
- MODE_INFO *const mi = xd->mi[0];
- const BLOCK_SIZE plane_bsize = get_plane_block_size(mi->mbmi.sb_type, pd);
- const int width = 4 * num_4x4_blocks_wide_lookup[plane_bsize];
- const int height = 4 * num_4x4_blocks_high_lookup[plane_bsize];
- int idx, idy;
- void (*fwd_txm4x4)(const int16_t *input, tran_low_t *output, int stride);
-
- const uint8_t *const src =
- &p->src.buf[vp10_raster_block_offset(BLOCK_8X8, i, p->src.stride)];
- uint8_t *const dst = &pd->dst.buf[vp10_raster_block_offset(BLOCK_8X8, i,
- pd->dst.stride)];
- int64_t thisdistortion = 0, thissse = 0;
- int thisrate = 0;
- TX_TYPE tx_type = get_tx_type(PLANE_TYPE_Y, xd, i);
- const scan_order *so = get_scan(TX_4X4, tx_type);
-
- vp10_build_inter_predictor_sub8x8(xd, 0, i, ir, ic, mi_row, mi_col);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- fwd_txm4x4 = xd->lossless[mi->mbmi.segment_id] ? vp10_highbd_fwht4x4
- : vpx_highbd_fdct4x4;
- } else {
- fwd_txm4x4 = xd->lossless[mi->mbmi.segment_id] ? vp10_fwht4x4 : vpx_fdct4x4;
- }
-#else
- fwd_txm4x4 = xd->lossless[mi->mbmi.segment_id] ? vp10_fwht4x4 : vpx_fdct4x4;
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- vpx_highbd_subtract_block(
- height, width, vp10_raster_block_offset_int16(BLOCK_8X8, i, p->src_diff),
- 8, src, p->src.stride, dst, pd->dst.stride, xd->bd);
- } else {
- vpx_subtract_block(
- height, width, vp10_raster_block_offset_int16(BLOCK_8X8, i, p->src_diff),
- 8, src, p->src.stride, dst, pd->dst.stride);
- }
-#else
- vpx_subtract_block(height, width,
- vp10_raster_block_offset_int16(BLOCK_8X8, i, p->src_diff),
- 8, src, p->src.stride, dst, pd->dst.stride);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- k = i;
- for (idy = 0; idy < height / 4; ++idy) {
- for (idx = 0; idx < width / 4; ++idx) {
- int64_t ssz, rd, rd1, rd2;
- tran_low_t* coeff;
-
- k += (idy * 2 + idx);
- coeff = BLOCK_OFFSET(p->coeff, k);
- fwd_txm4x4(vp10_raster_block_offset_int16(BLOCK_8X8, k, p->src_diff),
- coeff, 8);
- vp10_regular_quantize_b_4x4(x, 0, k, so->scan, so->iscan);
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- thisdistortion += vp10_highbd_block_error(coeff,
- BLOCK_OFFSET(pd->dqcoeff, k),
- 16, &ssz, xd->bd);
- } else {
- thisdistortion += vp10_block_error(coeff, BLOCK_OFFSET(pd->dqcoeff, k),
- 16, &ssz);
- }
-#else
- thisdistortion += vp10_block_error(coeff, BLOCK_OFFSET(pd->dqcoeff, k),
- 16, &ssz);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- thissse += ssz;
- thisrate += cost_coeffs(x, 0, k, ta + (k & 1), tl + (k >> 1), TX_4X4,
- so->scan, so->neighbors,
- cpi->sf.use_fast_coef_costing);
- rd1 = RDCOST(x->rdmult, x->rddiv, thisrate, thisdistortion >> 2);
- rd2 = RDCOST(x->rdmult, x->rddiv, 0, thissse >> 2);
- rd = VPXMIN(rd1, rd2);
- if (rd >= best_yrd)
- return INT64_MAX;
- }
- }
-
- *distortion = thisdistortion >> 2;
- *labelyrate = thisrate;
- *sse = thissse >> 2;
-
- return RDCOST(x->rdmult, x->rddiv, *labelyrate, *distortion);
-}
-
-typedef struct {
- int eobs;
- int brate;
- int byrate;
- int64_t bdist;
- int64_t bsse;
- int64_t brdcost;
- int_mv mvs[2];
- ENTROPY_CONTEXT ta[2];
- ENTROPY_CONTEXT tl[2];
-} SEG_RDSTAT;
-
-typedef struct {
- int_mv *ref_mv[2];
- int_mv mvp;
-
- int64_t segment_rd;
- int r;
- int64_t d;
- int64_t sse;
- int segment_yrate;
- PREDICTION_MODE modes[4];
- SEG_RDSTAT rdstat[4][INTER_MODES];
- int mvthresh;
-} BEST_SEG_INFO;
-
-static INLINE int mv_check_bounds(const MACROBLOCK *x, const MV *mv) {
- return (mv->row >> 3) < x->mv_row_min ||
- (mv->row >> 3) > x->mv_row_max ||
- (mv->col >> 3) < x->mv_col_min ||
- (mv->col >> 3) > x->mv_col_max;
-}
-
-static INLINE void mi_buf_shift(MACROBLOCK *x, int i) {
- MB_MODE_INFO *const mbmi = &x->e_mbd.mi[0]->mbmi;
- struct macroblock_plane *const p = &x->plane[0];
- struct macroblockd_plane *const pd = &x->e_mbd.plane[0];
-
- p->src.buf = &p->src.buf[vp10_raster_block_offset(BLOCK_8X8, i,
- p->src.stride)];
- assert(((intptr_t)pd->pre[0].buf & 0x7) == 0);
- pd->pre[0].buf = &pd->pre[0].buf[vp10_raster_block_offset(BLOCK_8X8, i,
- pd->pre[0].stride)];
- if (has_second_ref(mbmi))
- pd->pre[1].buf = &pd->pre[1].buf[vp10_raster_block_offset(BLOCK_8X8, i,
- pd->pre[1].stride)];
-}
-
-static INLINE void mi_buf_restore(MACROBLOCK *x, struct buf_2d orig_src,
- struct buf_2d orig_pre[2]) {
- MB_MODE_INFO *mbmi = &x->e_mbd.mi[0]->mbmi;
- x->plane[0].src = orig_src;
- x->e_mbd.plane[0].pre[0] = orig_pre[0];
- if (has_second_ref(mbmi))
- x->e_mbd.plane[0].pre[1] = orig_pre[1];
-}
-
-static INLINE int mv_has_subpel(const MV *mv) {
- return (mv->row & 0x0F) || (mv->col & 0x0F);
-}
-
-// Check if NEARESTMV/NEARMV/ZEROMV is the cheapest way encode zero motion.
-// TODO(aconverse): Find out if this is still productive then clean up or remove
-static int check_best_zero_mv(
- const VP10_COMP *cpi, const uint8_t mode_context[MAX_REF_FRAMES],
- int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES], int this_mode,
- const MV_REFERENCE_FRAME ref_frames[2]) {
- if ((this_mode == NEARMV || this_mode == NEARESTMV || this_mode == ZEROMV) &&
- frame_mv[this_mode][ref_frames[0]].as_int == 0 &&
- (ref_frames[1] == NONE ||
- frame_mv[this_mode][ref_frames[1]].as_int == 0)) {
- int rfc = mode_context[ref_frames[0]];
- int c1 = cost_mv_ref(cpi, NEARMV, rfc);
- int c2 = cost_mv_ref(cpi, NEARESTMV, rfc);
- int c3 = cost_mv_ref(cpi, ZEROMV, rfc);
-
- if (this_mode == NEARMV) {
- if (c1 > c3) return 0;
- } else if (this_mode == NEARESTMV) {
- if (c2 > c3) return 0;
- } else {
- assert(this_mode == ZEROMV);
- if (ref_frames[1] == NONE) {
- if ((c3 >= c2 && frame_mv[NEARESTMV][ref_frames[0]].as_int == 0) ||
- (c3 >= c1 && frame_mv[NEARMV][ref_frames[0]].as_int == 0))
- return 0;
- } else {
- if ((c3 >= c2 && frame_mv[NEARESTMV][ref_frames[0]].as_int == 0 &&
- frame_mv[NEARESTMV][ref_frames[1]].as_int == 0) ||
- (c3 >= c1 && frame_mv[NEARMV][ref_frames[0]].as_int == 0 &&
- frame_mv[NEARMV][ref_frames[1]].as_int == 0))
- return 0;
- }
- }
- }
- return 1;
-}
-
-static void joint_motion_search(VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bsize,
- int_mv *frame_mv,
- int mi_row, int mi_col,
- int_mv single_newmv[MAX_REF_FRAMES],
- int *rate_mv) {
- const VP10_COMMON *const cm = &cpi->common;
- const int pw = 4 * num_4x4_blocks_wide_lookup[bsize];
- const int ph = 4 * num_4x4_blocks_high_lookup[bsize];
- MACROBLOCKD *xd = &x->e_mbd;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- const int refs[2] = {mbmi->ref_frame[0],
- mbmi->ref_frame[1] < 0 ? 0 : mbmi->ref_frame[1]};
- int_mv ref_mv[2];
- int ite, ref;
- const InterpKernel *kernel = vp10_filter_kernels[mbmi->interp_filter];
- struct scale_factors sf;
-
- // Do joint motion search in compound mode to get more accurate mv.
- struct buf_2d backup_yv12[2][MAX_MB_PLANE];
- int last_besterr[2] = {INT_MAX, INT_MAX};
- const YV12_BUFFER_CONFIG *const scaled_ref_frame[2] = {
- vp10_get_scaled_ref_frame(cpi, mbmi->ref_frame[0]),
- vp10_get_scaled_ref_frame(cpi, mbmi->ref_frame[1])
- };
-
- // Prediction buffer from second frame.
-#if CONFIG_VP9_HIGHBITDEPTH
- DECLARE_ALIGNED(16, uint16_t, second_pred_alloc_16[64 * 64]);
- uint8_t *second_pred;
-#else
- DECLARE_ALIGNED(16, uint8_t, second_pred[64 * 64]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- for (ref = 0; ref < 2; ++ref) {
- ref_mv[ref] = x->mbmi_ext->ref_mvs[refs[ref]][0];
-
- if (scaled_ref_frame[ref]) {
- int i;
- // Swap out the reference frame for a version that's been scaled to
- // match the resolution of the current frame, allowing the existing
- // motion search code to be used without additional modifications.
- for (i = 0; i < MAX_MB_PLANE; i++)
- backup_yv12[ref][i] = xd->plane[i].pre[ref];
- vp10_setup_pre_planes(xd, ref, scaled_ref_frame[ref], mi_row, mi_col,
- NULL);
- }
-
- frame_mv[refs[ref]].as_int = single_newmv[refs[ref]].as_int;
- }
-
- // Since we have scaled the reference frames to match the size of the current
- // frame we must use a unit scaling factor during mode selection.
-#if CONFIG_VP9_HIGHBITDEPTH
- vp10_setup_scale_factors_for_frame(&sf, cm->width, cm->height,
- cm->width, cm->height,
- cm->use_highbitdepth);
-#else
- vp10_setup_scale_factors_for_frame(&sf, cm->width, cm->height,
- cm->width, cm->height);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Allow joint search multiple times iteratively for each reference frame
- // and break out of the search loop if it couldn't find a better mv.
- for (ite = 0; ite < 4; ite++) {
- struct buf_2d ref_yv12[2];
- int bestsme = INT_MAX;
- int sadpb = x->sadperbit16;
- MV tmp_mv;
- int search_range = 3;
-
- int tmp_col_min = x->mv_col_min;
- int tmp_col_max = x->mv_col_max;
- int tmp_row_min = x->mv_row_min;
- int tmp_row_max = x->mv_row_max;
- int id = ite % 2; // Even iterations search in the first reference frame,
- // odd iterations search in the second. The predictor
- // found for the 'other' reference frame is factored in.
-
- // Initialized here because of compiler problem in Visual Studio.
- ref_yv12[0] = xd->plane[0].pre[0];
- ref_yv12[1] = xd->plane[0].pre[1];
-
- // Get the prediction block from the 'other' reference frame.
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- second_pred = CONVERT_TO_BYTEPTR(second_pred_alloc_16);
- vp10_highbd_build_inter_predictor(ref_yv12[!id].buf,
- ref_yv12[!id].stride,
- second_pred, pw,
- &frame_mv[refs[!id]].as_mv,
- &sf, pw, ph, 0,
- kernel, MV_PRECISION_Q3,
- mi_col * MI_SIZE, mi_row * MI_SIZE,
- xd->bd);
- } else {
- second_pred = (uint8_t *)second_pred_alloc_16;
- vp10_build_inter_predictor(ref_yv12[!id].buf,
- ref_yv12[!id].stride,
- second_pred, pw,
- &frame_mv[refs[!id]].as_mv,
- &sf, pw, ph, 0,
- kernel, MV_PRECISION_Q3,
- mi_col * MI_SIZE, mi_row * MI_SIZE);
- }
-#else
- vp10_build_inter_predictor(ref_yv12[!id].buf,
- ref_yv12[!id].stride,
- second_pred, pw,
- &frame_mv[refs[!id]].as_mv,
- &sf, pw, ph, 0,
- kernel, MV_PRECISION_Q3,
- mi_col * MI_SIZE, mi_row * MI_SIZE);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- // Do compound motion search on the current reference frame.
- if (id)
- xd->plane[0].pre[0] = ref_yv12[id];
- vp10_set_mv_search_range(x, &ref_mv[id].as_mv);
-
- // Use the mv result from the single mode as mv predictor.
- tmp_mv = frame_mv[refs[id]].as_mv;
-
- tmp_mv.col >>= 3;
- tmp_mv.row >>= 3;
-
- // Small-range full-pixel motion search.
- bestsme = vp10_refining_search_8p_c(x, &tmp_mv, sadpb,
- search_range,
- &cpi->fn_ptr[bsize],
- &ref_mv[id].as_mv, second_pred);
- if (bestsme < INT_MAX)
- bestsme = vp10_get_mvpred_av_var(x, &tmp_mv, &ref_mv[id].as_mv,
- second_pred, &cpi->fn_ptr[bsize], 1);
-
- x->mv_col_min = tmp_col_min;
- x->mv_col_max = tmp_col_max;
- x->mv_row_min = tmp_row_min;
- x->mv_row_max = tmp_row_max;
-
- if (bestsme < INT_MAX) {
- int dis; /* TODO: use dis in distortion calculation later. */
- unsigned int sse;
- bestsme = cpi->find_fractional_mv_step(
- x, &tmp_mv,
- &ref_mv[id].as_mv,
- cpi->common.allow_high_precision_mv,
- x->errorperbit,
- &cpi->fn_ptr[bsize],
- 0, cpi->sf.mv.subpel_iters_per_step,
- NULL,
- x->nmvjointcost, x->mvcost,
- &dis, &sse, second_pred,
- pw, ph);
- }
-
- // Restore the pointer to the first (possibly scaled) prediction buffer.
- if (id)
- xd->plane[0].pre[0] = ref_yv12[0];
-
- if (bestsme < last_besterr[id]) {
- frame_mv[refs[id]].as_mv = tmp_mv;
- last_besterr[id] = bestsme;
- } else {
- break;
- }
- }
-
- *rate_mv = 0;
-
- for (ref = 0; ref < 2; ++ref) {
- if (scaled_ref_frame[ref]) {
- // Restore the prediction frame pointers to their unscaled versions.
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++)
- xd->plane[i].pre[ref] = backup_yv12[ref][i];
- }
-
- *rate_mv += vp10_mv_bit_cost(&frame_mv[refs[ref]].as_mv,
- &x->mbmi_ext->ref_mvs[refs[ref]][0].as_mv,
- x->nmvjointcost, x->mvcost, MV_COST_WEIGHT);
- }
-}
-
-static int64_t rd_pick_best_sub8x8_mode(VP10_COMP *cpi, MACROBLOCK *x,
- int_mv *best_ref_mv,
- int_mv *second_best_ref_mv,
- int64_t best_rd, int *returntotrate,
- int *returnyrate,
- int64_t *returndistortion,
- int *skippable, int64_t *psse,
- int mvthresh,
- int_mv seg_mvs[4][MAX_REF_FRAMES],
- BEST_SEG_INFO *bsi_buf, int filter_idx,
- int mi_row, int mi_col) {
- int i;
- BEST_SEG_INFO *bsi = bsi_buf + filter_idx;
- MACROBLOCKD *xd = &x->e_mbd;
- MODE_INFO *mi = xd->mi[0];
- MB_MODE_INFO *mbmi = &mi->mbmi;
- int mode_idx;
- int k, br = 0, idx, idy;
- int64_t bd = 0, block_sse = 0;
- PREDICTION_MODE this_mode;
- VP10_COMMON *cm = &cpi->common;
- struct macroblock_plane *const p = &x->plane[0];
- struct macroblockd_plane *const pd = &xd->plane[0];
- const int label_count = 4;
- int64_t this_segment_rd = 0;
- int label_mv_thresh;
- int segmentyrate = 0;
- const BLOCK_SIZE bsize = mbmi->sb_type;
- const int num_4x4_blocks_wide = num_4x4_blocks_wide_lookup[bsize];
- const int num_4x4_blocks_high = num_4x4_blocks_high_lookup[bsize];
- ENTROPY_CONTEXT t_above[2], t_left[2];
- int subpelmv = 1, have_ref = 0;
- const int has_second_rf = has_second_ref(mbmi);
- const int inter_mode_mask = cpi->sf.inter_mode_mask[bsize];
- MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
-
- vp10_zero(*bsi);
-
- bsi->segment_rd = best_rd;
- bsi->ref_mv[0] = best_ref_mv;
- bsi->ref_mv[1] = second_best_ref_mv;
- bsi->mvp.as_int = best_ref_mv->as_int;
- bsi->mvthresh = mvthresh;
-
- for (i = 0; i < 4; i++)
- bsi->modes[i] = ZEROMV;
-
- memcpy(t_above, pd->above_context, sizeof(t_above));
- memcpy(t_left, pd->left_context, sizeof(t_left));
-
- // 64 makes this threshold really big effectively
- // making it so that we very rarely check mvs on
- // segments. setting this to 1 would make mv thresh
- // roughly equal to what it is for macroblocks
- label_mv_thresh = 1 * bsi->mvthresh / label_count;
-
- // Segmentation method overheads
- for (idy = 0; idy < 2; idy += num_4x4_blocks_high) {
- for (idx = 0; idx < 2; idx += num_4x4_blocks_wide) {
- // TODO(jingning,rbultje): rewrite the rate-distortion optimization
- // loop for 4x4/4x8/8x4 block coding. to be replaced with new rd loop
- int_mv mode_mv[MB_MODE_COUNT][2];
- int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES];
- PREDICTION_MODE mode_selected = ZEROMV;
- int64_t best_rd = INT64_MAX;
- const int i = idy * 2 + idx;
- int ref;
-
- for (ref = 0; ref < 1 + has_second_rf; ++ref) {
- const MV_REFERENCE_FRAME frame = mbmi->ref_frame[ref];
- frame_mv[ZEROMV][frame].as_int = 0;
- vp10_append_sub8x8_mvs_for_idx(cm, xd, i, ref, mi_row, mi_col,
- &frame_mv[NEARESTMV][frame],
- &frame_mv[NEARMV][frame],
- mbmi_ext->mode_context);
- }
-
- // search for the best motion vector on this segment
- for (this_mode = NEARESTMV; this_mode <= NEWMV; ++this_mode) {
- const struct buf_2d orig_src = x->plane[0].src;
- struct buf_2d orig_pre[2];
-
- mode_idx = INTER_OFFSET(this_mode);
- bsi->rdstat[i][mode_idx].brdcost = INT64_MAX;
- if (!(inter_mode_mask & (1 << this_mode)))
- continue;
-
- if (!check_best_zero_mv(cpi, mbmi_ext->mode_context, frame_mv,
- this_mode, mbmi->ref_frame))
- continue;
-
- memcpy(orig_pre, pd->pre, sizeof(orig_pre));
- memcpy(bsi->rdstat[i][mode_idx].ta, t_above,
- sizeof(bsi->rdstat[i][mode_idx].ta));
- memcpy(bsi->rdstat[i][mode_idx].tl, t_left,
- sizeof(bsi->rdstat[i][mode_idx].tl));
-
- // motion search for newmv (single predictor case only)
- if (!has_second_rf && this_mode == NEWMV &&
- seg_mvs[i][mbmi->ref_frame[0]].as_int == INVALID_MV) {
- MV *const new_mv = &mode_mv[NEWMV][0].as_mv;
- int step_param = 0;
- int thissme, bestsme = INT_MAX;
- int sadpb = x->sadperbit4;
- MV mvp_full;
- int max_mv;
- int cost_list[5];
-
- /* Is the best so far sufficiently good that we cant justify doing
- * and new motion search. */
- if (best_rd < label_mv_thresh)
- break;
-
- if (cpi->oxcf.mode != BEST) {
- // use previous block's result as next block's MV predictor.
- if (i > 0) {
- bsi->mvp.as_int = mi->bmi[i - 1].as_mv[0].as_int;
- if (i == 2)
- bsi->mvp.as_int = mi->bmi[i - 2].as_mv[0].as_int;
- }
- }
- if (i == 0)
- max_mv = x->max_mv_context[mbmi->ref_frame[0]];
- else
- max_mv =
- VPXMAX(abs(bsi->mvp.as_mv.row), abs(bsi->mvp.as_mv.col)) >> 3;
-
- if (cpi->sf.mv.auto_mv_step_size && cm->show_frame) {
- // Take wtd average of the step_params based on the last frame's
- // max mv magnitude and the best ref mvs of the current block for
- // the given reference.
- step_param = (vp10_init_search_range(max_mv) +
- cpi->mv_step_param) / 2;
- } else {
- step_param = cpi->mv_step_param;
- }
-
- mvp_full.row = bsi->mvp.as_mv.row >> 3;
- mvp_full.col = bsi->mvp.as_mv.col >> 3;
-
- if (cpi->sf.adaptive_motion_search) {
- mvp_full.row = x->pred_mv[mbmi->ref_frame[0]].row >> 3;
- mvp_full.col = x->pred_mv[mbmi->ref_frame[0]].col >> 3;
- step_param = VPXMAX(step_param, 8);
- }
-
- // adjust src pointer for this block
- mi_buf_shift(x, i);
-
- vp10_set_mv_search_range(x, &bsi->ref_mv[0]->as_mv);
-
- bestsme = vp10_full_pixel_search(
- cpi, x, bsize, &mvp_full, step_param, sadpb,
- cpi->sf.mv.subpel_search_method != SUBPEL_TREE ? cost_list : NULL,
- &bsi->ref_mv[0]->as_mv, new_mv,
- INT_MAX, 1);
-
- // Should we do a full search (best quality only)
- if (cpi->oxcf.mode == BEST) {
- int_mv *const best_mv = &mi->bmi[i].as_mv[0];
- /* Check if mvp_full is within the range. */
- clamp_mv(&mvp_full, x->mv_col_min, x->mv_col_max,
- x->mv_row_min, x->mv_row_max);
- thissme = cpi->full_search_sad(x, &mvp_full,
- sadpb, 16, &cpi->fn_ptr[bsize],
- &bsi->ref_mv[0]->as_mv,
- &best_mv->as_mv);
- cost_list[1] = cost_list[2] = cost_list[3] = cost_list[4] = INT_MAX;
- if (thissme < bestsme) {
- bestsme = thissme;
- *new_mv = best_mv->as_mv;
- } else {
- // The full search result is actually worse so re-instate the
- // previous best vector
- best_mv->as_mv = *new_mv;
- }
- }
-
- if (bestsme < INT_MAX) {
- int distortion;
- cpi->find_fractional_mv_step(
- x,
- new_mv,
- &bsi->ref_mv[0]->as_mv,
- cm->allow_high_precision_mv,
- x->errorperbit, &cpi->fn_ptr[bsize],
- cpi->sf.mv.subpel_force_stop,
- cpi->sf.mv.subpel_iters_per_step,
- cond_cost_list(cpi, cost_list),
- x->nmvjointcost, x->mvcost,
- &distortion,
- &x->pred_sse[mbmi->ref_frame[0]],
- NULL, 0, 0);
-
- // save motion search result for use in compound prediction
- seg_mvs[i][mbmi->ref_frame[0]].as_mv = *new_mv;
- }
-
- if (cpi->sf.adaptive_motion_search)
- x->pred_mv[mbmi->ref_frame[0]] = *new_mv;
-
- // restore src pointers
- mi_buf_restore(x, orig_src, orig_pre);
- }
-
- if (has_second_rf) {
- if (seg_mvs[i][mbmi->ref_frame[1]].as_int == INVALID_MV ||
- seg_mvs[i][mbmi->ref_frame[0]].as_int == INVALID_MV)
- continue;
- }
-
- if (has_second_rf && this_mode == NEWMV &&
- mbmi->interp_filter == EIGHTTAP) {
- // adjust src pointers
- mi_buf_shift(x, i);
- if (cpi->sf.comp_inter_joint_search_thresh <= bsize) {
- int rate_mv;
- joint_motion_search(cpi, x, bsize, frame_mv[this_mode],
- mi_row, mi_col, seg_mvs[i],
- &rate_mv);
- seg_mvs[i][mbmi->ref_frame[0]].as_int =
- frame_mv[this_mode][mbmi->ref_frame[0]].as_int;
- seg_mvs[i][mbmi->ref_frame[1]].as_int =
- frame_mv[this_mode][mbmi->ref_frame[1]].as_int;
- }
- // restore src pointers
- mi_buf_restore(x, orig_src, orig_pre);
- }
-
- bsi->rdstat[i][mode_idx].brate =
- set_and_cost_bmi_mvs(cpi, x, xd, i, this_mode, mode_mv[this_mode],
- frame_mv, seg_mvs[i], bsi->ref_mv,
- x->nmvjointcost, x->mvcost);
-
- for (ref = 0; ref < 1 + has_second_rf; ++ref) {
- bsi->rdstat[i][mode_idx].mvs[ref].as_int =
- mode_mv[this_mode][ref].as_int;
- if (num_4x4_blocks_wide > 1)
- bsi->rdstat[i + 1][mode_idx].mvs[ref].as_int =
- mode_mv[this_mode][ref].as_int;
- if (num_4x4_blocks_high > 1)
- bsi->rdstat[i + 2][mode_idx].mvs[ref].as_int =
- mode_mv[this_mode][ref].as_int;
- }
-
- // Trap vectors that reach beyond the UMV borders
- if (mv_check_bounds(x, &mode_mv[this_mode][0].as_mv) ||
- (has_second_rf &&
- mv_check_bounds(x, &mode_mv[this_mode][1].as_mv)))
- continue;
-
- if (filter_idx > 0) {
- BEST_SEG_INFO *ref_bsi = bsi_buf;
- subpelmv = 0;
- have_ref = 1;
-
- for (ref = 0; ref < 1 + has_second_rf; ++ref) {
- subpelmv |= mv_has_subpel(&mode_mv[this_mode][ref].as_mv);
- have_ref &= mode_mv[this_mode][ref].as_int ==
- ref_bsi->rdstat[i][mode_idx].mvs[ref].as_int;
- }
-
- if (filter_idx > 1 && !subpelmv && !have_ref) {
- ref_bsi = bsi_buf + 1;
- have_ref = 1;
- for (ref = 0; ref < 1 + has_second_rf; ++ref)
- have_ref &= mode_mv[this_mode][ref].as_int ==
- ref_bsi->rdstat[i][mode_idx].mvs[ref].as_int;
- }
-
- if (!subpelmv && have_ref &&
- ref_bsi->rdstat[i][mode_idx].brdcost < INT64_MAX) {
- memcpy(&bsi->rdstat[i][mode_idx], &ref_bsi->rdstat[i][mode_idx],
- sizeof(SEG_RDSTAT));
- if (num_4x4_blocks_wide > 1)
- bsi->rdstat[i + 1][mode_idx].eobs =
- ref_bsi->rdstat[i + 1][mode_idx].eobs;
- if (num_4x4_blocks_high > 1)
- bsi->rdstat[i + 2][mode_idx].eobs =
- ref_bsi->rdstat[i + 2][mode_idx].eobs;
-
- if (bsi->rdstat[i][mode_idx].brdcost < best_rd) {
- mode_selected = this_mode;
- best_rd = bsi->rdstat[i][mode_idx].brdcost;
- }
- continue;
- }
- }
-
- bsi->rdstat[i][mode_idx].brdcost =
- encode_inter_mb_segment(cpi, x,
- bsi->segment_rd - this_segment_rd, i,
- &bsi->rdstat[i][mode_idx].byrate,
- &bsi->rdstat[i][mode_idx].bdist,
- &bsi->rdstat[i][mode_idx].bsse,
- bsi->rdstat[i][mode_idx].ta,
- bsi->rdstat[i][mode_idx].tl,
- idy, idx,
- mi_row, mi_col);
- if (bsi->rdstat[i][mode_idx].brdcost < INT64_MAX) {
- bsi->rdstat[i][mode_idx].brdcost += RDCOST(x->rdmult, x->rddiv,
- bsi->rdstat[i][mode_idx].brate, 0);
- bsi->rdstat[i][mode_idx].brate += bsi->rdstat[i][mode_idx].byrate;
- bsi->rdstat[i][mode_idx].eobs = p->eobs[i];
- if (num_4x4_blocks_wide > 1)
- bsi->rdstat[i + 1][mode_idx].eobs = p->eobs[i + 1];
- if (num_4x4_blocks_high > 1)
- bsi->rdstat[i + 2][mode_idx].eobs = p->eobs[i + 2];
- }
-
- if (bsi->rdstat[i][mode_idx].brdcost < best_rd) {
- mode_selected = this_mode;
- best_rd = bsi->rdstat[i][mode_idx].brdcost;
- }
- } /*for each 4x4 mode*/
-
- if (best_rd == INT64_MAX) {
- int iy, midx;
- for (iy = i + 1; iy < 4; ++iy)
- for (midx = 0; midx < INTER_MODES; ++midx)
- bsi->rdstat[iy][midx].brdcost = INT64_MAX;
- bsi->segment_rd = INT64_MAX;
- return INT64_MAX;
- }
-
- mode_idx = INTER_OFFSET(mode_selected);
- memcpy(t_above, bsi->rdstat[i][mode_idx].ta, sizeof(t_above));
- memcpy(t_left, bsi->rdstat[i][mode_idx].tl, sizeof(t_left));
-
- set_and_cost_bmi_mvs(cpi, x, xd, i, mode_selected, mode_mv[mode_selected],
- frame_mv, seg_mvs[i], bsi->ref_mv, x->nmvjointcost,
- x->mvcost);
-
- br += bsi->rdstat[i][mode_idx].brate;
- bd += bsi->rdstat[i][mode_idx].bdist;
- block_sse += bsi->rdstat[i][mode_idx].bsse;
- segmentyrate += bsi->rdstat[i][mode_idx].byrate;
- this_segment_rd += bsi->rdstat[i][mode_idx].brdcost;
-
- if (this_segment_rd > bsi->segment_rd) {
- int iy, midx;
- for (iy = i + 1; iy < 4; ++iy)
- for (midx = 0; midx < INTER_MODES; ++midx)
- bsi->rdstat[iy][midx].brdcost = INT64_MAX;
- bsi->segment_rd = INT64_MAX;
- return INT64_MAX;
- }
- }
- } /* for each label */
-
- bsi->r = br;
- bsi->d = bd;
- bsi->segment_yrate = segmentyrate;
- bsi->segment_rd = this_segment_rd;
- bsi->sse = block_sse;
-
- // update the coding decisions
- for (k = 0; k < 4; ++k)
- bsi->modes[k] = mi->bmi[k].as_mode;
-
- if (bsi->segment_rd > best_rd)
- return INT64_MAX;
- /* set it to the best */
- for (i = 0; i < 4; i++) {
- mode_idx = INTER_OFFSET(bsi->modes[i]);
- mi->bmi[i].as_mv[0].as_int = bsi->rdstat[i][mode_idx].mvs[0].as_int;
- if (has_second_ref(mbmi))
- mi->bmi[i].as_mv[1].as_int = bsi->rdstat[i][mode_idx].mvs[1].as_int;
- x->plane[0].eobs[i] = bsi->rdstat[i][mode_idx].eobs;
- mi->bmi[i].as_mode = bsi->modes[i];
- }
-
- /*
- * used to set mbmi->mv.as_int
- */
- *returntotrate = bsi->r;
- *returndistortion = bsi->d;
- *returnyrate = bsi->segment_yrate;
- *skippable = vp10_is_skippable_in_plane(x, BLOCK_8X8, 0);
- *psse = bsi->sse;
- mbmi->mode = bsi->modes[3];
-
- return bsi->segment_rd;
-}
-
-static void estimate_ref_frame_costs(const VP10_COMMON *cm,
- const MACROBLOCKD *xd,
- int segment_id,
- unsigned int *ref_costs_single,
- unsigned int *ref_costs_comp,
- vpx_prob *comp_mode_p) {
- int seg_ref_active = segfeature_active(&cm->seg, segment_id,
- SEG_LVL_REF_FRAME);
- if (seg_ref_active) {
- memset(ref_costs_single, 0, MAX_REF_FRAMES * sizeof(*ref_costs_single));
- memset(ref_costs_comp, 0, MAX_REF_FRAMES * sizeof(*ref_costs_comp));
- *comp_mode_p = 128;
- } else {
- vpx_prob intra_inter_p = vp10_get_intra_inter_prob(cm, xd);
- vpx_prob comp_inter_p = 128;
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT) {
- comp_inter_p = vp10_get_reference_mode_prob(cm, xd);
- *comp_mode_p = comp_inter_p;
- } else {
- *comp_mode_p = 128;
- }
-
- ref_costs_single[INTRA_FRAME] = vp10_cost_bit(intra_inter_p, 0);
-
- if (cm->reference_mode != COMPOUND_REFERENCE) {
- vpx_prob ref_single_p1 = vp10_get_pred_prob_single_ref_p1(cm, xd);
- vpx_prob ref_single_p2 = vp10_get_pred_prob_single_ref_p2(cm, xd);
- unsigned int base_cost = vp10_cost_bit(intra_inter_p, 1);
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- base_cost += vp10_cost_bit(comp_inter_p, 0);
-
- ref_costs_single[LAST_FRAME] = ref_costs_single[GOLDEN_FRAME] =
- ref_costs_single[ALTREF_FRAME] = base_cost;
- ref_costs_single[LAST_FRAME] += vp10_cost_bit(ref_single_p1, 0);
- ref_costs_single[GOLDEN_FRAME] += vp10_cost_bit(ref_single_p1, 1);
- ref_costs_single[ALTREF_FRAME] += vp10_cost_bit(ref_single_p1, 1);
- ref_costs_single[GOLDEN_FRAME] += vp10_cost_bit(ref_single_p2, 0);
- ref_costs_single[ALTREF_FRAME] += vp10_cost_bit(ref_single_p2, 1);
- } else {
- ref_costs_single[LAST_FRAME] = 512;
- ref_costs_single[GOLDEN_FRAME] = 512;
- ref_costs_single[ALTREF_FRAME] = 512;
- }
- if (cm->reference_mode != SINGLE_REFERENCE) {
- vpx_prob ref_comp_p = vp10_get_pred_prob_comp_ref_p(cm, xd);
- unsigned int base_cost = vp10_cost_bit(intra_inter_p, 1);
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- base_cost += vp10_cost_bit(comp_inter_p, 1);
-
- ref_costs_comp[LAST_FRAME] = base_cost + vp10_cost_bit(ref_comp_p, 0);
- ref_costs_comp[GOLDEN_FRAME] = base_cost + vp10_cost_bit(ref_comp_p, 1);
- } else {
- ref_costs_comp[LAST_FRAME] = 512;
- ref_costs_comp[GOLDEN_FRAME] = 512;
- }
- }
-}
-
-static void store_coding_context(MACROBLOCK *x, PICK_MODE_CONTEXT *ctx,
- int mode_index,
- int64_t comp_pred_diff[REFERENCE_MODES],
- int64_t best_filter_diff[SWITCHABLE_FILTER_CONTEXTS],
- int skippable) {
- MACROBLOCKD *const xd = &x->e_mbd;
-
- // Take a snapshot of the coding context so it can be
- // restored if we decide to encode this way
- ctx->skip = x->skip;
- ctx->skippable = skippable;
- ctx->best_mode_index = mode_index;
- ctx->mic = *xd->mi[0];
- ctx->mbmi_ext = *x->mbmi_ext;
- ctx->single_pred_diff = (int)comp_pred_diff[SINGLE_REFERENCE];
- ctx->comp_pred_diff = (int)comp_pred_diff[COMPOUND_REFERENCE];
- ctx->hybrid_pred_diff = (int)comp_pred_diff[REFERENCE_MODE_SELECT];
-
- memcpy(ctx->best_filter_diff, best_filter_diff,
- sizeof(*best_filter_diff) * SWITCHABLE_FILTER_CONTEXTS);
-}
-
-static void setup_buffer_inter(VP10_COMP *cpi, MACROBLOCK *x,
- MV_REFERENCE_FRAME ref_frame,
- BLOCK_SIZE block_size,
- int mi_row, int mi_col,
- int_mv frame_nearest_mv[MAX_REF_FRAMES],
- int_mv frame_near_mv[MAX_REF_FRAMES],
- struct buf_2d yv12_mb[4][MAX_MB_PLANE]) {
- const VP10_COMMON *cm = &cpi->common;
- const YV12_BUFFER_CONFIG *yv12 = get_ref_frame_buffer(cpi, ref_frame);
- MACROBLOCKD *const xd = &x->e_mbd;
- MODE_INFO *const mi = xd->mi[0];
- int_mv *const candidates = x->mbmi_ext->ref_mvs[ref_frame];
- const struct scale_factors *const sf = &cm->frame_refs[ref_frame - 1].sf;
- MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
-
- assert(yv12 != NULL);
-
- // TODO(jkoleszar): Is the UV buffer ever used here? If so, need to make this
- // use the UV scaling factors.
- vp10_setup_pred_block(xd, yv12_mb[ref_frame], yv12, mi_row, mi_col, sf, sf);
-
- // Gets an initial list of candidate vectors from neighbours and orders them
- vp10_find_mv_refs(cm, xd, mi, ref_frame, candidates, mi_row, mi_col,
- NULL, NULL, mbmi_ext->mode_context);
-
- // Candidate refinement carried out at encoder and decoder
- vp10_find_best_ref_mvs(cm->allow_high_precision_mv, candidates,
- &frame_nearest_mv[ref_frame],
- &frame_near_mv[ref_frame]);
-
- // Further refinement that is encode side only to test the top few candidates
- // in full and choose the best as the centre point for subsequent searches.
- // The current implementation doesn't support scaling.
- if (!vp10_is_scaled(sf) && block_size >= BLOCK_8X8)
- vp10_mv_pred(cpi, x, yv12_mb[ref_frame][0].buf, yv12->y_stride,
- ref_frame, block_size);
-}
-
-static void single_motion_search(VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bsize,
- int mi_row, int mi_col,
- int_mv *tmp_mv, int *rate_mv) {
- MACROBLOCKD *xd = &x->e_mbd;
- const VP10_COMMON *cm = &cpi->common;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- struct buf_2d backup_yv12[MAX_MB_PLANE] = {{0, 0}};
- int bestsme = INT_MAX;
- int step_param;
- int sadpb = x->sadperbit16;
- MV mvp_full;
- int ref = mbmi->ref_frame[0];
- MV ref_mv = x->mbmi_ext->ref_mvs[ref][0].as_mv;
-
- int tmp_col_min = x->mv_col_min;
- int tmp_col_max = x->mv_col_max;
- int tmp_row_min = x->mv_row_min;
- int tmp_row_max = x->mv_row_max;
- int cost_list[5];
-
- const YV12_BUFFER_CONFIG *scaled_ref_frame = vp10_get_scaled_ref_frame(cpi,
- ref);
-
- MV pred_mv[3];
- pred_mv[0] = x->mbmi_ext->ref_mvs[ref][0].as_mv;
- pred_mv[1] = x->mbmi_ext->ref_mvs[ref][1].as_mv;
- pred_mv[2] = x->pred_mv[ref];
-
- if (scaled_ref_frame) {
- int i;
- // Swap out the reference frame for a version that's been scaled to
- // match the resolution of the current frame, allowing the existing
- // motion search code to be used without additional modifications.
- for (i = 0; i < MAX_MB_PLANE; i++)
- backup_yv12[i] = xd->plane[i].pre[0];
-
- vp10_setup_pre_planes(xd, 0, scaled_ref_frame, mi_row, mi_col, NULL);
- }
-
- vp10_set_mv_search_range(x, &ref_mv);
-
- // Work out the size of the first step in the mv step search.
- // 0 here is maximum length first step. 1 is VPXMAX >> 1 etc.
- if (cpi->sf.mv.auto_mv_step_size && cm->show_frame) {
- // Take wtd average of the step_params based on the last frame's
- // max mv magnitude and that based on the best ref mvs of the current
- // block for the given reference.
- step_param = (vp10_init_search_range(x->max_mv_context[ref]) +
- cpi->mv_step_param) / 2;
- } else {
- step_param = cpi->mv_step_param;
- }
-
- if (cpi->sf.adaptive_motion_search && bsize < BLOCK_64X64) {
- int boffset =
- 2 * (b_width_log2_lookup[BLOCK_64X64] -
- VPXMIN(b_height_log2_lookup[bsize], b_width_log2_lookup[bsize]));
- step_param = VPXMAX(step_param, boffset);
- }
-
- if (cpi->sf.adaptive_motion_search) {
- int bwl = b_width_log2_lookup[bsize];
- int bhl = b_height_log2_lookup[bsize];
- int tlevel = x->pred_mv_sad[ref] >> (bwl + bhl + 4);
-
- if (tlevel < 5)
- step_param += 2;
-
- // prev_mv_sad is not setup for dynamically scaled frames.
- if (cpi->oxcf.resize_mode != RESIZE_DYNAMIC) {
- int i;
- for (i = LAST_FRAME; i <= ALTREF_FRAME && cm->show_frame; ++i) {
- if ((x->pred_mv_sad[ref] >> 3) > x->pred_mv_sad[i]) {
- x->pred_mv[ref].row = 0;
- x->pred_mv[ref].col = 0;
- tmp_mv->as_int = INVALID_MV;
-
- if (scaled_ref_frame) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; ++i)
- xd->plane[i].pre[0] = backup_yv12[i];
- }
- return;
- }
- }
- }
- }
-
- mvp_full = pred_mv[x->mv_best_ref_index[ref]];
-
- mvp_full.col >>= 3;
- mvp_full.row >>= 3;
-
- bestsme = vp10_full_pixel_search(cpi, x, bsize, &mvp_full, step_param, sadpb,
- cond_cost_list(cpi, cost_list),
- &ref_mv, &tmp_mv->as_mv, INT_MAX, 1);
-
- x->mv_col_min = tmp_col_min;
- x->mv_col_max = tmp_col_max;
- x->mv_row_min = tmp_row_min;
- x->mv_row_max = tmp_row_max;
-
- if (bestsme < INT_MAX) {
- int dis; /* TODO: use dis in distortion calculation later. */
- cpi->find_fractional_mv_step(x, &tmp_mv->as_mv, &ref_mv,
- cm->allow_high_precision_mv,
- x->errorperbit,
- &cpi->fn_ptr[bsize],
- cpi->sf.mv.subpel_force_stop,
- cpi->sf.mv.subpel_iters_per_step,
- cond_cost_list(cpi, cost_list),
- x->nmvjointcost, x->mvcost,
- &dis, &x->pred_sse[ref], NULL, 0, 0);
- }
- *rate_mv = vp10_mv_bit_cost(&tmp_mv->as_mv, &ref_mv,
- x->nmvjointcost, x->mvcost, MV_COST_WEIGHT);
-
- if (cpi->sf.adaptive_motion_search)
- x->pred_mv[ref] = tmp_mv->as_mv;
-
- if (scaled_ref_frame) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++)
- xd->plane[i].pre[0] = backup_yv12[i];
- }
-}
-
-
-
-static INLINE void restore_dst_buf(MACROBLOCKD *xd,
- uint8_t *orig_dst[MAX_MB_PLANE],
- int orig_dst_stride[MAX_MB_PLANE]) {
- int i;
- for (i = 0; i < MAX_MB_PLANE; i++) {
- xd->plane[i].dst.buf = orig_dst[i];
- xd->plane[i].dst.stride = orig_dst_stride[i];
- }
-}
-
-// In some situations we want to discount tha pparent cost of a new motion
-// vector. Where there is a subtle motion field and especially where there is
-// low spatial complexity then it can be hard to cover the cost of a new motion
-// vector in a single block, even if that motion vector reduces distortion.
-// However, once established that vector may be usable through the nearest and
-// near mv modes to reduce distortion in subsequent blocks and also improve
-// visual quality.
-static int discount_newmv_test(const VP10_COMP *cpi,
- int this_mode,
- int_mv this_mv,
- int_mv (*mode_mv)[MAX_REF_FRAMES],
- int ref_frame) {
- return (!cpi->rc.is_src_frame_alt_ref &&
- (this_mode == NEWMV) &&
- (this_mv.as_int != 0) &&
- ((mode_mv[NEARESTMV][ref_frame].as_int == 0) ||
- (mode_mv[NEARESTMV][ref_frame].as_int == INVALID_MV)) &&
- ((mode_mv[NEARMV][ref_frame].as_int == 0) ||
- (mode_mv[NEARMV][ref_frame].as_int == INVALID_MV)));
-}
-
-#define LEFT_TOP_MARGIN ((VP9_ENC_BORDER_IN_PIXELS - VP9_INTERP_EXTEND) << 3)
-#define RIGHT_BOTTOM_MARGIN ((VP9_ENC_BORDER_IN_PIXELS -\
- VP9_INTERP_EXTEND) << 3)
-
-// TODO(jingning): this mv clamping function should be block size dependent.
-static INLINE void clamp_mv2(MV *mv, const MACROBLOCKD *xd) {
- clamp_mv(mv, xd->mb_to_left_edge - LEFT_TOP_MARGIN,
- xd->mb_to_right_edge + RIGHT_BOTTOM_MARGIN,
- xd->mb_to_top_edge - LEFT_TOP_MARGIN,
- xd->mb_to_bottom_edge + RIGHT_BOTTOM_MARGIN);
-}
-
-static int64_t handle_inter_mode(VP10_COMP *cpi, MACROBLOCK *x,
- BLOCK_SIZE bsize,
- int *rate2, int64_t *distortion,
- int *skippable,
- int *rate_y, int *rate_uv,
- int *disable_skip,
- int_mv (*mode_mv)[MAX_REF_FRAMES],
- int mi_row, int mi_col,
- int_mv single_newmv[MAX_REF_FRAMES],
- INTERP_FILTER (*single_filter)[MAX_REF_FRAMES],
- int (*single_skippable)[MAX_REF_FRAMES],
- int64_t *psse,
- const int64_t ref_best_rd,
- int64_t *mask_filter,
- int64_t filter_cache[]) {
- VP10_COMMON *cm = &cpi->common;
- MACROBLOCKD *xd = &x->e_mbd;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
- const int is_comp_pred = has_second_ref(mbmi);
- const int this_mode = mbmi->mode;
- int_mv *frame_mv = mode_mv[this_mode];
- int i;
- int refs[2] = { mbmi->ref_frame[0],
- (mbmi->ref_frame[1] < 0 ? 0 : mbmi->ref_frame[1]) };
- int_mv cur_mv[2];
-#if CONFIG_VP9_HIGHBITDEPTH
- DECLARE_ALIGNED(16, uint16_t, tmp_buf16[MAX_MB_PLANE * 64 * 64]);
- uint8_t *tmp_buf;
-#else
- DECLARE_ALIGNED(16, uint8_t, tmp_buf[MAX_MB_PLANE * 64 * 64]);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- int pred_exists = 0;
- int intpel_mv;
- int64_t rd, tmp_rd, best_rd = INT64_MAX;
- int best_needs_copy = 0;
- uint8_t *orig_dst[MAX_MB_PLANE];
- int orig_dst_stride[MAX_MB_PLANE];
- int rs = 0;
- INTERP_FILTER best_filter = SWITCHABLE;
- uint8_t skip_txfm[MAX_MB_PLANE << 2] = {0};
- int64_t bsse[MAX_MB_PLANE << 2] = {0};
-
- int bsl = mi_width_log2_lookup[bsize];
- int pred_filter_search = cpi->sf.cb_pred_filter_search ?
- (((mi_row + mi_col) >> bsl) +
- get_chessboard_index(cm->current_video_frame)) & 0x1 : 0;
-
- int skip_txfm_sb = 0;
- int64_t skip_sse_sb = INT64_MAX;
- int64_t distortion_y = 0, distortion_uv = 0;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- tmp_buf = CONVERT_TO_BYTEPTR(tmp_buf16);
- } else {
- tmp_buf = (uint8_t *)tmp_buf16;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- if (pred_filter_search) {
- INTERP_FILTER af = SWITCHABLE, lf = SWITCHABLE;
- if (xd->up_available)
- af = xd->mi[-xd->mi_stride]->mbmi.interp_filter;
- if (xd->left_available)
- lf = xd->mi[-1]->mbmi.interp_filter;
-
- if ((this_mode != NEWMV) || (af == lf))
- best_filter = af;
- }
-
- if (is_comp_pred) {
- if (frame_mv[refs[0]].as_int == INVALID_MV ||
- frame_mv[refs[1]].as_int == INVALID_MV)
- return INT64_MAX;
-
- if (cpi->sf.adaptive_mode_search) {
- if (single_filter[this_mode][refs[0]] ==
- single_filter[this_mode][refs[1]])
- best_filter = single_filter[this_mode][refs[0]];
- }
- }
-
- if (this_mode == NEWMV) {
- int rate_mv;
- if (is_comp_pred) {
- // Initialize mv using single prediction mode result.
- frame_mv[refs[0]].as_int = single_newmv[refs[0]].as_int;
- frame_mv[refs[1]].as_int = single_newmv[refs[1]].as_int;
-
- if (cpi->sf.comp_inter_joint_search_thresh <= bsize) {
- joint_motion_search(cpi, x, bsize, frame_mv,
- mi_row, mi_col, single_newmv, &rate_mv);
- } else {
- rate_mv = vp10_mv_bit_cost(&frame_mv[refs[0]].as_mv,
- &x->mbmi_ext->ref_mvs[refs[0]][0].as_mv,
- x->nmvjointcost, x->mvcost, MV_COST_WEIGHT);
- rate_mv += vp10_mv_bit_cost(&frame_mv[refs[1]].as_mv,
- &x->mbmi_ext->ref_mvs[refs[1]][0].as_mv,
- x->nmvjointcost, x->mvcost, MV_COST_WEIGHT);
- }
- *rate2 += rate_mv;
- } else {
- int_mv tmp_mv;
- single_motion_search(cpi, x, bsize, mi_row, mi_col,
- &tmp_mv, &rate_mv);
- if (tmp_mv.as_int == INVALID_MV)
- return INT64_MAX;
-
- frame_mv[refs[0]].as_int =
- xd->mi[0]->bmi[0].as_mv[0].as_int = tmp_mv.as_int;
- single_newmv[refs[0]].as_int = tmp_mv.as_int;
-
- // Estimate the rate implications of a new mv but discount this
- // under certain circumstances where we want to help initiate a weak
- // motion field, where the distortion gain for a single block may not
- // be enough to overcome the cost of a new mv.
- if (discount_newmv_test(cpi, this_mode, tmp_mv, mode_mv, refs[0])) {
- *rate2 += VPXMAX((rate_mv / NEW_MV_DISCOUNT_FACTOR), 1);
- } else {
- *rate2 += rate_mv;
- }
- }
- }
-
- for (i = 0; i < is_comp_pred + 1; ++i) {
- cur_mv[i] = frame_mv[refs[i]];
- // Clip "next_nearest" so that it does not extend to far out of image
- if (this_mode != NEWMV)
- clamp_mv2(&cur_mv[i].as_mv, xd);
-
- if (mv_check_bounds(x, &cur_mv[i].as_mv))
- return INT64_MAX;
- mbmi->mv[i].as_int = cur_mv[i].as_int;
- }
-
- // do first prediction into the destination buffer. Do the next
- // prediction into a temporary buffer. Then keep track of which one
- // of these currently holds the best predictor, and use the other
- // one for future predictions. In the end, copy from tmp_buf to
- // dst if necessary.
- for (i = 0; i < MAX_MB_PLANE; i++) {
- orig_dst[i] = xd->plane[i].dst.buf;
- orig_dst_stride[i] = xd->plane[i].dst.stride;
- }
-
- // We don't include the cost of the second reference here, because there
- // are only three options: Last/Golden, ARF/Last or Golden/ARF, or in other
- // words if you present them in that order, the second one is always known
- // if the first is known.
- //
- // Under some circumstances we discount the cost of new mv mode to encourage
- // initiation of a motion field.
- if (discount_newmv_test(cpi, this_mode, frame_mv[refs[0]],
- mode_mv, refs[0])) {
- *rate2 += VPXMIN(cost_mv_ref(cpi, this_mode,
- mbmi_ext->mode_context[refs[0]]),
- cost_mv_ref(cpi, NEARESTMV,
- mbmi_ext->mode_context[refs[0]]));
- } else {
- *rate2 += cost_mv_ref(cpi, this_mode, mbmi_ext->mode_context[refs[0]]);
- }
-
- if (RDCOST(x->rdmult, x->rddiv, *rate2, 0) > ref_best_rd &&
- mbmi->mode != NEARESTMV)
- return INT64_MAX;
-
- pred_exists = 0;
- // Are all MVs integer pel for Y and UV
- intpel_mv = !mv_has_subpel(&mbmi->mv[0].as_mv);
- if (is_comp_pred)
- intpel_mv &= !mv_has_subpel(&mbmi->mv[1].as_mv);
-
- // Search for best switchable filter by checking the variance of
- // pred error irrespective of whether the filter will be used
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- filter_cache[i] = INT64_MAX;
-
- if (cm->interp_filter != BILINEAR) {
- if (x->source_variance < cpi->sf.disable_filter_search_var_thresh) {
- best_filter = EIGHTTAP;
- } else if (best_filter == SWITCHABLE) {
- int newbest;
- int tmp_rate_sum = 0;
- int64_t tmp_dist_sum = 0;
-
- for (i = 0; i < SWITCHABLE_FILTERS; ++i) {
- int j;
- int64_t rs_rd;
- int tmp_skip_sb = 0;
- int64_t tmp_skip_sse = INT64_MAX;
-
- mbmi->interp_filter = i;
- rs = vp10_get_switchable_rate(cpi, xd);
- rs_rd = RDCOST(x->rdmult, x->rddiv, rs, 0);
-
- if (i > 0 && intpel_mv) {
- rd = RDCOST(x->rdmult, x->rddiv, tmp_rate_sum, tmp_dist_sum);
- filter_cache[i] = rd;
- filter_cache[SWITCHABLE_FILTERS] =
- VPXMIN(filter_cache[SWITCHABLE_FILTERS], rd + rs_rd);
- if (cm->interp_filter == SWITCHABLE)
- rd += rs_rd;
- *mask_filter = VPXMAX(*mask_filter, rd);
- } else {
- int rate_sum = 0;
- int64_t dist_sum = 0;
- if (i > 0 && cpi->sf.adaptive_interp_filter_search &&
- (cpi->sf.interp_filter_search_mask & (1 << i))) {
- rate_sum = INT_MAX;
- dist_sum = INT64_MAX;
- continue;
- }
-
- if ((cm->interp_filter == SWITCHABLE &&
- (!i || best_needs_copy)) ||
- (cm->interp_filter != SWITCHABLE &&
- (cm->interp_filter == mbmi->interp_filter ||
- (i == 0 && intpel_mv)))) {
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- } else {
- for (j = 0; j < MAX_MB_PLANE; j++) {
- xd->plane[j].dst.buf = tmp_buf + j * 64 * 64;
- xd->plane[j].dst.stride = 64;
- }
- }
- vp10_build_inter_predictors_sb(xd, mi_row, mi_col, bsize);
- model_rd_for_sb(cpi, bsize, x, xd, &rate_sum, &dist_sum,
- &tmp_skip_sb, &tmp_skip_sse);
-
- rd = RDCOST(x->rdmult, x->rddiv, rate_sum, dist_sum);
- filter_cache[i] = rd;
- filter_cache[SWITCHABLE_FILTERS] =
- VPXMIN(filter_cache[SWITCHABLE_FILTERS], rd + rs_rd);
- if (cm->interp_filter == SWITCHABLE)
- rd += rs_rd;
- *mask_filter = VPXMAX(*mask_filter, rd);
-
- if (i == 0 && intpel_mv) {
- tmp_rate_sum = rate_sum;
- tmp_dist_sum = dist_sum;
- }
- }
-
- if (i == 0 && cpi->sf.use_rd_breakout && ref_best_rd < INT64_MAX) {
- if (rd / 2 > ref_best_rd) {
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- return INT64_MAX;
- }
- }
- newbest = i == 0 || rd < best_rd;
-
- if (newbest) {
- best_rd = rd;
- best_filter = mbmi->interp_filter;
- if (cm->interp_filter == SWITCHABLE && i && !intpel_mv)
- best_needs_copy = !best_needs_copy;
- }
-
- if ((cm->interp_filter == SWITCHABLE && newbest) ||
- (cm->interp_filter != SWITCHABLE &&
- cm->interp_filter == mbmi->interp_filter)) {
- pred_exists = 1;
- tmp_rd = best_rd;
-
- skip_txfm_sb = tmp_skip_sb;
- skip_sse_sb = tmp_skip_sse;
- memcpy(skip_txfm, x->skip_txfm, sizeof(skip_txfm));
- memcpy(bsse, x->bsse, sizeof(bsse));
- }
- }
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- }
- }
- // Set the appropriate filter
- mbmi->interp_filter = cm->interp_filter != SWITCHABLE ?
- cm->interp_filter : best_filter;
- rs = cm->interp_filter == SWITCHABLE ? vp10_get_switchable_rate(cpi, xd) : 0;
-
- if (pred_exists) {
- if (best_needs_copy) {
- // again temporarily set the buffers to local memory to prevent a memcpy
- for (i = 0; i < MAX_MB_PLANE; i++) {
- xd->plane[i].dst.buf = tmp_buf + i * 64 * 64;
- xd->plane[i].dst.stride = 64;
- }
- }
- rd = tmp_rd + RDCOST(x->rdmult, x->rddiv, rs, 0);
- } else {
- int tmp_rate;
- int64_t tmp_dist;
- // Handles the special case when a filter that is not in the
- // switchable list (ex. bilinear) is indicated at the frame level, or
- // skip condition holds.
- vp10_build_inter_predictors_sb(xd, mi_row, mi_col, bsize);
- model_rd_for_sb(cpi, bsize, x, xd, &tmp_rate, &tmp_dist,
- &skip_txfm_sb, &skip_sse_sb);
- rd = RDCOST(x->rdmult, x->rddiv, rs + tmp_rate, tmp_dist);
- memcpy(skip_txfm, x->skip_txfm, sizeof(skip_txfm));
- memcpy(bsse, x->bsse, sizeof(bsse));
- }
-
- if (!is_comp_pred)
- single_filter[this_mode][refs[0]] = mbmi->interp_filter;
-
- if (cpi->sf.adaptive_mode_search)
- if (is_comp_pred)
- if (single_skippable[this_mode][refs[0]] &&
- single_skippable[this_mode][refs[1]])
- memset(skip_txfm, SKIP_TXFM_AC_DC, sizeof(skip_txfm));
-
- if (cpi->sf.use_rd_breakout && ref_best_rd < INT64_MAX) {
- // if current pred_error modeled rd is substantially more than the best
- // so far, do not bother doing full rd
- if (rd / 2 > ref_best_rd) {
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- return INT64_MAX;
- }
- }
-
- if (cm->interp_filter == SWITCHABLE)
- *rate2 += rs;
-
- memcpy(x->skip_txfm, skip_txfm, sizeof(skip_txfm));
- memcpy(x->bsse, bsse, sizeof(bsse));
-
- if (!skip_txfm_sb) {
- int skippable_y, skippable_uv;
- int64_t sseuv = INT64_MAX;
- int64_t rdcosty = INT64_MAX;
-
- // Y cost and distortion
- vp10_subtract_plane(x, bsize, 0);
- super_block_yrd(cpi, x, rate_y, &distortion_y, &skippable_y, psse,
- bsize, ref_best_rd);
-
- if (*rate_y == INT_MAX) {
- *rate2 = INT_MAX;
- *distortion = INT64_MAX;
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- return INT64_MAX;
- }
-
- *rate2 += *rate_y;
- *distortion += distortion_y;
-
- rdcosty = RDCOST(x->rdmult, x->rddiv, *rate2, *distortion);
- rdcosty = VPXMIN(rdcosty, RDCOST(x->rdmult, x->rddiv, 0, *psse));
-
- if (!super_block_uvrd(cpi, x, rate_uv, &distortion_uv, &skippable_uv,
- &sseuv, bsize, ref_best_rd - rdcosty)) {
- *rate2 = INT_MAX;
- *distortion = INT64_MAX;
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- return INT64_MAX;
- }
-
- *psse += sseuv;
- *rate2 += *rate_uv;
- *distortion += distortion_uv;
- *skippable = skippable_y && skippable_uv;
- } else {
- x->skip = 1;
- *disable_skip = 1;
-
- // The cost of skip bit needs to be added.
- *rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 1);
-
- *distortion = skip_sse_sb;
- }
-
- if (!is_comp_pred)
- single_skippable[this_mode][refs[0]] = *skippable;
-
- restore_dst_buf(xd, orig_dst, orig_dst_stride);
- return 0; // The rate-distortion cost will be re-calculated by caller.
-}
-
-void vp10_rd_pick_intra_mode_sb(VP10_COMP *cpi, MACROBLOCK *x,
- RD_COST *rd_cost, BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx, int64_t best_rd) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- struct macroblockd_plane *const pd = xd->plane;
- int rate_y = 0, rate_uv = 0, rate_y_tokenonly = 0, rate_uv_tokenonly = 0;
- int y_skip = 0, uv_skip = 0;
- int64_t dist_y = 0, dist_uv = 0;
- TX_SIZE max_uv_tx_size;
- ctx->skip = 0;
- xd->mi[0]->mbmi.ref_frame[0] = INTRA_FRAME;
- xd->mi[0]->mbmi.ref_frame[1] = NONE;
-
- if (bsize >= BLOCK_8X8) {
- if (rd_pick_intra_sby_mode(cpi, x, &rate_y, &rate_y_tokenonly,
- &dist_y, &y_skip, bsize,
- best_rd) >= best_rd) {
- rd_cost->rate = INT_MAX;
- return;
- }
- } else {
- y_skip = 0;
- if (rd_pick_intra_sub_8x8_y_mode(cpi, x, &rate_y, &rate_y_tokenonly,
- &dist_y, best_rd) >= best_rd) {
- rd_cost->rate = INT_MAX;
- return;
- }
- }
- max_uv_tx_size = get_uv_tx_size_impl(xd->mi[0]->mbmi.tx_size, bsize,
- pd[1].subsampling_x,
- pd[1].subsampling_y);
- rd_pick_intra_sbuv_mode(cpi, x, ctx, &rate_uv, &rate_uv_tokenonly,
- &dist_uv, &uv_skip, VPXMAX(BLOCK_8X8, bsize),
- max_uv_tx_size);
-
- if (y_skip && uv_skip) {
- rd_cost->rate = rate_y + rate_uv - rate_y_tokenonly - rate_uv_tokenonly +
- vp10_cost_bit(vp10_get_skip_prob(cm, xd), 1);
- rd_cost->dist = dist_y + dist_uv;
- } else {
- rd_cost->rate = rate_y + rate_uv +
- vp10_cost_bit(vp10_get_skip_prob(cm, xd), 0);
- rd_cost->dist = dist_y + dist_uv;
- }
-
- ctx->mic = *xd->mi[0];
- ctx->mbmi_ext = *x->mbmi_ext;
- rd_cost->rdcost = RDCOST(x->rdmult, x->rddiv, rd_cost->rate, rd_cost->dist);
-}
-
-// This function is designed to apply a bias or adjustment to an rd value based
-// on the relative variance of the source and reconstruction.
-#define LOW_VAR_THRESH 16
-#define VLOW_ADJ_MAX 25
-#define VHIGH_ADJ_MAX 8
-static void rd_variance_adjustment(VP10_COMP *cpi,
- MACROBLOCK *x,
- BLOCK_SIZE bsize,
- int64_t *this_rd,
- MV_REFERENCE_FRAME ref_frame,
- unsigned int source_variance) {
- MACROBLOCKD *const xd = &x->e_mbd;
- unsigned int recon_variance;
- unsigned int absvar_diff = 0;
- int64_t var_error = 0;
- int64_t var_factor = 0;
-
- if (*this_rd == INT64_MAX)
- return;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- recon_variance =
- vp10_high_get_sby_perpixel_variance(cpi, &xd->plane[0].dst, bsize, xd->bd);
- } else {
- recon_variance =
- vp10_get_sby_perpixel_variance(cpi, &xd->plane[0].dst, bsize);
- }
-#else
- recon_variance =
- vp10_get_sby_perpixel_variance(cpi, &xd->plane[0].dst, bsize);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
- if ((source_variance + recon_variance) > LOW_VAR_THRESH) {
- absvar_diff = (source_variance > recon_variance)
- ? (source_variance - recon_variance)
- : (recon_variance - source_variance);
-
- var_error = (200 * source_variance * recon_variance) /
- ((source_variance * source_variance) +
- (recon_variance * recon_variance));
- var_error = 100 - var_error;
- }
-
- // Source variance above a threshold and ref frame is intra.
- // This case is targeted mainly at discouraging intra modes that give rise
- // to a predictor with a low spatial complexity compared to the source.
- if ((source_variance > LOW_VAR_THRESH) && (ref_frame == INTRA_FRAME) &&
- (source_variance > recon_variance)) {
- var_factor = VPXMIN(absvar_diff, VPXMIN(VLOW_ADJ_MAX, var_error));
- // A second possible case of interest is where the source variance
- // is very low and we wish to discourage false texture or motion trails.
- } else if ((source_variance < (LOW_VAR_THRESH >> 1)) &&
- (recon_variance > source_variance)) {
- var_factor = VPXMIN(absvar_diff, VPXMIN(VHIGH_ADJ_MAX, var_error));
- }
- *this_rd += (*this_rd * var_factor) / 100;
-}
-
-
-// Do we have an internal image edge (e.g. formatting bars).
-int vp10_internal_image_edge(VP10_COMP *cpi) {
- return (cpi->oxcf.pass == 2) &&
- ((cpi->twopass.this_frame_stats.inactive_zone_rows > 0) ||
- (cpi->twopass.this_frame_stats.inactive_zone_cols > 0));
-}
-
-// Checks to see if a super block is on a horizontal image edge.
-// In most cases this is the "real" edge unless there are formatting
-// bars embedded in the stream.
-int vp10_active_h_edge(VP10_COMP *cpi, int mi_row, int mi_step) {
- int top_edge = 0;
- int bottom_edge = cpi->common.mi_rows;
- int is_active_h_edge = 0;
-
- // For two pass account for any formatting bars detected.
- if (cpi->oxcf.pass == 2) {
- TWO_PASS *twopass = &cpi->twopass;
-
- // The inactive region is specified in MBs not mi units.
- // The image edge is in the following MB row.
- top_edge += (int)(twopass->this_frame_stats.inactive_zone_rows * 2);
-
- bottom_edge -= (int)(twopass->this_frame_stats.inactive_zone_rows * 2);
- bottom_edge = VPXMAX(top_edge, bottom_edge);
- }
-
- if (((top_edge >= mi_row) && (top_edge < (mi_row + mi_step))) ||
- ((bottom_edge >= mi_row) && (bottom_edge < (mi_row + mi_step)))) {
- is_active_h_edge = 1;
- }
- return is_active_h_edge;
-}
-
-// Checks to see if a super block is on a vertical image edge.
-// In most cases this is the "real" edge unless there are formatting
-// bars embedded in the stream.
-int vp10_active_v_edge(VP10_COMP *cpi, int mi_col, int mi_step) {
- int left_edge = 0;
- int right_edge = cpi->common.mi_cols;
- int is_active_v_edge = 0;
-
- // For two pass account for any formatting bars detected.
- if (cpi->oxcf.pass == 2) {
- TWO_PASS *twopass = &cpi->twopass;
-
- // The inactive region is specified in MBs not mi units.
- // The image edge is in the following MB row.
- left_edge += (int)(twopass->this_frame_stats.inactive_zone_cols * 2);
-
- right_edge -= (int)(twopass->this_frame_stats.inactive_zone_cols * 2);
- right_edge = VPXMAX(left_edge, right_edge);
- }
-
- if (((left_edge >= mi_col) && (left_edge < (mi_col + mi_step))) ||
- ((right_edge >= mi_col) && (right_edge < (mi_col + mi_step)))) {
- is_active_v_edge = 1;
- }
- return is_active_v_edge;
-}
-
-// Checks to see if a super block is at the edge of the active image.
-// In most cases this is the "real" edge unless there are formatting
-// bars embedded in the stream.
-int vp10_active_edge_sb(VP10_COMP *cpi,
- int mi_row, int mi_col) {
- return vp10_active_h_edge(cpi, mi_row, MI_BLOCK_SIZE) ||
- vp10_active_v_edge(cpi, mi_col, MI_BLOCK_SIZE);
-}
-
-void vp10_rd_pick_inter_mode_sb(VP10_COMP *cpi,
- TileDataEnc *tile_data,
- MACROBLOCK *x,
- int mi_row, int mi_col,
- RD_COST *rd_cost, BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx,
- int64_t best_rd_so_far) {
- VP10_COMMON *const cm = &cpi->common;
- RD_OPT *const rd_opt = &cpi->rd;
- SPEED_FEATURES *const sf = &cpi->sf;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- MB_MODE_INFO_EXT *const mbmi_ext = x->mbmi_ext;
- const struct segmentation *const seg = &cm->seg;
- PREDICTION_MODE this_mode;
- MV_REFERENCE_FRAME ref_frame, second_ref_frame;
- unsigned char segment_id = mbmi->segment_id;
- int comp_pred, i, k;
- int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES];
- struct buf_2d yv12_mb[4][MAX_MB_PLANE];
- int_mv single_newmv[MAX_REF_FRAMES] = { { 0 } };
- INTERP_FILTER single_inter_filter[MB_MODE_COUNT][MAX_REF_FRAMES];
- int single_skippable[MB_MODE_COUNT][MAX_REF_FRAMES];
- static const int flag_list[4] = { 0, VP9_LAST_FLAG, VP9_GOLD_FLAG,
- VP9_ALT_FLAG };
- int64_t best_rd = best_rd_so_far;
- int64_t best_pred_diff[REFERENCE_MODES];
- int64_t best_pred_rd[REFERENCE_MODES];
- int64_t best_filter_rd[SWITCHABLE_FILTER_CONTEXTS];
- int64_t best_filter_diff[SWITCHABLE_FILTER_CONTEXTS];
- MB_MODE_INFO best_mbmode;
- int best_mode_skippable = 0;
- int midx, best_mode_index = -1;
- unsigned int ref_costs_single[MAX_REF_FRAMES], ref_costs_comp[MAX_REF_FRAMES];
- vpx_prob comp_mode_p;
- int64_t best_intra_rd = INT64_MAX;
- unsigned int best_pred_sse = UINT_MAX;
- PREDICTION_MODE best_intra_mode = DC_PRED;
- int rate_uv_intra[TX_SIZES], rate_uv_tokenonly[TX_SIZES];
- int64_t dist_uv[TX_SIZES];
- int skip_uv[TX_SIZES];
- PREDICTION_MODE mode_uv[TX_SIZES];
- const int intra_cost_penalty = vp10_get_intra_cost_penalty(
- cm->base_qindex, cm->y_dc_delta_q, cm->bit_depth);
- int best_skip2 = 0;
- uint8_t ref_frame_skip_mask[2] = { 0 };
- uint16_t mode_skip_mask[MAX_REF_FRAMES] = { 0 };
- int mode_skip_start = sf->mode_skip_start + 1;
- const int *const rd_threshes = rd_opt->threshes[segment_id][bsize];
- const int *const rd_thresh_freq_fact = tile_data->thresh_freq_fact[bsize];
- int64_t mode_threshold[MAX_MODES];
- int *mode_map = tile_data->mode_map[bsize];
- const int mode_search_skip_flags = sf->mode_search_skip_flags;
- int64_t mask_filter = 0;
- int64_t filter_cache[SWITCHABLE_FILTER_CONTEXTS];
-
- vp10_zero(best_mbmode);
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- filter_cache[i] = INT64_MAX;
-
- estimate_ref_frame_costs(cm, xd, segment_id, ref_costs_single, ref_costs_comp,
- &comp_mode_p);
-
- for (i = 0; i < REFERENCE_MODES; ++i)
- best_pred_rd[i] = INT64_MAX;
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- best_filter_rd[i] = INT64_MAX;
- for (i = 0; i < TX_SIZES; i++)
- rate_uv_intra[i] = INT_MAX;
- for (i = 0; i < MAX_REF_FRAMES; ++i)
- x->pred_sse[i] = INT_MAX;
- for (i = 0; i < MB_MODE_COUNT; ++i) {
- for (k = 0; k < MAX_REF_FRAMES; ++k) {
- single_inter_filter[i][k] = SWITCHABLE;
- single_skippable[i][k] = 0;
- }
- }
-
- rd_cost->rate = INT_MAX;
-
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ++ref_frame) {
- x->pred_mv_sad[ref_frame] = INT_MAX;
- if (cpi->ref_frame_flags & flag_list[ref_frame]) {
- assert(get_ref_frame_buffer(cpi, ref_frame) != NULL);
- setup_buffer_inter(cpi, x, ref_frame, bsize, mi_row, mi_col,
- frame_mv[NEARESTMV], frame_mv[NEARMV], yv12_mb);
- }
- frame_mv[NEWMV][ref_frame].as_int = INVALID_MV;
- frame_mv[ZEROMV][ref_frame].as_int = 0;
- }
-
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ++ref_frame) {
- if (!(cpi->ref_frame_flags & flag_list[ref_frame])) {
- // Skip checking missing references in both single and compound reference
- // modes. Note that a mode will be skipped iff both reference frames
- // are masked out.
- ref_frame_skip_mask[0] |= (1 << ref_frame);
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- } else {
- for (i = LAST_FRAME; i <= ALTREF_FRAME; ++i) {
- // Skip fixed mv modes for poor references
- if ((x->pred_mv_sad[ref_frame] >> 2) > x->pred_mv_sad[i]) {
- mode_skip_mask[ref_frame] |= INTER_NEAREST_NEAR_ZERO;
- break;
- }
- }
- }
- // If the segment reference frame feature is enabled....
- // then do nothing if the current ref frame is not allowed..
- if (segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME) &&
- get_segdata(seg, segment_id, SEG_LVL_REF_FRAME) != (int)ref_frame) {
- ref_frame_skip_mask[0] |= (1 << ref_frame);
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- }
- }
-
- // Disable this drop out case if the ref frame
- // segment level feature is enabled for this segment. This is to
- // prevent the possibility that we end up unable to pick any mode.
- if (!segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME)) {
- // Only consider ZEROMV/ALTREF_FRAME for alt ref frame,
- // unless ARNR filtering is enabled in which case we want
- // an unfiltered alternative. We allow near/nearest as well
- // because they may result in zero-zero MVs but be cheaper.
- if (cpi->rc.is_src_frame_alt_ref && (cpi->oxcf.arnr_max_frames == 0)) {
- ref_frame_skip_mask[0] = (1 << LAST_FRAME) | (1 << GOLDEN_FRAME);
- ref_frame_skip_mask[1] = SECOND_REF_FRAME_MASK;
- mode_skip_mask[ALTREF_FRAME] = ~INTER_NEAREST_NEAR_ZERO;
- if (frame_mv[NEARMV][ALTREF_FRAME].as_int != 0)
- mode_skip_mask[ALTREF_FRAME] |= (1 << NEARMV);
- if (frame_mv[NEARESTMV][ALTREF_FRAME].as_int != 0)
- mode_skip_mask[ALTREF_FRAME] |= (1 << NEARESTMV);
- }
- }
-
- if (cpi->rc.is_src_frame_alt_ref) {
- if (sf->alt_ref_search_fp) {
- mode_skip_mask[ALTREF_FRAME] = 0;
- ref_frame_skip_mask[0] = ~(1 << ALTREF_FRAME);
- ref_frame_skip_mask[1] = SECOND_REF_FRAME_MASK;
- }
- }
-
- if (sf->alt_ref_search_fp)
- if (!cm->show_frame && x->pred_mv_sad[GOLDEN_FRAME] < INT_MAX)
- if (x->pred_mv_sad[ALTREF_FRAME] > (x->pred_mv_sad[GOLDEN_FRAME] << 1))
- mode_skip_mask[ALTREF_FRAME] |= INTER_ALL;
-
- if (sf->adaptive_mode_search) {
- if (cm->show_frame && !cpi->rc.is_src_frame_alt_ref &&
- cpi->rc.frames_since_golden >= 3)
- if (x->pred_mv_sad[GOLDEN_FRAME] > (x->pred_mv_sad[LAST_FRAME] << 1))
- mode_skip_mask[GOLDEN_FRAME] |= INTER_ALL;
- }
-
- if (bsize > sf->max_intra_bsize) {
- ref_frame_skip_mask[0] |= (1 << INTRA_FRAME);
- ref_frame_skip_mask[1] |= (1 << INTRA_FRAME);
- }
-
- mode_skip_mask[INTRA_FRAME] |=
- ~(sf->intra_y_mode_mask[max_txsize_lookup[bsize]]);
-
- for (i = 0; i <= LAST_NEW_MV_INDEX; ++i)
- mode_threshold[i] = 0;
- for (i = LAST_NEW_MV_INDEX + 1; i < MAX_MODES; ++i)
- mode_threshold[i] = ((int64_t)rd_threshes[i] * rd_thresh_freq_fact[i]) >> 5;
-
- midx = sf->schedule_mode_search ? mode_skip_start : 0;
- while (midx > 4) {
- uint8_t end_pos = 0;
- for (i = 5; i < midx; ++i) {
- if (mode_threshold[mode_map[i - 1]] > mode_threshold[mode_map[i]]) {
- uint8_t tmp = mode_map[i];
- mode_map[i] = mode_map[i - 1];
- mode_map[i - 1] = tmp;
- end_pos = i;
- }
- }
- midx = end_pos;
- }
-
- mbmi->palette_mode_info.palette_size[0] = 0;
- mbmi->palette_mode_info.palette_size[1] = 0;
- for (midx = 0; midx < MAX_MODES; ++midx) {
- int mode_index = mode_map[midx];
- int mode_excluded = 0;
- int64_t this_rd = INT64_MAX;
- int disable_skip = 0;
- int compmode_cost = 0;
- int rate2 = 0, rate_y = 0, rate_uv = 0;
- int64_t distortion2 = 0, distortion_y = 0, distortion_uv = 0;
- int skippable = 0;
- int this_skip2 = 0;
- int64_t total_sse = INT64_MAX;
- int early_term = 0;
-
- this_mode = vp10_mode_order[mode_index].mode;
- ref_frame = vp10_mode_order[mode_index].ref_frame[0];
- second_ref_frame = vp10_mode_order[mode_index].ref_frame[1];
-
- // Look at the reference frame of the best mode so far and set the
- // skip mask to look at a subset of the remaining modes.
- if (midx == mode_skip_start && best_mode_index >= 0) {
- switch (best_mbmode.ref_frame[0]) {
- case INTRA_FRAME:
- break;
- case LAST_FRAME:
- ref_frame_skip_mask[0] |= LAST_FRAME_MODE_MASK;
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- break;
- case GOLDEN_FRAME:
- ref_frame_skip_mask[0] |= GOLDEN_FRAME_MODE_MASK;
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- break;
- case ALTREF_FRAME:
- ref_frame_skip_mask[0] |= ALT_REF_MODE_MASK;
- break;
- case NONE:
- case MAX_REF_FRAMES:
- assert(0 && "Invalid Reference frame");
- break;
- }
- }
-
- if ((ref_frame_skip_mask[0] & (1 << ref_frame)) &&
- (ref_frame_skip_mask[1] & (1 << VPXMAX(0, second_ref_frame))))
- continue;
-
- if (mode_skip_mask[ref_frame] & (1 << this_mode))
- continue;
-
- // Test best rd so far against threshold for trying this mode.
- if (best_mode_skippable && sf->schedule_mode_search)
- mode_threshold[mode_index] <<= 1;
-
- if (best_rd < mode_threshold[mode_index])
- continue;
-
- comp_pred = second_ref_frame > INTRA_FRAME;
- if (comp_pred) {
- if (!cpi->allow_comp_inter_inter)
- continue;
-
- // Skip compound inter modes if ARF is not available.
- if (!(cpi->ref_frame_flags & flag_list[second_ref_frame]))
- continue;
-
- // Do not allow compound prediction if the segment level reference frame
- // feature is in use as in this case there can only be one reference.
- if (segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME))
- continue;
-
- if ((mode_search_skip_flags & FLAG_SKIP_COMP_BESTINTRA) &&
- best_mode_index >= 0 && best_mbmode.ref_frame[0] == INTRA_FRAME)
- continue;
-
- mode_excluded = cm->reference_mode == SINGLE_REFERENCE;
- } else {
- if (ref_frame != INTRA_FRAME)
- mode_excluded = cm->reference_mode == COMPOUND_REFERENCE;
- }
-
- if (ref_frame == INTRA_FRAME) {
- if (sf->adaptive_mode_search)
- if ((x->source_variance << num_pels_log2_lookup[bsize]) > best_pred_sse)
- continue;
-
- if (this_mode != DC_PRED) {
- // Disable intra modes other than DC_PRED for blocks with low variance
- // Threshold for intra skipping based on source variance
- // TODO(debargha): Specialize the threshold for super block sizes
- const unsigned int skip_intra_var_thresh = 64;
- if ((mode_search_skip_flags & FLAG_SKIP_INTRA_LOWVAR) &&
- x->source_variance < skip_intra_var_thresh)
- continue;
- // Only search the oblique modes if the best so far is
- // one of the neighboring directional modes
- if ((mode_search_skip_flags & FLAG_SKIP_INTRA_BESTINTER) &&
- (this_mode >= D45_PRED && this_mode <= TM_PRED)) {
- if (best_mode_index >= 0 &&
- best_mbmode.ref_frame[0] > INTRA_FRAME)
- continue;
- }
- if (mode_search_skip_flags & FLAG_SKIP_INTRA_DIRMISMATCH) {
- if (conditional_skipintra(this_mode, best_intra_mode))
- continue;
- }
- }
- } else {
- const MV_REFERENCE_FRAME ref_frames[2] = {ref_frame, second_ref_frame};
- if (!check_best_zero_mv(cpi, mbmi_ext->mode_context, frame_mv,
- this_mode, ref_frames))
- continue;
- }
-
- mbmi->mode = this_mode;
- mbmi->uv_mode = DC_PRED;
- mbmi->ref_frame[0] = ref_frame;
- mbmi->ref_frame[1] = second_ref_frame;
- // Evaluate all sub-pel filters irrespective of whether we can use
- // them for this frame.
- mbmi->interp_filter = cm->interp_filter == SWITCHABLE ? EIGHTTAP
- : cm->interp_filter;
- mbmi->mv[0].as_int = mbmi->mv[1].as_int = 0;
-
- x->skip = 0;
- set_ref_ptrs(cm, xd, ref_frame, second_ref_frame);
-
- // Select prediction reference frames.
- for (i = 0; i < MAX_MB_PLANE; i++) {
- xd->plane[i].pre[0] = yv12_mb[ref_frame][i];
- if (comp_pred)
- xd->plane[i].pre[1] = yv12_mb[second_ref_frame][i];
- }
-
- if (ref_frame == INTRA_FRAME) {
- TX_SIZE uv_tx;
- struct macroblockd_plane *const pd = &xd->plane[1];
- memset(x->skip_txfm, 0, sizeof(x->skip_txfm));
- super_block_yrd(cpi, x, &rate_y, &distortion_y, &skippable,
- NULL, bsize, best_rd);
- if (rate_y == INT_MAX)
- continue;
-
- uv_tx = get_uv_tx_size_impl(mbmi->tx_size, bsize, pd->subsampling_x,
- pd->subsampling_y);
- if (rate_uv_intra[uv_tx] == INT_MAX) {
- choose_intra_uv_mode(cpi, x, ctx, bsize, uv_tx,
- &rate_uv_intra[uv_tx], &rate_uv_tokenonly[uv_tx],
- &dist_uv[uv_tx], &skip_uv[uv_tx], &mode_uv[uv_tx]);
- }
-
- rate_uv = rate_uv_tokenonly[uv_tx];
- distortion_uv = dist_uv[uv_tx];
- skippable = skippable && skip_uv[uv_tx];
- mbmi->uv_mode = mode_uv[uv_tx];
-
- rate2 = rate_y + cpi->mbmode_cost[mbmi->mode] + rate_uv_intra[uv_tx];
- if (this_mode != DC_PRED && this_mode != TM_PRED)
- rate2 += intra_cost_penalty;
- distortion2 = distortion_y + distortion_uv;
- } else {
- this_rd = handle_inter_mode(cpi, x, bsize,
- &rate2, &distortion2, &skippable,
- &rate_y, &rate_uv,
- &disable_skip, frame_mv,
- mi_row, mi_col,
- single_newmv, single_inter_filter,
- single_skippable, &total_sse, best_rd,
- &mask_filter, filter_cache);
- if (this_rd == INT64_MAX)
- continue;
-
- compmode_cost = vp10_cost_bit(comp_mode_p, comp_pred);
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- rate2 += compmode_cost;
- }
-
- // Estimate the reference frame signaling cost and add it
- // to the rolling cost variable.
- if (comp_pred) {
- rate2 += ref_costs_comp[ref_frame];
- } else {
- rate2 += ref_costs_single[ref_frame];
- }
-
- if (!disable_skip) {
- if (skippable) {
- // Back out the coefficient coding costs
- rate2 -= (rate_y + rate_uv);
-
- // Cost the skip mb case
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 1);
- } else if (ref_frame != INTRA_FRAME && !xd->lossless[mbmi->segment_id]) {
- if (RDCOST(x->rdmult, x->rddiv, rate_y + rate_uv, distortion2) <
- RDCOST(x->rdmult, x->rddiv, 0, total_sse)) {
- // Add in the cost of the no skip flag.
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 0);
- } else {
- // FIXME(rbultje) make this work for splitmv also
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 1);
- distortion2 = total_sse;
- assert(total_sse >= 0);
- rate2 -= (rate_y + rate_uv);
- this_skip2 = 1;
- }
- } else {
- // Add in the cost of the no skip flag.
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 0);
- }
-
- // Calculate the final RD estimate for this mode.
- this_rd = RDCOST(x->rdmult, x->rddiv, rate2, distortion2);
- }
-
- // Apply an adjustment to the rd value based on the similarity of the
- // source variance and reconstructed variance.
- rd_variance_adjustment(cpi, x, bsize, &this_rd,
- ref_frame, x->source_variance);
-
- if (ref_frame == INTRA_FRAME) {
- // Keep record of best intra rd
- if (this_rd < best_intra_rd) {
- best_intra_rd = this_rd;
- best_intra_mode = mbmi->mode;
- }
- }
-
- if (!disable_skip && ref_frame == INTRA_FRAME) {
- for (i = 0; i < REFERENCE_MODES; ++i)
- best_pred_rd[i] = VPXMIN(best_pred_rd[i], this_rd);
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- best_filter_rd[i] = VPXMIN(best_filter_rd[i], this_rd);
- }
-
- // Did this mode help.. i.e. is it the new best mode
- if (this_rd < best_rd || x->skip) {
- int max_plane = MAX_MB_PLANE;
- if (!mode_excluded) {
- // Note index of best mode so far
- best_mode_index = mode_index;
-
- if (ref_frame == INTRA_FRAME) {
- /* required for left and above block mv */
- mbmi->mv[0].as_int = 0;
- max_plane = 1;
- } else {
- best_pred_sse = x->pred_sse[ref_frame];
- }
-
- rd_cost->rate = rate2;
- rd_cost->dist = distortion2;
- rd_cost->rdcost = this_rd;
- best_rd = this_rd;
- best_mbmode = *mbmi;
- best_skip2 = this_skip2;
- best_mode_skippable = skippable;
-
- if (!x->select_tx_size)
- swap_block_ptr(x, ctx, 1, 0, 0, max_plane);
- memcpy(ctx->zcoeff_blk, x->zcoeff_blk[mbmi->tx_size],
- sizeof(ctx->zcoeff_blk[0]) * ctx->num_4x4_blk);
-
- // TODO(debargha): enhance this test with a better distortion prediction
- // based on qp, activity mask and history
- if ((mode_search_skip_flags & FLAG_EARLY_TERMINATE) &&
- (mode_index > MIN_EARLY_TERM_INDEX)) {
- int qstep = xd->plane[0].dequant[1];
- // TODO(debargha): Enhance this by specializing for each mode_index
- int scale = 4;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- qstep >>= (xd->bd - 8);
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- if (x->source_variance < UINT_MAX) {
- const int var_adjust = (x->source_variance < 16);
- scale -= var_adjust;
- }
- if (ref_frame > INTRA_FRAME &&
- distortion2 * scale < qstep * qstep) {
- early_term = 1;
- }
- }
- }
- }
-
- /* keep record of best compound/single-only prediction */
- if (!disable_skip && ref_frame != INTRA_FRAME) {
- int64_t single_rd, hybrid_rd, single_rate, hybrid_rate;
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT) {
- single_rate = rate2 - compmode_cost;
- hybrid_rate = rate2;
- } else {
- single_rate = rate2;
- hybrid_rate = rate2 + compmode_cost;
- }
-
- single_rd = RDCOST(x->rdmult, x->rddiv, single_rate, distortion2);
- hybrid_rd = RDCOST(x->rdmult, x->rddiv, hybrid_rate, distortion2);
-
- if (!comp_pred) {
- if (single_rd < best_pred_rd[SINGLE_REFERENCE])
- best_pred_rd[SINGLE_REFERENCE] = single_rd;
- } else {
- if (single_rd < best_pred_rd[COMPOUND_REFERENCE])
- best_pred_rd[COMPOUND_REFERENCE] = single_rd;
- }
- if (hybrid_rd < best_pred_rd[REFERENCE_MODE_SELECT])
- best_pred_rd[REFERENCE_MODE_SELECT] = hybrid_rd;
-
- /* keep record of best filter type */
- if (!mode_excluded && cm->interp_filter != BILINEAR) {
- int64_t ref = filter_cache[cm->interp_filter == SWITCHABLE ?
- SWITCHABLE_FILTERS : cm->interp_filter];
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++) {
- int64_t adj_rd;
- if (ref == INT64_MAX)
- adj_rd = 0;
- else if (filter_cache[i] == INT64_MAX)
- // when early termination is triggered, the encoder does not have
- // access to the rate-distortion cost. it only knows that the cost
- // should be above the maximum valid value. hence it takes the known
- // maximum plus an arbitrary constant as the rate-distortion cost.
- adj_rd = mask_filter - ref + 10;
- else
- adj_rd = filter_cache[i] - ref;
-
- adj_rd += this_rd;
- best_filter_rd[i] = VPXMIN(best_filter_rd[i], adj_rd);
- }
- }
- }
-
- if (early_term)
- break;
-
- if (x->skip && !comp_pred)
- break;
- }
-
- // The inter modes' rate costs are not calculated precisely in some cases.
- // Therefore, sometimes, NEWMV is chosen instead of NEARESTMV, NEARMV, and
- // ZEROMV. Here, checks are added for those cases, and the mode decisions
- // are corrected.
- if (best_mbmode.mode == NEWMV) {
- const MV_REFERENCE_FRAME refs[2] = {best_mbmode.ref_frame[0],
- best_mbmode.ref_frame[1]};
- int comp_pred_mode = refs[1] > INTRA_FRAME;
-
- if (frame_mv[NEARESTMV][refs[0]].as_int == best_mbmode.mv[0].as_int &&
- ((comp_pred_mode && frame_mv[NEARESTMV][refs[1]].as_int ==
- best_mbmode.mv[1].as_int) || !comp_pred_mode))
- best_mbmode.mode = NEARESTMV;
- else if (frame_mv[NEARMV][refs[0]].as_int == best_mbmode.mv[0].as_int &&
- ((comp_pred_mode && frame_mv[NEARMV][refs[1]].as_int ==
- best_mbmode.mv[1].as_int) || !comp_pred_mode))
- best_mbmode.mode = NEARMV;
- else if (best_mbmode.mv[0].as_int == 0 &&
- ((comp_pred_mode && best_mbmode.mv[1].as_int == 0) || !comp_pred_mode))
- best_mbmode.mode = ZEROMV;
- }
-
- if (best_mode_index < 0 || best_rd >= best_rd_so_far) {
- rd_cost->rate = INT_MAX;
- rd_cost->rdcost = INT64_MAX;
- return;
- }
-
- // If we used an estimate for the uv intra rd in the loop above...
- if (sf->use_uv_intra_rd_estimate) {
- // Do Intra UV best rd mode selection if best mode choice above was intra.
- if (best_mbmode.ref_frame[0] == INTRA_FRAME) {
- TX_SIZE uv_tx_size;
- *mbmi = best_mbmode;
- uv_tx_size = get_uv_tx_size(mbmi, &xd->plane[1]);
- rd_pick_intra_sbuv_mode(cpi, x, ctx, &rate_uv_intra[uv_tx_size],
- &rate_uv_tokenonly[uv_tx_size],
- &dist_uv[uv_tx_size],
- &skip_uv[uv_tx_size],
- bsize < BLOCK_8X8 ? BLOCK_8X8 : bsize,
- uv_tx_size);
- }
- }
-
- assert((cm->interp_filter == SWITCHABLE) ||
- (cm->interp_filter == best_mbmode.interp_filter) ||
- !is_inter_block(&best_mbmode));
-
- if (!cpi->rc.is_src_frame_alt_ref)
- vp10_update_rd_thresh_fact(tile_data->thresh_freq_fact,
- sf->adaptive_rd_thresh, bsize, best_mode_index);
-
- // macroblock modes
- *mbmi = best_mbmode;
- x->skip |= best_skip2;
-
- for (i = 0; i < REFERENCE_MODES; ++i) {
- if (best_pred_rd[i] == INT64_MAX)
- best_pred_diff[i] = INT_MIN;
- else
- best_pred_diff[i] = best_rd - best_pred_rd[i];
- }
-
- if (!x->skip) {
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++) {
- if (best_filter_rd[i] == INT64_MAX)
- best_filter_diff[i] = 0;
- else
- best_filter_diff[i] = best_rd - best_filter_rd[i];
- }
- if (cm->interp_filter == SWITCHABLE)
- assert(best_filter_diff[SWITCHABLE_FILTERS] == 0);
- } else {
- vp10_zero(best_filter_diff);
- }
-
- // TODO(yunqingwang): Moving this line in front of the above best_filter_diff
- // updating code causes PSNR loss. Need to figure out the confliction.
- x->skip |= best_mode_skippable;
-
- if (!x->skip && !x->select_tx_size) {
- int has_high_freq_coeff = 0;
- int plane;
- int max_plane = is_inter_block(&xd->mi[0]->mbmi)
- ? MAX_MB_PLANE : 1;
- for (plane = 0; plane < max_plane; ++plane) {
- x->plane[plane].eobs = ctx->eobs_pbuf[plane][1];
- has_high_freq_coeff |= vp10_has_high_freq_in_plane(x, bsize, plane);
- }
-
- for (plane = max_plane; plane < MAX_MB_PLANE; ++plane) {
- x->plane[plane].eobs = ctx->eobs_pbuf[plane][2];
- has_high_freq_coeff |= vp10_has_high_freq_in_plane(x, bsize, plane);
- }
-
- best_mode_skippable |= !has_high_freq_coeff;
- }
-
- assert(best_mode_index >= 0);
-
- store_coding_context(x, ctx, best_mode_index, best_pred_diff,
- best_filter_diff, best_mode_skippable);
-}
-
-void vp10_rd_pick_inter_mode_sb_seg_skip(VP10_COMP *cpi,
- TileDataEnc *tile_data,
- MACROBLOCK *x,
- RD_COST *rd_cost,
- BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx,
- int64_t best_rd_so_far) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- unsigned char segment_id = mbmi->segment_id;
- const int comp_pred = 0;
- int i;
- int64_t best_pred_diff[REFERENCE_MODES];
- int64_t best_filter_diff[SWITCHABLE_FILTER_CONTEXTS];
- unsigned int ref_costs_single[MAX_REF_FRAMES], ref_costs_comp[MAX_REF_FRAMES];
- vpx_prob comp_mode_p;
- INTERP_FILTER best_filter = SWITCHABLE;
- int64_t this_rd = INT64_MAX;
- int rate2 = 0;
- const int64_t distortion2 = 0;
-
- estimate_ref_frame_costs(cm, xd, segment_id, ref_costs_single, ref_costs_comp,
- &comp_mode_p);
-
- for (i = 0; i < MAX_REF_FRAMES; ++i)
- x->pred_sse[i] = INT_MAX;
- for (i = LAST_FRAME; i < MAX_REF_FRAMES; ++i)
- x->pred_mv_sad[i] = INT_MAX;
-
- rd_cost->rate = INT_MAX;
-
- assert(segfeature_active(&cm->seg, segment_id, SEG_LVL_SKIP));
-
- mbmi->palette_mode_info.palette_size[0] = 0;
- mbmi->palette_mode_info.palette_size[1] = 0;
- mbmi->mode = ZEROMV;
- mbmi->uv_mode = DC_PRED;
- mbmi->ref_frame[0] = LAST_FRAME;
- mbmi->ref_frame[1] = NONE;
- mbmi->mv[0].as_int = 0;
- x->skip = 1;
-
- if (cm->interp_filter != BILINEAR) {
- best_filter = EIGHTTAP;
- if (cm->interp_filter == SWITCHABLE &&
- x->source_variance >= cpi->sf.disable_filter_search_var_thresh) {
- int rs;
- int best_rs = INT_MAX;
- for (i = 0; i < SWITCHABLE_FILTERS; ++i) {
- mbmi->interp_filter = i;
- rs = vp10_get_switchable_rate(cpi, xd);
- if (rs < best_rs) {
- best_rs = rs;
- best_filter = mbmi->interp_filter;
- }
- }
- }
- }
- // Set the appropriate filter
- if (cm->interp_filter == SWITCHABLE) {
- mbmi->interp_filter = best_filter;
- rate2 += vp10_get_switchable_rate(cpi, xd);
- } else {
- mbmi->interp_filter = cm->interp_filter;
- }
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- rate2 += vp10_cost_bit(comp_mode_p, comp_pred);
-
- // Estimate the reference frame signaling cost and add it
- // to the rolling cost variable.
- rate2 += ref_costs_single[LAST_FRAME];
- this_rd = RDCOST(x->rdmult, x->rddiv, rate2, distortion2);
-
- rd_cost->rate = rate2;
- rd_cost->dist = distortion2;
- rd_cost->rdcost = this_rd;
-
- if (this_rd >= best_rd_so_far) {
- rd_cost->rate = INT_MAX;
- rd_cost->rdcost = INT64_MAX;
- return;
- }
-
- assert((cm->interp_filter == SWITCHABLE) ||
- (cm->interp_filter == mbmi->interp_filter));
-
- vp10_update_rd_thresh_fact(tile_data->thresh_freq_fact,
- cpi->sf.adaptive_rd_thresh, bsize, THR_ZEROMV);
-
- vp10_zero(best_pred_diff);
- vp10_zero(best_filter_diff);
-
- if (!x->select_tx_size)
- swap_block_ptr(x, ctx, 1, 0, 0, MAX_MB_PLANE);
- store_coding_context(x, ctx, THR_ZEROMV,
- best_pred_diff, best_filter_diff, 0);
-}
-
-void vp10_rd_pick_inter_mode_sub8x8(VP10_COMP *cpi,
- TileDataEnc *tile_data,
- MACROBLOCK *x,
- int mi_row, int mi_col,
- RD_COST *rd_cost,
- BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx,
- int64_t best_rd_so_far) {
- VP10_COMMON *const cm = &cpi->common;
- RD_OPT *const rd_opt = &cpi->rd;
- SPEED_FEATURES *const sf = &cpi->sf;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- const struct segmentation *const seg = &cm->seg;
- MV_REFERENCE_FRAME ref_frame, second_ref_frame;
- unsigned char segment_id = mbmi->segment_id;
- int comp_pred, i;
- int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES];
- struct buf_2d yv12_mb[4][MAX_MB_PLANE];
- static const int flag_list[4] = { 0, VP9_LAST_FLAG, VP9_GOLD_FLAG,
- VP9_ALT_FLAG };
- int64_t best_rd = best_rd_so_far;
- int64_t best_yrd = best_rd_so_far; // FIXME(rbultje) more precise
- int64_t best_pred_diff[REFERENCE_MODES];
- int64_t best_pred_rd[REFERENCE_MODES];
- int64_t best_filter_rd[SWITCHABLE_FILTER_CONTEXTS];
- int64_t best_filter_diff[SWITCHABLE_FILTER_CONTEXTS];
- MB_MODE_INFO best_mbmode;
- int ref_index, best_ref_index = 0;
- unsigned int ref_costs_single[MAX_REF_FRAMES], ref_costs_comp[MAX_REF_FRAMES];
- vpx_prob comp_mode_p;
- INTERP_FILTER tmp_best_filter = SWITCHABLE;
- int rate_uv_intra, rate_uv_tokenonly;
- int64_t dist_uv;
- int skip_uv;
- PREDICTION_MODE mode_uv = DC_PRED;
- const int intra_cost_penalty = vp10_get_intra_cost_penalty(
- cm->base_qindex, cm->y_dc_delta_q, cm->bit_depth);
- int_mv seg_mvs[4][MAX_REF_FRAMES];
- b_mode_info best_bmodes[4];
- int best_skip2 = 0;
- int ref_frame_skip_mask[2] = { 0 };
- int64_t mask_filter = 0;
- int64_t filter_cache[SWITCHABLE_FILTER_CONTEXTS];
- int internal_active_edge =
- vp10_active_edge_sb(cpi, mi_row, mi_col) && vp10_internal_image_edge(cpi);
-
- memset(x->zcoeff_blk[TX_4X4], 0, 4);
- vp10_zero(best_mbmode);
-
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- filter_cache[i] = INT64_MAX;
-
- for (i = 0; i < 4; i++) {
- int j;
- for (j = 0; j < MAX_REF_FRAMES; j++)
- seg_mvs[i][j].as_int = INVALID_MV;
- }
-
- estimate_ref_frame_costs(cm, xd, segment_id, ref_costs_single, ref_costs_comp,
- &comp_mode_p);
-
- for (i = 0; i < REFERENCE_MODES; ++i)
- best_pred_rd[i] = INT64_MAX;
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- best_filter_rd[i] = INT64_MAX;
- rate_uv_intra = INT_MAX;
-
- rd_cost->rate = INT_MAX;
-
- for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ref_frame++) {
- if (cpi->ref_frame_flags & flag_list[ref_frame]) {
- setup_buffer_inter(cpi, x, ref_frame, bsize, mi_row, mi_col,
- frame_mv[NEARESTMV], frame_mv[NEARMV],
- yv12_mb);
- } else {
- ref_frame_skip_mask[0] |= (1 << ref_frame);
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- }
- frame_mv[NEWMV][ref_frame].as_int = INVALID_MV;
- frame_mv[ZEROMV][ref_frame].as_int = 0;
- }
-
- mbmi->palette_mode_info.palette_size[0] = 0;
- mbmi->palette_mode_info.palette_size[1] = 0;
-
- for (ref_index = 0; ref_index < MAX_REFS; ++ref_index) {
- int mode_excluded = 0;
- int64_t this_rd = INT64_MAX;
- int disable_skip = 0;
- int compmode_cost = 0;
- int rate2 = 0, rate_y = 0, rate_uv = 0;
- int64_t distortion2 = 0, distortion_y = 0, distortion_uv = 0;
- int skippable = 0;
- int i;
- int this_skip2 = 0;
- int64_t total_sse = INT_MAX;
- int early_term = 0;
-
- ref_frame = vp10_ref_order[ref_index].ref_frame[0];
- second_ref_frame = vp10_ref_order[ref_index].ref_frame[1];
-
- // Look at the reference frame of the best mode so far and set the
- // skip mask to look at a subset of the remaining modes.
- if (ref_index > 2 && sf->mode_skip_start < MAX_MODES) {
- if (ref_index == 3) {
- switch (best_mbmode.ref_frame[0]) {
- case INTRA_FRAME:
- break;
- case LAST_FRAME:
- ref_frame_skip_mask[0] |= (1 << GOLDEN_FRAME) | (1 << ALTREF_FRAME);
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- break;
- case GOLDEN_FRAME:
- ref_frame_skip_mask[0] |= (1 << LAST_FRAME) | (1 << ALTREF_FRAME);
- ref_frame_skip_mask[1] |= SECOND_REF_FRAME_MASK;
- break;
- case ALTREF_FRAME:
- ref_frame_skip_mask[0] |= (1 << GOLDEN_FRAME) | (1 << LAST_FRAME);
- break;
- case NONE:
- case MAX_REF_FRAMES:
- assert(0 && "Invalid Reference frame");
- break;
- }
- }
- }
-
- if ((ref_frame_skip_mask[0] & (1 << ref_frame)) &&
- (ref_frame_skip_mask[1] & (1 << VPXMAX(0, second_ref_frame))))
- continue;
-
- // Test best rd so far against threshold for trying this mode.
- if (!internal_active_edge &&
- rd_less_than_thresh(best_rd,
- rd_opt->threshes[segment_id][bsize][ref_index],
- tile_data->thresh_freq_fact[bsize][ref_index]))
- continue;
-
- comp_pred = second_ref_frame > INTRA_FRAME;
- if (comp_pred) {
- if (!cpi->allow_comp_inter_inter)
- continue;
- if (!(cpi->ref_frame_flags & flag_list[second_ref_frame]))
- continue;
- // Do not allow compound prediction if the segment level reference frame
- // feature is in use as in this case there can only be one reference.
- if (segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME))
- continue;
-
- if ((sf->mode_search_skip_flags & FLAG_SKIP_COMP_BESTINTRA) &&
- best_mbmode.ref_frame[0] == INTRA_FRAME)
- continue;
- }
-
- // TODO(jingning, jkoleszar): scaling reference frame not supported for
- // sub8x8 blocks.
- if (ref_frame > INTRA_FRAME &&
- vp10_is_scaled(&cm->frame_refs[ref_frame - 1].sf))
- continue;
-
- if (second_ref_frame > INTRA_FRAME &&
- vp10_is_scaled(&cm->frame_refs[second_ref_frame - 1].sf))
- continue;
-
- if (comp_pred)
- mode_excluded = cm->reference_mode == SINGLE_REFERENCE;
- else if (ref_frame != INTRA_FRAME)
- mode_excluded = cm->reference_mode == COMPOUND_REFERENCE;
-
- // If the segment reference frame feature is enabled....
- // then do nothing if the current ref frame is not allowed..
- if (segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME) &&
- get_segdata(seg, segment_id, SEG_LVL_REF_FRAME) != (int)ref_frame) {
- continue;
- // Disable this drop out case if the ref frame
- // segment level feature is enabled for this segment. This is to
- // prevent the possibility that we end up unable to pick any mode.
- } else if (!segfeature_active(seg, segment_id, SEG_LVL_REF_FRAME)) {
- // Only consider ZEROMV/ALTREF_FRAME for alt ref frame,
- // unless ARNR filtering is enabled in which case we want
- // an unfiltered alternative. We allow near/nearest as well
- // because they may result in zero-zero MVs but be cheaper.
- if (cpi->rc.is_src_frame_alt_ref && (cpi->oxcf.arnr_max_frames == 0))
- continue;
- }
-
- mbmi->tx_size = TX_4X4;
- mbmi->uv_mode = DC_PRED;
- mbmi->ref_frame[0] = ref_frame;
- mbmi->ref_frame[1] = second_ref_frame;
- // Evaluate all sub-pel filters irrespective of whether we can use
- // them for this frame.
- mbmi->interp_filter = cm->interp_filter == SWITCHABLE ? EIGHTTAP
- : cm->interp_filter;
- x->skip = 0;
- set_ref_ptrs(cm, xd, ref_frame, second_ref_frame);
-
- // Select prediction reference frames.
- for (i = 0; i < MAX_MB_PLANE; i++) {
- xd->plane[i].pre[0] = yv12_mb[ref_frame][i];
- if (comp_pred)
- xd->plane[i].pre[1] = yv12_mb[second_ref_frame][i];
- }
-
- if (ref_frame == INTRA_FRAME) {
- int rate;
- if (rd_pick_intra_sub_8x8_y_mode(cpi, x, &rate, &rate_y,
- &distortion_y, best_rd) >= best_rd)
- continue;
- rate2 += rate;
- rate2 += intra_cost_penalty;
- distortion2 += distortion_y;
-
- if (rate_uv_intra == INT_MAX) {
- choose_intra_uv_mode(cpi, x, ctx, bsize, TX_4X4,
- &rate_uv_intra,
- &rate_uv_tokenonly,
- &dist_uv, &skip_uv,
- &mode_uv);
- }
- rate2 += rate_uv_intra;
- rate_uv = rate_uv_tokenonly;
- distortion2 += dist_uv;
- distortion_uv = dist_uv;
- mbmi->uv_mode = mode_uv;
- } else {
- int rate;
- int64_t distortion;
- int64_t this_rd_thresh;
- int64_t tmp_rd, tmp_best_rd = INT64_MAX, tmp_best_rdu = INT64_MAX;
- int tmp_best_rate = INT_MAX, tmp_best_ratey = INT_MAX;
- int64_t tmp_best_distortion = INT_MAX, tmp_best_sse, uv_sse;
- int tmp_best_skippable = 0;
- int switchable_filter_index;
- int_mv *second_ref = comp_pred ?
- &x->mbmi_ext->ref_mvs[second_ref_frame][0] : NULL;
- b_mode_info tmp_best_bmodes[16];
- MB_MODE_INFO tmp_best_mbmode;
- BEST_SEG_INFO bsi[SWITCHABLE_FILTERS];
- int pred_exists = 0;
- int uv_skippable;
-
- this_rd_thresh = (ref_frame == LAST_FRAME) ?
- rd_opt->threshes[segment_id][bsize][THR_LAST] :
- rd_opt->threshes[segment_id][bsize][THR_ALTR];
- this_rd_thresh = (ref_frame == GOLDEN_FRAME) ?
- rd_opt->threshes[segment_id][bsize][THR_GOLD] : this_rd_thresh;
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; ++i)
- filter_cache[i] = INT64_MAX;
-
- if (cm->interp_filter != BILINEAR) {
- tmp_best_filter = EIGHTTAP;
- if (x->source_variance < sf->disable_filter_search_var_thresh) {
- tmp_best_filter = EIGHTTAP;
- } else if (sf->adaptive_pred_interp_filter == 1 &&
- ctx->pred_interp_filter < SWITCHABLE) {
- tmp_best_filter = ctx->pred_interp_filter;
- } else if (sf->adaptive_pred_interp_filter == 2) {
- tmp_best_filter = ctx->pred_interp_filter < SWITCHABLE ?
- ctx->pred_interp_filter : 0;
- } else {
- for (switchable_filter_index = 0;
- switchable_filter_index < SWITCHABLE_FILTERS;
- ++switchable_filter_index) {
- int newbest, rs;
- int64_t rs_rd;
- MB_MODE_INFO_EXT *mbmi_ext = x->mbmi_ext;
- mbmi->interp_filter = switchable_filter_index;
- tmp_rd = rd_pick_best_sub8x8_mode(cpi, x,
- &mbmi_ext->ref_mvs[ref_frame][0],
- second_ref, best_yrd, &rate,
- &rate_y, &distortion,
- &skippable, &total_sse,
- (int) this_rd_thresh, seg_mvs,
- bsi, switchable_filter_index,
- mi_row, mi_col);
-
- if (tmp_rd == INT64_MAX)
- continue;
- rs = vp10_get_switchable_rate(cpi, xd);
- rs_rd = RDCOST(x->rdmult, x->rddiv, rs, 0);
- filter_cache[switchable_filter_index] = tmp_rd;
- filter_cache[SWITCHABLE_FILTERS] =
- VPXMIN(filter_cache[SWITCHABLE_FILTERS], tmp_rd + rs_rd);
- if (cm->interp_filter == SWITCHABLE)
- tmp_rd += rs_rd;
-
- mask_filter = VPXMAX(mask_filter, tmp_rd);
-
- newbest = (tmp_rd < tmp_best_rd);
- if (newbest) {
- tmp_best_filter = mbmi->interp_filter;
- tmp_best_rd = tmp_rd;
- }
- if ((newbest && cm->interp_filter == SWITCHABLE) ||
- (mbmi->interp_filter == cm->interp_filter &&
- cm->interp_filter != SWITCHABLE)) {
- tmp_best_rdu = tmp_rd;
- tmp_best_rate = rate;
- tmp_best_ratey = rate_y;
- tmp_best_distortion = distortion;
- tmp_best_sse = total_sse;
- tmp_best_skippable = skippable;
- tmp_best_mbmode = *mbmi;
- for (i = 0; i < 4; i++) {
- tmp_best_bmodes[i] = xd->mi[0]->bmi[i];
- x->zcoeff_blk[TX_4X4][i] = !x->plane[0].eobs[i];
- }
- pred_exists = 1;
- if (switchable_filter_index == 0 &&
- sf->use_rd_breakout &&
- best_rd < INT64_MAX) {
- if (tmp_best_rdu / 2 > best_rd) {
- // skip searching the other filters if the first is
- // already substantially larger than the best so far
- tmp_best_filter = mbmi->interp_filter;
- tmp_best_rdu = INT64_MAX;
- break;
- }
- }
- }
- } // switchable_filter_index loop
- }
- }
-
- if (tmp_best_rdu == INT64_MAX && pred_exists)
- continue;
-
- mbmi->interp_filter = (cm->interp_filter == SWITCHABLE ?
- tmp_best_filter : cm->interp_filter);
- if (!pred_exists) {
- // Handles the special case when a filter that is not in the
- // switchable list (bilinear, 6-tap) is indicated at the frame level
- tmp_rd = rd_pick_best_sub8x8_mode(cpi, x,
- &x->mbmi_ext->ref_mvs[ref_frame][0],
- second_ref, best_yrd, &rate, &rate_y,
- &distortion, &skippable, &total_sse,
- (int) this_rd_thresh, seg_mvs, bsi, 0,
- mi_row, mi_col);
- if (tmp_rd == INT64_MAX)
- continue;
- } else {
- total_sse = tmp_best_sse;
- rate = tmp_best_rate;
- rate_y = tmp_best_ratey;
- distortion = tmp_best_distortion;
- skippable = tmp_best_skippable;
- *mbmi = tmp_best_mbmode;
- for (i = 0; i < 4; i++)
- xd->mi[0]->bmi[i] = tmp_best_bmodes[i];
- }
-
- rate2 += rate;
- distortion2 += distortion;
-
- if (cm->interp_filter == SWITCHABLE)
- rate2 += vp10_get_switchable_rate(cpi, xd);
-
- if (!mode_excluded)
- mode_excluded = comp_pred ? cm->reference_mode == SINGLE_REFERENCE
- : cm->reference_mode == COMPOUND_REFERENCE;
-
- compmode_cost = vp10_cost_bit(comp_mode_p, comp_pred);
-
- tmp_best_rdu = best_rd -
- VPXMIN(RDCOST(x->rdmult, x->rddiv, rate2, distortion2),
- RDCOST(x->rdmult, x->rddiv, 0, total_sse));
-
- if (tmp_best_rdu > 0) {
- // If even the 'Y' rd value of split is higher than best so far
- // then dont bother looking at UV
- vp10_build_inter_predictors_sbuv(&x->e_mbd, mi_row, mi_col,
- BLOCK_8X8);
- memset(x->skip_txfm, SKIP_TXFM_NONE, sizeof(x->skip_txfm));
- if (!super_block_uvrd(cpi, x, &rate_uv, &distortion_uv, &uv_skippable,
- &uv_sse, BLOCK_8X8, tmp_best_rdu))
- continue;
-
- rate2 += rate_uv;
- distortion2 += distortion_uv;
- skippable = skippable && uv_skippable;
- total_sse += uv_sse;
- }
- }
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT)
- rate2 += compmode_cost;
-
- // Estimate the reference frame signaling cost and add it
- // to the rolling cost variable.
- if (second_ref_frame > INTRA_FRAME) {
- rate2 += ref_costs_comp[ref_frame];
- } else {
- rate2 += ref_costs_single[ref_frame];
- }
-
- if (!disable_skip) {
- // Skip is never coded at the segment level for sub8x8 blocks and instead
- // always coded in the bitstream at the mode info level.
-
- if (ref_frame != INTRA_FRAME && !xd->lossless[mbmi->segment_id]) {
- if (RDCOST(x->rdmult, x->rddiv, rate_y + rate_uv, distortion2) <
- RDCOST(x->rdmult, x->rddiv, 0, total_sse)) {
- // Add in the cost of the no skip flag.
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 0);
- } else {
- // FIXME(rbultje) make this work for splitmv also
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 1);
- distortion2 = total_sse;
- assert(total_sse >= 0);
- rate2 -= (rate_y + rate_uv);
- rate_y = 0;
- rate_uv = 0;
- this_skip2 = 1;
- }
- } else {
- // Add in the cost of the no skip flag.
- rate2 += vp10_cost_bit(vp10_get_skip_prob(cm, xd), 0);
- }
-
- // Calculate the final RD estimate for this mode.
- this_rd = RDCOST(x->rdmult, x->rddiv, rate2, distortion2);
- }
-
- if (!disable_skip && ref_frame == INTRA_FRAME) {
- for (i = 0; i < REFERENCE_MODES; ++i)
- best_pred_rd[i] = VPXMIN(best_pred_rd[i], this_rd);
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++)
- best_filter_rd[i] = VPXMIN(best_filter_rd[i], this_rd);
- }
-
- // Did this mode help.. i.e. is it the new best mode
- if (this_rd < best_rd || x->skip) {
- if (!mode_excluded) {
- int max_plane = MAX_MB_PLANE;
- // Note index of best mode so far
- best_ref_index = ref_index;
-
- if (ref_frame == INTRA_FRAME) {
- /* required for left and above block mv */
- mbmi->mv[0].as_int = 0;
- max_plane = 1;
- }
-
- rd_cost->rate = rate2;
- rd_cost->dist = distortion2;
- rd_cost->rdcost = this_rd;
- best_rd = this_rd;
- best_yrd = best_rd -
- RDCOST(x->rdmult, x->rddiv, rate_uv, distortion_uv);
- best_mbmode = *mbmi;
- best_skip2 = this_skip2;
- if (!x->select_tx_size)
- swap_block_ptr(x, ctx, 1, 0, 0, max_plane);
- memcpy(ctx->zcoeff_blk, x->zcoeff_blk[TX_4X4],
- sizeof(ctx->zcoeff_blk[0]) * ctx->num_4x4_blk);
-
- for (i = 0; i < 4; i++)
- best_bmodes[i] = xd->mi[0]->bmi[i];
-
- // TODO(debargha): enhance this test with a better distortion prediction
- // based on qp, activity mask and history
- if ((sf->mode_search_skip_flags & FLAG_EARLY_TERMINATE) &&
- (ref_index > MIN_EARLY_TERM_INDEX)) {
- int qstep = xd->plane[0].dequant[1];
- // TODO(debargha): Enhance this by specializing for each mode_index
- int scale = 4;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- qstep >>= (xd->bd - 8);
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- if (x->source_variance < UINT_MAX) {
- const int var_adjust = (x->source_variance < 16);
- scale -= var_adjust;
- }
- if (ref_frame > INTRA_FRAME &&
- distortion2 * scale < qstep * qstep) {
- early_term = 1;
- }
- }
- }
- }
-
- /* keep record of best compound/single-only prediction */
- if (!disable_skip && ref_frame != INTRA_FRAME) {
- int64_t single_rd, hybrid_rd, single_rate, hybrid_rate;
-
- if (cm->reference_mode == REFERENCE_MODE_SELECT) {
- single_rate = rate2 - compmode_cost;
- hybrid_rate = rate2;
- } else {
- single_rate = rate2;
- hybrid_rate = rate2 + compmode_cost;
- }
-
- single_rd = RDCOST(x->rdmult, x->rddiv, single_rate, distortion2);
- hybrid_rd = RDCOST(x->rdmult, x->rddiv, hybrid_rate, distortion2);
-
- if (!comp_pred && single_rd < best_pred_rd[SINGLE_REFERENCE])
- best_pred_rd[SINGLE_REFERENCE] = single_rd;
- else if (comp_pred && single_rd < best_pred_rd[COMPOUND_REFERENCE])
- best_pred_rd[COMPOUND_REFERENCE] = single_rd;
-
- if (hybrid_rd < best_pred_rd[REFERENCE_MODE_SELECT])
- best_pred_rd[REFERENCE_MODE_SELECT] = hybrid_rd;
- }
-
- /* keep record of best filter type */
- if (!mode_excluded && !disable_skip && ref_frame != INTRA_FRAME &&
- cm->interp_filter != BILINEAR) {
- int64_t ref = filter_cache[cm->interp_filter == SWITCHABLE ?
- SWITCHABLE_FILTERS : cm->interp_filter];
- int64_t adj_rd;
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++) {
- if (ref == INT64_MAX)
- adj_rd = 0;
- else if (filter_cache[i] == INT64_MAX)
- // when early termination is triggered, the encoder does not have
- // access to the rate-distortion cost. it only knows that the cost
- // should be above the maximum valid value. hence it takes the known
- // maximum plus an arbitrary constant as the rate-distortion cost.
- adj_rd = mask_filter - ref + 10;
- else
- adj_rd = filter_cache[i] - ref;
-
- adj_rd += this_rd;
- best_filter_rd[i] = VPXMIN(best_filter_rd[i], adj_rd);
- }
- }
-
- if (early_term)
- break;
-
- if (x->skip && !comp_pred)
- break;
- }
-
- if (best_rd >= best_rd_so_far) {
- rd_cost->rate = INT_MAX;
- rd_cost->rdcost = INT64_MAX;
- return;
- }
-
- // If we used an estimate for the uv intra rd in the loop above...
- if (sf->use_uv_intra_rd_estimate) {
- // Do Intra UV best rd mode selection if best mode choice above was intra.
- if (best_mbmode.ref_frame[0] == INTRA_FRAME) {
- *mbmi = best_mbmode;
- rd_pick_intra_sbuv_mode(cpi, x, ctx, &rate_uv_intra,
- &rate_uv_tokenonly,
- &dist_uv,
- &skip_uv,
- BLOCK_8X8, TX_4X4);
- }
- }
-
- if (best_rd == INT64_MAX) {
- rd_cost->rate = INT_MAX;
- rd_cost->dist = INT64_MAX;
- rd_cost->rdcost = INT64_MAX;
- return;
- }
-
- assert((cm->interp_filter == SWITCHABLE) ||
- (cm->interp_filter == best_mbmode.interp_filter) ||
- !is_inter_block(&best_mbmode));
-
- vp10_update_rd_thresh_fact(tile_data->thresh_freq_fact,
- sf->adaptive_rd_thresh, bsize, best_ref_index);
-
- // macroblock modes
- *mbmi = best_mbmode;
- x->skip |= best_skip2;
- if (!is_inter_block(&best_mbmode)) {
- for (i = 0; i < 4; i++)
- xd->mi[0]->bmi[i].as_mode = best_bmodes[i].as_mode;
- } else {
- for (i = 0; i < 4; ++i)
- memcpy(&xd->mi[0]->bmi[i], &best_bmodes[i], sizeof(b_mode_info));
-
- mbmi->mv[0].as_int = xd->mi[0]->bmi[3].as_mv[0].as_int;
- mbmi->mv[1].as_int = xd->mi[0]->bmi[3].as_mv[1].as_int;
- }
-
- for (i = 0; i < REFERENCE_MODES; ++i) {
- if (best_pred_rd[i] == INT64_MAX)
- best_pred_diff[i] = INT_MIN;
- else
- best_pred_diff[i] = best_rd - best_pred_rd[i];
- }
-
- if (!x->skip) {
- for (i = 0; i < SWITCHABLE_FILTER_CONTEXTS; i++) {
- if (best_filter_rd[i] == INT64_MAX)
- best_filter_diff[i] = 0;
- else
- best_filter_diff[i] = best_rd - best_filter_rd[i];
- }
- if (cm->interp_filter == SWITCHABLE)
- assert(best_filter_diff[SWITCHABLE_FILTERS] == 0);
- } else {
- vp10_zero(best_filter_diff);
- }
-
- store_coding_context(x, ctx, best_ref_index,
- best_pred_diff, best_filter_diff, 0);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_RDOPT_H_
-#define VP10_ENCODER_RDOPT_H_
-
-#include "vp10/common/blockd.h"
-
-#include "vp10/encoder/block.h"
-#include "vp10/encoder/context_tree.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct TileInfo;
-struct VP10_COMP;
-struct macroblock;
-struct RD_COST;
-
-void vp10_rd_pick_intra_mode_sb(struct VP10_COMP *cpi, struct macroblock *x,
- struct RD_COST *rd_cost, BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx, int64_t best_rd);
-
-unsigned int vp10_get_sby_perpixel_variance(VP10_COMP *cpi,
- const struct buf_2d *ref,
- BLOCK_SIZE bs);
-#if CONFIG_VP9_HIGHBITDEPTH
-unsigned int vp10_high_get_sby_perpixel_variance(VP10_COMP *cpi,
- const struct buf_2d *ref,
- BLOCK_SIZE bs, int bd);
-#endif
-
-void vp10_rd_pick_inter_mode_sb(struct VP10_COMP *cpi,
- struct TileDataEnc *tile_data,
- struct macroblock *x,
- int mi_row, int mi_col,
- struct RD_COST *rd_cost,
- BLOCK_SIZE bsize, PICK_MODE_CONTEXT *ctx,
- int64_t best_rd_so_far);
-
-void vp10_rd_pick_inter_mode_sb_seg_skip(struct VP10_COMP *cpi,
- struct TileDataEnc *tile_data,
- struct macroblock *x,
- struct RD_COST *rd_cost,
- BLOCK_SIZE bsize,
- PICK_MODE_CONTEXT *ctx,
- int64_t best_rd_so_far);
-
-int vp10_internal_image_edge(struct VP10_COMP *cpi);
-int vp10_active_h_edge(struct VP10_COMP *cpi, int mi_row, int mi_step);
-int vp10_active_v_edge(struct VP10_COMP *cpi, int mi_col, int mi_step);
-int vp10_active_edge_sb(struct VP10_COMP *cpi, int mi_row, int mi_col);
-
-void vp10_rd_pick_inter_mode_sub8x8(struct VP10_COMP *cpi,
- struct TileDataEnc *tile_data,
- struct macroblock *x,
- int mi_row, int mi_col,
- struct RD_COST *rd_cost,
- BLOCK_SIZE bsize, PICK_MODE_CONTEXT *ctx,
- int64_t best_rd_so_far);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_RDOPT_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <limits.h>
-#include <math.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#if CONFIG_VP9_HIGHBITDEPTH
-#include "vpx_dsp/vpx_dsp_common.h"
-#endif // CONFIG_VP9_HIGHBITDEPTH
-#include "vpx_ports/mem.h"
-#include "vp10/common/common.h"
-#include "vp10/encoder/resize.h"
-
-#define FILTER_BITS 7
-
-#define INTERP_TAPS 8
-#define SUBPEL_BITS 5
-#define SUBPEL_MASK ((1 << SUBPEL_BITS) - 1)
-#define INTERP_PRECISION_BITS 32
-
-typedef int16_t interp_kernel[INTERP_TAPS];
-
-// Filters for interpolation (0.5-band) - note this also filters integer pels.
-static const interp_kernel filteredinterp_filters500[(1 << SUBPEL_BITS)] = {
- {-3, 0, 35, 64, 35, 0, -3, 0},
- {-3, -1, 34, 64, 36, 1, -3, 0},
- {-3, -1, 32, 64, 38, 1, -3, 0},
- {-2, -2, 31, 63, 39, 2, -3, 0},
- {-2, -2, 29, 63, 41, 2, -3, 0},
- {-2, -2, 28, 63, 42, 3, -4, 0},
- {-2, -3, 27, 63, 43, 4, -4, 0},
- {-2, -3, 25, 62, 45, 5, -4, 0},
- {-2, -3, 24, 62, 46, 5, -4, 0},
- {-2, -3, 23, 61, 47, 6, -4, 0},
- {-2, -3, 21, 60, 49, 7, -4, 0},
- {-1, -4, 20, 60, 50, 8, -4, -1},
- {-1, -4, 19, 59, 51, 9, -4, -1},
- {-1, -4, 17, 58, 52, 10, -4, 0},
- {-1, -4, 16, 57, 53, 12, -4, -1},
- {-1, -4, 15, 56, 54, 13, -4, -1},
- {-1, -4, 14, 55, 55, 14, -4, -1},
- {-1, -4, 13, 54, 56, 15, -4, -1},
- {-1, -4, 12, 53, 57, 16, -4, -1},
- {0, -4, 10, 52, 58, 17, -4, -1},
- {-1, -4, 9, 51, 59, 19, -4, -1},
- {-1, -4, 8, 50, 60, 20, -4, -1},
- {0, -4, 7, 49, 60, 21, -3, -2},
- {0, -4, 6, 47, 61, 23, -3, -2},
- {0, -4, 5, 46, 62, 24, -3, -2},
- {0, -4, 5, 45, 62, 25, -3, -2},
- {0, -4, 4, 43, 63, 27, -3, -2},
- {0, -4, 3, 42, 63, 28, -2, -2},
- {0, -3, 2, 41, 63, 29, -2, -2},
- {0, -3, 2, 39, 63, 31, -2, -2},
- {0, -3, 1, 38, 64, 32, -1, -3},
- {0, -3, 1, 36, 64, 34, -1, -3}
-};
-
-// Filters for interpolation (0.625-band) - note this also filters integer pels.
-static const interp_kernel filteredinterp_filters625[(1 << SUBPEL_BITS)] = {
- {-1, -8, 33, 80, 33, -8, -1, 0},
- {-1, -8, 30, 80, 35, -8, -1, 1},
- {-1, -8, 28, 80, 37, -7, -2, 1},
- {0, -8, 26, 79, 39, -7, -2, 1},
- {0, -8, 24, 79, 41, -7, -2, 1},
- {0, -8, 22, 78, 43, -6, -2, 1},
- {0, -8, 20, 78, 45, -5, -3, 1},
- {0, -8, 18, 77, 48, -5, -3, 1},
- {0, -8, 16, 76, 50, -4, -3, 1},
- {0, -8, 15, 75, 52, -3, -4, 1},
- {0, -7, 13, 74, 54, -3, -4, 1},
- {0, -7, 11, 73, 56, -2, -4, 1},
- {0, -7, 10, 71, 58, -1, -4, 1},
- {1, -7, 8, 70, 60, 0, -5, 1},
- {1, -6, 6, 68, 62, 1, -5, 1},
- {1, -6, 5, 67, 63, 2, -5, 1},
- {1, -6, 4, 65, 65, 4, -6, 1},
- {1, -5, 2, 63, 67, 5, -6, 1},
- {1, -5, 1, 62, 68, 6, -6, 1},
- {1, -5, 0, 60, 70, 8, -7, 1},
- {1, -4, -1, 58, 71, 10, -7, 0},
- {1, -4, -2, 56, 73, 11, -7, 0},
- {1, -4, -3, 54, 74, 13, -7, 0},
- {1, -4, -3, 52, 75, 15, -8, 0},
- {1, -3, -4, 50, 76, 16, -8, 0},
- {1, -3, -5, 48, 77, 18, -8, 0},
- {1, -3, -5, 45, 78, 20, -8, 0},
- {1, -2, -6, 43, 78, 22, -8, 0},
- {1, -2, -7, 41, 79, 24, -8, 0},
- {1, -2, -7, 39, 79, 26, -8, 0},
- {1, -2, -7, 37, 80, 28, -8, -1},
- {1, -1, -8, 35, 80, 30, -8, -1},
-};
-
-// Filters for interpolation (0.75-band) - note this also filters integer pels.
-static const interp_kernel filteredinterp_filters750[(1 << SUBPEL_BITS)] = {
- {2, -11, 25, 96, 25, -11, 2, 0},
- {2, -11, 22, 96, 28, -11, 2, 0},
- {2, -10, 19, 95, 31, -11, 2, 0},
- {2, -10, 17, 95, 34, -12, 2, 0},
- {2, -9, 14, 94, 37, -12, 2, 0},
- {2, -8, 12, 93, 40, -12, 1, 0},
- {2, -8, 9, 92, 43, -12, 1, 1},
- {2, -7, 7, 91, 46, -12, 1, 0},
- {2, -7, 5, 90, 49, -12, 1, 0},
- {2, -6, 3, 88, 52, -12, 0, 1},
- {2, -5, 1, 86, 55, -12, 0, 1},
- {2, -5, -1, 84, 58, -11, 0, 1},
- {2, -4, -2, 82, 61, -11, -1, 1},
- {2, -4, -4, 80, 64, -10, -1, 1},
- {1, -3, -5, 77, 67, -9, -1, 1},
- {1, -3, -6, 75, 70, -8, -2, 1},
- {1, -2, -7, 72, 72, -7, -2, 1},
- {1, -2, -8, 70, 75, -6, -3, 1},
- {1, -1, -9, 67, 77, -5, -3, 1},
- {1, -1, -10, 64, 80, -4, -4, 2},
- {1, -1, -11, 61, 82, -2, -4, 2},
- {1, 0, -11, 58, 84, -1, -5, 2},
- {1, 0, -12, 55, 86, 1, -5, 2},
- {1, 0, -12, 52, 88, 3, -6, 2},
- {0, 1, -12, 49, 90, 5, -7, 2},
- {0, 1, -12, 46, 91, 7, -7, 2},
- {1, 1, -12, 43, 92, 9, -8, 2},
- {0, 1, -12, 40, 93, 12, -8, 2},
- {0, 2, -12, 37, 94, 14, -9, 2},
- {0, 2, -12, 34, 95, 17, -10, 2},
- {0, 2, -11, 31, 95, 19, -10, 2},
- {0, 2, -11, 28, 96, 22, -11, 2}
-};
-
-// Filters for interpolation (0.875-band) - note this also filters integer pels.
-static const interp_kernel filteredinterp_filters875[(1 << SUBPEL_BITS)] = {
- {3, -8, 13, 112, 13, -8, 3, 0},
- {3, -7, 10, 112, 17, -9, 3, -1},
- {2, -6, 7, 111, 21, -9, 3, -1},
- {2, -5, 4, 111, 24, -10, 3, -1},
- {2, -4, 1, 110, 28, -11, 3, -1},
- {1, -3, -1, 108, 32, -12, 4, -1},
- {1, -2, -3, 106, 36, -13, 4, -1},
- {1, -1, -6, 105, 40, -14, 4, -1},
- {1, -1, -7, 102, 44, -14, 4, -1},
- {1, 0, -9, 100, 48, -15, 4, -1},
- {1, 1, -11, 97, 53, -16, 4, -1},
- {0, 1, -12, 95, 57, -16, 4, -1},
- {0, 2, -13, 91, 61, -16, 4, -1},
- {0, 2, -14, 88, 65, -16, 4, -1},
- {0, 3, -15, 84, 69, -17, 4, 0},
- {0, 3, -16, 81, 73, -16, 3, 0},
- {0, 3, -16, 77, 77, -16, 3, 0},
- {0, 3, -16, 73, 81, -16, 3, 0},
- {0, 4, -17, 69, 84, -15, 3, 0},
- {-1, 4, -16, 65, 88, -14, 2, 0},
- {-1, 4, -16, 61, 91, -13, 2, 0},
- {-1, 4, -16, 57, 95, -12, 1, 0},
- {-1, 4, -16, 53, 97, -11, 1, 1},
- {-1, 4, -15, 48, 100, -9, 0, 1},
- {-1, 4, -14, 44, 102, -7, -1, 1},
- {-1, 4, -14, 40, 105, -6, -1, 1},
- {-1, 4, -13, 36, 106, -3, -2, 1},
- {-1, 4, -12, 32, 108, -1, -3, 1},
- {-1, 3, -11, 28, 110, 1, -4, 2},
- {-1, 3, -10, 24, 111, 4, -5, 2},
- {-1, 3, -9, 21, 111, 7, -6, 2},
- {-1, 3, -9, 17, 112, 10, -7, 3}
-};
-
-// Filters for interpolation (full-band) - no filtering for integer pixels
-static const interp_kernel filteredinterp_filters1000[(1 << SUBPEL_BITS)] = {
- {0, 0, 0, 128, 0, 0, 0, 0},
- {0, 1, -3, 128, 3, -1, 0, 0},
- {-1, 2, -6, 127, 7, -2, 1, 0},
- {-1, 3, -9, 126, 12, -4, 1, 0},
- {-1, 4, -12, 125, 16, -5, 1, 0},
- {-1, 4, -14, 123, 20, -6, 2, 0},
- {-1, 5, -15, 120, 25, -8, 2, 0},
- {-1, 5, -17, 118, 30, -9, 3, -1},
- {-1, 6, -18, 114, 35, -10, 3, -1},
- {-1, 6, -19, 111, 41, -12, 3, -1},
- {-1, 6, -20, 107, 46, -13, 4, -1},
- {-1, 6, -21, 103, 52, -14, 4, -1},
- {-1, 6, -21, 99, 57, -16, 5, -1},
- {-1, 6, -21, 94, 63, -17, 5, -1},
- {-1, 6, -20, 89, 68, -18, 5, -1},
- {-1, 6, -20, 84, 73, -19, 6, -1},
- {-1, 6, -20, 79, 79, -20, 6, -1},
- {-1, 6, -19, 73, 84, -20, 6, -1},
- {-1, 5, -18, 68, 89, -20, 6, -1},
- {-1, 5, -17, 63, 94, -21, 6, -1},
- {-1, 5, -16, 57, 99, -21, 6, -1},
- {-1, 4, -14, 52, 103, -21, 6, -1},
- {-1, 4, -13, 46, 107, -20, 6, -1},
- {-1, 3, -12, 41, 111, -19, 6, -1},
- {-1, 3, -10, 35, 114, -18, 6, -1},
- {-1, 3, -9, 30, 118, -17, 5, -1},
- {0, 2, -8, 25, 120, -15, 5, -1},
- {0, 2, -6, 20, 123, -14, 4, -1},
- {0, 1, -5, 16, 125, -12, 4, -1},
- {0, 1, -4, 12, 126, -9, 3, -1},
- {0, 1, -2, 7, 127, -6, 2, -1},
- {0, 0, -1, 3, 128, -3, 1, 0}
-};
-
-// Filters for factor of 2 downsampling.
-static const int16_t vp10_down2_symeven_half_filter[] = {56, 12, -3, -1};
-static const int16_t vp10_down2_symodd_half_filter[] = {64, 35, 0, -3};
-
-static const interp_kernel *choose_interp_filter(int inlength, int outlength) {
- int outlength16 = outlength * 16;
- if (outlength16 >= inlength * 16)
- return filteredinterp_filters1000;
- else if (outlength16 >= inlength * 13)
- return filteredinterp_filters875;
- else if (outlength16 >= inlength * 11)
- return filteredinterp_filters750;
- else if (outlength16 >= inlength * 9)
- return filteredinterp_filters625;
- else
- return filteredinterp_filters500;
-}
-
-static void interpolate(const uint8_t *const input, int inlength,
- uint8_t *output, int outlength) {
- const int64_t delta = (((uint64_t)inlength << 32) + outlength / 2) /
- outlength;
- const int64_t offset = inlength > outlength ?
- (((int64_t)(inlength - outlength) << 31) + outlength / 2) / outlength :
- -(((int64_t)(outlength - inlength) << 31) + outlength / 2) / outlength;
- uint8_t *optr = output;
- int x, x1, x2, sum, k, int_pel, sub_pel;
- int64_t y;
-
- const interp_kernel *interp_filters =
- choose_interp_filter(inlength, outlength);
-
- x = 0;
- y = offset;
- while ((y >> INTERP_PRECISION_BITS) < (INTERP_TAPS / 2 - 1)) {
- x++;
- y += delta;
- }
- x1 = x;
- x = outlength - 1;
- y = delta * x + offset;
- while ((y >> INTERP_PRECISION_BITS) +
- (int64_t)(INTERP_TAPS / 2) >= inlength) {
- x--;
- y -= delta;
- }
- x2 = x;
- if (x1 > x2) {
- for (x = 0, y = offset; x < outlength; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k) {
- const int pk = int_pel - INTERP_TAPS / 2 + 1 + k;
- sum += filter[k] * input[(pk < 0 ? 0 :
- (pk >= inlength ? inlength - 1 : pk))];
- }
- *optr++ = clip_pixel(ROUND_POWER_OF_TWO(sum, FILTER_BITS));
- }
- } else {
- // Initial part.
- for (x = 0, y = offset; x < x1; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k)
- sum += filter[k] * input[(int_pel - INTERP_TAPS / 2 + 1 + k < 0 ?
- 0 :
- int_pel - INTERP_TAPS / 2 + 1 + k)];
- *optr++ = clip_pixel(ROUND_POWER_OF_TWO(sum, FILTER_BITS));
- }
- // Middle part.
- for (; x <= x2; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k)
- sum += filter[k] * input[int_pel - INTERP_TAPS / 2 + 1 + k];
- *optr++ = clip_pixel(ROUND_POWER_OF_TWO(sum, FILTER_BITS));
- }
- // End part.
- for (; x < outlength; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k)
- sum += filter[k] * input[(int_pel - INTERP_TAPS / 2 + 1 + k >=
- inlength ? inlength - 1 :
- int_pel - INTERP_TAPS / 2 + 1 + k)];
- *optr++ = clip_pixel(ROUND_POWER_OF_TWO(sum, FILTER_BITS));
- }
- }
-}
-
-static void down2_symeven(const uint8_t *const input, int length,
- uint8_t *output) {
- // Actual filter len = 2 * filter_len_half.
- const int16_t *filter = vp10_down2_symeven_half_filter;
- const int filter_len_half = sizeof(vp10_down2_symeven_half_filter) / 2;
- int i, j;
- uint8_t *optr = output;
- int l1 = filter_len_half;
- int l2 = (length - filter_len_half);
- l1 += (l1 & 1);
- l2 += (l2 & 1);
- if (l1 > l2) {
- // Short input length.
- for (i = 0; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] +
- input[(i + 1 + j >= length ? length - 1 : i + 1 + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- } else {
- // Initial part.
- for (i = 0; i < l1; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] + input[i + 1 + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- // Middle part.
- for (; i < l2; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[i - j] + input[i + 1 + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- // End part.
- for (; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[i - j] +
- input[(i + 1 + j >= length ? length - 1 : i + 1 + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- }
-}
-
-static void down2_symodd(const uint8_t *const input, int length,
- uint8_t *output) {
- // Actual filter len = 2 * filter_len_half - 1.
- const int16_t *filter = vp10_down2_symodd_half_filter;
- const int filter_len_half = sizeof(vp10_down2_symodd_half_filter) / 2;
- int i, j;
- uint8_t *optr = output;
- int l1 = filter_len_half - 1;
- int l2 = (length - filter_len_half + 1);
- l1 += (l1 & 1);
- l2 += (l2 & 1);
- if (l1 > l2) {
- // Short input length.
- for (i = 0; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] +
- input[(i + j >= length ? length - 1 : i + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- } else {
- // Initial part.
- for (i = 0; i < l1; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] + input[i + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- // Middle part.
- for (; i < l2; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[i - j] + input[i + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- // End part.
- for (; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[i - j] + input[(i + j >= length ? length - 1 : i + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel(sum);
- }
- }
-}
-
-static int get_down2_length(int length, int steps) {
- int s;
- for (s = 0; s < steps; ++s)
- length = (length + 1) >> 1;
- return length;
-}
-
-static int get_down2_steps(int in_length, int out_length) {
- int steps = 0;
- int proj_in_length;
- while ((proj_in_length = get_down2_length(in_length, 1)) >= out_length) {
- ++steps;
- in_length = proj_in_length;
- }
- return steps;
-}
-
-static void resize_multistep(const uint8_t *const input,
- int length,
- uint8_t *output,
- int olength,
- uint8_t *buf) {
- int steps;
- if (length == olength) {
- memcpy(output, input, sizeof(output[0]) * length);
- return;
- }
- steps = get_down2_steps(length, olength);
-
- if (steps > 0) {
- int s;
- uint8_t *out = NULL;
- uint8_t *tmpbuf = NULL;
- uint8_t *otmp, *otmp2;
- int filteredlength = length;
- if (!tmpbuf) {
- tmpbuf = (uint8_t *)malloc(sizeof(uint8_t) * length);
- otmp = tmpbuf;
- } else {
- otmp = buf;
- }
- otmp2 = otmp + get_down2_length(length, 1);
- for (s = 0; s < steps; ++s) {
- const int proj_filteredlength = get_down2_length(filteredlength, 1);
- const uint8_t *const in = (s == 0 ? input : out);
- if (s == steps - 1 && proj_filteredlength == olength)
- out = output;
- else
- out = (s & 1 ? otmp2 : otmp);
- if (filteredlength & 1)
- down2_symodd(in, filteredlength, out);
- else
- down2_symeven(in, filteredlength, out);
- filteredlength = proj_filteredlength;
- }
- if (filteredlength != olength) {
- interpolate(out, filteredlength, output, olength);
- }
- if (tmpbuf)
- free(tmpbuf);
- } else {
- interpolate(input, length, output, olength);
- }
-}
-
-static void fill_col_to_arr(uint8_t *img, int stride, int len, uint8_t *arr) {
- int i;
- uint8_t *iptr = img;
- uint8_t *aptr = arr;
- for (i = 0; i < len; ++i, iptr += stride) {
- *aptr++ = *iptr;
- }
-}
-
-static void fill_arr_to_col(uint8_t *img, int stride, int len, uint8_t *arr) {
- int i;
- uint8_t *iptr = img;
- uint8_t *aptr = arr;
- for (i = 0; i < len; ++i, iptr += stride) {
- *iptr = *aptr++;
- }
-}
-
-void vp10_resize_plane(const uint8_t *const input,
- int height,
- int width,
- int in_stride,
- uint8_t *output,
- int height2,
- int width2,
- int out_stride) {
- int i;
- uint8_t *intbuf = (uint8_t *)malloc(sizeof(uint8_t) * width2 * height);
- uint8_t *tmpbuf = (uint8_t *)malloc(sizeof(uint8_t) *
- (width < height ? height : width));
- uint8_t *arrbuf = (uint8_t *)malloc(sizeof(uint8_t) * (height + height2));
- assert(width > 0);
- assert(height > 0);
- assert(width2 > 0);
- assert(height2 > 0);
- for (i = 0; i < height; ++i)
- resize_multistep(input + in_stride * i, width,
- intbuf + width2 * i, width2, tmpbuf);
- for (i = 0; i < width2; ++i) {
- fill_col_to_arr(intbuf + i, width2, height, arrbuf);
- resize_multistep(arrbuf, height, arrbuf + height, height2, tmpbuf);
- fill_arr_to_col(output + i, out_stride, height2, arrbuf + height);
- }
- free(intbuf);
- free(tmpbuf);
- free(arrbuf);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static void highbd_interpolate(const uint16_t *const input, int inlength,
- uint16_t *output, int outlength, int bd) {
- const int64_t delta =
- (((uint64_t)inlength << 32) + outlength / 2) / outlength;
- const int64_t offset = inlength > outlength ?
- (((int64_t)(inlength - outlength) << 31) + outlength / 2) / outlength :
- -(((int64_t)(outlength - inlength) << 31) + outlength / 2) / outlength;
- uint16_t *optr = output;
- int x, x1, x2, sum, k, int_pel, sub_pel;
- int64_t y;
-
- const interp_kernel *interp_filters =
- choose_interp_filter(inlength, outlength);
-
- x = 0;
- y = offset;
- while ((y >> INTERP_PRECISION_BITS) < (INTERP_TAPS / 2 - 1)) {
- x++;
- y += delta;
- }
- x1 = x;
- x = outlength - 1;
- y = delta * x + offset;
- while ((y >> INTERP_PRECISION_BITS) +
- (int64_t)(INTERP_TAPS / 2) >= inlength) {
- x--;
- y -= delta;
- }
- x2 = x;
- if (x1 > x2) {
- for (x = 0, y = offset; x < outlength; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k) {
- const int pk = int_pel - INTERP_TAPS / 2 + 1 + k;
- sum += filter[k] *
- input[(pk < 0 ? 0 : (pk >= inlength ? inlength - 1 : pk))];
- }
- *optr++ = clip_pixel_highbd(ROUND_POWER_OF_TWO(sum, FILTER_BITS), bd);
- }
- } else {
- // Initial part.
- for (x = 0, y = offset; x < x1; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k)
- sum += filter[k] *
- input[(int_pel - INTERP_TAPS / 2 + 1 + k < 0 ?
- 0 : int_pel - INTERP_TAPS / 2 + 1 + k)];
- *optr++ = clip_pixel_highbd(ROUND_POWER_OF_TWO(sum, FILTER_BITS), bd);
- }
- // Middle part.
- for (; x <= x2; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k)
- sum += filter[k] * input[int_pel - INTERP_TAPS / 2 + 1 + k];
- *optr++ = clip_pixel_highbd(ROUND_POWER_OF_TWO(sum, FILTER_BITS), bd);
- }
- // End part.
- for (; x < outlength; ++x, y += delta) {
- const int16_t *filter;
- int_pel = y >> INTERP_PRECISION_BITS;
- sub_pel = (y >> (INTERP_PRECISION_BITS - SUBPEL_BITS)) & SUBPEL_MASK;
- filter = interp_filters[sub_pel];
- sum = 0;
- for (k = 0; k < INTERP_TAPS; ++k)
- sum += filter[k] * input[(int_pel - INTERP_TAPS / 2 + 1 + k >=
- inlength ? inlength - 1 :
- int_pel - INTERP_TAPS / 2 + 1 + k)];
- *optr++ = clip_pixel_highbd(ROUND_POWER_OF_TWO(sum, FILTER_BITS), bd);
- }
- }
-}
-
-static void highbd_down2_symeven(const uint16_t *const input, int length,
- uint16_t *output, int bd) {
- // Actual filter len = 2 * filter_len_half.
- static const int16_t *filter = vp10_down2_symeven_half_filter;
- const int filter_len_half = sizeof(vp10_down2_symeven_half_filter) / 2;
- int i, j;
- uint16_t *optr = output;
- int l1 = filter_len_half;
- int l2 = (length - filter_len_half);
- l1 += (l1 & 1);
- l2 += (l2 & 1);
- if (l1 > l2) {
- // Short input length.
- for (i = 0; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] +
- input[(i + 1 + j >= length ? length - 1 : i + 1 + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- } else {
- // Initial part.
- for (i = 0; i < l1; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] + input[i + 1 + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- // Middle part.
- for (; i < l2; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[i - j] + input[i + 1 + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- // End part.
- for (; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1));
- for (j = 0; j < filter_len_half; ++j) {
- sum += (input[i - j] +
- input[(i + 1 + j >= length ? length - 1 : i + 1 + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- }
-}
-
-static void highbd_down2_symodd(const uint16_t *const input, int length,
- uint16_t *output, int bd) {
- // Actual filter len = 2 * filter_len_half - 1.
- static const int16_t *filter = vp10_down2_symodd_half_filter;
- const int filter_len_half = sizeof(vp10_down2_symodd_half_filter) / 2;
- int i, j;
- uint16_t *optr = output;
- int l1 = filter_len_half - 1;
- int l2 = (length - filter_len_half + 1);
- l1 += (l1 & 1);
- l2 += (l2 & 1);
- if (l1 > l2) {
- // Short input length.
- for (i = 0; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] +
- input[(i + j >= length ? length - 1 : i + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- } else {
- // Initial part.
- for (i = 0; i < l1; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[(i - j < 0 ? 0 : i - j)] + input[i + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- // Middle part.
- for (; i < l2; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[i - j] + input[i + j]) * filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- // End part.
- for (; i < length; i += 2) {
- int sum = (1 << (FILTER_BITS - 1)) + input[i] * filter[0];
- for (j = 1; j < filter_len_half; ++j) {
- sum += (input[i - j] + input[(i + j >= length ? length - 1 : i + j)]) *
- filter[j];
- }
- sum >>= FILTER_BITS;
- *optr++ = clip_pixel_highbd(sum, bd);
- }
- }
-}
-
-static void highbd_resize_multistep(const uint16_t *const input,
- int length,
- uint16_t *output,
- int olength,
- uint16_t *buf,
- int bd) {
- int steps;
- if (length == olength) {
- memcpy(output, input, sizeof(output[0]) * length);
- return;
- }
- steps = get_down2_steps(length, olength);
-
- if (steps > 0) {
- int s;
- uint16_t *out = NULL;
- uint16_t *tmpbuf = NULL;
- uint16_t *otmp, *otmp2;
- int filteredlength = length;
- if (!tmpbuf) {
- tmpbuf = (uint16_t *)malloc(sizeof(uint16_t) * length);
- otmp = tmpbuf;
- } else {
- otmp = buf;
- }
- otmp2 = otmp + get_down2_length(length, 1);
- for (s = 0; s < steps; ++s) {
- const int proj_filteredlength = get_down2_length(filteredlength, 1);
- const uint16_t *const in = (s == 0 ? input : out);
- if (s == steps - 1 && proj_filteredlength == olength)
- out = output;
- else
- out = (s & 1 ? otmp2 : otmp);
- if (filteredlength & 1)
- highbd_down2_symodd(in, filteredlength, out, bd);
- else
- highbd_down2_symeven(in, filteredlength, out, bd);
- filteredlength = proj_filteredlength;
- }
- if (filteredlength != olength) {
- highbd_interpolate(out, filteredlength, output, olength, bd);
- }
- if (tmpbuf)
- free(tmpbuf);
- } else {
- highbd_interpolate(input, length, output, olength, bd);
- }
-}
-
-static void highbd_fill_col_to_arr(uint16_t *img, int stride, int len,
- uint16_t *arr) {
- int i;
- uint16_t *iptr = img;
- uint16_t *aptr = arr;
- for (i = 0; i < len; ++i, iptr += stride) {
- *aptr++ = *iptr;
- }
-}
-
-static void highbd_fill_arr_to_col(uint16_t *img, int stride, int len,
- uint16_t *arr) {
- int i;
- uint16_t *iptr = img;
- uint16_t *aptr = arr;
- for (i = 0; i < len; ++i, iptr += stride) {
- *iptr = *aptr++;
- }
-}
-
-void vp10_highbd_resize_plane(const uint8_t *const input,
- int height,
- int width,
- int in_stride,
- uint8_t *output,
- int height2,
- int width2,
- int out_stride,
- int bd) {
- int i;
- uint16_t *intbuf = (uint16_t *)malloc(sizeof(uint16_t) * width2 * height);
- uint16_t *tmpbuf = (uint16_t *)malloc(sizeof(uint16_t) *
- (width < height ? height : width));
- uint16_t *arrbuf = (uint16_t *)malloc(sizeof(uint16_t) * (height + height2));
- for (i = 0; i < height; ++i) {
- highbd_resize_multistep(CONVERT_TO_SHORTPTR(input + in_stride * i), width,
- intbuf + width2 * i, width2, tmpbuf, bd);
- }
- for (i = 0; i < width2; ++i) {
- highbd_fill_col_to_arr(intbuf + i, width2, height, arrbuf);
- highbd_resize_multistep(arrbuf, height, arrbuf + height, height2, tmpbuf,
- bd);
- highbd_fill_arr_to_col(CONVERT_TO_SHORTPTR(output + i), out_stride, height2,
- arrbuf + height);
- }
- free(intbuf);
- free(tmpbuf);
- free(arrbuf);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-void vp10_resize_frame420(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u, const uint8_t *const v,
- int uv_stride,
- int height, int width,
- uint8_t *oy, int oy_stride,
- uint8_t *ou, uint8_t *ov, int ouv_stride,
- int oheight, int owidth) {
- vp10_resize_plane(y, height, width, y_stride,
- oy, oheight, owidth, oy_stride);
- vp10_resize_plane(u, height / 2, width / 2, uv_stride,
- ou, oheight / 2, owidth / 2, ouv_stride);
- vp10_resize_plane(v, height / 2, width / 2, uv_stride,
- ov, oheight / 2, owidth / 2, ouv_stride);
-}
-
-void vp10_resize_frame422(const uint8_t *const y, int y_stride,
- const uint8_t *const u, const uint8_t *const v,
- int uv_stride,
- int height, int width,
- uint8_t *oy, int oy_stride,
- uint8_t *ou, uint8_t *ov, int ouv_stride,
- int oheight, int owidth) {
- vp10_resize_plane(y, height, width, y_stride,
- oy, oheight, owidth, oy_stride);
- vp10_resize_plane(u, height, width / 2, uv_stride,
- ou, oheight, owidth / 2, ouv_stride);
- vp10_resize_plane(v, height, width / 2, uv_stride,
- ov, oheight, owidth / 2, ouv_stride);
-}
-
-void vp10_resize_frame444(const uint8_t *const y, int y_stride,
- const uint8_t *const u, const uint8_t *const v,
- int uv_stride,
- int height, int width,
- uint8_t *oy, int oy_stride,
- uint8_t *ou, uint8_t *ov, int ouv_stride,
- int oheight, int owidth) {
- vp10_resize_plane(y, height, width, y_stride,
- oy, oheight, owidth, oy_stride);
- vp10_resize_plane(u, height, width, uv_stride,
- ou, oheight, owidth, ouv_stride);
- vp10_resize_plane(v, height, width, uv_stride,
- ov, oheight, owidth, ouv_stride);
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_resize_frame420(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u, const uint8_t *const v,
- int uv_stride,
- int height, int width,
- uint8_t *oy, int oy_stride,
- uint8_t *ou, uint8_t *ov, int ouv_stride,
- int oheight, int owidth, int bd) {
- vp10_highbd_resize_plane(y, height, width, y_stride,
- oy, oheight, owidth, oy_stride, bd);
- vp10_highbd_resize_plane(u, height / 2, width / 2, uv_stride,
- ou, oheight / 2, owidth / 2, ouv_stride, bd);
- vp10_highbd_resize_plane(v, height / 2, width / 2, uv_stride,
- ov, oheight / 2, owidth / 2, ouv_stride, bd);
-}
-
-void vp10_highbd_resize_frame422(const uint8_t *const y, int y_stride,
- const uint8_t *const u, const uint8_t *const v,
- int uv_stride,
- int height, int width,
- uint8_t *oy, int oy_stride,
- uint8_t *ou, uint8_t *ov, int ouv_stride,
- int oheight, int owidth, int bd) {
- vp10_highbd_resize_plane(y, height, width, y_stride,
- oy, oheight, owidth, oy_stride, bd);
- vp10_highbd_resize_plane(u, height, width / 2, uv_stride,
- ou, oheight, owidth / 2, ouv_stride, bd);
- vp10_highbd_resize_plane(v, height, width / 2, uv_stride,
- ov, oheight, owidth / 2, ouv_stride, bd);
-}
-
-void vp10_highbd_resize_frame444(const uint8_t *const y, int y_stride,
- const uint8_t *const u, const uint8_t *const v,
- int uv_stride,
- int height, int width,
- uint8_t *oy, int oy_stride,
- uint8_t *ou, uint8_t *ov, int ouv_stride,
- int oheight, int owidth, int bd) {
- vp10_highbd_resize_plane(y, height, width, y_stride,
- oy, oheight, owidth, oy_stride, bd);
- vp10_highbd_resize_plane(u, height, width, uv_stride,
- ou, oheight, owidth, ouv_stride, bd);
- vp10_highbd_resize_plane(v, height, width, uv_stride,
- ov, oheight, owidth, ouv_stride, bd);
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_RESIZE_H_
-#define VP10_ENCODER_RESIZE_H_
-
-#include <stdio.h>
-#include "vpx/vpx_integer.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_resize_plane(const uint8_t *const input,
- int height,
- int width,
- int in_stride,
- uint8_t *output,
- int height2,
- int width2,
- int out_stride);
-void vp10_resize_frame420(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u,
- const uint8_t *const v,
- int uv_stride,
- int height,
- int width,
- uint8_t *oy,
- int oy_stride,
- uint8_t *ou,
- uint8_t *ov,
- int ouv_stride,
- int oheight,
- int owidth);
-void vp10_resize_frame422(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u,
- const uint8_t *const v,
- int uv_stride,
- int height,
- int width,
- uint8_t *oy,
- int oy_stride,
- uint8_t *ou,
- uint8_t *ov,
- int ouv_stride,
- int oheight,
- int owidth);
-void vp10_resize_frame444(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u,
- const uint8_t *const v,
- int uv_stride,
- int height,
- int width,
- uint8_t *oy,
- int oy_stride,
- uint8_t *ou,
- uint8_t *ov,
- int ouv_stride,
- int oheight,
- int owidth);
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_resize_plane(const uint8_t *const input,
- int height,
- int width,
- int in_stride,
- uint8_t *output,
- int height2,
- int width2,
- int out_stride,
- int bd);
-void vp10_highbd_resize_frame420(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u,
- const uint8_t *const v,
- int uv_stride,
- int height,
- int width,
- uint8_t *oy,
- int oy_stride,
- uint8_t *ou,
- uint8_t *ov,
- int ouv_stride,
- int oheight,
- int owidth,
- int bd);
-void vp10_highbd_resize_frame422(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u,
- const uint8_t *const v,
- int uv_stride,
- int height,
- int width,
- uint8_t *oy,
- int oy_stride,
- uint8_t *ou,
- uint8_t *ov,
- int ouv_stride,
- int oheight,
- int owidth,
- int bd);
-void vp10_highbd_resize_frame444(const uint8_t *const y,
- int y_stride,
- const uint8_t *const u,
- const uint8_t *const v,
- int uv_stride,
- int height,
- int width,
- uint8_t *oy,
- int oy_stride,
- uint8_t *ou,
- uint8_t *ov,
- int ouv_stride,
- int oheight,
- int owidth,
- int bd);
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_RESIZE_H_
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#include <limits.h>
-
-#include "vpx_mem/vpx_mem.h"
-
-#include "vp10/common/pred_common.h"
-#include "vp10/common/tile_common.h"
-
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/segmentation.h"
-#include "vp10/encoder/subexp.h"
-
-void vp10_enable_segmentation(struct segmentation *seg) {
- seg->enabled = 1;
- seg->update_map = 1;
- seg->update_data = 1;
-}
-
-void vp10_disable_segmentation(struct segmentation *seg) {
- seg->enabled = 0;
- seg->update_map = 0;
- seg->update_data = 0;
-}
-
-void vp10_set_segment_data(struct segmentation *seg,
- signed char *feature_data,
- unsigned char abs_delta) {
- seg->abs_delta = abs_delta;
-
- memcpy(seg->feature_data, feature_data, sizeof(seg->feature_data));
-}
-void vp10_disable_segfeature(struct segmentation *seg, int segment_id,
- SEG_LVL_FEATURES feature_id) {
- seg->feature_mask[segment_id] &= ~(1 << feature_id);
-}
-
-void vp10_clear_segdata(struct segmentation *seg, int segment_id,
- SEG_LVL_FEATURES feature_id) {
- seg->feature_data[segment_id][feature_id] = 0;
-}
-
-// Based on set of segment counts calculate a probability tree
-static void calc_segtree_probs(unsigned *segcounts,
- vpx_prob *segment_tree_probs, const vpx_prob *cur_tree_probs) {
- // Work out probabilities of each segment
- const unsigned cc[4] = {
- segcounts[0] + segcounts[1], segcounts[2] + segcounts[3],
- segcounts[4] + segcounts[5], segcounts[6] + segcounts[7]
- };
- const unsigned ccc[2] = { cc[0] + cc[1], cc[2] + cc[3] };
-#if CONFIG_MISC_FIXES
- int i;
-#endif
-
- segment_tree_probs[0] = get_binary_prob(ccc[0], ccc[1]);
- segment_tree_probs[1] = get_binary_prob(cc[0], cc[1]);
- segment_tree_probs[2] = get_binary_prob(cc[2], cc[3]);
- segment_tree_probs[3] = get_binary_prob(segcounts[0], segcounts[1]);
- segment_tree_probs[4] = get_binary_prob(segcounts[2], segcounts[3]);
- segment_tree_probs[5] = get_binary_prob(segcounts[4], segcounts[5]);
- segment_tree_probs[6] = get_binary_prob(segcounts[6], segcounts[7]);
-
-#if CONFIG_MISC_FIXES
- for (i = 0; i < 7; i++) {
- const unsigned *ct = i == 0 ? ccc : i < 3 ? cc + (i & 2)
- : segcounts + (i - 3) * 2;
- vp10_prob_diff_update_savings_search(ct,
- cur_tree_probs[i], &segment_tree_probs[i], DIFF_UPDATE_PROB);
- }
-#else
- (void) cur_tree_probs;
-#endif
-}
-
-// Based on set of segment counts and probabilities calculate a cost estimate
-static int cost_segmap(unsigned *segcounts, vpx_prob *probs) {
- const int c01 = segcounts[0] + segcounts[1];
- const int c23 = segcounts[2] + segcounts[3];
- const int c45 = segcounts[4] + segcounts[5];
- const int c67 = segcounts[6] + segcounts[7];
- const int c0123 = c01 + c23;
- const int c4567 = c45 + c67;
-
- // Cost the top node of the tree
- int cost = c0123 * vp10_cost_zero(probs[0]) +
- c4567 * vp10_cost_one(probs[0]);
-
- // Cost subsequent levels
- if (c0123 > 0) {
- cost += c01 * vp10_cost_zero(probs[1]) +
- c23 * vp10_cost_one(probs[1]);
-
- if (c01 > 0)
- cost += segcounts[0] * vp10_cost_zero(probs[3]) +
- segcounts[1] * vp10_cost_one(probs[3]);
- if (c23 > 0)
- cost += segcounts[2] * vp10_cost_zero(probs[4]) +
- segcounts[3] * vp10_cost_one(probs[4]);
- }
-
- if (c4567 > 0) {
- cost += c45 * vp10_cost_zero(probs[2]) +
- c67 * vp10_cost_one(probs[2]);
-
- if (c45 > 0)
- cost += segcounts[4] * vp10_cost_zero(probs[5]) +
- segcounts[5] * vp10_cost_one(probs[5]);
- if (c67 > 0)
- cost += segcounts[6] * vp10_cost_zero(probs[6]) +
- segcounts[7] * vp10_cost_one(probs[6]);
- }
-
- return cost;
-}
-
-static void count_segs(const VP10_COMMON *cm, MACROBLOCKD *xd,
- const TileInfo *tile, MODE_INFO **mi,
- unsigned *no_pred_segcounts,
- unsigned (*temporal_predictor_count)[2],
- unsigned *t_unpred_seg_counts,
- int bw, int bh, int mi_row, int mi_col) {
- int segment_id;
-
- if (mi_row >= cm->mi_rows || mi_col >= cm->mi_cols)
- return;
-
- xd->mi = mi;
- segment_id = xd->mi[0]->mbmi.segment_id;
-
- set_mi_row_col(xd, tile, mi_row, bh, mi_col, bw, cm->mi_rows, cm->mi_cols);
-
- // Count the number of hits on each segment with no prediction
- no_pred_segcounts[segment_id]++;
-
- // Temporal prediction not allowed on key frames
- if (cm->frame_type != KEY_FRAME) {
- const BLOCK_SIZE bsize = xd->mi[0]->mbmi.sb_type;
- // Test to see if the segment id matches the predicted value.
- const int pred_segment_id = get_segment_id(cm, cm->last_frame_seg_map,
- bsize, mi_row, mi_col);
- const int pred_flag = pred_segment_id == segment_id;
- const int pred_context = vp10_get_pred_context_seg_id(xd);
-
- // Store the prediction status for this mb and update counts
- // as appropriate
- xd->mi[0]->mbmi.seg_id_predicted = pred_flag;
- temporal_predictor_count[pred_context][pred_flag]++;
-
- // Update the "unpredicted" segment count
- if (!pred_flag)
- t_unpred_seg_counts[segment_id]++;
- }
-}
-
-static void count_segs_sb(const VP10_COMMON *cm, MACROBLOCKD *xd,
- const TileInfo *tile, MODE_INFO **mi,
- unsigned *no_pred_segcounts,
- unsigned (*temporal_predictor_count)[2],
- unsigned *t_unpred_seg_counts,
- int mi_row, int mi_col,
- BLOCK_SIZE bsize) {
- const int mis = cm->mi_stride;
- int bw, bh;
- const int bs = num_8x8_blocks_wide_lookup[bsize], hbs = bs / 2;
-
- if (mi_row >= cm->mi_rows || mi_col >= cm->mi_cols)
- return;
-
- bw = num_8x8_blocks_wide_lookup[mi[0]->mbmi.sb_type];
- bh = num_8x8_blocks_high_lookup[mi[0]->mbmi.sb_type];
-
- if (bw == bs && bh == bs) {
- count_segs(cm, xd, tile, mi, no_pred_segcounts, temporal_predictor_count,
- t_unpred_seg_counts, bs, bs, mi_row, mi_col);
- } else if (bw == bs && bh < bs) {
- count_segs(cm, xd, tile, mi, no_pred_segcounts, temporal_predictor_count,
- t_unpred_seg_counts, bs, hbs, mi_row, mi_col);
- count_segs(cm, xd, tile, mi + hbs * mis, no_pred_segcounts,
- temporal_predictor_count, t_unpred_seg_counts, bs, hbs,
- mi_row + hbs, mi_col);
- } else if (bw < bs && bh == bs) {
- count_segs(cm, xd, tile, mi, no_pred_segcounts, temporal_predictor_count,
- t_unpred_seg_counts, hbs, bs, mi_row, mi_col);
- count_segs(cm, xd, tile, mi + hbs,
- no_pred_segcounts, temporal_predictor_count, t_unpred_seg_counts,
- hbs, bs, mi_row, mi_col + hbs);
- } else {
- const BLOCK_SIZE subsize = subsize_lookup[PARTITION_SPLIT][bsize];
- int n;
-
- assert(bw < bs && bh < bs);
-
- for (n = 0; n < 4; n++) {
- const int mi_dc = hbs * (n & 1);
- const int mi_dr = hbs * (n >> 1);
-
- count_segs_sb(cm, xd, tile, &mi[mi_dr * mis + mi_dc],
- no_pred_segcounts, temporal_predictor_count,
- t_unpred_seg_counts,
- mi_row + mi_dr, mi_col + mi_dc, subsize);
- }
- }
-}
-
-void vp10_choose_segmap_coding_method(VP10_COMMON *cm, MACROBLOCKD *xd) {
- struct segmentation *seg = &cm->seg;
-#if CONFIG_MISC_FIXES
- struct segmentation_probs *segp = &cm->fc->seg;
-#else
- struct segmentation_probs *segp = &cm->segp;
-#endif
-
- int no_pred_cost;
- int t_pred_cost = INT_MAX;
-
- int i, tile_col, mi_row, mi_col;
-
-#if CONFIG_MISC_FIXES
- unsigned (*temporal_predictor_count)[2] = cm->counts.seg.pred;
- unsigned *no_pred_segcounts = cm->counts.seg.tree_total;
- unsigned *t_unpred_seg_counts = cm->counts.seg.tree_mispred;
-#else
- unsigned temporal_predictor_count[PREDICTION_PROBS][2] = { { 0 } };
- unsigned no_pred_segcounts[MAX_SEGMENTS] = { 0 };
- unsigned t_unpred_seg_counts[MAX_SEGMENTS] = { 0 };
-#endif
-
- vpx_prob no_pred_tree[SEG_TREE_PROBS];
- vpx_prob t_pred_tree[SEG_TREE_PROBS];
- vpx_prob t_nopred_prob[PREDICTION_PROBS];
-
-#if CONFIG_MISC_FIXES
- (void) xd;
-#else
- // Set default state for the segment tree probabilities and the
- // temporal coding probabilities
- memset(segp->tree_probs, 255, sizeof(segp->tree_probs));
- memset(segp->pred_probs, 255, sizeof(segp->pred_probs));
-#endif
-
- // First of all generate stats regarding how well the last segment map
- // predicts this one
- for (tile_col = 0; tile_col < 1 << cm->log2_tile_cols; tile_col++) {
- TileInfo tile;
- MODE_INFO **mi_ptr;
- vp10_tile_init(&tile, cm, 0, tile_col);
-
- mi_ptr = cm->mi_grid_visible + tile.mi_col_start;
- for (mi_row = 0; mi_row < cm->mi_rows;
- mi_row += 8, mi_ptr += 8 * cm->mi_stride) {
- MODE_INFO **mi = mi_ptr;
- for (mi_col = tile.mi_col_start; mi_col < tile.mi_col_end;
- mi_col += 8, mi += 8)
- count_segs_sb(cm, xd, &tile, mi, no_pred_segcounts,
- temporal_predictor_count, t_unpred_seg_counts,
- mi_row, mi_col, BLOCK_64X64);
- }
- }
-
- // Work out probability tree for coding segments without prediction
- // and the cost.
- calc_segtree_probs(no_pred_segcounts, no_pred_tree, segp->tree_probs);
- no_pred_cost = cost_segmap(no_pred_segcounts, no_pred_tree);
-
- // Key frames cannot use temporal prediction
- if (!frame_is_intra_only(cm)) {
- // Work out probability tree for coding those segments not
- // predicted using the temporal method and the cost.
- calc_segtree_probs(t_unpred_seg_counts, t_pred_tree, segp->tree_probs);
- t_pred_cost = cost_segmap(t_unpred_seg_counts, t_pred_tree);
-
- // Add in the cost of the signaling for each prediction context.
- for (i = 0; i < PREDICTION_PROBS; i++) {
- const int count0 = temporal_predictor_count[i][0];
- const int count1 = temporal_predictor_count[i][1];
-
-#if CONFIG_MISC_FIXES
- vp10_prob_diff_update_savings_search(temporal_predictor_count[i],
- segp->pred_probs[i],
- &t_nopred_prob[i], DIFF_UPDATE_PROB);
-#else
- t_nopred_prob[i] = get_binary_prob(count0, count1);
-#endif
-
- // Add in the predictor signaling cost
- t_pred_cost += count0 * vp10_cost_zero(t_nopred_prob[i]) +
- count1 * vp10_cost_one(t_nopred_prob[i]);
- }
- }
-
- // Now choose which coding method to use.
- if (t_pred_cost < no_pred_cost) {
- seg->temporal_update = 1;
-#if !CONFIG_MISC_FIXES
- memcpy(segp->tree_probs, t_pred_tree, sizeof(t_pred_tree));
- memcpy(segp->pred_probs, t_nopred_prob, sizeof(t_nopred_prob));
-#endif
- } else {
- seg->temporal_update = 0;
-#if !CONFIG_MISC_FIXES
- memcpy(segp->tree_probs, no_pred_tree, sizeof(no_pred_tree));
-#endif
- }
-}
-
-void vp10_reset_segment_features(VP10_COMMON *cm) {
- struct segmentation *seg = &cm->seg;
-#if !CONFIG_MISC_FIXES
- struct segmentation_probs *segp = &cm->segp;
-#endif
-
- // Set up default state for MB feature flags
- seg->enabled = 0;
- seg->update_map = 0;
- seg->update_data = 0;
-#if !CONFIG_MISC_FIXES
- memset(segp->tree_probs, 255, sizeof(segp->tree_probs));
-#endif
- vp10_clearall_segfeatures(seg);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_SEGMENTATION_H_
-#define VP10_ENCODER_SEGMENTATION_H_
-
-#include "vp10/common/blockd.h"
-#include "vp10/encoder/encoder.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_enable_segmentation(struct segmentation *seg);
-void vp10_disable_segmentation(struct segmentation *seg);
-
-void vp10_disable_segfeature(struct segmentation *seg,
- int segment_id,
- SEG_LVL_FEATURES feature_id);
-void vp10_clear_segdata(struct segmentation *seg,
- int segment_id,
- SEG_LVL_FEATURES feature_id);
-
-// The values given for each segment can be either deltas (from the default
-// value chosen for the frame) or absolute values.
-//
-// Valid range for abs values is (0-127 for MB_LVL_ALT_Q), (0-63 for
-// SEGMENT_ALT_LF)
-// Valid range for delta values are (+/-127 for MB_LVL_ALT_Q), (+/-63 for
-// SEGMENT_ALT_LF)
-//
-// abs_delta = SEGMENT_DELTADATA (deltas) abs_delta = SEGMENT_ABSDATA (use
-// the absolute values given).
-void vp10_set_segment_data(struct segmentation *seg, signed char *feature_data,
- unsigned char abs_delta);
-
-void vp10_choose_segmap_coding_method(VP10_COMMON *cm, MACROBLOCKD *xd);
-
-void vp10_reset_segment_features(VP10_COMMON *cm);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_SEGMENTATION_H_
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-#include <math.h>
-
-#include "vp10/common/blockd.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/skin_detection.h"
-
-// Fixed-point skin color model parameters.
-static const int skin_mean[2] = {7463, 9614}; // q6
-static const int skin_inv_cov[4] = {4107, 1663, 1663, 2157}; // q16
-static const int skin_threshold = 1570636; // q18
-
-// Thresholds on luminance.
-static const int y_low = 20;
-static const int y_high = 220;
-
-// Evaluates the Mahalanobis distance measure for the input CbCr values.
-static int evaluate_skin_color_difference(int cb, int cr) {
- const int cb_q6 = cb << 6;
- const int cr_q6 = cr << 6;
- const int cb_diff_q12 = (cb_q6 - skin_mean[0]) * (cb_q6 - skin_mean[0]);
- const int cbcr_diff_q12 = (cb_q6 - skin_mean[0]) * (cr_q6 - skin_mean[1]);
- const int cr_diff_q12 = (cr_q6 - skin_mean[1]) * (cr_q6 - skin_mean[1]);
- const int cb_diff_q2 = (cb_diff_q12 + (1 << 9)) >> 10;
- const int cbcr_diff_q2 = (cbcr_diff_q12 + (1 << 9)) >> 10;
- const int cr_diff_q2 = (cr_diff_q12 + (1 << 9)) >> 10;
- const int skin_diff = skin_inv_cov[0] * cb_diff_q2 +
- skin_inv_cov[1] * cbcr_diff_q2 +
- skin_inv_cov[2] * cbcr_diff_q2 +
- skin_inv_cov[3] * cr_diff_q2;
- return skin_diff;
-}
-
-int vp10_skin_pixel(const uint8_t y, const uint8_t cb, const uint8_t cr) {
- if (y < y_low || y > y_high)
- return 0;
- else
- return (evaluate_skin_color_difference(cb, cr) < skin_threshold);
-}
-
-#ifdef OUTPUT_YUV_SKINMAP
-// For viewing skin map on input source.
-void vp10_compute_skin_map(VP10_COMP *const cpi, FILE *yuv_skinmap_file) {
- int i, j, mi_row, mi_col;
- VP10_COMMON *const cm = &cpi->common;
- uint8_t *y;
- const uint8_t *src_y = cpi->Source->y_buffer;
- const uint8_t *src_u = cpi->Source->u_buffer;
- const uint8_t *src_v = cpi->Source->v_buffer;
- const int src_ystride = cpi->Source->y_stride;
- const int src_uvstride = cpi->Source->uv_stride;
- YV12_BUFFER_CONFIG skinmap;
- memset(&skinmap, 0, sizeof(YV12_BUFFER_CONFIG));
- if (vpx_alloc_frame_buffer(&skinmap, cm->width, cm->height,
- cm->subsampling_x, cm->subsampling_y,
- VP9_ENC_BORDER_IN_PIXELS, cm->byte_alignment)) {
- vpx_free_frame_buffer(&skinmap);
- return;
- }
- memset(skinmap.buffer_alloc, 128, skinmap.frame_size);
- y = skinmap.y_buffer;
- // Loop through 8x8 blocks and set skin map based on center pixel of block.
- // Set y to white for skin block, otherwise set to source with gray scale.
- // Ignore rightmost/bottom boundary blocks.
- for (mi_row = 0; mi_row < cm->mi_rows - 1; ++mi_row) {
- for (mi_col = 0; mi_col < cm->mi_cols - 1; ++mi_col) {
- // Use middle pixel for each 8x8 block for skin detection.
- // If middle pixel is skin, assign whole 8x8 block to skin.
- const uint8_t ysource = src_y[4 * src_ystride + 4];
- const uint8_t usource = src_u[2 * src_uvstride + 2];
- const uint8_t vsource = src_v[2 * src_uvstride + 2];
- const int is_skin = vp10_skin_pixel(ysource, usource, vsource);
- for (i = 0; i < 8; i++) {
- for (j = 0; j < 8; j++) {
- if (is_skin)
- y[i * src_ystride + j] = 255;
- else
- y[i * src_ystride + j] = src_y[i * src_ystride + j];
- }
- }
- y += 8;
- src_y += 8;
- src_u += 4;
- src_v += 4;
- }
- y += (src_ystride << 3) - ((cm->mi_cols - 1) << 3);
- src_y += (src_ystride << 3) - ((cm->mi_cols - 1) << 3);
- src_u += (src_uvstride << 2) - ((cm->mi_cols - 1) << 2);
- src_v += (src_uvstride << 2) - ((cm->mi_cols - 1) << 2);
- }
- vp10_write_yuv_frame_420(&skinmap, yuv_skinmap_file);
- vpx_free_frame_buffer(&skinmap);
-}
-#endif
+++ /dev/null
-/*
- * Copyright (c) 2015 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_SKIN_MAP_H_
-#define VP10_ENCODER_SKIN_MAP_H_
-
-#include "vp10/common/blockd.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-struct VP10_COMP;
-
-// #define OUTPUT_YUV_SKINMAP
-
-int vp10_skin_pixel(const uint8_t y, const uint8_t cb, const uint8_t cr);
-
-#ifdef OUTPUT_YUV_SKINMAP
-// For viewing skin map on input source.
-void vp10_compute_skin_map(VP10_COMP *const cpi, FILE *yuv_skinmap_file);
-#endif
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_SKIN_MAP_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <limits.h>
-
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/speed_features.h"
-#include "vp10/encoder/rdopt.h"
-
-#include "vpx_dsp/vpx_dsp_common.h"
-
-// Intra only frames, golden frames (except alt ref overlays) and
-// alt ref frames tend to be coded at a higher than ambient quality
-static int frame_is_boosted(const VP10_COMP *cpi) {
- return frame_is_kf_gf_arf(cpi);
-}
-
-// Sets a partition size down to which the auto partition code will always
-// search (can go lower), based on the image dimensions. The logic here
-// is that the extent to which ringing artefacts are offensive, depends
-// partly on the screen area that over which they propogate. Propogation is
-// limited by transform block size but the screen area take up by a given block
-// size will be larger for a small image format stretched to full screen.
-static BLOCK_SIZE set_partition_min_limit(VP10_COMMON *const cm) {
- unsigned int screen_area = (cm->width * cm->height);
-
- // Select block size based on image format size.
- if (screen_area < 1280 * 720) {
- // Formats smaller in area than 720P
- return BLOCK_4X4;
- } else if (screen_area < 1920 * 1080) {
- // Format >= 720P and < 1080P
- return BLOCK_8X8;
- } else {
- // Formats 1080P and up
- return BLOCK_16X16;
- }
-}
-
-static void set_good_speed_feature_framesize_dependent(VP10_COMP *cpi,
- SPEED_FEATURES *sf,
- int speed) {
- VP10_COMMON *const cm = &cpi->common;
-
- if (speed >= 1) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->disable_split_mask = cm->show_frame ? DISABLE_ALL_SPLIT
- : DISABLE_ALL_INTER_SPLIT;
- sf->partition_search_breakout_dist_thr = (1 << 23);
- } else {
- sf->disable_split_mask = DISABLE_COMPOUND_SPLIT;
- sf->partition_search_breakout_dist_thr = (1 << 21);
- }
- }
-
- if (speed >= 2) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->disable_split_mask = cm->show_frame ? DISABLE_ALL_SPLIT
- : DISABLE_ALL_INTER_SPLIT;
- sf->adaptive_pred_interp_filter = 0;
- sf->partition_search_breakout_dist_thr = (1 << 24);
- sf->partition_search_breakout_rate_thr = 120;
- } else {
- sf->disable_split_mask = LAST_AND_INTRA_SPLIT_ONLY;
- sf->partition_search_breakout_dist_thr = (1 << 22);
- sf->partition_search_breakout_rate_thr = 100;
- }
- sf->rd_auto_partition_min_limit = set_partition_min_limit(cm);
- }
-
- if (speed >= 3) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->disable_split_mask = DISABLE_ALL_SPLIT;
- sf->schedule_mode_search = cm->base_qindex < 220 ? 1 : 0;
- sf->partition_search_breakout_dist_thr = (1 << 25);
- sf->partition_search_breakout_rate_thr = 200;
- } else {
- sf->max_intra_bsize = BLOCK_32X32;
- sf->disable_split_mask = DISABLE_ALL_INTER_SPLIT;
- sf->schedule_mode_search = cm->base_qindex < 175 ? 1 : 0;
- sf->partition_search_breakout_dist_thr = (1 << 23);
- sf->partition_search_breakout_rate_thr = 120;
- }
- }
-
- // If this is a two pass clip that fits the criteria for animated or
- // graphics content then reset disable_split_mask for speeds 1-4.
- // Also if the image edge is internal to the coded area.
- if ((speed >= 1) && (cpi->oxcf.pass == 2) &&
- ((cpi->twopass.fr_content_type == FC_GRAPHICS_ANIMATION) ||
- (vp10_internal_image_edge(cpi)))) {
- sf->disable_split_mask = DISABLE_COMPOUND_SPLIT;
- }
-
- if (speed >= 4) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->partition_search_breakout_dist_thr = (1 << 26);
- } else {
- sf->partition_search_breakout_dist_thr = (1 << 24);
- }
- sf->disable_split_mask = DISABLE_ALL_SPLIT;
- }
-}
-
-static void set_good_speed_feature(VP10_COMP *cpi, VP10_COMMON *cm,
- SPEED_FEATURES *sf, int speed) {
- const int boosted = frame_is_boosted(cpi);
-
- sf->adaptive_rd_thresh = 1;
- sf->allow_skip_recode = 1;
-
- if (speed >= 1) {
- if ((cpi->twopass.fr_content_type == FC_GRAPHICS_ANIMATION) ||
- vp10_internal_image_edge(cpi)) {
- sf->use_square_partition_only = !frame_is_boosted(cpi);
- } else {
- sf->use_square_partition_only = !frame_is_intra_only(cm);
- }
-
- sf->less_rectangular_check = 1;
-
- sf->use_rd_breakout = 1;
- sf->adaptive_motion_search = 1;
- sf->mv.auto_mv_step_size = 1;
- sf->adaptive_rd_thresh = 2;
- sf->mv.subpel_iters_per_step = 1;
- sf->mode_skip_start = 10;
- sf->adaptive_pred_interp_filter = 1;
-
- sf->recode_loop = ALLOW_RECODE_KFARFGF;
- sf->intra_y_mode_mask[TX_32X32] = INTRA_DC_H_V;
- sf->intra_uv_mode_mask[TX_32X32] = INTRA_DC_H_V;
- sf->intra_y_mode_mask[TX_16X16] = INTRA_DC_H_V;
- sf->intra_uv_mode_mask[TX_16X16] = INTRA_DC_H_V;
-
- sf->tx_size_search_breakout = 1;
- sf->partition_search_breakout_rate_thr = 80;
- }
-
- if (speed >= 2) {
- sf->tx_size_search_method = frame_is_boosted(cpi) ? USE_FULL_RD
- : USE_LARGESTALL;
-
- sf->mode_search_skip_flags = (cm->frame_type == KEY_FRAME) ? 0 :
- FLAG_SKIP_INTRA_DIRMISMATCH |
- FLAG_SKIP_INTRA_BESTINTER |
- FLAG_SKIP_COMP_BESTINTRA |
- FLAG_SKIP_INTRA_LOWVAR;
- sf->disable_filter_search_var_thresh = 100;
- sf->comp_inter_joint_search_thresh = BLOCK_SIZES;
- sf->auto_min_max_partition_size = RELAXED_NEIGHBORING_MIN_MAX;
- sf->allow_partition_search_skip = 1;
- }
-
- if (speed >= 3) {
- sf->use_square_partition_only = !frame_is_intra_only(cm);
- sf->tx_size_search_method = frame_is_intra_only(cm) ? USE_FULL_RD
- : USE_LARGESTALL;
- sf->mv.subpel_search_method = SUBPEL_TREE_PRUNED;
- sf->adaptive_pred_interp_filter = 0;
- sf->adaptive_mode_search = 1;
- sf->cb_partition_search = !boosted;
- sf->cb_pred_filter_search = 1;
- sf->alt_ref_search_fp = 1;
- sf->recode_loop = ALLOW_RECODE_KFMAXBW;
- sf->adaptive_rd_thresh = 3;
- sf->mode_skip_start = 6;
- sf->intra_y_mode_mask[TX_32X32] = INTRA_DC;
- sf->intra_uv_mode_mask[TX_32X32] = INTRA_DC;
- sf->adaptive_interp_filter_search = 1;
- }
-
- if (speed >= 4) {
- sf->use_square_partition_only = 1;
- sf->tx_size_search_method = USE_LARGESTALL;
- sf->mv.search_method = BIGDIA;
- sf->mv.subpel_search_method = SUBPEL_TREE_PRUNED_MORE;
- sf->adaptive_rd_thresh = 4;
- if (cm->frame_type != KEY_FRAME)
- sf->mode_search_skip_flags |= FLAG_EARLY_TERMINATE;
- sf->disable_filter_search_var_thresh = 200;
- sf->use_lp32x32fdct = 1;
- sf->use_fast_coef_updates = ONE_LOOP_REDUCED;
- sf->use_fast_coef_costing = 1;
- sf->partition_search_breakout_rate_thr = 300;
- }
-
- if (speed >= 5) {
- int i;
- sf->optimize_coefficients = 0;
- sf->mv.search_method = HEX;
- sf->disable_filter_search_var_thresh = 500;
- for (i = 0; i < TX_SIZES; ++i) {
- sf->intra_y_mode_mask[i] = INTRA_DC;
- sf->intra_uv_mode_mask[i] = INTRA_DC;
- }
- sf->partition_search_breakout_rate_thr = 500;
- sf->mv.reduce_first_step_size = 1;
- sf->simple_model_rd_from_var = 1;
- }
-}
-
-static void set_rt_speed_feature_framesize_dependent(VP10_COMP *cpi,
- SPEED_FEATURES *sf, int speed) {
- VP10_COMMON *const cm = &cpi->common;
-
- if (speed >= 1) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->disable_split_mask = cm->show_frame ? DISABLE_ALL_SPLIT
- : DISABLE_ALL_INTER_SPLIT;
- } else {
- sf->disable_split_mask = DISABLE_COMPOUND_SPLIT;
- }
- }
-
- if (speed >= 2) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->disable_split_mask = cm->show_frame ? DISABLE_ALL_SPLIT
- : DISABLE_ALL_INTER_SPLIT;
- } else {
- sf->disable_split_mask = LAST_AND_INTRA_SPLIT_ONLY;
- }
- }
-
- if (speed >= 5) {
- if (VPXMIN(cm->width, cm->height) >= 720) {
- sf->partition_search_breakout_dist_thr = (1 << 25);
- } else {
- sf->partition_search_breakout_dist_thr = (1 << 23);
- }
- }
-
- if (speed >= 7) {
- sf->encode_breakout_thresh = (VPXMIN(cm->width, cm->height) >= 720) ?
- 800 : 300;
- }
-}
-
-static void set_rt_speed_feature(VP10_COMP *cpi, SPEED_FEATURES *sf,
- int speed, vp9e_tune_content content) {
- VP10_COMMON *const cm = &cpi->common;
- const int is_keyframe = cm->frame_type == KEY_FRAME;
- const int frames_since_key = is_keyframe ? 0 : cpi->rc.frames_since_key;
- sf->static_segmentation = 0;
- sf->adaptive_rd_thresh = 1;
- sf->use_fast_coef_costing = 1;
-
- if (speed >= 1) {
- sf->use_square_partition_only = !frame_is_intra_only(cm);
- sf->less_rectangular_check = 1;
- sf->tx_size_search_method = frame_is_intra_only(cm) ? USE_FULL_RD
- : USE_LARGESTALL;
-
- sf->use_rd_breakout = 1;
-
- sf->adaptive_motion_search = 1;
- sf->adaptive_pred_interp_filter = 1;
- sf->mv.auto_mv_step_size = 1;
- sf->adaptive_rd_thresh = 2;
- sf->intra_y_mode_mask[TX_32X32] = INTRA_DC_H_V;
- sf->intra_uv_mode_mask[TX_32X32] = INTRA_DC_H_V;
- sf->intra_uv_mode_mask[TX_16X16] = INTRA_DC_H_V;
- }
-
- if (speed >= 2) {
- sf->mode_search_skip_flags = (cm->frame_type == KEY_FRAME) ? 0 :
- FLAG_SKIP_INTRA_DIRMISMATCH |
- FLAG_SKIP_INTRA_BESTINTER |
- FLAG_SKIP_COMP_BESTINTRA |
- FLAG_SKIP_INTRA_LOWVAR;
- sf->adaptive_pred_interp_filter = 2;
- sf->disable_filter_search_var_thresh = 50;
- sf->comp_inter_joint_search_thresh = BLOCK_SIZES;
- sf->auto_min_max_partition_size = RELAXED_NEIGHBORING_MIN_MAX;
- sf->lf_motion_threshold = LOW_MOTION_THRESHOLD;
- sf->adjust_partitioning_from_last_frame = 1;
- sf->last_partitioning_redo_frequency = 3;
- sf->use_lp32x32fdct = 1;
- sf->mode_skip_start = 11;
- sf->intra_y_mode_mask[TX_16X16] = INTRA_DC_H_V;
- }
-
- if (speed >= 3) {
- sf->use_square_partition_only = 1;
- sf->disable_filter_search_var_thresh = 100;
- sf->use_uv_intra_rd_estimate = 1;
- sf->mv.subpel_iters_per_step = 1;
- sf->adaptive_rd_thresh = 4;
- sf->mode_skip_start = 6;
- sf->allow_skip_recode = 0;
- sf->optimize_coefficients = 0;
- sf->disable_split_mask = DISABLE_ALL_SPLIT;
- sf->lpf_pick = LPF_PICK_FROM_Q;
- }
-
- if (speed >= 4) {
- int i;
- sf->last_partitioning_redo_frequency = 4;
- sf->adaptive_rd_thresh = 5;
- sf->use_fast_coef_costing = 0;
- sf->auto_min_max_partition_size = STRICT_NEIGHBORING_MIN_MAX;
- sf->adjust_partitioning_from_last_frame =
- cm->last_frame_type != cm->frame_type || (0 ==
- (frames_since_key + 1) % sf->last_partitioning_redo_frequency);
- sf->mv.subpel_force_stop = 1;
- for (i = 0; i < TX_SIZES; i++) {
- sf->intra_y_mode_mask[i] = INTRA_DC_H_V;
- sf->intra_uv_mode_mask[i] = INTRA_DC;
- }
- sf->intra_y_mode_mask[TX_32X32] = INTRA_DC;
- sf->frame_parameter_update = 0;
- sf->mv.search_method = FAST_HEX;
-
- sf->inter_mode_mask[BLOCK_32X32] = INTER_NEAREST_NEAR_NEW;
- sf->inter_mode_mask[BLOCK_32X64] = INTER_NEAREST;
- sf->inter_mode_mask[BLOCK_64X32] = INTER_NEAREST;
- sf->inter_mode_mask[BLOCK_64X64] = INTER_NEAREST;
- sf->max_intra_bsize = BLOCK_32X32;
- sf->allow_skip_recode = 1;
- }
-
- if (speed >= 5) {
- sf->use_quant_fp = !is_keyframe;
- sf->auto_min_max_partition_size = is_keyframe ? RELAXED_NEIGHBORING_MIN_MAX
- : STRICT_NEIGHBORING_MIN_MAX;
- sf->default_max_partition_size = BLOCK_32X32;
- sf->default_min_partition_size = BLOCK_8X8;
- sf->force_frame_boost = is_keyframe ||
- (frames_since_key % (sf->last_partitioning_redo_frequency << 1) == 1);
- sf->max_delta_qindex = is_keyframe ? 20 : 15;
- sf->partition_search_type = REFERENCE_PARTITION;
- sf->allow_skip_recode = 0;
- sf->inter_mode_mask[BLOCK_32X32] = INTER_NEAREST_NEW_ZERO;
- sf->inter_mode_mask[BLOCK_32X64] = INTER_NEAREST_NEW_ZERO;
- sf->inter_mode_mask[BLOCK_64X32] = INTER_NEAREST_NEW_ZERO;
- sf->inter_mode_mask[BLOCK_64X64] = INTER_NEAREST_NEW_ZERO;
- sf->adaptive_rd_thresh = 2;
- // This feature is only enabled when partition search is disabled.
- sf->reuse_inter_pred_sby = 1;
- sf->partition_search_breakout_rate_thr = 200;
- sf->coeff_prob_appx_step = 4;
- sf->use_fast_coef_updates = is_keyframe ? TWO_LOOP : ONE_LOOP_REDUCED;
- sf->mode_search_skip_flags = FLAG_SKIP_INTRA_DIRMISMATCH;
- sf->tx_size_search_method = is_keyframe ? USE_LARGESTALL : USE_TX_8X8;
- sf->simple_model_rd_from_var = 1;
-
- if (!is_keyframe) {
- int i;
- if (content == VP9E_CONTENT_SCREEN) {
- for (i = 0; i < BLOCK_SIZES; ++i)
- sf->intra_y_mode_bsize_mask[i] = INTRA_DC_TM_H_V;
- } else {
- for (i = 0; i < BLOCK_SIZES; ++i)
- if (i >= BLOCK_16X16)
- sf->intra_y_mode_bsize_mask[i] = INTRA_DC;
- else
- // Use H and V intra mode for block sizes <= 16X16.
- sf->intra_y_mode_bsize_mask[i] = INTRA_DC_H_V;
- }
- }
- }
-
- if (speed >= 6) {
- // Adaptively switch between SOURCE_VAR_BASED_PARTITION and FIXED_PARTITION.
- sf->partition_search_type = VAR_BASED_PARTITION;
- // Turn on this to use non-RD key frame coding mode.
- sf->mv.search_method = NSTEP;
- sf->mv.reduce_first_step_size = 1;
- }
-
- if (speed >= 7) {
- sf->adaptive_rd_thresh = 3;
- sf->mv.search_method = FAST_DIAMOND;
- sf->mv.fullpel_search_step_param = 10;
- }
- if (speed >= 8) {
- sf->adaptive_rd_thresh = 4;
- sf->mv.subpel_force_stop = 2;
- sf->lpf_pick = LPF_PICK_MINIMAL_LPF;
- }
-}
-
-void vp10_set_speed_features_framesize_dependent(VP10_COMP *cpi) {
- SPEED_FEATURES *const sf = &cpi->sf;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- RD_OPT *const rd = &cpi->rd;
- int i;
-
- if (oxcf->mode == REALTIME) {
- set_rt_speed_feature_framesize_dependent(cpi, sf, oxcf->speed);
- } else if (oxcf->mode == GOOD) {
- set_good_speed_feature_framesize_dependent(cpi, sf, oxcf->speed);
- }
-
- if (sf->disable_split_mask == DISABLE_ALL_SPLIT) {
- sf->adaptive_pred_interp_filter = 0;
- }
-
- if (cpi->encode_breakout && oxcf->mode == REALTIME &&
- sf->encode_breakout_thresh > cpi->encode_breakout) {
- cpi->encode_breakout = sf->encode_breakout_thresh;
- }
-
- // Check for masked out split cases.
- for (i = 0; i < MAX_REFS; ++i) {
- if (sf->disable_split_mask & (1 << i)) {
- rd->thresh_mult_sub8x8[i] = INT_MAX;
- }
- }
-}
-
-void vp10_set_speed_features_framesize_independent(VP10_COMP *cpi) {
- SPEED_FEATURES *const sf = &cpi->sf;
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCK *const x = &cpi->td.mb;
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- int i;
-
- // best quality defaults
- sf->frame_parameter_update = 1;
- sf->mv.search_method = NSTEP;
- sf->recode_loop = ALLOW_RECODE;
- sf->mv.subpel_search_method = SUBPEL_TREE;
- sf->mv.subpel_iters_per_step = 2;
- sf->mv.subpel_force_stop = 0;
- sf->optimize_coefficients = !is_lossless_requested(&cpi->oxcf);
- sf->mv.reduce_first_step_size = 0;
- sf->coeff_prob_appx_step = 1;
- sf->mv.auto_mv_step_size = 0;
- sf->mv.fullpel_search_step_param = 6;
- sf->comp_inter_joint_search_thresh = BLOCK_4X4;
- sf->adaptive_rd_thresh = 0;
- sf->tx_size_search_method = USE_FULL_RD;
- sf->use_lp32x32fdct = 0;
- sf->adaptive_motion_search = 0;
- sf->adaptive_pred_interp_filter = 0;
- sf->adaptive_mode_search = 0;
- sf->cb_pred_filter_search = 0;
- sf->cb_partition_search = 0;
- sf->alt_ref_search_fp = 0;
- sf->use_quant_fp = 0;
- sf->partition_search_type = SEARCH_PARTITION;
- sf->less_rectangular_check = 0;
- sf->use_square_partition_only = 0;
- sf->auto_min_max_partition_size = NOT_IN_USE;
- sf->rd_auto_partition_min_limit = BLOCK_4X4;
- sf->default_max_partition_size = BLOCK_64X64;
- sf->default_min_partition_size = BLOCK_4X4;
- sf->adjust_partitioning_from_last_frame = 0;
- sf->last_partitioning_redo_frequency = 4;
- sf->disable_split_mask = 0;
- sf->mode_search_skip_flags = 0;
- sf->force_frame_boost = 0;
- sf->max_delta_qindex = 0;
- sf->disable_filter_search_var_thresh = 0;
- sf->adaptive_interp_filter_search = 0;
- sf->allow_partition_search_skip = 0;
-
- for (i = 0; i < TX_SIZES; i++) {
- sf->intra_y_mode_mask[i] = INTRA_ALL;
- sf->intra_uv_mode_mask[i] = INTRA_ALL;
- }
- sf->use_rd_breakout = 0;
- sf->use_uv_intra_rd_estimate = 0;
- sf->allow_skip_recode = 0;
- sf->lpf_pick = LPF_PICK_FROM_FULL_IMAGE;
- sf->use_fast_coef_updates = TWO_LOOP;
- sf->use_fast_coef_costing = 0;
- sf->mode_skip_start = MAX_MODES; // Mode index at which mode skip mask set
- sf->schedule_mode_search = 0;
- for (i = 0; i < BLOCK_SIZES; ++i)
- sf->inter_mode_mask[i] = INTER_ALL;
- sf->max_intra_bsize = BLOCK_64X64;
- sf->reuse_inter_pred_sby = 0;
- // This setting only takes effect when partition_search_type is set
- // to FIXED_PARTITION.
- sf->always_this_block_size = BLOCK_16X16;
- sf->search_type_check_frequency = 50;
- sf->encode_breakout_thresh = 0;
- // Recode loop tolerance %.
- sf->recode_tolerance = 25;
- sf->default_interp_filter = SWITCHABLE;
- sf->tx_size_search_breakout = 0;
- sf->partition_search_breakout_dist_thr = 0;
- sf->partition_search_breakout_rate_thr = 0;
- sf->simple_model_rd_from_var = 0;
-
- if (oxcf->mode == REALTIME)
- set_rt_speed_feature(cpi, sf, oxcf->speed, oxcf->content);
- else if (oxcf->mode == GOOD)
- set_good_speed_feature(cpi, cm, sf, oxcf->speed);
-
- cpi->full_search_sad = vp10_full_search_sad;
- cpi->diamond_search_sad = oxcf->mode == BEST ? vp10_full_range_search
- : vp10_diamond_search_sad;
-
- // Slow quant, dct and trellis not worthwhile for first pass
- // so make sure they are always turned off.
- if (oxcf->pass == 1)
- sf->optimize_coefficients = 0;
-
- // No recode for 1 pass.
- if (oxcf->pass == 0) {
- sf->recode_loop = DISALLOW_RECODE;
- sf->optimize_coefficients = 0;
- }
-
- if (sf->mv.subpel_search_method == SUBPEL_TREE) {
- cpi->find_fractional_mv_step = vp10_find_best_sub_pixel_tree;
- } else if (sf->mv.subpel_search_method == SUBPEL_TREE_PRUNED) {
- cpi->find_fractional_mv_step = vp10_find_best_sub_pixel_tree_pruned;
- } else if (sf->mv.subpel_search_method == SUBPEL_TREE_PRUNED_MORE) {
- cpi->find_fractional_mv_step = vp10_find_best_sub_pixel_tree_pruned_more;
- } else if (sf->mv.subpel_search_method == SUBPEL_TREE_PRUNED_EVENMORE) {
- cpi->find_fractional_mv_step = vp10_find_best_sub_pixel_tree_pruned_evenmore;
- }
-
- x->optimize = sf->optimize_coefficients == 1 && oxcf->pass != 1;
-
- x->min_partition_size = sf->default_min_partition_size;
- x->max_partition_size = sf->default_max_partition_size;
-
- if (!cpi->oxcf.frame_periodic_boost) {
- sf->max_delta_qindex = 0;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_SPEED_FEATURES_H_
-#define VP10_ENCODER_SPEED_FEATURES_H_
-
-#include "vp10/common/enums.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-enum {
- INTRA_ALL = (1 << DC_PRED) |
- (1 << V_PRED) | (1 << H_PRED) |
- (1 << D45_PRED) | (1 << D135_PRED) |
- (1 << D117_PRED) | (1 << D153_PRED) |
- (1 << D207_PRED) | (1 << D63_PRED) |
- (1 << TM_PRED),
- INTRA_DC = (1 << DC_PRED),
- INTRA_DC_TM = (1 << DC_PRED) | (1 << TM_PRED),
- INTRA_DC_H_V = (1 << DC_PRED) | (1 << V_PRED) | (1 << H_PRED),
- INTRA_DC_TM_H_V = (1 << DC_PRED) | (1 << TM_PRED) | (1 << V_PRED) |
- (1 << H_PRED)
-};
-
-enum {
- INTER_ALL = (1 << NEARESTMV) | (1 << NEARMV) | (1 << ZEROMV) | (1 << NEWMV),
- INTER_NEAREST = (1 << NEARESTMV),
- INTER_NEAREST_NEW = (1 << NEARESTMV) | (1 << NEWMV),
- INTER_NEAREST_ZERO = (1 << NEARESTMV) | (1 << ZEROMV),
- INTER_NEAREST_NEW_ZERO = (1 << NEARESTMV) | (1 << ZEROMV) | (1 << NEWMV),
- INTER_NEAREST_NEAR_NEW = (1 << NEARESTMV) | (1 << NEARMV) | (1 << NEWMV),
- INTER_NEAREST_NEAR_ZERO = (1 << NEARESTMV) | (1 << NEARMV) | (1 << ZEROMV),
-};
-
-enum {
- DISABLE_ALL_INTER_SPLIT = (1 << THR_COMP_GA) |
- (1 << THR_COMP_LA) |
- (1 << THR_ALTR) |
- (1 << THR_GOLD) |
- (1 << THR_LAST),
-
- DISABLE_ALL_SPLIT = (1 << THR_INTRA) | DISABLE_ALL_INTER_SPLIT,
-
- DISABLE_COMPOUND_SPLIT = (1 << THR_COMP_GA) | (1 << THR_COMP_LA),
-
- LAST_AND_INTRA_SPLIT_ONLY = (1 << THR_COMP_GA) |
- (1 << THR_COMP_LA) |
- (1 << THR_ALTR) |
- (1 << THR_GOLD)
-};
-
-typedef enum {
- DIAMOND = 0,
- NSTEP = 1,
- HEX = 2,
- BIGDIA = 3,
- SQUARE = 4,
- FAST_HEX = 5,
- FAST_DIAMOND = 6
-} SEARCH_METHODS;
-
-typedef enum {
- // No recode.
- DISALLOW_RECODE = 0,
- // Allow recode for KF and exceeding maximum frame bandwidth.
- ALLOW_RECODE_KFMAXBW = 1,
- // Allow recode only for KF/ARF/GF frames.
- ALLOW_RECODE_KFARFGF = 2,
- // Allow recode for all frames based on bitrate constraints.
- ALLOW_RECODE = 3,
-} RECODE_LOOP_TYPE;
-
-typedef enum {
- SUBPEL_TREE = 0,
- SUBPEL_TREE_PRUNED = 1, // Prunes 1/2-pel searches
- SUBPEL_TREE_PRUNED_MORE = 2, // Prunes 1/2-pel searches more aggressively
- SUBPEL_TREE_PRUNED_EVENMORE = 3, // Prunes 1/2- and 1/4-pel searches
- // Other methods to come
-} SUBPEL_SEARCH_METHODS;
-
-typedef enum {
- NO_MOTION_THRESHOLD = 0,
- LOW_MOTION_THRESHOLD = 7
-} MOTION_THRESHOLD;
-
-typedef enum {
- USE_FULL_RD = 0,
- USE_LARGESTALL,
- USE_TX_8X8
-} TX_SIZE_SEARCH_METHOD;
-
-typedef enum {
- NOT_IN_USE = 0,
- RELAXED_NEIGHBORING_MIN_MAX = 1,
- STRICT_NEIGHBORING_MIN_MAX = 2
-} AUTO_MIN_MAX_MODE;
-
-typedef enum {
- // Try the full image with different values.
- LPF_PICK_FROM_FULL_IMAGE,
- // Try a small portion of the image with different values.
- LPF_PICK_FROM_SUBIMAGE,
- // Estimate the level based on quantizer and frame type
- LPF_PICK_FROM_Q,
- // Pick 0 to disable LPF if LPF was enabled last frame
- LPF_PICK_MINIMAL_LPF
-} LPF_PICK_METHOD;
-
-typedef enum {
- // Terminate search early based on distortion so far compared to
- // qp step, distortion in the neighborhood of the frame, etc.
- FLAG_EARLY_TERMINATE = 1 << 0,
-
- // Skips comp inter modes if the best so far is an intra mode.
- FLAG_SKIP_COMP_BESTINTRA = 1 << 1,
-
- // Skips oblique intra modes if the best so far is an inter mode.
- FLAG_SKIP_INTRA_BESTINTER = 1 << 3,
-
- // Skips oblique intra modes at angles 27, 63, 117, 153 if the best
- // intra so far is not one of the neighboring directions.
- FLAG_SKIP_INTRA_DIRMISMATCH = 1 << 4,
-
- // Skips intra modes other than DC_PRED if the source variance is small
- FLAG_SKIP_INTRA_LOWVAR = 1 << 5,
-} MODE_SEARCH_SKIP_LOGIC;
-
-typedef enum {
- FLAG_SKIP_EIGHTTAP = 1 << EIGHTTAP,
- FLAG_SKIP_EIGHTTAP_SMOOTH = 1 << EIGHTTAP_SMOOTH,
- FLAG_SKIP_EIGHTTAP_SHARP = 1 << EIGHTTAP_SHARP,
-} INTERP_FILTER_MASK;
-
-typedef enum {
- // Search partitions using RD criterion
- SEARCH_PARTITION,
-
- // Always use a fixed size partition
- FIXED_PARTITION,
-
- REFERENCE_PARTITION,
-
- // Use an arbitrary partitioning scheme based on source variance within
- // a 64X64 SB
- VAR_BASED_PARTITION,
-
- // Use non-fixed partitions based on source variance
- SOURCE_VAR_BASED_PARTITION
-} PARTITION_SEARCH_TYPE;
-
-typedef enum {
- // Does a dry run to see if any of the contexts need to be updated or not,
- // before the final run.
- TWO_LOOP = 0,
-
- // No dry run, also only half the coef contexts and bands are updated.
- // The rest are not updated at all.
- ONE_LOOP_REDUCED = 1
-} FAST_COEFF_UPDATE;
-
-typedef struct MV_SPEED_FEATURES {
- // Motion search method (Diamond, NSTEP, Hex, Big Diamond, Square, etc).
- SEARCH_METHODS search_method;
-
- // This parameter controls which step in the n-step process we start at.
- // It's changed adaptively based on circumstances.
- int reduce_first_step_size;
-
- // If this is set to 1, we limit the motion search range to 2 times the
- // largest motion vector found in the last frame.
- int auto_mv_step_size;
-
- // Subpel_search_method can only be subpel_tree which does a subpixel
- // logarithmic search that keeps stepping at 1/2 pixel units until
- // you stop getting a gain, and then goes on to 1/4 and repeats
- // the same process. Along the way it skips many diagonals.
- SUBPEL_SEARCH_METHODS subpel_search_method;
-
- // Maximum number of steps in logarithmic subpel search before giving up.
- int subpel_iters_per_step;
-
- // Control when to stop subpel search
- int subpel_force_stop;
-
- // This variable sets the step_param used in full pel motion search.
- int fullpel_search_step_param;
-} MV_SPEED_FEATURES;
-
-typedef struct SPEED_FEATURES {
- MV_SPEED_FEATURES mv;
-
- // Frame level coding parameter update
- int frame_parameter_update;
-
- RECODE_LOOP_TYPE recode_loop;
-
- // Trellis (dynamic programming) optimization of quantized values (+1, 0).
- int optimize_coefficients;
-
- // Always set to 0. If on it enables 0 cost background transmission
- // (except for the initial transmission of the segmentation). The feature is
- // disabled because the addition of very large block sizes make the
- // backgrounds very to cheap to encode, and the segmentation we have
- // adds overhead.
- int static_segmentation;
-
- // If 1 we iterate finding a best reference for 2 ref frames together - via
- // a log search that iterates 4 times (check around mv for last for best
- // error of combined predictor then check around mv for alt). If 0 we
- // we just use the best motion vector found for each frame by itself.
- BLOCK_SIZE comp_inter_joint_search_thresh;
-
- // This variable is used to cap the maximum number of times we skip testing a
- // mode to be evaluated. A high value means we will be faster.
- int adaptive_rd_thresh;
-
- // Speed feature to allow or disallow skipping of recode at block
- // level within a frame.
- int allow_skip_recode;
-
- // Coefficient probability model approximation step size
- int coeff_prob_appx_step;
-
- // The threshold is to determine how slow the motino is, it is used when
- // use_lastframe_partitioning is set to LAST_FRAME_PARTITION_LOW_MOTION
- MOTION_THRESHOLD lf_motion_threshold;
-
- // Determine which method we use to determine transform size. We can choose
- // between options like full rd, largest for prediction size, largest
- // for intra and model coefs for the rest.
- TX_SIZE_SEARCH_METHOD tx_size_search_method;
-
- // Low precision 32x32 fdct keeps everything in 16 bits and thus is less
- // precise but significantly faster than the non lp version.
- int use_lp32x32fdct;
-
- // After looking at the first set of modes (set by index here), skip
- // checking modes for reference frames that don't match the reference frame
- // of the best so far.
- int mode_skip_start;
-
- PARTITION_SEARCH_TYPE partition_search_type;
-
- // Used if partition_search_type = FIXED_SIZE_PARTITION
- BLOCK_SIZE always_this_block_size;
-
- // Skip rectangular partition test when partition type none gives better
- // rd than partition type split.
- int less_rectangular_check;
-
- // Disable testing non square partitions. (eg 16x32)
- int use_square_partition_only;
-
- // Sets min and max partition sizes for this 64x64 region based on the
- // same 64x64 in last encoded frame, and the left and above neighbor.
- AUTO_MIN_MAX_MODE auto_min_max_partition_size;
- // Ensures the rd based auto partition search will always
- // go down at least to the specified level.
- BLOCK_SIZE rd_auto_partition_min_limit;
-
- // Min and max partition size we enable (block_size) as per auto
- // min max, but also used by adjust partitioning, and pick_partitioning.
- BLOCK_SIZE default_min_partition_size;
- BLOCK_SIZE default_max_partition_size;
-
- // Whether or not we allow partitions one smaller or one greater than the last
- // frame's partitioning. Only used if use_lastframe_partitioning is set.
- int adjust_partitioning_from_last_frame;
-
- // How frequently we re do the partitioning from scratch. Only used if
- // use_lastframe_partitioning is set.
- int last_partitioning_redo_frequency;
-
- // Disables sub 8x8 blocksizes in different scenarios: Choices are to disable
- // it always, to allow it for only Last frame and Intra, disable it for all
- // inter modes or to enable it always.
- int disable_split_mask;
-
- // TODO(jingning): combine the related motion search speed features
- // This allows us to use motion search at other sizes as a starting
- // point for this motion search and limits the search range around it.
- int adaptive_motion_search;
-
- int schedule_mode_search;
-
- // Allows sub 8x8 modes to use the prediction filter that was determined
- // best for 8x8 mode. If set to 0 we always re check all the filters for
- // sizes less than 8x8, 1 means we check all filter modes if no 8x8 filter
- // was selected, and 2 means we use 8 tap if no 8x8 filter mode was selected.
- int adaptive_pred_interp_filter;
-
- // Adaptive prediction mode search
- int adaptive_mode_search;
-
- // Chessboard pattern prediction filter type search
- int cb_pred_filter_search;
-
- int cb_partition_search;
-
- int alt_ref_search_fp;
-
- // Fast quantization process path
- int use_quant_fp;
-
- // Use finer quantizer in every other few frames that run variable block
- // partition type search.
- int force_frame_boost;
-
- // Maximally allowed base quantization index fluctuation.
- int max_delta_qindex;
-
- // Implements various heuristics to skip searching modes
- // The heuristics selected are based on flags
- // defined in the MODE_SEARCH_SKIP_HEURISTICS enum
- unsigned int mode_search_skip_flags;
-
- // A source variance threshold below which filter search is disabled
- // Choose a very large value (UINT_MAX) to use 8-tap always
- unsigned int disable_filter_search_var_thresh;
-
- // These bit masks allow you to enable or disable intra modes for each
- // transform size separately.
- int intra_y_mode_mask[TX_SIZES];
- int intra_uv_mode_mask[TX_SIZES];
-
- // These bit masks allow you to enable or disable intra modes for each
- // prediction block size separately.
- int intra_y_mode_bsize_mask[BLOCK_SIZES];
-
- // This variable enables an early break out of mode testing if the model for
- // rd built from the prediction signal indicates a value that's much
- // higher than the best rd we've seen so far.
- int use_rd_breakout;
-
- // This enables us to use an estimate for intra rd based on dc mode rather
- // than choosing an actual uv mode in the stage of encoding before the actual
- // final encode.
- int use_uv_intra_rd_estimate;
-
- // This feature controls how the loop filter level is determined.
- LPF_PICK_METHOD lpf_pick;
-
- // This feature limits the number of coefficients updates we actually do
- // by only looking at counts from 1/2 the bands.
- FAST_COEFF_UPDATE use_fast_coef_updates;
-
- // A binary mask indicating if NEARESTMV, NEARMV, ZEROMV, NEWMV
- // modes are used in order from LSB to MSB for each BLOCK_SIZE.
- int inter_mode_mask[BLOCK_SIZES];
-
- // This feature controls whether we do the expensive context update and
- // calculation in the rd coefficient costing loop.
- int use_fast_coef_costing;
-
- // This feature controls the tolerence vs target used in deciding whether to
- // recode a frame. It has no meaning if recode is disabled.
- int recode_tolerance;
-
- // This variable controls the maximum block size where intra blocks can be
- // used in inter frames.
- // TODO(aconverse): Fold this into one of the other many mode skips
- BLOCK_SIZE max_intra_bsize;
-
- // The frequency that we check if SOURCE_VAR_BASED_PARTITION or
- // FIXED_PARTITION search type should be used.
- int search_type_check_frequency;
-
- // When partition is pre-set, the inter prediction result from pick_inter_mode
- // can be reused in final block encoding process. It is enabled only for real-
- // time mode speed 6.
- int reuse_inter_pred_sby;
-
- // This variable sets the encode_breakout threshold. Currently, it is only
- // enabled in real time mode.
- int encode_breakout_thresh;
-
- // default interp filter choice
- INTERP_FILTER default_interp_filter;
-
- // Early termination in transform size search, which only applies while
- // tx_size_search_method is USE_FULL_RD.
- int tx_size_search_breakout;
-
- // adaptive interp_filter search to allow skip of certain filter types.
- int adaptive_interp_filter_search;
-
- // mask for skip evaluation of certain interp_filter type.
- INTERP_FILTER_MASK interp_filter_search_mask;
-
- // Partition search early breakout thresholds.
- int64_t partition_search_breakout_dist_thr;
- int partition_search_breakout_rate_thr;
-
- // Allow skipping partition search for still image frame
- int allow_partition_search_skip;
-
- // Fast approximation of vp10_model_rd_from_var_lapndz
- int simple_model_rd_from_var;
-} SPEED_FEATURES;
-
-struct VP10_COMP;
-
-void vp10_set_speed_features_framesize_independent(struct VP10_COMP *cpi);
-void vp10_set_speed_features_framesize_dependent(struct VP10_COMP *cpi);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_SPEED_FEATURES_H_
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#include "vpx_dsp/bitwriter.h"
-
-#include "vp10/common/common.h"
-#include "vp10/common/entropy.h"
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/subexp.h"
-
-#define vp10_cost_upd256 ((int)(vp10_cost_one(upd) - vp10_cost_zero(upd)))
-
-static const int update_bits[255] = {
- 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
- 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
- 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
- 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
- 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
- 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
- 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
- 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 0,
-};
-
-static int recenter_nonneg(int v, int m) {
- if (v > (m << 1))
- return v;
- else if (v >= m)
- return ((v - m) << 1);
- else
- return ((m - v) << 1) - 1;
-}
-
-static int remap_prob(int v, int m) {
- int i;
- static const int map_table[MAX_PROB - 1] = {
- // generated by:
- // map_table[j] = split_index(j, MAX_PROB - 1, MODULUS_PARAM);
- 20, 21, 22, 23, 24, 25, 0, 26, 27, 28, 29, 30, 31, 32, 33,
- 34, 35, 36, 37, 1, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 2, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61,
- 3, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 4, 74,
- 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 5, 86, 87, 88,
- 89, 90, 91, 92, 93, 94, 95, 96, 97, 6, 98, 99, 100, 101, 102,
- 103, 104, 105, 106, 107, 108, 109, 7, 110, 111, 112, 113, 114, 115, 116,
- 117, 118, 119, 120, 121, 8, 122, 123, 124, 125, 126, 127, 128, 129, 130,
- 131, 132, 133, 9, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144,
- 145, 10, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 11,
- 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 12, 170, 171,
- 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 13, 182, 183, 184, 185,
- 186, 187, 188, 189, 190, 191, 192, 193, 14, 194, 195, 196, 197, 198, 199,
- 200, 201, 202, 203, 204, 205, 15, 206, 207, 208, 209, 210, 211, 212, 213,
- 214, 215, 216, 217, 16, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227,
- 228, 229, 17, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241,
- 18, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 19,
- };
- v--;
- m--;
- if ((m << 1) <= MAX_PROB)
- i = recenter_nonneg(v, m) - 1;
- else
- i = recenter_nonneg(MAX_PROB - 1 - v, MAX_PROB - 1 - m) - 1;
-
- i = map_table[i];
- return i;
-}
-
-static int prob_diff_update_cost(vpx_prob newp, vpx_prob oldp) {
- int delp = remap_prob(newp, oldp);
- return update_bits[delp] * 256;
-}
-
-static void encode_uniform(vpx_writer *w, int v) {
- const int l = 8;
- const int m = (1 << l) - 191;
- if (v < m) {
- vpx_write_literal(w, v, l - 1);
- } else {
- vpx_write_literal(w, m + ((v - m) >> 1), l - 1);
- vpx_write_literal(w, (v - m) & 1, 1);
- }
-}
-
-static INLINE int write_bit_gte(vpx_writer *w, int word, int test) {
- vpx_write_literal(w, word >= test, 1);
- return word >= test;
-}
-
-static void encode_term_subexp(vpx_writer *w, int word) {
- if (!write_bit_gte(w, word, 16)) {
- vpx_write_literal(w, word, 4);
- } else if (!write_bit_gte(w, word, 32)) {
- vpx_write_literal(w, word - 16, 4);
- } else if (!write_bit_gte(w, word, 64)) {
- vpx_write_literal(w, word - 32, 5);
- } else {
- encode_uniform(w, word - 64);
- }
-}
-
-void vp10_write_prob_diff_update(vpx_writer *w, vpx_prob newp, vpx_prob oldp) {
- const int delp = remap_prob(newp, oldp);
- encode_term_subexp(w, delp);
-}
-
-int vp10_prob_diff_update_savings_search(const unsigned int *ct,
- vpx_prob oldp, vpx_prob *bestp,
- vpx_prob upd) {
- const int old_b = cost_branch256(ct, oldp);
- int bestsavings = 0;
- vpx_prob newp, bestnewp = oldp;
- const int step = *bestp > oldp ? -1 : 1;
-
- for (newp = *bestp; newp != oldp; newp += step) {
- const int new_b = cost_branch256(ct, newp);
- const int update_b = prob_diff_update_cost(newp, oldp) + vp10_cost_upd256;
- const int savings = old_b - new_b - update_b;
- if (savings > bestsavings) {
- bestsavings = savings;
- bestnewp = newp;
- }
- }
- *bestp = bestnewp;
- return bestsavings;
-}
-
-int vp10_prob_diff_update_savings_search_model(const unsigned int *ct,
- const vpx_prob *oldp,
- vpx_prob *bestp,
- vpx_prob upd,
- int stepsize) {
- int i, old_b, new_b, update_b, savings, bestsavings, step;
- int newp;
- vpx_prob bestnewp, newplist[ENTROPY_NODES], oldplist[ENTROPY_NODES];
- vp10_model_to_full_probs(oldp, oldplist);
- memcpy(newplist, oldp, sizeof(vpx_prob) * UNCONSTRAINED_NODES);
- for (i = UNCONSTRAINED_NODES, old_b = 0; i < ENTROPY_NODES; ++i)
- old_b += cost_branch256(ct + 2 * i, oldplist[i]);
- old_b += cost_branch256(ct + 2 * PIVOT_NODE, oldplist[PIVOT_NODE]);
-
- bestsavings = 0;
- bestnewp = oldp[PIVOT_NODE];
-
- if (*bestp > oldp[PIVOT_NODE]) {
- step = -stepsize;
- for (newp = *bestp; newp > oldp[PIVOT_NODE]; newp += step) {
- if (newp < 1 || newp > 255)
- continue;
- newplist[PIVOT_NODE] = newp;
- vp10_model_to_full_probs(newplist, newplist);
- for (i = UNCONSTRAINED_NODES, new_b = 0; i < ENTROPY_NODES; ++i)
- new_b += cost_branch256(ct + 2 * i, newplist[i]);
- new_b += cost_branch256(ct + 2 * PIVOT_NODE, newplist[PIVOT_NODE]);
- update_b = prob_diff_update_cost(newp, oldp[PIVOT_NODE]) +
- vp10_cost_upd256;
- savings = old_b - new_b - update_b;
- if (savings > bestsavings) {
- bestsavings = savings;
- bestnewp = newp;
- }
- }
- } else {
- step = stepsize;
- for (newp = *bestp; newp < oldp[PIVOT_NODE]; newp += step) {
- if (newp < 1 || newp > 255)
- continue;
- newplist[PIVOT_NODE] = newp;
- vp10_model_to_full_probs(newplist, newplist);
- for (i = UNCONSTRAINED_NODES, new_b = 0; i < ENTROPY_NODES; ++i)
- new_b += cost_branch256(ct + 2 * i, newplist[i]);
- new_b += cost_branch256(ct + 2 * PIVOT_NODE, newplist[PIVOT_NODE]);
- update_b = prob_diff_update_cost(newp, oldp[PIVOT_NODE]) +
- vp10_cost_upd256;
- savings = old_b - new_b - update_b;
- if (savings > bestsavings) {
- bestsavings = savings;
- bestnewp = newp;
- }
- }
- }
-
- *bestp = bestnewp;
- return bestsavings;
-}
-
-void vp10_cond_prob_diff_update(vpx_writer *w, vpx_prob *oldp,
- const unsigned int ct[2]) {
- const vpx_prob upd = DIFF_UPDATE_PROB;
- vpx_prob newp = get_binary_prob(ct[0], ct[1]);
- const int savings = vp10_prob_diff_update_savings_search(ct, *oldp, &newp,
- upd);
- assert(newp >= 1);
- if (savings > 0) {
- vpx_write(w, 1, upd);
- vp10_write_prob_diff_update(w, newp, *oldp);
- *oldp = newp;
- } else {
- vpx_write(w, 0, upd);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-
-#ifndef VP10_ENCODER_SUBEXP_H_
-#define VP10_ENCODER_SUBEXP_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#include "vpx_dsp/prob.h"
-
-struct vpx_writer;
-
-void vp10_write_prob_diff_update(struct vpx_writer *w,
- vpx_prob newp, vpx_prob oldp);
-
-void vp10_cond_prob_diff_update(struct vpx_writer *w, vpx_prob *oldp,
- const unsigned int ct[2]);
-
-int vp10_prob_diff_update_savings_search(const unsigned int *ct,
- vpx_prob oldp, vpx_prob *bestp,
- vpx_prob upd);
-
-
-int vp10_prob_diff_update_savings_search_model(const unsigned int *ct,
- const vpx_prob *oldp,
- vpx_prob *bestp,
- vpx_prob upd,
- int stepsize);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_SUBEXP_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <math.h>
-#include <limits.h>
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/onyxc_int.h"
-#include "vp10/common/quant_common.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/encoder/extend.h"
-#include "vp10/encoder/firstpass.h"
-#include "vp10/encoder/mcomp.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/quantize.h"
-#include "vp10/encoder/ratectrl.h"
-#include "vp10/encoder/segmentation.h"
-#include "vp10/encoder/temporal_filter.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_mem/vpx_mem.h"
-#include "vpx_ports/mem.h"
-#include "vpx_ports/vpx_timer.h"
-#include "vpx_scale/vpx_scale.h"
-
-static int fixed_divide[512];
-
-static void temporal_filter_predictors_mb_c(MACROBLOCKD *xd,
- uint8_t *y_mb_ptr,
- uint8_t *u_mb_ptr,
- uint8_t *v_mb_ptr,
- int stride,
- int uv_block_width,
- int uv_block_height,
- int mv_row,
- int mv_col,
- uint8_t *pred,
- struct scale_factors *scale,
- int x, int y) {
- const int which_mv = 0;
- const MV mv = { mv_row, mv_col };
- const InterpKernel *const kernel =
- vp10_filter_kernels[xd->mi[0]->mbmi.interp_filter];
-
- enum mv_precision mv_precision_uv;
- int uv_stride;
- if (uv_block_width == 8) {
- uv_stride = (stride + 1) >> 1;
- mv_precision_uv = MV_PRECISION_Q4;
- } else {
- uv_stride = stride;
- mv_precision_uv = MV_PRECISION_Q3;
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (xd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- vp10_highbd_build_inter_predictor(y_mb_ptr, stride,
- &pred[0], 16,
- &mv,
- scale,
- 16, 16,
- which_mv,
- kernel, MV_PRECISION_Q3, x, y, xd->bd);
-
- vp10_highbd_build_inter_predictor(u_mb_ptr, uv_stride,
- &pred[256], uv_block_width,
- &mv,
- scale,
- uv_block_width, uv_block_height,
- which_mv,
- kernel, mv_precision_uv, x, y, xd->bd);
-
- vp10_highbd_build_inter_predictor(v_mb_ptr, uv_stride,
- &pred[512], uv_block_width,
- &mv,
- scale,
- uv_block_width, uv_block_height,
- which_mv,
- kernel, mv_precision_uv, x, y, xd->bd);
- return;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- vp10_build_inter_predictor(y_mb_ptr, stride,
- &pred[0], 16,
- &mv,
- scale,
- 16, 16,
- which_mv,
- kernel, MV_PRECISION_Q3, x, y);
-
- vp10_build_inter_predictor(u_mb_ptr, uv_stride,
- &pred[256], uv_block_width,
- &mv,
- scale,
- uv_block_width, uv_block_height,
- which_mv,
- kernel, mv_precision_uv, x, y);
-
- vp10_build_inter_predictor(v_mb_ptr, uv_stride,
- &pred[512], uv_block_width,
- &mv,
- scale,
- uv_block_width, uv_block_height,
- which_mv,
- kernel, mv_precision_uv, x, y);
-}
-
-void vp10_temporal_filter_init(void) {
- int i;
-
- fixed_divide[0] = 0;
- for (i = 1; i < 512; ++i)
- fixed_divide[i] = 0x80000 / i;
-}
-
-void vp10_temporal_filter_apply_c(uint8_t *frame1,
- unsigned int stride,
- uint8_t *frame2,
- unsigned int block_width,
- unsigned int block_height,
- int strength,
- int filter_weight,
- unsigned int *accumulator,
- uint16_t *count) {
- unsigned int i, j, k;
- int modifier;
- int byte = 0;
- const int rounding = strength > 0 ? 1 << (strength - 1) : 0;
-
- for (i = 0, k = 0; i < block_height; i++) {
- for (j = 0; j < block_width; j++, k++) {
- int src_byte = frame1[byte];
- int pixel_value = *frame2++;
-
- modifier = src_byte - pixel_value;
- // This is an integer approximation of:
- // float coeff = (3.0 * modifer * modifier) / pow(2, strength);
- // modifier = (int)roundf(coeff > 16 ? 0 : 16-coeff);
- modifier *= modifier;
- modifier *= 3;
- modifier += rounding;
- modifier >>= strength;
-
- if (modifier > 16)
- modifier = 16;
-
- modifier = 16 - modifier;
- modifier *= filter_weight;
-
- count[k] += modifier;
- accumulator[k] += modifier * pixel_value;
-
- byte++;
- }
-
- byte += stride - block_width;
- }
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-void vp10_highbd_temporal_filter_apply_c(uint8_t *frame1_8,
- unsigned int stride,
- uint8_t *frame2_8,
- unsigned int block_width,
- unsigned int block_height,
- int strength,
- int filter_weight,
- unsigned int *accumulator,
- uint16_t *count) {
- uint16_t *frame1 = CONVERT_TO_SHORTPTR(frame1_8);
- uint16_t *frame2 = CONVERT_TO_SHORTPTR(frame2_8);
- unsigned int i, j, k;
- int modifier;
- int byte = 0;
- const int rounding = strength > 0 ? 1 << (strength - 1) : 0;
-
- for (i = 0, k = 0; i < block_height; i++) {
- for (j = 0; j < block_width; j++, k++) {
- int src_byte = frame1[byte];
- int pixel_value = *frame2++;
-
- modifier = src_byte - pixel_value;
- // This is an integer approximation of:
- // float coeff = (3.0 * modifer * modifier) / pow(2, strength);
- // modifier = (int)roundf(coeff > 16 ? 0 : 16-coeff);
- modifier *= modifier;
- modifier *= 3;
- modifier += rounding;
- modifier >>= strength;
-
- if (modifier > 16)
- modifier = 16;
-
- modifier = 16 - modifier;
- modifier *= filter_weight;
-
- count[k] += modifier;
- accumulator[k] += modifier * pixel_value;
-
- byte++;
- }
-
- byte += stride - block_width;
- }
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static int temporal_filter_find_matching_mb_c(VP10_COMP *cpi,
- uint8_t *arf_frame_buf,
- uint8_t *frame_ptr_buf,
- int stride) {
- MACROBLOCK *const x = &cpi->td.mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- const MV_SPEED_FEATURES *const mv_sf = &cpi->sf.mv;
- int step_param;
- int sadpb = x->sadperbit16;
- int bestsme = INT_MAX;
- int distortion;
- unsigned int sse;
- int cost_list[5];
-
- MV best_ref_mv1 = {0, 0};
- MV best_ref_mv1_full; /* full-pixel value of best_ref_mv1 */
- MV *ref_mv = &x->e_mbd.mi[0]->bmi[0].as_mv[0].as_mv;
-
- // Save input state
- struct buf_2d src = x->plane[0].src;
- struct buf_2d pre = xd->plane[0].pre[0];
-
- best_ref_mv1_full.col = best_ref_mv1.col >> 3;
- best_ref_mv1_full.row = best_ref_mv1.row >> 3;
-
- // Setup frame pointers
- x->plane[0].src.buf = arf_frame_buf;
- x->plane[0].src.stride = stride;
- xd->plane[0].pre[0].buf = frame_ptr_buf;
- xd->plane[0].pre[0].stride = stride;
-
- step_param = mv_sf->reduce_first_step_size;
- step_param = VPXMIN(step_param, MAX_MVSEARCH_STEPS - 2);
-
- // Ignore mv costing by sending NULL pointer instead of cost arrays
- vp10_hex_search(x, &best_ref_mv1_full, step_param, sadpb, 1,
- cond_cost_list(cpi, cost_list),
- &cpi->fn_ptr[BLOCK_16X16], 0, &best_ref_mv1, ref_mv);
-
- // Ignore mv costing by sending NULL pointer instead of cost array
- bestsme = cpi->find_fractional_mv_step(x, ref_mv,
- &best_ref_mv1,
- cpi->common.allow_high_precision_mv,
- x->errorperbit,
- &cpi->fn_ptr[BLOCK_16X16],
- 0, mv_sf->subpel_iters_per_step,
- cond_cost_list(cpi, cost_list),
- NULL, NULL,
- &distortion, &sse, NULL, 0, 0);
-
- // Restore input state
- x->plane[0].src = src;
- xd->plane[0].pre[0] = pre;
-
- return bestsme;
-}
-
-static void temporal_filter_iterate_c(VP10_COMP *cpi,
- YV12_BUFFER_CONFIG **frames,
- int frame_count,
- int alt_ref_index,
- int strength,
- struct scale_factors *scale) {
- int byte;
- int frame;
- int mb_col, mb_row;
- unsigned int filter_weight;
- int mb_cols = (frames[alt_ref_index]->y_crop_width + 15) >> 4;
- int mb_rows = (frames[alt_ref_index]->y_crop_height + 15) >> 4;
- int mb_y_offset = 0;
- int mb_uv_offset = 0;
- DECLARE_ALIGNED(16, unsigned int, accumulator[16 * 16 * 3]);
- DECLARE_ALIGNED(16, uint16_t, count[16 * 16 * 3]);
- MACROBLOCKD *mbd = &cpi->td.mb.e_mbd;
- YV12_BUFFER_CONFIG *f = frames[alt_ref_index];
- uint8_t *dst1, *dst2;
-#if CONFIG_VP9_HIGHBITDEPTH
- DECLARE_ALIGNED(16, uint16_t, predictor16[16 * 16 * 3]);
- DECLARE_ALIGNED(16, uint8_t, predictor8[16 * 16 * 3]);
- uint8_t *predictor;
-#else
- DECLARE_ALIGNED(16, uint8_t, predictor[16 * 16 * 3]);
-#endif
- const int mb_uv_height = 16 >> mbd->plane[1].subsampling_y;
- const int mb_uv_width = 16 >> mbd->plane[1].subsampling_x;
-
- // Save input state
- uint8_t* input_buffer[MAX_MB_PLANE];
- int i;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (mbd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- predictor = CONVERT_TO_BYTEPTR(predictor16);
- } else {
- predictor = predictor8;
- }
-#endif
-
- for (i = 0; i < MAX_MB_PLANE; i++)
- input_buffer[i] = mbd->plane[i].pre[0].buf;
-
- for (mb_row = 0; mb_row < mb_rows; mb_row++) {
- // Source frames are extended to 16 pixels. This is different than
- // L/A/G reference frames that have a border of 32 (VP9ENCBORDERINPIXELS)
- // A 6/8 tap filter is used for motion search. This requires 2 pixels
- // before and 3 pixels after. So the largest Y mv on a border would
- // then be 16 - VP9_INTERP_EXTEND. The UV blocks are half the size of the
- // Y and therefore only extended by 8. The largest mv that a UV block
- // can support is 8 - VP9_INTERP_EXTEND. A UV mv is half of a Y mv.
- // (16 - VP9_INTERP_EXTEND) >> 1 which is greater than
- // 8 - VP9_INTERP_EXTEND.
- // To keep the mv in play for both Y and UV planes the max that it
- // can be on a border is therefore 16 - (2*VP9_INTERP_EXTEND+1).
- cpi->td.mb.mv_row_min = -((mb_row * 16) + (17 - 2 * VP9_INTERP_EXTEND));
- cpi->td.mb.mv_row_max = ((mb_rows - 1 - mb_row) * 16)
- + (17 - 2 * VP9_INTERP_EXTEND);
-
- for (mb_col = 0; mb_col < mb_cols; mb_col++) {
- int i, j, k;
- int stride;
-
- memset(accumulator, 0, 16 * 16 * 3 * sizeof(accumulator[0]));
- memset(count, 0, 16 * 16 * 3 * sizeof(count[0]));
-
- cpi->td.mb.mv_col_min = -((mb_col * 16) + (17 - 2 * VP9_INTERP_EXTEND));
- cpi->td.mb.mv_col_max = ((mb_cols - 1 - mb_col) * 16)
- + (17 - 2 * VP9_INTERP_EXTEND);
-
- for (frame = 0; frame < frame_count; frame++) {
- const int thresh_low = 10000;
- const int thresh_high = 20000;
-
- if (frames[frame] == NULL)
- continue;
-
- mbd->mi[0]->bmi[0].as_mv[0].as_mv.row = 0;
- mbd->mi[0]->bmi[0].as_mv[0].as_mv.col = 0;
-
- if (frame == alt_ref_index) {
- filter_weight = 2;
- } else {
- // Find best match in this frame by MC
- int err = temporal_filter_find_matching_mb_c(cpi,
- frames[alt_ref_index]->y_buffer + mb_y_offset,
- frames[frame]->y_buffer + mb_y_offset,
- frames[frame]->y_stride);
-
- // Assign higher weight to matching MB if it's error
- // score is lower. If not applying MC default behavior
- // is to weight all MBs equal.
- filter_weight = err < thresh_low
- ? 2 : err < thresh_high ? 1 : 0;
- }
-
- if (filter_weight != 0) {
- // Construct the predictors
- temporal_filter_predictors_mb_c(mbd,
- frames[frame]->y_buffer + mb_y_offset,
- frames[frame]->u_buffer + mb_uv_offset,
- frames[frame]->v_buffer + mb_uv_offset,
- frames[frame]->y_stride,
- mb_uv_width, mb_uv_height,
- mbd->mi[0]->bmi[0].as_mv[0].as_mv.row,
- mbd->mi[0]->bmi[0].as_mv[0].as_mv.col,
- predictor, scale,
- mb_col * 16, mb_row * 16);
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (mbd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- int adj_strength = strength + 2 * (mbd->bd - 8);
- // Apply the filter (YUV)
- vp10_highbd_temporal_filter_apply(f->y_buffer + mb_y_offset,
- f->y_stride,
- predictor, 16, 16, adj_strength,
- filter_weight,
- accumulator, count);
- vp10_highbd_temporal_filter_apply(f->u_buffer + mb_uv_offset,
- f->uv_stride, predictor + 256,
- mb_uv_width, mb_uv_height,
- adj_strength,
- filter_weight, accumulator + 256,
- count + 256);
- vp10_highbd_temporal_filter_apply(f->v_buffer + mb_uv_offset,
- f->uv_stride, predictor + 512,
- mb_uv_width, mb_uv_height,
- adj_strength, filter_weight,
- accumulator + 512, count + 512);
- } else {
- // Apply the filter (YUV)
- vp10_temporal_filter_apply(f->y_buffer + mb_y_offset, f->y_stride,
- predictor, 16, 16,
- strength, filter_weight,
- accumulator, count);
- vp10_temporal_filter_apply(f->u_buffer + mb_uv_offset, f->uv_stride,
- predictor + 256,
- mb_uv_width, mb_uv_height, strength,
- filter_weight, accumulator + 256,
- count + 256);
- vp10_temporal_filter_apply(f->v_buffer + mb_uv_offset, f->uv_stride,
- predictor + 512,
- mb_uv_width, mb_uv_height, strength,
- filter_weight, accumulator + 512,
- count + 512);
- }
-#else
- // Apply the filter (YUV)
- vp10_temporal_filter_apply(f->y_buffer + mb_y_offset, f->y_stride,
- predictor, 16, 16,
- strength, filter_weight,
- accumulator, count);
- vp10_temporal_filter_apply(f->u_buffer + mb_uv_offset, f->uv_stride,
- predictor + 256,
- mb_uv_width, mb_uv_height, strength,
- filter_weight, accumulator + 256,
- count + 256);
- vp10_temporal_filter_apply(f->v_buffer + mb_uv_offset, f->uv_stride,
- predictor + 512,
- mb_uv_width, mb_uv_height, strength,
- filter_weight, accumulator + 512,
- count + 512);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
- }
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (mbd->cur_buf->flags & YV12_FLAG_HIGHBITDEPTH) {
- uint16_t *dst1_16;
- uint16_t *dst2_16;
- // Normalize filter output to produce AltRef frame
- dst1 = cpi->alt_ref_buffer.y_buffer;
- dst1_16 = CONVERT_TO_SHORTPTR(dst1);
- stride = cpi->alt_ref_buffer.y_stride;
- byte = mb_y_offset;
- for (i = 0, k = 0; i < 16; i++) {
- for (j = 0; j < 16; j++, k++) {
- unsigned int pval = accumulator[k] + (count[k] >> 1);
- pval *= fixed_divide[count[k]];
- pval >>= 19;
-
- dst1_16[byte] = (uint16_t)pval;
-
- // move to next pixel
- byte++;
- }
-
- byte += stride - 16;
- }
-
- dst1 = cpi->alt_ref_buffer.u_buffer;
- dst2 = cpi->alt_ref_buffer.v_buffer;
- dst1_16 = CONVERT_TO_SHORTPTR(dst1);
- dst2_16 = CONVERT_TO_SHORTPTR(dst2);
- stride = cpi->alt_ref_buffer.uv_stride;
- byte = mb_uv_offset;
- for (i = 0, k = 256; i < mb_uv_height; i++) {
- for (j = 0; j < mb_uv_width; j++, k++) {
- int m = k + 256;
-
- // U
- unsigned int pval = accumulator[k] + (count[k] >> 1);
- pval *= fixed_divide[count[k]];
- pval >>= 19;
- dst1_16[byte] = (uint16_t)pval;
-
- // V
- pval = accumulator[m] + (count[m] >> 1);
- pval *= fixed_divide[count[m]];
- pval >>= 19;
- dst2_16[byte] = (uint16_t)pval;
-
- // move to next pixel
- byte++;
- }
-
- byte += stride - mb_uv_width;
- }
- } else {
- // Normalize filter output to produce AltRef frame
- dst1 = cpi->alt_ref_buffer.y_buffer;
- stride = cpi->alt_ref_buffer.y_stride;
- byte = mb_y_offset;
- for (i = 0, k = 0; i < 16; i++) {
- for (j = 0; j < 16; j++, k++) {
- unsigned int pval = accumulator[k] + (count[k] >> 1);
- pval *= fixed_divide[count[k]];
- pval >>= 19;
-
- dst1[byte] = (uint8_t)pval;
-
- // move to next pixel
- byte++;
- }
- byte += stride - 16;
- }
-
- dst1 = cpi->alt_ref_buffer.u_buffer;
- dst2 = cpi->alt_ref_buffer.v_buffer;
- stride = cpi->alt_ref_buffer.uv_stride;
- byte = mb_uv_offset;
- for (i = 0, k = 256; i < mb_uv_height; i++) {
- for (j = 0; j < mb_uv_width; j++, k++) {
- int m = k + 256;
-
- // U
- unsigned int pval = accumulator[k] + (count[k] >> 1);
- pval *= fixed_divide[count[k]];
- pval >>= 19;
- dst1[byte] = (uint8_t)pval;
-
- // V
- pval = accumulator[m] + (count[m] >> 1);
- pval *= fixed_divide[count[m]];
- pval >>= 19;
- dst2[byte] = (uint8_t)pval;
-
- // move to next pixel
- byte++;
- }
- byte += stride - mb_uv_width;
- }
- }
-#else
- // Normalize filter output to produce AltRef frame
- dst1 = cpi->alt_ref_buffer.y_buffer;
- stride = cpi->alt_ref_buffer.y_stride;
- byte = mb_y_offset;
- for (i = 0, k = 0; i < 16; i++) {
- for (j = 0; j < 16; j++, k++) {
- unsigned int pval = accumulator[k] + (count[k] >> 1);
- pval *= fixed_divide[count[k]];
- pval >>= 19;
-
- dst1[byte] = (uint8_t)pval;
-
- // move to next pixel
- byte++;
- }
- byte += stride - 16;
- }
-
- dst1 = cpi->alt_ref_buffer.u_buffer;
- dst2 = cpi->alt_ref_buffer.v_buffer;
- stride = cpi->alt_ref_buffer.uv_stride;
- byte = mb_uv_offset;
- for (i = 0, k = 256; i < mb_uv_height; i++) {
- for (j = 0; j < mb_uv_width; j++, k++) {
- int m = k + 256;
-
- // U
- unsigned int pval = accumulator[k] + (count[k] >> 1);
- pval *= fixed_divide[count[k]];
- pval >>= 19;
- dst1[byte] = (uint8_t)pval;
-
- // V
- pval = accumulator[m] + (count[m] >> 1);
- pval *= fixed_divide[count[m]];
- pval >>= 19;
- dst2[byte] = (uint8_t)pval;
-
- // move to next pixel
- byte++;
- }
- byte += stride - mb_uv_width;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- mb_y_offset += 16;
- mb_uv_offset += mb_uv_width;
- }
- mb_y_offset += 16 * (f->y_stride - mb_cols);
- mb_uv_offset += mb_uv_height * f->uv_stride - mb_uv_width * mb_cols;
- }
-
- // Restore input state
- for (i = 0; i < MAX_MB_PLANE; i++)
- mbd->plane[i].pre[0].buf = input_buffer[i];
-}
-
-// Apply buffer limits and context specific adjustments to arnr filter.
-static void adjust_arnr_filter(VP10_COMP *cpi,
- int distance, int group_boost,
- int *arnr_frames, int *arnr_strength) {
- const VP10EncoderConfig *const oxcf = &cpi->oxcf;
- const int frames_after_arf =
- vp10_lookahead_depth(cpi->lookahead) - distance - 1;
- int frames_fwd = (cpi->oxcf.arnr_max_frames - 1) >> 1;
- int frames_bwd;
- int q, frames, strength;
-
- // Define the forward and backwards filter limits for this arnr group.
- if (frames_fwd > frames_after_arf)
- frames_fwd = frames_after_arf;
- if (frames_fwd > distance)
- frames_fwd = distance;
-
- frames_bwd = frames_fwd;
-
- // For even length filter there is one more frame backward
- // than forward: e.g. len=6 ==> bbbAff, len=7 ==> bbbAfff.
- if (frames_bwd < distance)
- frames_bwd += (oxcf->arnr_max_frames + 1) & 0x1;
-
- // Set the baseline active filter size.
- frames = frames_bwd + 1 + frames_fwd;
-
- // Adjust the strength based on active max q.
- if (cpi->common.current_video_frame > 1)
- q = ((int)vp10_convert_qindex_to_q(
- cpi->rc.avg_frame_qindex[INTER_FRAME], cpi->common.bit_depth));
- else
- q = ((int)vp10_convert_qindex_to_q(
- cpi->rc.avg_frame_qindex[KEY_FRAME], cpi->common.bit_depth));
- if (q > 16) {
- strength = oxcf->arnr_strength;
- } else {
- strength = oxcf->arnr_strength - ((16 - q) / 2);
- if (strength < 0)
- strength = 0;
- }
-
- // Adjust number of frames in filter and strength based on gf boost level.
- if (frames > group_boost / 150) {
- frames = group_boost / 150;
- frames += !(frames & 1);
- }
-
- if (strength > group_boost / 300) {
- strength = group_boost / 300;
- }
-
- // Adjustments for second level arf in multi arf case.
- if (cpi->oxcf.pass == 2 && cpi->multi_arf_allowed) {
- const GF_GROUP *const gf_group = &cpi->twopass.gf_group;
- if (gf_group->rf_level[gf_group->index] != GF_ARF_STD) {
- strength >>= 1;
- }
- }
-
- *arnr_frames = frames;
- *arnr_strength = strength;
-}
-
-void vp10_temporal_filter(VP10_COMP *cpi, int distance) {
- RATE_CONTROL *const rc = &cpi->rc;
- int frame;
- int frames_to_blur;
- int start_frame;
- int strength;
- int frames_to_blur_backward;
- int frames_to_blur_forward;
- struct scale_factors sf;
- YV12_BUFFER_CONFIG *frames[MAX_LAG_BUFFERS] = {NULL};
-
- // Apply context specific adjustments to the arnr filter parameters.
- adjust_arnr_filter(cpi, distance, rc->gfu_boost, &frames_to_blur, &strength);
- frames_to_blur_backward = (frames_to_blur / 2);
- frames_to_blur_forward = ((frames_to_blur - 1) / 2);
- start_frame = distance + frames_to_blur_forward;
-
- // Setup frame pointers, NULL indicates frame not included in filter.
- for (frame = 0; frame < frames_to_blur; ++frame) {
- const int which_buffer = start_frame - frame;
- struct lookahead_entry *buf = vp10_lookahead_peek(cpi->lookahead,
- which_buffer);
- frames[frames_to_blur - 1 - frame] = &buf->img;
- }
-
- if (frames_to_blur > 0) {
- // Setup scaling factors. Scaling on each of the arnr frames is not
- // supported.
- // ARF is produced at the native frame size and resized when coded.
-#if CONFIG_VP9_HIGHBITDEPTH
- vp10_setup_scale_factors_for_frame(&sf,
- frames[0]->y_crop_width,
- frames[0]->y_crop_height,
- frames[0]->y_crop_width,
- frames[0]->y_crop_height,
- cpi->common.use_highbitdepth);
-#else
- vp10_setup_scale_factors_for_frame(&sf,
- frames[0]->y_crop_width,
- frames[0]->y_crop_height,
- frames[0]->y_crop_width,
- frames[0]->y_crop_height);
-#endif // CONFIG_VP9_HIGHBITDEPTH
- }
-
- temporal_filter_iterate_c(cpi, frames, frames_to_blur,
- frames_to_blur_backward, strength, &sf);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_TEMPORAL_FILTER_H_
-#define VP10_ENCODER_TEMPORAL_FILTER_H_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_temporal_filter_init(void);
-void vp10_temporal_filter(VP10_COMP *cpi, int distance);
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_TEMPORAL_FILTER_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <math.h>
-#include <stdio.h>
-#include <string.h>
-
-#include "vpx_mem/vpx_mem.h"
-
-#include "vp10/common/entropy.h"
-#include "vp10/common/pred_common.h"
-#include "vp10/common/scan.h"
-#include "vp10/common/seg_common.h"
-
-#include "vp10/encoder/cost.h"
-#include "vp10/encoder/encoder.h"
-#include "vp10/encoder/tokenize.h"
-
-static const TOKENVALUE dct_cat_lt_10_value_tokens[] = {
- {9, 63}, {9, 61}, {9, 59}, {9, 57}, {9, 55}, {9, 53}, {9, 51}, {9, 49},
- {9, 47}, {9, 45}, {9, 43}, {9, 41}, {9, 39}, {9, 37}, {9, 35}, {9, 33},
- {9, 31}, {9, 29}, {9, 27}, {9, 25}, {9, 23}, {9, 21}, {9, 19}, {9, 17},
- {9, 15}, {9, 13}, {9, 11}, {9, 9}, {9, 7}, {9, 5}, {9, 3}, {9, 1},
- {8, 31}, {8, 29}, {8, 27}, {8, 25}, {8, 23}, {8, 21},
- {8, 19}, {8, 17}, {8, 15}, {8, 13}, {8, 11}, {8, 9},
- {8, 7}, {8, 5}, {8, 3}, {8, 1},
- {7, 15}, {7, 13}, {7, 11}, {7, 9}, {7, 7}, {7, 5}, {7, 3}, {7, 1},
- {6, 7}, {6, 5}, {6, 3}, {6, 1}, {5, 3}, {5, 1},
- {4, 1}, {3, 1}, {2, 1}, {1, 1}, {0, 0},
- {1, 0}, {2, 0}, {3, 0}, {4, 0},
- {5, 0}, {5, 2}, {6, 0}, {6, 2}, {6, 4}, {6, 6},
- {7, 0}, {7, 2}, {7, 4}, {7, 6}, {7, 8}, {7, 10}, {7, 12}, {7, 14},
- {8, 0}, {8, 2}, {8, 4}, {8, 6}, {8, 8}, {8, 10}, {8, 12},
- {8, 14}, {8, 16}, {8, 18}, {8, 20}, {8, 22}, {8, 24},
- {8, 26}, {8, 28}, {8, 30}, {9, 0}, {9, 2},
- {9, 4}, {9, 6}, {9, 8}, {9, 10}, {9, 12}, {9, 14}, {9, 16},
- {9, 18}, {9, 20}, {9, 22}, {9, 24}, {9, 26}, {9, 28},
- {9, 30}, {9, 32}, {9, 34}, {9, 36}, {9, 38}, {9, 40},
- {9, 42}, {9, 44}, {9, 46}, {9, 48}, {9, 50}, {9, 52},
- {9, 54}, {9, 56}, {9, 58}, {9, 60}, {9, 62}
-};
-const TOKENVALUE *vp10_dct_cat_lt_10_value_tokens = dct_cat_lt_10_value_tokens +
- (sizeof(dct_cat_lt_10_value_tokens) / sizeof(*dct_cat_lt_10_value_tokens))
- / 2;
-
-// Array indices are identical to previously-existing CONTEXT_NODE indices
-const vpx_tree_index vp10_coef_tree[TREE_SIZE(ENTROPY_TOKENS)] = {
- -EOB_TOKEN, 2, // 0 = EOB
- -ZERO_TOKEN, 4, // 1 = ZERO
- -ONE_TOKEN, 6, // 2 = ONE
- 8, 12, // 3 = LOW_VAL
- -TWO_TOKEN, 10, // 4 = TWO
- -THREE_TOKEN, -FOUR_TOKEN, // 5 = THREE
- 14, 16, // 6 = HIGH_LOW
- -CATEGORY1_TOKEN, -CATEGORY2_TOKEN, // 7 = CAT_ONE
- 18, 20, // 8 = CAT_THREEFOUR
- -CATEGORY3_TOKEN, -CATEGORY4_TOKEN, // 9 = CAT_THREE
- -CATEGORY5_TOKEN, -CATEGORY6_TOKEN // 10 = CAT_FIVE
-};
-
-static const vpx_tree_index cat1[2] = {0, 0};
-static const vpx_tree_index cat2[4] = {2, 2, 0, 0};
-static const vpx_tree_index cat3[6] = {2, 2, 4, 4, 0, 0};
-static const vpx_tree_index cat4[8] = {2, 2, 4, 4, 6, 6, 0, 0};
-static const vpx_tree_index cat5[10] = {2, 2, 4, 4, 6, 6, 8, 8, 0, 0};
-static const vpx_tree_index cat6[28] = {2, 2, 4, 4, 6, 6, 8, 8, 10, 10, 12, 12,
- 14, 14, 16, 16, 18, 18, 20, 20, 22, 22, 24, 24, 26, 26, 0, 0};
-
-static const int16_t zero_cost[] = {0};
-static const int16_t one_cost[] = {255, 257};
-static const int16_t two_cost[] = {255, 257};
-static const int16_t three_cost[] = {255, 257};
-static const int16_t four_cost[] = {255, 257};
-static const int16_t cat1_cost[] = {429, 431, 616, 618};
-static const int16_t cat2_cost[] = {624, 626, 727, 729, 848, 850, 951, 953};
-static const int16_t cat3_cost[] = {
- 820, 822, 893, 895, 940, 942, 1013, 1015, 1096, 1098, 1169, 1171, 1216, 1218,
- 1289, 1291
-};
-static const int16_t cat4_cost[] = {
- 1032, 1034, 1075, 1077, 1105, 1107, 1148, 1150, 1194, 1196, 1237, 1239,
- 1267, 1269, 1310, 1312, 1328, 1330, 1371, 1373, 1401, 1403, 1444, 1446,
- 1490, 1492, 1533, 1535, 1563, 1565, 1606, 1608
-};
-static const int16_t cat5_cost[] = {
- 1269, 1271, 1283, 1285, 1306, 1308, 1320,
- 1322, 1347, 1349, 1361, 1363, 1384, 1386, 1398, 1400, 1443, 1445, 1457,
- 1459, 1480, 1482, 1494, 1496, 1521, 1523, 1535, 1537, 1558, 1560, 1572,
- 1574, 1592, 1594, 1606, 1608, 1629, 1631, 1643, 1645, 1670, 1672, 1684,
- 1686, 1707, 1709, 1721, 1723, 1766, 1768, 1780, 1782, 1803, 1805, 1817,
- 1819, 1844, 1846, 1858, 1860, 1881, 1883, 1895, 1897
-};
-const int16_t vp10_cat6_low_cost[256] = {
- 1638, 1640, 1646, 1648, 1652, 1654, 1660, 1662,
- 1670, 1672, 1678, 1680, 1684, 1686, 1692, 1694, 1711, 1713, 1719, 1721,
- 1725, 1727, 1733, 1735, 1743, 1745, 1751, 1753, 1757, 1759, 1765, 1767,
- 1787, 1789, 1795, 1797, 1801, 1803, 1809, 1811, 1819, 1821, 1827, 1829,
- 1833, 1835, 1841, 1843, 1860, 1862, 1868, 1870, 1874, 1876, 1882, 1884,
- 1892, 1894, 1900, 1902, 1906, 1908, 1914, 1916, 1940, 1942, 1948, 1950,
- 1954, 1956, 1962, 1964, 1972, 1974, 1980, 1982, 1986, 1988, 1994, 1996,
- 2013, 2015, 2021, 2023, 2027, 2029, 2035, 2037, 2045, 2047, 2053, 2055,
- 2059, 2061, 2067, 2069, 2089, 2091, 2097, 2099, 2103, 2105, 2111, 2113,
- 2121, 2123, 2129, 2131, 2135, 2137, 2143, 2145, 2162, 2164, 2170, 2172,
- 2176, 2178, 2184, 2186, 2194, 2196, 2202, 2204, 2208, 2210, 2216, 2218,
- 2082, 2084, 2090, 2092, 2096, 2098, 2104, 2106, 2114, 2116, 2122, 2124,
- 2128, 2130, 2136, 2138, 2155, 2157, 2163, 2165, 2169, 2171, 2177, 2179,
- 2187, 2189, 2195, 2197, 2201, 2203, 2209, 2211, 2231, 2233, 2239, 2241,
- 2245, 2247, 2253, 2255, 2263, 2265, 2271, 2273, 2277, 2279, 2285, 2287,
- 2304, 2306, 2312, 2314, 2318, 2320, 2326, 2328, 2336, 2338, 2344, 2346,
- 2350, 2352, 2358, 2360, 2384, 2386, 2392, 2394, 2398, 2400, 2406, 2408,
- 2416, 2418, 2424, 2426, 2430, 2432, 2438, 2440, 2457, 2459, 2465, 2467,
- 2471, 2473, 2479, 2481, 2489, 2491, 2497, 2499, 2503, 2505, 2511, 2513,
- 2533, 2535, 2541, 2543, 2547, 2549, 2555, 2557, 2565, 2567, 2573, 2575,
- 2579, 2581, 2587, 2589, 2606, 2608, 2614, 2616, 2620, 2622, 2628, 2630,
- 2638, 2640, 2646, 2648, 2652, 2654, 2660, 2662
-};
-const int16_t vp10_cat6_high_cost[128] = {
- 72, 892, 1183, 2003, 1448, 2268, 2559, 3379,
- 1709, 2529, 2820, 3640, 3085, 3905, 4196, 5016, 2118, 2938, 3229, 4049,
- 3494, 4314, 4605, 5425, 3755, 4575, 4866, 5686, 5131, 5951, 6242, 7062,
- 2118, 2938, 3229, 4049, 3494, 4314, 4605, 5425, 3755, 4575, 4866, 5686,
- 5131, 5951, 6242, 7062, 4164, 4984, 5275, 6095, 5540, 6360, 6651, 7471,
- 5801, 6621, 6912, 7732, 7177, 7997, 8288, 9108, 2118, 2938, 3229, 4049,
- 3494, 4314, 4605, 5425, 3755, 4575, 4866, 5686, 5131, 5951, 6242, 7062,
- 4164, 4984, 5275, 6095, 5540, 6360, 6651, 7471, 5801, 6621, 6912, 7732,
- 7177, 7997, 8288, 9108, 4164, 4984, 5275, 6095, 5540, 6360, 6651, 7471,
- 5801, 6621, 6912, 7732, 7177, 7997, 8288, 9108, 6210, 7030, 7321, 8141,
- 7586, 8406, 8697, 9517, 7847, 8667, 8958, 9778, 9223, 10043, 10334, 11154
-};
-
-#if CONFIG_VP9_HIGHBITDEPTH
-const int16_t vp10_cat6_high10_high_cost[512] = {
- 74, 894, 1185, 2005, 1450, 2270, 2561,
- 3381, 1711, 2531, 2822, 3642, 3087, 3907, 4198, 5018, 2120, 2940, 3231,
- 4051, 3496, 4316, 4607, 5427, 3757, 4577, 4868, 5688, 5133, 5953, 6244,
- 7064, 2120, 2940, 3231, 4051, 3496, 4316, 4607, 5427, 3757, 4577, 4868,
- 5688, 5133, 5953, 6244, 7064, 4166, 4986, 5277, 6097, 5542, 6362, 6653,
- 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290, 9110, 2120, 2940, 3231,
- 4051, 3496, 4316, 4607, 5427, 3757, 4577, 4868, 5688, 5133, 5953, 6244,
- 7064, 4166, 4986, 5277, 6097, 5542, 6362, 6653, 7473, 5803, 6623, 6914,
- 7734, 7179, 7999, 8290, 9110, 4166, 4986, 5277, 6097, 5542, 6362, 6653,
- 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290, 9110, 6212, 7032, 7323,
- 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960, 9780, 9225, 10045, 10336,
- 11156, 2120, 2940, 3231, 4051, 3496, 4316, 4607, 5427, 3757, 4577, 4868,
- 5688, 5133, 5953, 6244, 7064, 4166, 4986, 5277, 6097, 5542, 6362, 6653,
- 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290, 9110, 4166, 4986, 5277,
- 6097, 5542, 6362, 6653, 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290,
- 9110, 6212, 7032, 7323, 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960,
- 9780, 9225, 10045, 10336, 11156, 4166, 4986, 5277, 6097, 5542, 6362, 6653,
- 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290, 9110, 6212, 7032, 7323,
- 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960, 9780, 9225, 10045, 10336,
- 11156, 6212, 7032, 7323, 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960,
- 9780, 9225, 10045, 10336, 11156, 8258, 9078, 9369, 10189, 9634, 10454,
- 10745, 11565, 9895, 10715, 11006, 11826, 11271, 12091, 12382, 13202, 2120,
- 2940, 3231, 4051, 3496, 4316, 4607, 5427, 3757, 4577, 4868, 5688, 5133,
- 5953, 6244, 7064, 4166, 4986, 5277, 6097, 5542, 6362, 6653, 7473, 5803,
- 6623, 6914, 7734, 7179, 7999, 8290, 9110, 4166, 4986, 5277, 6097, 5542,
- 6362, 6653, 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290, 9110, 6212,
- 7032, 7323, 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960, 9780, 9225,
- 10045, 10336, 11156, 4166, 4986, 5277, 6097, 5542, 6362, 6653, 7473, 5803,
- 6623, 6914, 7734, 7179, 7999, 8290, 9110, 6212, 7032, 7323, 8143, 7588,
- 8408, 8699, 9519, 7849, 8669, 8960, 9780, 9225, 10045, 10336, 11156, 6212,
- 7032, 7323, 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960, 9780, 9225,
- 10045, 10336, 11156, 8258, 9078, 9369, 10189, 9634, 10454, 10745, 11565,
- 9895, 10715, 11006, 11826, 11271, 12091, 12382, 13202, 4166, 4986, 5277,
- 6097, 5542, 6362, 6653, 7473, 5803, 6623, 6914, 7734, 7179, 7999, 8290,
- 9110, 6212, 7032, 7323, 8143, 7588, 8408, 8699, 9519, 7849, 8669, 8960,
- 9780, 9225, 10045, 10336, 11156, 6212, 7032, 7323, 8143, 7588, 8408, 8699,
- 9519, 7849, 8669, 8960, 9780, 9225, 10045, 10336, 11156, 8258, 9078, 9369,
- 10189, 9634, 10454, 10745, 11565, 9895, 10715, 11006, 11826, 11271, 12091,
- 12382, 13202, 6212, 7032, 7323, 8143, 7588, 8408, 8699, 9519, 7849, 8669,
- 8960, 9780, 9225, 10045, 10336, 11156, 8258, 9078, 9369, 10189, 9634, 10454,
- 10745, 11565, 9895, 10715, 11006, 11826, 11271, 12091, 12382, 13202, 8258,
- 9078, 9369, 10189, 9634, 10454, 10745, 11565, 9895, 10715, 11006, 11826,
- 11271, 12091, 12382, 13202, 10304, 11124, 11415, 12235, 11680, 12500, 12791,
- 13611, 11941, 12761, 13052, 13872, 13317, 14137, 14428, 15248,
-};
-const int16_t vp10_cat6_high12_high_cost[2048] = {
- 76, 896, 1187, 2007, 1452, 2272, 2563,
- 3383, 1713, 2533, 2824, 3644, 3089, 3909, 4200, 5020, 2122, 2942, 3233,
- 4053, 3498, 4318, 4609, 5429, 3759, 4579, 4870, 5690, 5135, 5955, 6246,
- 7066, 2122, 2942, 3233, 4053, 3498, 4318, 4609, 5429, 3759, 4579, 4870,
- 5690, 5135, 5955, 6246, 7066, 4168, 4988, 5279, 6099, 5544, 6364, 6655,
- 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 2122, 2942, 3233,
- 4053, 3498, 4318, 4609, 5429, 3759, 4579, 4870, 5690, 5135, 5955, 6246,
- 7066, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916,
- 7736, 7181, 8001, 8292, 9112, 4168, 4988, 5279, 6099, 5544, 6364, 6655,
- 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325,
- 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338,
- 11158, 2122, 2942, 3233, 4053, 3498, 4318, 4609, 5429, 3759, 4579, 4870,
- 5690, 5135, 5955, 6246, 7066, 4168, 4988, 5279, 6099, 5544, 6364, 6655,
- 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 4168, 4988, 5279,
- 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292,
- 9112, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962,
- 9782, 9227, 10047, 10338, 11158, 4168, 4988, 5279, 6099, 5544, 6364, 6655,
- 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325,
- 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338,
- 11158, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962,
- 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456,
- 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 2122,
- 2942, 3233, 4053, 3498, 4318, 4609, 5429, 3759, 4579, 4870, 5690, 5135,
- 5955, 6246, 7066, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805,
- 6625, 6916, 7736, 7181, 8001, 8292, 9112, 4168, 4988, 5279, 6099, 5544,
- 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 6214,
- 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227,
- 10047, 10338, 11158, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805,
- 6625, 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325, 8145, 7590,
- 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 6214,
- 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227,
- 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567,
- 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 4168, 4988, 5279,
- 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292,
- 9112, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962,
- 9782, 9227, 10047, 10338, 11158, 6214, 7034, 7325, 8145, 7590, 8410, 8701,
- 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371,
- 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093,
- 12384, 13204, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671,
- 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456,
- 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 8260,
- 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828,
- 11273, 12093, 12384, 13204, 10306, 11126, 11417, 12237, 11682, 12502, 12793,
- 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250, 2122, 2942,
- 3233, 4053, 3498, 4318, 4609, 5429, 3759, 4579, 4870, 5690, 5135, 5955,
- 6246, 7066, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625,
- 6916, 7736, 7181, 8001, 8292, 9112, 4168, 4988, 5279, 6099, 5544, 6364,
- 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034,
- 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047,
- 10338, 11158, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625,
- 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325, 8145, 7590, 8410,
- 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 6214, 7034,
- 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047,
- 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897,
- 10717, 11008, 11828, 11273, 12093, 12384, 13204, 4168, 4988, 5279, 6099,
- 5544, 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112,
- 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782,
- 9227, 10047, 10338, 11158, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521,
- 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191,
- 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384,
- 13204, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962,
- 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456,
- 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 8260,
- 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828,
- 11273, 12093, 12384, 13204, 10306, 11126, 11417, 12237, 11682, 12502, 12793,
- 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250, 4168, 4988,
- 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001,
- 8292, 9112, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671,
- 8962, 9782, 9227, 10047, 10338, 11158, 6214, 7034, 7325, 8145, 7590, 8410,
- 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080,
- 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273,
- 12093, 12384, 13204, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851,
- 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636,
- 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204,
- 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008,
- 11828, 11273, 12093, 12384, 13204, 10306, 11126, 11417, 12237, 11682, 12502,
- 12793, 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250, 6214,
- 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227,
- 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567,
- 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 8260, 9080, 9371,
- 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093,
- 12384, 13204, 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943,
- 12763, 13054, 13874, 13319, 14139, 14430, 15250, 8260, 9080, 9371, 10191,
- 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384,
- 13204, 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763,
- 13054, 13874, 13319, 14139, 14430, 15250, 10306, 11126, 11417, 12237, 11682,
- 12502, 12793, 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250,
- 12352, 13172, 13463, 14283, 13728, 14548, 14839, 15659, 13989, 14809, 15100,
- 15920, 15365, 16185, 16476, 17296, 2122, 2942, 3233, 4053, 3498, 4318, 4609,
- 5429, 3759, 4579, 4870, 5690, 5135, 5955, 6246, 7066, 4168, 4988, 5279,
- 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292,
- 9112, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916,
- 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325, 8145, 7590, 8410, 8701,
- 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 4168, 4988, 5279,
- 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916, 7736, 7181, 8001, 8292,
- 9112, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962,
- 9782, 9227, 10047, 10338, 11158, 6214, 7034, 7325, 8145, 7590, 8410, 8701,
- 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371,
- 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093,
- 12384, 13204, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625,
- 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325, 8145, 7590, 8410,
- 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 6214, 7034,
- 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047,
- 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897,
- 10717, 11008, 11828, 11273, 12093, 12384, 13204, 6214, 7034, 7325, 8145,
- 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158,
- 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008,
- 11828, 11273, 12093, 12384, 13204, 8260, 9080, 9371, 10191, 9636, 10456,
- 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 10306,
- 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874,
- 13319, 14139, 14430, 15250, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475,
- 5805, 6625, 6916, 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325, 8145,
- 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158,
- 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782,
- 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747,
- 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 6214, 7034,
- 7325, 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047,
- 10338, 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897,
- 10717, 11008, 11828, 11273, 12093, 12384, 13204, 8260, 9080, 9371, 10191,
- 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384,
- 13204, 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763,
- 13054, 13874, 13319, 14139, 14430, 15250, 6214, 7034, 7325, 8145, 7590,
- 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260,
- 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828,
- 11273, 12093, 12384, 13204, 8260, 9080, 9371, 10191, 9636, 10456, 10747,
- 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 10306, 11126,
- 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874, 13319,
- 14139, 14430, 15250, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567,
- 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 10306, 11126, 11417,
- 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874, 13319, 14139,
- 14430, 15250, 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943,
- 12763, 13054, 13874, 13319, 14139, 14430, 15250, 12352, 13172, 13463, 14283,
- 13728, 14548, 14839, 15659, 13989, 14809, 15100, 15920, 15365, 16185, 16476,
- 17296, 4168, 4988, 5279, 6099, 5544, 6364, 6655, 7475, 5805, 6625, 6916,
- 7736, 7181, 8001, 8292, 9112, 6214, 7034, 7325, 8145, 7590, 8410, 8701,
- 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 6214, 7034, 7325,
- 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338,
- 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717,
- 11008, 11828, 11273, 12093, 12384, 13204, 6214, 7034, 7325, 8145, 7590,
- 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260,
- 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828,
- 11273, 12093, 12384, 13204, 8260, 9080, 9371, 10191, 9636, 10456, 10747,
- 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 10306, 11126,
- 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874, 13319,
- 14139, 14430, 15250, 6214, 7034, 7325, 8145, 7590, 8410, 8701, 9521, 7851,
- 8671, 8962, 9782, 9227, 10047, 10338, 11158, 8260, 9080, 9371, 10191, 9636,
- 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204,
- 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008,
- 11828, 11273, 12093, 12384, 13204, 10306, 11126, 11417, 12237, 11682, 12502,
- 12793, 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250, 8260,
- 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717, 11008, 11828,
- 11273, 12093, 12384, 13204, 10306, 11126, 11417, 12237, 11682, 12502, 12793,
- 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250, 10306, 11126,
- 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874, 13319,
- 14139, 14430, 15250, 12352, 13172, 13463, 14283, 13728, 14548, 14839, 15659,
- 13989, 14809, 15100, 15920, 15365, 16185, 16476, 17296, 6214, 7034, 7325,
- 8145, 7590, 8410, 8701, 9521, 7851, 8671, 8962, 9782, 9227, 10047, 10338,
- 11158, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567, 9897, 10717,
- 11008, 11828, 11273, 12093, 12384, 13204, 8260, 9080, 9371, 10191, 9636,
- 10456, 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204,
- 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054,
- 13874, 13319, 14139, 14430, 15250, 8260, 9080, 9371, 10191, 9636, 10456,
- 10747, 11567, 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 10306,
- 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874,
- 13319, 14139, 14430, 15250, 10306, 11126, 11417, 12237, 11682, 12502, 12793,
- 13613, 11943, 12763, 13054, 13874, 13319, 14139, 14430, 15250, 12352, 13172,
- 13463, 14283, 13728, 14548, 14839, 15659, 13989, 14809, 15100, 15920, 15365,
- 16185, 16476, 17296, 8260, 9080, 9371, 10191, 9636, 10456, 10747, 11567,
- 9897, 10717, 11008, 11828, 11273, 12093, 12384, 13204, 10306, 11126, 11417,
- 12237, 11682, 12502, 12793, 13613, 11943, 12763, 13054, 13874, 13319, 14139,
- 14430, 15250, 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943,
- 12763, 13054, 13874, 13319, 14139, 14430, 15250, 12352, 13172, 13463, 14283,
- 13728, 14548, 14839, 15659, 13989, 14809, 15100, 15920, 15365, 16185, 16476,
- 17296, 10306, 11126, 11417, 12237, 11682, 12502, 12793, 13613, 11943, 12763,
- 13054, 13874, 13319, 14139, 14430, 15250, 12352, 13172, 13463, 14283, 13728,
- 14548, 14839, 15659, 13989, 14809, 15100, 15920, 15365, 16185, 16476, 17296,
- 12352, 13172, 13463, 14283, 13728, 14548, 14839, 15659, 13989, 14809, 15100,
- 15920, 15365, 16185, 16476, 17296, 14398, 15218, 15509, 16329, 15774, 16594,
- 16885, 17705, 16035, 16855, 17146, 17966, 17411, 18231, 18522, 19342
-};
-#endif
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static const vpx_tree_index cat1_high10[2] = {0, 0};
-static const vpx_tree_index cat2_high10[4] = {2, 2, 0, 0};
-static const vpx_tree_index cat3_high10[6] = {2, 2, 4, 4, 0, 0};
-static const vpx_tree_index cat4_high10[8] = {2, 2, 4, 4, 6, 6, 0, 0};
-static const vpx_tree_index cat5_high10[10] = {2, 2, 4, 4, 6, 6, 8, 8, 0, 0};
-static const vpx_tree_index cat6_high10[32] = {2, 2, 4, 4, 6, 6, 8, 8, 10, 10,
- 12, 12, 14, 14, 16, 16, 18, 18, 20, 20, 22, 22, 24, 24, 26, 26, 28, 28,
- 30, 30, 0, 0};
-static const vpx_tree_index cat1_high12[2] = {0, 0};
-static const vpx_tree_index cat2_high12[4] = {2, 2, 0, 0};
-static const vpx_tree_index cat3_high12[6] = {2, 2, 4, 4, 0, 0};
-static const vpx_tree_index cat4_high12[8] = {2, 2, 4, 4, 6, 6, 0, 0};
-static const vpx_tree_index cat5_high12[10] = {2, 2, 4, 4, 6, 6, 8, 8, 0, 0};
-static const vpx_tree_index cat6_high12[36] = {2, 2, 4, 4, 6, 6, 8, 8, 10, 10,
- 12, 12, 14, 14, 16, 16, 18, 18, 20, 20, 22, 22, 24, 24, 26, 26, 28, 28,
- 30, 30, 32, 32, 34, 34, 0, 0};
-#endif
-
-const vp10_extra_bit vp10_extra_bits[ENTROPY_TOKENS] = {
- {0, 0, 0, 0, zero_cost}, // ZERO_TOKEN
- {0, 0, 0, 1, one_cost}, // ONE_TOKEN
- {0, 0, 0, 2, two_cost}, // TWO_TOKEN
- {0, 0, 0, 3, three_cost}, // THREE_TOKEN
- {0, 0, 0, 4, four_cost}, // FOUR_TOKEN
- {cat1, vp10_cat1_prob, 1, CAT1_MIN_VAL, cat1_cost}, // CATEGORY1_TOKEN
- {cat2, vp10_cat2_prob, 2, CAT2_MIN_VAL, cat2_cost}, // CATEGORY2_TOKEN
- {cat3, vp10_cat3_prob, 3, CAT3_MIN_VAL, cat3_cost}, // CATEGORY3_TOKEN
- {cat4, vp10_cat4_prob, 4, CAT4_MIN_VAL, cat4_cost}, // CATEGORY4_TOKEN
- {cat5, vp10_cat5_prob, 5, CAT5_MIN_VAL, cat5_cost}, // CATEGORY5_TOKEN
- {cat6, vp10_cat6_prob, 14, CAT6_MIN_VAL, 0}, // CATEGORY6_TOKEN
- {0, 0, 0, 0, zero_cost} // EOB_TOKEN
-};
-
-#if CONFIG_VP9_HIGHBITDEPTH
-const vp10_extra_bit vp10_extra_bits_high10[ENTROPY_TOKENS] = {
- {0, 0, 0, 0, zero_cost}, // ZERO
- {0, 0, 0, 1, one_cost}, // ONE
- {0, 0, 0, 2, two_cost}, // TWO
- {0, 0, 0, 3, three_cost}, // THREE
- {0, 0, 0, 4, four_cost}, // FOUR
- {cat1_high10, vp10_cat1_prob_high10, 1, CAT1_MIN_VAL, cat1_cost}, // CAT1
- {cat2_high10, vp10_cat2_prob_high10, 2, CAT2_MIN_VAL, cat2_cost}, // CAT2
- {cat3_high10, vp10_cat3_prob_high10, 3, CAT3_MIN_VAL, cat3_cost}, // CAT3
- {cat4_high10, vp10_cat4_prob_high10, 4, CAT4_MIN_VAL, cat4_cost}, // CAT4
- {cat5_high10, vp10_cat5_prob_high10, 5, CAT5_MIN_VAL, cat5_cost}, // CAT5
- {cat6_high10, vp10_cat6_prob_high10, 16, CAT6_MIN_VAL, 0}, // CAT6
- {0, 0, 0, 0, zero_cost} // EOB
-};
-const vp10_extra_bit vp10_extra_bits_high12[ENTROPY_TOKENS] = {
- {0, 0, 0, 0, zero_cost}, // ZERO
- {0, 0, 0, 1, one_cost}, // ONE
- {0, 0, 0, 2, two_cost}, // TWO
- {0, 0, 0, 3, three_cost}, // THREE
- {0, 0, 0, 4, four_cost}, // FOUR
- {cat1_high12, vp10_cat1_prob_high12, 1, CAT1_MIN_VAL, cat1_cost}, // CAT1
- {cat2_high12, vp10_cat2_prob_high12, 2, CAT2_MIN_VAL, cat2_cost}, // CAT2
- {cat3_high12, vp10_cat3_prob_high12, 3, CAT3_MIN_VAL, cat3_cost}, // CAT3
- {cat4_high12, vp10_cat4_prob_high12, 4, CAT4_MIN_VAL, cat4_cost}, // CAT4
- {cat5_high12, vp10_cat5_prob_high12, 5, CAT5_MIN_VAL, cat5_cost}, // CAT5
- {cat6_high12, vp10_cat6_prob_high12, 18, CAT6_MIN_VAL, 0}, // CAT6
- {0, 0, 0, 0, zero_cost} // EOB
-};
-#endif
-
-const struct vp10_token vp10_coef_encodings[ENTROPY_TOKENS] = {
- {2, 2}, {6, 3}, {28, 5}, {58, 6}, {59, 6}, {60, 6}, {61, 6}, {124, 7},
- {125, 7}, {126, 7}, {127, 7}, {0, 1}
-};
-
-
-struct tokenize_b_args {
- VP10_COMP *cpi;
- ThreadData *td;
- TOKENEXTRA **tp;
-};
-
-static void set_entropy_context_b(int plane, int block,
- int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg) {
- struct tokenize_b_args* const args = arg;
- ThreadData *const td = args->td;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- struct macroblock_plane *p = &x->plane[plane];
- struct macroblockd_plane *pd = &xd->plane[plane];
- vp10_set_contexts(xd, pd, plane_bsize, tx_size, p->eobs[block] > 0,
- blk_col, blk_row);
-}
-
-static INLINE void add_token(TOKENEXTRA **t, const vpx_prob *context_tree,
- int32_t extra, uint8_t token,
- uint8_t skip_eob_node,
- unsigned int *counts) {
- (*t)->token = token;
- (*t)->extra = extra;
- (*t)->context_tree = context_tree;
- (*t)->skip_eob_node = skip_eob_node;
- (*t)++;
- ++counts[token];
-}
-
-static INLINE void add_token_no_extra(TOKENEXTRA **t,
- const vpx_prob *context_tree,
- uint8_t token,
- uint8_t skip_eob_node,
- unsigned int *counts) {
- (*t)->token = token;
- (*t)->context_tree = context_tree;
- (*t)->skip_eob_node = skip_eob_node;
- (*t)++;
- ++counts[token];
-}
-
-static INLINE int get_tx_eob(const struct segmentation *seg, int segment_id,
- TX_SIZE tx_size) {
- const int eob_max = 16 << (tx_size << 1);
- return segfeature_active(seg, segment_id, SEG_LVL_SKIP) ? 0 : eob_max;
-}
-
-void vp10_tokenize_palette_sb(struct ThreadData *const td,
- BLOCK_SIZE bsize, int plane,
- TOKENEXTRA **t) {
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- uint8_t *color_map = xd->plane[0].color_index_map;
- PALETTE_MODE_INFO *pmi = &mbmi->palette_mode_info;
- int n = pmi->palette_size[plane != 0];
- int i, j, k;
- int color_new_idx = -1, color_ctx, color_order[PALETTE_MAX_SIZE];
- int rows = 4 * num_4x4_blocks_high_lookup[bsize];
- int cols = 4 * num_4x4_blocks_wide_lookup[bsize];
-
- for (i = 0; i < rows; ++i) {
- for (j = (i == 0 ? 1 : 0); j < cols; ++j) {
- color_ctx = vp10_get_palette_color_context(color_map, cols, i, j, n,
- color_order);
- for (k = 0; k < n; ++k)
- if (color_map[i * cols + j] == color_order[k]) {
- color_new_idx = k;
- break;
- }
- assert(color_new_idx >= 0 && color_new_idx < n);
-
- (*t)->token = color_new_idx;
- (*t)->context_tree = vp10_default_palette_y_color_prob[n - 2][color_ctx];
- (*t)->skip_eob_node = 0;
- ++(*t);
- }
- }
-}
-
-static void tokenize_b(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize,
- TX_SIZE tx_size, void *arg) {
- struct tokenize_b_args* const args = arg;
- VP10_COMP *cpi = args->cpi;
- ThreadData *const td = args->td;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- TOKENEXTRA **tp = args->tp;
- uint8_t token_cache[32 * 32];
- struct macroblock_plane *p = &x->plane[plane];
- struct macroblockd_plane *pd = &xd->plane[plane];
- MB_MODE_INFO *mbmi = &xd->mi[0]->mbmi;
- int pt; /* near block/prev token context index */
- int c;
- TOKENEXTRA *t = *tp; /* store tokens starting here */
- int eob = p->eobs[block];
- const PLANE_TYPE type = pd->plane_type;
- const tran_low_t *qcoeff = BLOCK_OFFSET(p->qcoeff, block);
- const int segment_id = mbmi->segment_id;
- const int16_t *scan, *nb;
- const TX_TYPE tx_type = get_tx_type(type, xd, block);
- const scan_order *const so = get_scan(tx_size, tx_type);
- const int ref = is_inter_block(mbmi);
- unsigned int (*const counts)[COEFF_CONTEXTS][ENTROPY_TOKENS] =
- td->rd_counts.coef_counts[tx_size][type][ref];
- vpx_prob (*const coef_probs)[COEFF_CONTEXTS][UNCONSTRAINED_NODES] =
- cpi->common.fc->coef_probs[tx_size][type][ref];
- unsigned int (*const eob_branch)[COEFF_CONTEXTS] =
- td->counts->eob_branch[tx_size][type][ref];
- const uint8_t *const band = get_band_translate(tx_size);
- const int seg_eob = get_tx_eob(&cpi->common.seg, segment_id, tx_size);
- int16_t token;
- EXTRABIT extra;
- pt = get_entropy_context(tx_size, pd->above_context + blk_col,
- pd->left_context + blk_row);
- scan = so->scan;
- nb = so->neighbors;
- c = 0;
-
- while (c < eob) {
- int v = 0;
- int skip_eob = 0;
- v = qcoeff[scan[c]];
-
- while (!v) {
- add_token_no_extra(&t, coef_probs[band[c]][pt], ZERO_TOKEN, skip_eob,
- counts[band[c]][pt]);
- eob_branch[band[c]][pt] += !skip_eob;
-
- skip_eob = 1;
- token_cache[scan[c]] = 0;
- ++c;
- pt = get_coef_context(nb, token_cache, c);
- v = qcoeff[scan[c]];
- }
-
- vp10_get_token_extra(v, &token, &extra);
-
- add_token(&t, coef_probs[band[c]][pt], extra, (uint8_t)token,
- (uint8_t)skip_eob, counts[band[c]][pt]);
- eob_branch[band[c]][pt] += !skip_eob;
-
- token_cache[scan[c]] = vp10_pt_energy_class[token];
- ++c;
- pt = get_coef_context(nb, token_cache, c);
- }
- if (c < seg_eob) {
- add_token_no_extra(&t, coef_probs[band[c]][pt], EOB_TOKEN, 0,
- counts[band[c]][pt]);
- ++eob_branch[band[c]][pt];
- }
-
- *tp = t;
-
- vp10_set_contexts(xd, pd, plane_bsize, tx_size, c > 0, blk_col, blk_row);
-}
-
-struct is_skippable_args {
- uint16_t *eobs;
- int *skippable;
-};
-static void is_skippable(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size,
- void *argv) {
- struct is_skippable_args *args = argv;
- (void)plane;
- (void)plane_bsize;
- (void)tx_size;
- (void)blk_row;
- (void)blk_col;
- args->skippable[0] &= (!args->eobs[block]);
-}
-
-// TODO(yaowu): rewrite and optimize this function to remove the usage of
-// vp10_foreach_transform_block() and simplify is_skippable().
-int vp10_is_skippable_in_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane) {
- int result = 1;
- struct is_skippable_args args = {x->plane[plane].eobs, &result};
- vp10_foreach_transformed_block_in_plane(&x->e_mbd, bsize, plane, is_skippable,
- &args);
- return result;
-}
-
-static void has_high_freq_coeff(int plane, int block, int blk_row, int blk_col,
- BLOCK_SIZE plane_bsize, TX_SIZE tx_size,
- void *argv) {
- struct is_skippable_args *args = argv;
- int eobs = (tx_size == TX_4X4) ? 3 : 10;
- (void) plane;
- (void) plane_bsize;
- (void) blk_row;
- (void) blk_col;
-
- *(args->skippable) |= (args->eobs[block] > eobs);
-}
-
-int vp10_has_high_freq_in_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane) {
- int result = 0;
- struct is_skippable_args args = {x->plane[plane].eobs, &result};
- vp10_foreach_transformed_block_in_plane(&x->e_mbd, bsize, plane,
- has_high_freq_coeff, &args);
- return result;
-}
-
-void vp10_tokenize_sb(VP10_COMP *cpi, ThreadData *td, TOKENEXTRA **t,
- int dry_run, BLOCK_SIZE bsize) {
- VP10_COMMON *const cm = &cpi->common;
- MACROBLOCK *const x = &td->mb;
- MACROBLOCKD *const xd = &x->e_mbd;
- MB_MODE_INFO *const mbmi = &xd->mi[0]->mbmi;
- const int ctx = vp10_get_skip_context(xd);
- const int skip_inc = !segfeature_active(&cm->seg, mbmi->segment_id,
- SEG_LVL_SKIP);
- struct tokenize_b_args arg = {cpi, td, t};
- if (mbmi->skip) {
- if (!dry_run)
- td->counts->skip[ctx][1] += skip_inc;
- reset_skip_context(xd, bsize);
- return;
- }
-
- if (!dry_run) {
- int plane;
-
- td->counts->skip[ctx][0] += skip_inc;
-
- for (plane = 0; plane < MAX_MB_PLANE; ++plane) {
- vp10_foreach_transformed_block_in_plane(xd, bsize, plane, tokenize_b,
- &arg);
- (*t)->token = EOSB_TOKEN;
- (*t)++;
- }
- } else {
- vp10_foreach_transformed_block(xd, bsize, set_entropy_context_b, &arg);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_TOKENIZE_H_
-#define VP10_ENCODER_TOKENIZE_H_
-
-#include "vp10/common/entropy.h"
-
-#include "vp10/encoder/block.h"
-#include "vp10/encoder/treewriter.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define EOSB_TOKEN 127 // Not signalled, encoder only
-
-#if CONFIG_VP9_HIGHBITDEPTH
- typedef int32_t EXTRABIT;
-#else
- typedef int16_t EXTRABIT;
-#endif
-
-
-typedef struct {
- int16_t token;
- EXTRABIT extra;
-} TOKENVALUE;
-
-typedef struct {
- const vpx_prob *context_tree;
- EXTRABIT extra;
- uint8_t token;
- uint8_t skip_eob_node;
-} TOKENEXTRA;
-
-extern const vpx_tree_index vp10_coef_tree[];
-extern const vpx_tree_index vp10_coef_con_tree[];
-extern const struct vp10_token vp10_coef_encodings[];
-
-int vp10_is_skippable_in_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane);
-int vp10_has_high_freq_in_plane(MACROBLOCK *x, BLOCK_SIZE bsize, int plane);
-
-struct VP10_COMP;
-struct ThreadData;
-
-void vp10_tokenize_palette_sb(struct ThreadData *const td,
- BLOCK_SIZE bsize, int plane,
- TOKENEXTRA **t);
-void vp10_tokenize_sb(struct VP10_COMP *cpi, struct ThreadData *td,
- TOKENEXTRA **t, int dry_run, BLOCK_SIZE bsize);
-
-extern const int16_t *vp10_dct_value_cost_ptr;
-/* TODO: The Token field should be broken out into a separate char array to
- * improve cache locality, since it's needed for costing when the rest of the
- * fields are not.
- */
-extern const TOKENVALUE *vp10_dct_value_tokens_ptr;
-extern const TOKENVALUE *vp10_dct_cat_lt_10_value_tokens;
-extern const int16_t vp10_cat6_low_cost[256];
-extern const int16_t vp10_cat6_high_cost[128];
-extern const int16_t vp10_cat6_high10_high_cost[512];
-extern const int16_t vp10_cat6_high12_high_cost[2048];
-static INLINE int16_t vp10_get_cost(int16_t token, EXTRABIT extrabits,
- const int16_t *cat6_high_table) {
- if (token != CATEGORY6_TOKEN)
- return vp10_extra_bits[token].cost[extrabits];
- return vp10_cat6_low_cost[extrabits & 0xff]
- + cat6_high_table[extrabits >> 8];
-}
-
-#if CONFIG_VP9_HIGHBITDEPTH
-static INLINE const int16_t* vp10_get_high_cost_table(int bit_depth) {
- return bit_depth == 8 ? vp10_cat6_high_cost
- : (bit_depth == 10 ? vp10_cat6_high10_high_cost :
- vp10_cat6_high12_high_cost);
-}
-#else
-static INLINE const int16_t* vp10_get_high_cost_table(int bit_depth) {
- (void) bit_depth;
- return vp10_cat6_high_cost;
-}
-#endif // CONFIG_VP9_HIGHBITDEPTH
-
-static INLINE void vp10_get_token_extra(int v, int16_t *token, EXTRABIT *extra) {
- if (v >= CAT6_MIN_VAL || v <= -CAT6_MIN_VAL) {
- *token = CATEGORY6_TOKEN;
- if (v >= CAT6_MIN_VAL)
- *extra = 2 * v - 2 * CAT6_MIN_VAL;
- else
- *extra = -2 * v - 2 * CAT6_MIN_VAL + 1;
- return;
- }
- *token = vp10_dct_cat_lt_10_value_tokens[v].token;
- *extra = vp10_dct_cat_lt_10_value_tokens[v].extra;
-}
-static INLINE int16_t vp10_get_token(int v) {
- if (v >= CAT6_MIN_VAL || v <= -CAT6_MIN_VAL)
- return 10;
- return vp10_dct_cat_lt_10_value_tokens[v].token;
-}
-
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_TOKENIZE_H_
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include "vp10/encoder/treewriter.h"
-
-static void tree2tok(struct vp10_token *tokens, const vpx_tree_index *tree,
- int i, int v, int l) {
- v += v;
- ++l;
-
- do {
- const vpx_tree_index j = tree[i++];
- if (j <= 0) {
- tokens[-j].value = v;
- tokens[-j].len = l;
- } else {
- tree2tok(tokens, tree, j, v, l);
- }
- } while (++v & 1);
-}
-
-void vp10_tokens_from_tree(struct vp10_token *tokens,
- const vpx_tree_index *tree) {
- tree2tok(tokens, tree, 0, 0, 0);
-}
-
-static unsigned int convert_distribution(unsigned int i, vpx_tree tree,
- unsigned int branch_ct[][2],
- const unsigned int num_events[]) {
- unsigned int left, right;
-
- if (tree[i] <= 0)
- left = num_events[-tree[i]];
- else
- left = convert_distribution(tree[i], tree, branch_ct, num_events);
-
- if (tree[i + 1] <= 0)
- right = num_events[-tree[i + 1]];
- else
- right = convert_distribution(tree[i + 1], tree, branch_ct, num_events);
-
- branch_ct[i >> 1][0] = left;
- branch_ct[i >> 1][1] = right;
- return left + right;
-}
-
-void vp10_tree_probs_from_distribution(vpx_tree tree,
- unsigned int branch_ct[/* n-1 */][2],
- const unsigned int num_events[/* n */]) {
- convert_distribution(0, tree, branch_ct, num_events);
-}
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#ifndef VP10_ENCODER_TREEWRITER_H_
-#define VP10_ENCODER_TREEWRITER_H_
-
-#include "vpx_dsp/bitwriter.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-void vp10_tree_probs_from_distribution(vpx_tree tree,
- unsigned int branch_ct[ /* n - 1 */ ][2],
- const unsigned int num_events[ /* n */ ]);
-
-struct vp10_token {
- int value;
- int len;
-};
-
-void vp10_tokens_from_tree(struct vp10_token*, const vpx_tree_index *);
-
-static INLINE void vp10_write_tree(vpx_writer *w, const vpx_tree_index *tree,
- const vpx_prob *probs, int bits, int len,
- vpx_tree_index i) {
- do {
- const int bit = (bits >> --len) & 1;
- vpx_write(w, bit, probs[i >> 1]);
- i = tree[i + bit];
- } while (len);
-}
-
-static INLINE void vp10_write_token(vpx_writer *w, const vpx_tree_index *tree,
- const vpx_prob *probs,
- const struct vp10_token *token) {
- vp10_write_tree(w, tree, probs, token->value, token->len, 0);
-}
-
-#ifdef __cplusplus
-} // extern "C"
-#endif
-
-#endif // VP10_ENCODER_TREEWRITER_H_
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h>
-
-#include "./vp10_rtcd.h"
-#include "vpx_ports/mem.h"
-
-void vp10_minmax_8x8_sse2(const uint8_t *s, int p, const uint8_t *d, int dp,
- int *min, int *max) {
- __m128i u0, s0, d0, diff, maxabsdiff, minabsdiff, negdiff, absdiff0, absdiff;
- u0 = _mm_setzero_si128();
- // Row 0
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff0 = _mm_max_epi16(diff, negdiff);
- // Row 1
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(absdiff0, absdiff);
- minabsdiff = _mm_min_epi16(absdiff0, absdiff);
- // Row 2
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 2 * p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + 2 * dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(maxabsdiff, absdiff);
- minabsdiff = _mm_min_epi16(minabsdiff, absdiff);
- // Row 3
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 3 * p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + 3 * dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(maxabsdiff, absdiff);
- minabsdiff = _mm_min_epi16(minabsdiff, absdiff);
- // Row 4
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 4 * p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + 4 * dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(maxabsdiff, absdiff);
- minabsdiff = _mm_min_epi16(minabsdiff, absdiff);
- // Row 5
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 5 * p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + 5 * dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(maxabsdiff, absdiff);
- minabsdiff = _mm_min_epi16(minabsdiff, absdiff);
- // Row 6
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 6 * p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + 6 * dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(maxabsdiff, absdiff);
- minabsdiff = _mm_min_epi16(minabsdiff, absdiff);
- // Row 7
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 7 * p)), u0);
- d0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(d + 7 * dp)), u0);
- diff = _mm_subs_epi16(s0, d0);
- negdiff = _mm_subs_epi16(u0, diff);
- absdiff = _mm_max_epi16(diff, negdiff);
- maxabsdiff = _mm_max_epi16(maxabsdiff, absdiff);
- minabsdiff = _mm_min_epi16(minabsdiff, absdiff);
-
- maxabsdiff = _mm_max_epi16(maxabsdiff, _mm_srli_si128(maxabsdiff, 8));
- maxabsdiff = _mm_max_epi16(maxabsdiff, _mm_srli_epi64(maxabsdiff, 32));
- maxabsdiff = _mm_max_epi16(maxabsdiff, _mm_srli_epi64(maxabsdiff, 16));
- *max = _mm_extract_epi16(maxabsdiff, 0);
-
- minabsdiff = _mm_min_epi16(minabsdiff, _mm_srli_si128(minabsdiff, 8));
- minabsdiff = _mm_min_epi16(minabsdiff, _mm_srli_epi64(minabsdiff, 32));
- minabsdiff = _mm_min_epi16(minabsdiff, _mm_srli_epi64(minabsdiff, 16));
- *min = _mm_extract_epi16(minabsdiff, 0);
-}
-
-unsigned int vp10_avg_8x8_sse2(const uint8_t *s, int p) {
- __m128i s0, s1, u0;
- unsigned int avg = 0;
- u0 = _mm_setzero_si128();
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s)), u0);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 2 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 3 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 4 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 5 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 6 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 7 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
-
- s0 = _mm_adds_epu16(s0, _mm_srli_si128(s0, 8));
- s0 = _mm_adds_epu16(s0, _mm_srli_epi64(s0, 32));
- s0 = _mm_adds_epu16(s0, _mm_srli_epi64(s0, 16));
- avg = _mm_extract_epi16(s0, 0);
- return (avg + 32) >> 6;
-}
-
-unsigned int vp10_avg_4x4_sse2(const uint8_t *s, int p) {
- __m128i s0, s1, u0;
- unsigned int avg = 0;
- u0 = _mm_setzero_si128();
- s0 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s)), u0);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 2 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
- s1 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(s + 3 * p)), u0);
- s0 = _mm_adds_epu16(s0, s1);
-
- s0 = _mm_adds_epu16(s0, _mm_srli_si128(s0, 4));
- s0 = _mm_adds_epu16(s0, _mm_srli_epi64(s0, 16));
- avg = _mm_extract_epi16(s0, 0);
- return (avg + 8) >> 4;
-}
-
-static void hadamard_col8_sse2(__m128i *in, int iter) {
- __m128i a0 = in[0];
- __m128i a1 = in[1];
- __m128i a2 = in[2];
- __m128i a3 = in[3];
- __m128i a4 = in[4];
- __m128i a5 = in[5];
- __m128i a6 = in[6];
- __m128i a7 = in[7];
-
- __m128i b0 = _mm_add_epi16(a0, a1);
- __m128i b1 = _mm_sub_epi16(a0, a1);
- __m128i b2 = _mm_add_epi16(a2, a3);
- __m128i b3 = _mm_sub_epi16(a2, a3);
- __m128i b4 = _mm_add_epi16(a4, a5);
- __m128i b5 = _mm_sub_epi16(a4, a5);
- __m128i b6 = _mm_add_epi16(a6, a7);
- __m128i b7 = _mm_sub_epi16(a6, a7);
-
- a0 = _mm_add_epi16(b0, b2);
- a1 = _mm_add_epi16(b1, b3);
- a2 = _mm_sub_epi16(b0, b2);
- a3 = _mm_sub_epi16(b1, b3);
- a4 = _mm_add_epi16(b4, b6);
- a5 = _mm_add_epi16(b5, b7);
- a6 = _mm_sub_epi16(b4, b6);
- a7 = _mm_sub_epi16(b5, b7);
-
- if (iter == 0) {
- b0 = _mm_add_epi16(a0, a4);
- b7 = _mm_add_epi16(a1, a5);
- b3 = _mm_add_epi16(a2, a6);
- b4 = _mm_add_epi16(a3, a7);
- b2 = _mm_sub_epi16(a0, a4);
- b6 = _mm_sub_epi16(a1, a5);
- b1 = _mm_sub_epi16(a2, a6);
- b5 = _mm_sub_epi16(a3, a7);
-
- a0 = _mm_unpacklo_epi16(b0, b1);
- a1 = _mm_unpacklo_epi16(b2, b3);
- a2 = _mm_unpackhi_epi16(b0, b1);
- a3 = _mm_unpackhi_epi16(b2, b3);
- a4 = _mm_unpacklo_epi16(b4, b5);
- a5 = _mm_unpacklo_epi16(b6, b7);
- a6 = _mm_unpackhi_epi16(b4, b5);
- a7 = _mm_unpackhi_epi16(b6, b7);
-
- b0 = _mm_unpacklo_epi32(a0, a1);
- b1 = _mm_unpacklo_epi32(a4, a5);
- b2 = _mm_unpackhi_epi32(a0, a1);
- b3 = _mm_unpackhi_epi32(a4, a5);
- b4 = _mm_unpacklo_epi32(a2, a3);
- b5 = _mm_unpacklo_epi32(a6, a7);
- b6 = _mm_unpackhi_epi32(a2, a3);
- b7 = _mm_unpackhi_epi32(a6, a7);
-
- in[0] = _mm_unpacklo_epi64(b0, b1);
- in[1] = _mm_unpackhi_epi64(b0, b1);
- in[2] = _mm_unpacklo_epi64(b2, b3);
- in[3] = _mm_unpackhi_epi64(b2, b3);
- in[4] = _mm_unpacklo_epi64(b4, b5);
- in[5] = _mm_unpackhi_epi64(b4, b5);
- in[6] = _mm_unpacklo_epi64(b6, b7);
- in[7] = _mm_unpackhi_epi64(b6, b7);
- } else {
- in[0] = _mm_add_epi16(a0, a4);
- in[7] = _mm_add_epi16(a1, a5);
- in[3] = _mm_add_epi16(a2, a6);
- in[4] = _mm_add_epi16(a3, a7);
- in[2] = _mm_sub_epi16(a0, a4);
- in[6] = _mm_sub_epi16(a1, a5);
- in[1] = _mm_sub_epi16(a2, a6);
- in[5] = _mm_sub_epi16(a3, a7);
- }
-}
-
-void vp10_hadamard_8x8_sse2(int16_t const *src_diff, int src_stride,
- int16_t *coeff) {
- __m128i src[8];
- src[0] = _mm_load_si128((const __m128i *)src_diff);
- src[1] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
- src[2] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
- src[3] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
- src[4] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
- src[5] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
- src[6] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
- src[7] = _mm_load_si128((const __m128i *)(src_diff += src_stride));
-
- hadamard_col8_sse2(src, 0);
- hadamard_col8_sse2(src, 1);
-
- _mm_store_si128((__m128i *)coeff, src[0]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[1]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[2]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[3]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[4]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[5]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[6]);
- coeff += 8;
- _mm_store_si128((__m128i *)coeff, src[7]);
-}
-
-void vp10_hadamard_16x16_sse2(int16_t const *src_diff, int src_stride,
- int16_t *coeff) {
- int idx;
- for (idx = 0; idx < 4; ++idx) {
- int16_t const *src_ptr = src_diff + (idx >> 1) * 8 * src_stride
- + (idx & 0x01) * 8;
- vp10_hadamard_8x8_sse2(src_ptr, src_stride, coeff + idx * 64);
- }
-
- for (idx = 0; idx < 64; idx += 8) {
- __m128i coeff0 = _mm_load_si128((const __m128i *)coeff);
- __m128i coeff1 = _mm_load_si128((const __m128i *)(coeff + 64));
- __m128i coeff2 = _mm_load_si128((const __m128i *)(coeff + 128));
- __m128i coeff3 = _mm_load_si128((const __m128i *)(coeff + 192));
-
- __m128i b0 = _mm_add_epi16(coeff0, coeff1);
- __m128i b1 = _mm_sub_epi16(coeff0, coeff1);
- __m128i b2 = _mm_add_epi16(coeff2, coeff3);
- __m128i b3 = _mm_sub_epi16(coeff2, coeff3);
-
- b0 = _mm_srai_epi16(b0, 1);
- b1 = _mm_srai_epi16(b1, 1);
- b2 = _mm_srai_epi16(b2, 1);
- b3 = _mm_srai_epi16(b3, 1);
-
- coeff0 = _mm_add_epi16(b0, b2);
- coeff1 = _mm_add_epi16(b1, b3);
- _mm_store_si128((__m128i *)coeff, coeff0);
- _mm_store_si128((__m128i *)(coeff + 64), coeff1);
-
- coeff2 = _mm_sub_epi16(b0, b2);
- coeff3 = _mm_sub_epi16(b1, b3);
- _mm_store_si128((__m128i *)(coeff + 128), coeff2);
- _mm_store_si128((__m128i *)(coeff + 192), coeff3);
-
- coeff += 8;
- }
-}
-
-int16_t vp10_satd_sse2(const int16_t *coeff, int length) {
- int i;
- __m128i sum = _mm_load_si128((const __m128i *)coeff);
- __m128i sign = _mm_srai_epi16(sum, 15);
- __m128i val = _mm_xor_si128(sum, sign);
- sum = _mm_sub_epi16(val, sign);
- coeff += 8;
-
- for (i = 8; i < length; i += 8) {
- __m128i src_line = _mm_load_si128((const __m128i *)coeff);
- sign = _mm_srai_epi16(src_line, 15);
- val = _mm_xor_si128(src_line, sign);
- val = _mm_sub_epi16(val, sign);
- sum = _mm_add_epi16(sum, val);
- coeff += 8;
- }
-
- val = _mm_srli_si128(sum, 8);
- sum = _mm_add_epi16(sum, val);
- val = _mm_srli_epi64(sum, 32);
- sum = _mm_add_epi16(sum, val);
- val = _mm_srli_epi32(sum, 16);
- sum = _mm_add_epi16(sum, val);
-
- return _mm_extract_epi16(sum, 0);
-}
-
-void vp10_int_pro_row_sse2(int16_t *hbuf, uint8_t const*ref,
- const int ref_stride, const int height) {
- int idx;
- __m128i zero = _mm_setzero_si128();
- __m128i src_line = _mm_loadu_si128((const __m128i *)ref);
- __m128i s0 = _mm_unpacklo_epi8(src_line, zero);
- __m128i s1 = _mm_unpackhi_epi8(src_line, zero);
- __m128i t0, t1;
- int height_1 = height - 1;
- ref += ref_stride;
-
- for (idx = 1; idx < height_1; idx += 2) {
- src_line = _mm_loadu_si128((const __m128i *)ref);
- t0 = _mm_unpacklo_epi8(src_line, zero);
- t1 = _mm_unpackhi_epi8(src_line, zero);
- s0 = _mm_adds_epu16(s0, t0);
- s1 = _mm_adds_epu16(s1, t1);
- ref += ref_stride;
-
- src_line = _mm_loadu_si128((const __m128i *)ref);
- t0 = _mm_unpacklo_epi8(src_line, zero);
- t1 = _mm_unpackhi_epi8(src_line, zero);
- s0 = _mm_adds_epu16(s0, t0);
- s1 = _mm_adds_epu16(s1, t1);
- ref += ref_stride;
- }
-
- src_line = _mm_loadu_si128((const __m128i *)ref);
- t0 = _mm_unpacklo_epi8(src_line, zero);
- t1 = _mm_unpackhi_epi8(src_line, zero);
- s0 = _mm_adds_epu16(s0, t0);
- s1 = _mm_adds_epu16(s1, t1);
-
- if (height == 64) {
- s0 = _mm_srai_epi16(s0, 5);
- s1 = _mm_srai_epi16(s1, 5);
- } else if (height == 32) {
- s0 = _mm_srai_epi16(s0, 4);
- s1 = _mm_srai_epi16(s1, 4);
- } else {
- s0 = _mm_srai_epi16(s0, 3);
- s1 = _mm_srai_epi16(s1, 3);
- }
-
- _mm_storeu_si128((__m128i *)hbuf, s0);
- hbuf += 8;
- _mm_storeu_si128((__m128i *)hbuf, s1);
-}
-
-int16_t vp10_int_pro_col_sse2(uint8_t const *ref, const int width) {
- __m128i zero = _mm_setzero_si128();
- __m128i src_line = _mm_load_si128((const __m128i *)ref);
- __m128i s0 = _mm_sad_epu8(src_line, zero);
- __m128i s1;
- int i;
-
- for (i = 16; i < width; i += 16) {
- ref += 16;
- src_line = _mm_load_si128((const __m128i *)ref);
- s1 = _mm_sad_epu8(src_line, zero);
- s0 = _mm_adds_epu16(s0, s1);
- }
-
- s1 = _mm_srli_si128(s0, 8);
- s0 = _mm_adds_epu16(s0, s1);
-
- return _mm_extract_epi16(s0, 0);
-}
-
-int vp10_vector_var_sse2(int16_t const *ref, int16_t const *src,
- const int bwl) {
- int idx;
- int width = 4 << bwl;
- int16_t mean;
- __m128i v0 = _mm_loadu_si128((const __m128i *)ref);
- __m128i v1 = _mm_load_si128((const __m128i *)src);
- __m128i diff = _mm_subs_epi16(v0, v1);
- __m128i sum = diff;
- __m128i sse = _mm_madd_epi16(diff, diff);
-
- ref += 8;
- src += 8;
-
- for (idx = 8; idx < width; idx += 8) {
- v0 = _mm_loadu_si128((const __m128i *)ref);
- v1 = _mm_load_si128((const __m128i *)src);
- diff = _mm_subs_epi16(v0, v1);
-
- sum = _mm_add_epi16(sum, diff);
- v0 = _mm_madd_epi16(diff, diff);
- sse = _mm_add_epi32(sse, v0);
-
- ref += 8;
- src += 8;
- }
-
- v0 = _mm_srli_si128(sum, 8);
- sum = _mm_add_epi16(sum, v0);
- v0 = _mm_srli_epi64(sum, 32);
- sum = _mm_add_epi16(sum, v0);
- v0 = _mm_srli_epi32(sum, 16);
- sum = _mm_add_epi16(sum, v0);
-
- v1 = _mm_srli_si128(sse, 8);
- sse = _mm_add_epi32(sse, v1);
- v1 = _mm_srli_epi64(sse, 32);
- sse = _mm_add_epi32(sse, v1);
-
- mean = _mm_extract_epi16(sum, 0);
-
- return _mm_cvtsi128_si32(sse) - ((mean * mean) >> (bwl + 2));
-}
+++ /dev/null
-;
-; Copyright (c) 2014 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-%define private_prefix vp10
-
-%include "third_party/x86inc/x86inc.asm"
-
-SECTION .text
-
-%macro TRANSFORM_COLS 0
- paddw m0, m1
- movq m4, m0
- psubw m3, m2
- psubw m4, m3
- psraw m4, 1
- movq m5, m4
- psubw m5, m1 ;b1
- psubw m4, m2 ;c1
- psubw m0, m4
- paddw m3, m5
- ; m0 a0
- SWAP 1, 4 ; m1 c1
- SWAP 2, 3 ; m2 d1
- SWAP 3, 5 ; m3 b1
-%endmacro
-
-%macro TRANSPOSE_4X4 0
- movq m4, m0
- movq m5, m2
- punpcklwd m4, m1
- punpckhwd m0, m1
- punpcklwd m5, m3
- punpckhwd m2, m3
- movq m1, m4
- movq m3, m0
- punpckldq m1, m5
- punpckhdq m4, m5
- punpckldq m3, m2
- punpckhdq m0, m2
- SWAP 2, 3, 0, 1, 4
-%endmacro
-
-INIT_MMX mmx
-cglobal fwht4x4, 3, 4, 8, input, output, stride
- lea r3q, [inputq + strideq*4]
- movq m0, [inputq] ;a1
- movq m1, [inputq + strideq*2] ;b1
- movq m2, [r3q] ;c1
- movq m3, [r3q + strideq*2] ;d1
-
- TRANSFORM_COLS
- TRANSPOSE_4X4
- TRANSFORM_COLS
- TRANSPOSE_4X4
-
- psllw m0, 2
- psllw m1, 2
- psllw m2, 2
- psllw m3, 2
-
-%if CONFIG_VP9_HIGHBITDEPTH
- pxor m4, m4
- pxor m5, m5
- pcmpgtw m4, m0
- pcmpgtw m5, m1
- movq m6, m0
- movq m7, m1
- punpcklwd m0, m4
- punpcklwd m1, m5
- punpckhwd m6, m4
- punpckhwd m7, m5
- movq [outputq], m0
- movq [outputq + 8], m6
- movq [outputq + 16], m1
- movq [outputq + 24], m7
- pxor m4, m4
- pxor m5, m5
- pcmpgtw m4, m2
- pcmpgtw m5, m3
- movq m6, m2
- movq m7, m3
- punpcklwd m2, m4
- punpcklwd m3, m5
- punpckhwd m6, m4
- punpckhwd m7, m5
- movq [outputq + 32], m2
- movq [outputq + 40], m6
- movq [outputq + 48], m3
- movq [outputq + 56], m7
-%else
- movq [outputq], m0
- movq [outputq + 8], m1
- movq [outputq + 16], m2
- movq [outputq + 24], m3
-%endif
-
- RET
+++ /dev/null
-/*
- * Copyright (c) 2012 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#include <emmintrin.h> // SSE2
-
-#include "./vp10_rtcd.h"
-#include "./vpx_dsp_rtcd.h"
-#include "vpx_dsp/txfm_common.h"
-#include "vpx_dsp/x86/fwd_txfm_sse2.h"
-#include "vpx_dsp/x86/txfm_common_sse2.h"
-#include "vpx_ports/mem.h"
-
-static INLINE void load_buffer_4x4(const int16_t *input, __m128i *in,
- int stride) {
- const __m128i k__nonzero_bias_a = _mm_setr_epi16(0, 1, 1, 1, 1, 1, 1, 1);
- const __m128i k__nonzero_bias_b = _mm_setr_epi16(1, 0, 0, 0, 0, 0, 0, 0);
- __m128i mask;
-
- in[0] = _mm_loadl_epi64((const __m128i *)(input + 0 * stride));
- in[1] = _mm_loadl_epi64((const __m128i *)(input + 1 * stride));
- in[2] = _mm_loadl_epi64((const __m128i *)(input + 2 * stride));
- in[3] = _mm_loadl_epi64((const __m128i *)(input + 3 * stride));
-
- in[0] = _mm_slli_epi16(in[0], 4);
- in[1] = _mm_slli_epi16(in[1], 4);
- in[2] = _mm_slli_epi16(in[2], 4);
- in[3] = _mm_slli_epi16(in[3], 4);
-
- mask = _mm_cmpeq_epi16(in[0], k__nonzero_bias_a);
- in[0] = _mm_add_epi16(in[0], mask);
- in[0] = _mm_add_epi16(in[0], k__nonzero_bias_b);
-}
-
-static INLINE void write_buffer_4x4(tran_low_t *output, __m128i *res) {
- const __m128i kOne = _mm_set1_epi16(1);
- __m128i in01 = _mm_unpacklo_epi64(res[0], res[1]);
- __m128i in23 = _mm_unpacklo_epi64(res[2], res[3]);
- __m128i out01 = _mm_add_epi16(in01, kOne);
- __m128i out23 = _mm_add_epi16(in23, kOne);
- out01 = _mm_srai_epi16(out01, 2);
- out23 = _mm_srai_epi16(out23, 2);
- store_output(&out01, (output + 0 * 8));
- store_output(&out23, (output + 1 * 8));
-}
-
-static INLINE void transpose_4x4(__m128i *res) {
- // Combine and transpose
- // 00 01 02 03 20 21 22 23
- // 10 11 12 13 30 31 32 33
- const __m128i tr0_0 = _mm_unpacklo_epi16(res[0], res[1]);
- const __m128i tr0_1 = _mm_unpackhi_epi16(res[0], res[1]);
-
- // 00 10 01 11 02 12 03 13
- // 20 30 21 31 22 32 23 33
- res[0] = _mm_unpacklo_epi32(tr0_0, tr0_1);
- res[2] = _mm_unpackhi_epi32(tr0_0, tr0_1);
-
- // 00 10 20 30 01 11 21 31
- // 02 12 22 32 03 13 23 33
- // only use the first 4 16-bit integers
- res[1] = _mm_unpackhi_epi64(res[0], res[0]);
- res[3] = _mm_unpackhi_epi64(res[2], res[2]);
-}
-
-static void fdct4_sse2(__m128i *in) {
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
-
- __m128i u[4], v[4];
- u[0]=_mm_unpacklo_epi16(in[0], in[1]);
- u[1]=_mm_unpacklo_epi16(in[3], in[2]);
-
- v[0] = _mm_add_epi16(u[0], u[1]);
- v[1] = _mm_sub_epi16(u[0], u[1]);
-
- u[0] = _mm_madd_epi16(v[0], k__cospi_p16_p16); // 0
- u[1] = _mm_madd_epi16(v[0], k__cospi_p16_m16); // 2
- u[2] = _mm_madd_epi16(v[1], k__cospi_p08_p24); // 1
- u[3] = _mm_madd_epi16(v[1], k__cospi_p24_m08); // 3
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
-
- in[0] = _mm_packs_epi32(u[0], u[1]);
- in[1] = _mm_packs_epi32(u[2], u[3]);
- transpose_4x4(in);
-}
-
-static void fadst4_sse2(__m128i *in) {
- const __m128i k__sinpi_p01_p02 = pair_set_epi16(sinpi_1_9, sinpi_2_9);
- const __m128i k__sinpi_p04_m01 = pair_set_epi16(sinpi_4_9, -sinpi_1_9);
- const __m128i k__sinpi_p03_p04 = pair_set_epi16(sinpi_3_9, sinpi_4_9);
- const __m128i k__sinpi_m03_p02 = pair_set_epi16(-sinpi_3_9, sinpi_2_9);
- const __m128i k__sinpi_p03_p03 = _mm_set1_epi16((int16_t)sinpi_3_9);
- const __m128i kZero = _mm_set1_epi16(0);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- __m128i u[8], v[8];
- __m128i in7 = _mm_add_epi16(in[0], in[1]);
-
- u[0] = _mm_unpacklo_epi16(in[0], in[1]);
- u[1] = _mm_unpacklo_epi16(in[2], in[3]);
- u[2] = _mm_unpacklo_epi16(in7, kZero);
- u[3] = _mm_unpacklo_epi16(in[2], kZero);
- u[4] = _mm_unpacklo_epi16(in[3], kZero);
-
- v[0] = _mm_madd_epi16(u[0], k__sinpi_p01_p02); // s0 + s2
- v[1] = _mm_madd_epi16(u[1], k__sinpi_p03_p04); // s4 + s5
- v[2] = _mm_madd_epi16(u[2], k__sinpi_p03_p03); // x1
- v[3] = _mm_madd_epi16(u[0], k__sinpi_p04_m01); // s1 - s3
- v[4] = _mm_madd_epi16(u[1], k__sinpi_m03_p02); // -s4 + s6
- v[5] = _mm_madd_epi16(u[3], k__sinpi_p03_p03); // s4
- v[6] = _mm_madd_epi16(u[4], k__sinpi_p03_p03);
-
- u[0] = _mm_add_epi32(v[0], v[1]);
- u[1] = _mm_sub_epi32(v[2], v[6]);
- u[2] = _mm_add_epi32(v[3], v[4]);
- u[3] = _mm_sub_epi32(u[2], u[0]);
- u[4] = _mm_slli_epi32(v[5], 2);
- u[5] = _mm_sub_epi32(u[4], v[5]);
- u[6] = _mm_add_epi32(u[3], u[5]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
-
- in[0] = _mm_packs_epi32(u[0], u[2]);
- in[1] = _mm_packs_epi32(u[1], u[3]);
- transpose_4x4(in);
-}
-
-void vp10_fht4x4_sse2(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- __m128i in[4];
-
- switch (tx_type) {
- case DCT_DCT:
- vpx_fdct4x4_sse2(input, output, stride);
- break;
- case ADST_DCT:
- load_buffer_4x4(input, in, stride);
- fadst4_sse2(in);
- fdct4_sse2(in);
- write_buffer_4x4(output, in);
- break;
- case DCT_ADST:
- load_buffer_4x4(input, in, stride);
- fdct4_sse2(in);
- fadst4_sse2(in);
- write_buffer_4x4(output, in);
- break;
- case ADST_ADST:
- load_buffer_4x4(input, in, stride);
- fadst4_sse2(in);
- fadst4_sse2(in);
- write_buffer_4x4(output, in);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-void vp10_fdct8x8_quant_sse2(const int16_t *input, int stride,
- int16_t* coeff_ptr, intptr_t n_coeffs,
- int skip_block, const int16_t* zbin_ptr,
- const int16_t* round_ptr, const int16_t* quant_ptr,
- const int16_t* quant_shift_ptr, int16_t* qcoeff_ptr,
- int16_t* dqcoeff_ptr, const int16_t* dequant_ptr,
- uint16_t* eob_ptr,
- const int16_t* scan_ptr,
- const int16_t* iscan_ptr) {
- __m128i zero;
- int pass;
- // Constants
- // When we use them, in one case, they are all the same. In all others
- // it's a pair of them that we need to repeat four times. This is done
- // by constructing the 32 bit constant corresponding to that pair.
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- // Load input
- __m128i in0 = _mm_load_si128((const __m128i *)(input + 0 * stride));
- __m128i in1 = _mm_load_si128((const __m128i *)(input + 1 * stride));
- __m128i in2 = _mm_load_si128((const __m128i *)(input + 2 * stride));
- __m128i in3 = _mm_load_si128((const __m128i *)(input + 3 * stride));
- __m128i in4 = _mm_load_si128((const __m128i *)(input + 4 * stride));
- __m128i in5 = _mm_load_si128((const __m128i *)(input + 5 * stride));
- __m128i in6 = _mm_load_si128((const __m128i *)(input + 6 * stride));
- __m128i in7 = _mm_load_si128((const __m128i *)(input + 7 * stride));
- __m128i *in[8];
- int index = 0;
-
- (void)scan_ptr;
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)coeff_ptr;
-
- // Pre-condition input (shift by two)
- in0 = _mm_slli_epi16(in0, 2);
- in1 = _mm_slli_epi16(in1, 2);
- in2 = _mm_slli_epi16(in2, 2);
- in3 = _mm_slli_epi16(in3, 2);
- in4 = _mm_slli_epi16(in4, 2);
- in5 = _mm_slli_epi16(in5, 2);
- in6 = _mm_slli_epi16(in6, 2);
- in7 = _mm_slli_epi16(in7, 2);
-
- in[0] = &in0;
- in[1] = &in1;
- in[2] = &in2;
- in[3] = &in3;
- in[4] = &in4;
- in[5] = &in5;
- in[6] = &in6;
- in[7] = &in7;
-
- // We do two passes, first the columns, then the rows. The results of the
- // first pass are transposed so that the same column code can be reused. The
- // results of the second pass are also transposed so that the rows (processed
- // as columns) are put back in row positions.
- for (pass = 0; pass < 2; pass++) {
- // To store results of each pass before the transpose.
- __m128i res0, res1, res2, res3, res4, res5, res6, res7;
- // Add/subtract
- const __m128i q0 = _mm_add_epi16(in0, in7);
- const __m128i q1 = _mm_add_epi16(in1, in6);
- const __m128i q2 = _mm_add_epi16(in2, in5);
- const __m128i q3 = _mm_add_epi16(in3, in4);
- const __m128i q4 = _mm_sub_epi16(in3, in4);
- const __m128i q5 = _mm_sub_epi16(in2, in5);
- const __m128i q6 = _mm_sub_epi16(in1, in6);
- const __m128i q7 = _mm_sub_epi16(in0, in7);
- // Work on first four results
- {
- // Add/subtract
- const __m128i r0 = _mm_add_epi16(q0, q3);
- const __m128i r1 = _mm_add_epi16(q1, q2);
- const __m128i r2 = _mm_sub_epi16(q1, q2);
- const __m128i r3 = _mm_sub_epi16(q0, q3);
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i t0 = _mm_unpacklo_epi16(r0, r1);
- const __m128i t1 = _mm_unpackhi_epi16(r0, r1);
- const __m128i t2 = _mm_unpacklo_epi16(r2, r3);
- const __m128i t3 = _mm_unpackhi_epi16(r2, r3);
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_p16_p16);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_p16_p16);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_p16_m16);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_p16_m16);
- const __m128i u4 = _mm_madd_epi16(t2, k__cospi_p24_p08);
- const __m128i u5 = _mm_madd_epi16(t3, k__cospi_p24_p08);
- const __m128i u6 = _mm_madd_epi16(t2, k__cospi_m08_p24);
- const __m128i u7 = _mm_madd_epi16(t3, k__cospi_m08_p24);
- // dct_const_round_shift
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- const __m128i v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- const __m128i v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- const __m128i v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- const __m128i v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- const __m128i w4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- const __m128i w5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- const __m128i w6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- const __m128i w7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- // Combine
- res0 = _mm_packs_epi32(w0, w1);
- res4 = _mm_packs_epi32(w2, w3);
- res2 = _mm_packs_epi32(w4, w5);
- res6 = _mm_packs_epi32(w6, w7);
- }
- // Work on next four results
- {
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i d0 = _mm_unpacklo_epi16(q6, q5);
- const __m128i d1 = _mm_unpackhi_epi16(q6, q5);
- const __m128i e0 = _mm_madd_epi16(d0, k__cospi_p16_m16);
- const __m128i e1 = _mm_madd_epi16(d1, k__cospi_p16_m16);
- const __m128i e2 = _mm_madd_epi16(d0, k__cospi_p16_p16);
- const __m128i e3 = _mm_madd_epi16(d1, k__cospi_p16_p16);
- // dct_const_round_shift
- const __m128i f0 = _mm_add_epi32(e0, k__DCT_CONST_ROUNDING);
- const __m128i f1 = _mm_add_epi32(e1, k__DCT_CONST_ROUNDING);
- const __m128i f2 = _mm_add_epi32(e2, k__DCT_CONST_ROUNDING);
- const __m128i f3 = _mm_add_epi32(e3, k__DCT_CONST_ROUNDING);
- const __m128i s0 = _mm_srai_epi32(f0, DCT_CONST_BITS);
- const __m128i s1 = _mm_srai_epi32(f1, DCT_CONST_BITS);
- const __m128i s2 = _mm_srai_epi32(f2, DCT_CONST_BITS);
- const __m128i s3 = _mm_srai_epi32(f3, DCT_CONST_BITS);
- // Combine
- const __m128i r0 = _mm_packs_epi32(s0, s1);
- const __m128i r1 = _mm_packs_epi32(s2, s3);
- // Add/subtract
- const __m128i x0 = _mm_add_epi16(q4, r0);
- const __m128i x1 = _mm_sub_epi16(q4, r0);
- const __m128i x2 = _mm_sub_epi16(q7, r1);
- const __m128i x3 = _mm_add_epi16(q7, r1);
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i t0 = _mm_unpacklo_epi16(x0, x3);
- const __m128i t1 = _mm_unpackhi_epi16(x0, x3);
- const __m128i t2 = _mm_unpacklo_epi16(x1, x2);
- const __m128i t3 = _mm_unpackhi_epi16(x1, x2);
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_p28_p04);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_p28_p04);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_m04_p28);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_m04_p28);
- const __m128i u4 = _mm_madd_epi16(t2, k__cospi_p12_p20);
- const __m128i u5 = _mm_madd_epi16(t3, k__cospi_p12_p20);
- const __m128i u6 = _mm_madd_epi16(t2, k__cospi_m20_p12);
- const __m128i u7 = _mm_madd_epi16(t3, k__cospi_m20_p12);
- // dct_const_round_shift
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- const __m128i v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- const __m128i v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- const __m128i v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- const __m128i v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- const __m128i w4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- const __m128i w5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- const __m128i w6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- const __m128i w7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- // Combine
- res1 = _mm_packs_epi32(w0, w1);
- res7 = _mm_packs_epi32(w2, w3);
- res5 = _mm_packs_epi32(w4, w5);
- res3 = _mm_packs_epi32(w6, w7);
- }
- // Transpose the 8x8.
- {
- // 00 01 02 03 04 05 06 07
- // 10 11 12 13 14 15 16 17
- // 20 21 22 23 24 25 26 27
- // 30 31 32 33 34 35 36 37
- // 40 41 42 43 44 45 46 47
- // 50 51 52 53 54 55 56 57
- // 60 61 62 63 64 65 66 67
- // 70 71 72 73 74 75 76 77
- const __m128i tr0_0 = _mm_unpacklo_epi16(res0, res1);
- const __m128i tr0_1 = _mm_unpacklo_epi16(res2, res3);
- const __m128i tr0_2 = _mm_unpackhi_epi16(res0, res1);
- const __m128i tr0_3 = _mm_unpackhi_epi16(res2, res3);
- const __m128i tr0_4 = _mm_unpacklo_epi16(res4, res5);
- const __m128i tr0_5 = _mm_unpacklo_epi16(res6, res7);
- const __m128i tr0_6 = _mm_unpackhi_epi16(res4, res5);
- const __m128i tr0_7 = _mm_unpackhi_epi16(res6, res7);
- // 00 10 01 11 02 12 03 13
- // 20 30 21 31 22 32 23 33
- // 04 14 05 15 06 16 07 17
- // 24 34 25 35 26 36 27 37
- // 40 50 41 51 42 52 43 53
- // 60 70 61 71 62 72 63 73
- // 54 54 55 55 56 56 57 57
- // 64 74 65 75 66 76 67 77
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_2, tr0_3);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_2, tr0_3);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5);
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7);
- // 00 10 20 30 01 11 21 31
- // 40 50 60 70 41 51 61 71
- // 02 12 22 32 03 13 23 33
- // 42 52 62 72 43 53 63 73
- // 04 14 24 34 05 15 21 36
- // 44 54 64 74 45 55 61 76
- // 06 16 26 36 07 17 27 37
- // 46 56 66 76 47 57 67 77
- in0 = _mm_unpacklo_epi64(tr1_0, tr1_4);
- in1 = _mm_unpackhi_epi64(tr1_0, tr1_4);
- in2 = _mm_unpacklo_epi64(tr1_2, tr1_6);
- in3 = _mm_unpackhi_epi64(tr1_2, tr1_6);
- in4 = _mm_unpacklo_epi64(tr1_1, tr1_5);
- in5 = _mm_unpackhi_epi64(tr1_1, tr1_5);
- in6 = _mm_unpacklo_epi64(tr1_3, tr1_7);
- in7 = _mm_unpackhi_epi64(tr1_3, tr1_7);
- // 00 10 20 30 40 50 60 70
- // 01 11 21 31 41 51 61 71
- // 02 12 22 32 42 52 62 72
- // 03 13 23 33 43 53 63 73
- // 04 14 24 34 44 54 64 74
- // 05 15 25 35 45 55 65 75
- // 06 16 26 36 46 56 66 76
- // 07 17 27 37 47 57 67 77
- }
- }
- // Post-condition output and store it
- {
- // Post-condition (division by two)
- // division of two 16 bits signed numbers using shifts
- // n / 2 = (n - (n >> 15)) >> 1
- const __m128i sign_in0 = _mm_srai_epi16(in0, 15);
- const __m128i sign_in1 = _mm_srai_epi16(in1, 15);
- const __m128i sign_in2 = _mm_srai_epi16(in2, 15);
- const __m128i sign_in3 = _mm_srai_epi16(in3, 15);
- const __m128i sign_in4 = _mm_srai_epi16(in4, 15);
- const __m128i sign_in5 = _mm_srai_epi16(in5, 15);
- const __m128i sign_in6 = _mm_srai_epi16(in6, 15);
- const __m128i sign_in7 = _mm_srai_epi16(in7, 15);
- in0 = _mm_sub_epi16(in0, sign_in0);
- in1 = _mm_sub_epi16(in1, sign_in1);
- in2 = _mm_sub_epi16(in2, sign_in2);
- in3 = _mm_sub_epi16(in3, sign_in3);
- in4 = _mm_sub_epi16(in4, sign_in4);
- in5 = _mm_sub_epi16(in5, sign_in5);
- in6 = _mm_sub_epi16(in6, sign_in6);
- in7 = _mm_sub_epi16(in7, sign_in7);
- in0 = _mm_srai_epi16(in0, 1);
- in1 = _mm_srai_epi16(in1, 1);
- in2 = _mm_srai_epi16(in2, 1);
- in3 = _mm_srai_epi16(in3, 1);
- in4 = _mm_srai_epi16(in4, 1);
- in5 = _mm_srai_epi16(in5, 1);
- in6 = _mm_srai_epi16(in6, 1);
- in7 = _mm_srai_epi16(in7, 1);
- }
-
- iscan_ptr += n_coeffs;
- qcoeff_ptr += n_coeffs;
- dqcoeff_ptr += n_coeffs;
- n_coeffs = -n_coeffs;
- zero = _mm_setzero_si128();
-
- if (!skip_block) {
- __m128i eob;
- __m128i round, quant, dequant;
- {
- __m128i coeff0, coeff1;
-
- // Setup global values
- {
- round = _mm_load_si128((const __m128i*)round_ptr);
- quant = _mm_load_si128((const __m128i*)quant_ptr);
- dequant = _mm_load_si128((const __m128i*)dequant_ptr);
- }
-
- {
- __m128i coeff0_sign, coeff1_sign;
- __m128i qcoeff0, qcoeff1;
- __m128i qtmp0, qtmp1;
- // Do DC and first 15 AC
- coeff0 = *in[0];
- coeff1 = *in[1];
-
- // Poor man's sign extract
- coeff0_sign = _mm_srai_epi16(coeff0, 15);
- coeff1_sign = _mm_srai_epi16(coeff1, 15);
- qcoeff0 = _mm_xor_si128(coeff0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(coeff1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- qcoeff0 = _mm_adds_epi16(qcoeff0, round);
- round = _mm_unpackhi_epi64(round, round);
- qcoeff1 = _mm_adds_epi16(qcoeff1, round);
- qtmp0 = _mm_mulhi_epi16(qcoeff0, quant);
- quant = _mm_unpackhi_epi64(quant, quant);
- qtmp1 = _mm_mulhi_epi16(qcoeff1, quant);
-
- // Reinsert signs
- qcoeff0 = _mm_xor_si128(qtmp0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(qtmp1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), qcoeff0);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, qcoeff1);
-
- coeff0 = _mm_mullo_epi16(qcoeff0, dequant);
- dequant = _mm_unpackhi_epi64(dequant, dequant);
- coeff1 = _mm_mullo_epi16(qcoeff1, dequant);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), coeff0);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, coeff1);
- }
-
- {
- // Scan for eob
- __m128i zero_coeff0, zero_coeff1;
- __m128i nzero_coeff0, nzero_coeff1;
- __m128i iscan0, iscan1;
- __m128i eob1;
- zero_coeff0 = _mm_cmpeq_epi16(coeff0, zero);
- zero_coeff1 = _mm_cmpeq_epi16(coeff1, zero);
- nzero_coeff0 = _mm_cmpeq_epi16(zero_coeff0, zero);
- nzero_coeff1 = _mm_cmpeq_epi16(zero_coeff1, zero);
- iscan0 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs));
- iscan1 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs) + 1);
- // Add one to convert from indices to counts
- iscan0 = _mm_sub_epi16(iscan0, nzero_coeff0);
- iscan1 = _mm_sub_epi16(iscan1, nzero_coeff1);
- eob = _mm_and_si128(iscan0, nzero_coeff0);
- eob1 = _mm_and_si128(iscan1, nzero_coeff1);
- eob = _mm_max_epi16(eob, eob1);
- }
- n_coeffs += 8 * 2;
- }
-
- // AC only loop
- index = 2;
- while (n_coeffs < 0) {
- __m128i coeff0, coeff1;
- {
- __m128i coeff0_sign, coeff1_sign;
- __m128i qcoeff0, qcoeff1;
- __m128i qtmp0, qtmp1;
-
- assert(index < (int)(sizeof(in) / sizeof(in[0])) - 1);
- coeff0 = *in[index];
- coeff1 = *in[index + 1];
-
- // Poor man's sign extract
- coeff0_sign = _mm_srai_epi16(coeff0, 15);
- coeff1_sign = _mm_srai_epi16(coeff1, 15);
- qcoeff0 = _mm_xor_si128(coeff0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(coeff1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- qcoeff0 = _mm_adds_epi16(qcoeff0, round);
- qcoeff1 = _mm_adds_epi16(qcoeff1, round);
- qtmp0 = _mm_mulhi_epi16(qcoeff0, quant);
- qtmp1 = _mm_mulhi_epi16(qcoeff1, quant);
-
- // Reinsert signs
- qcoeff0 = _mm_xor_si128(qtmp0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(qtmp1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), qcoeff0);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, qcoeff1);
-
- coeff0 = _mm_mullo_epi16(qcoeff0, dequant);
- coeff1 = _mm_mullo_epi16(qcoeff1, dequant);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), coeff0);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, coeff1);
- }
-
- {
- // Scan for eob
- __m128i zero_coeff0, zero_coeff1;
- __m128i nzero_coeff0, nzero_coeff1;
- __m128i iscan0, iscan1;
- __m128i eob0, eob1;
- zero_coeff0 = _mm_cmpeq_epi16(coeff0, zero);
- zero_coeff1 = _mm_cmpeq_epi16(coeff1, zero);
- nzero_coeff0 = _mm_cmpeq_epi16(zero_coeff0, zero);
- nzero_coeff1 = _mm_cmpeq_epi16(zero_coeff1, zero);
- iscan0 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs));
- iscan1 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs) + 1);
- // Add one to convert from indices to counts
- iscan0 = _mm_sub_epi16(iscan0, nzero_coeff0);
- iscan1 = _mm_sub_epi16(iscan1, nzero_coeff1);
- eob0 = _mm_and_si128(iscan0, nzero_coeff0);
- eob1 = _mm_and_si128(iscan1, nzero_coeff1);
- eob0 = _mm_max_epi16(eob0, eob1);
- eob = _mm_max_epi16(eob, eob0);
- }
- n_coeffs += 8 * 2;
- index += 2;
- }
-
- // Accumulate EOB
- {
- __m128i eob_shuffled;
- eob_shuffled = _mm_shuffle_epi32(eob, 0xe);
- eob = _mm_max_epi16(eob, eob_shuffled);
- eob_shuffled = _mm_shufflelo_epi16(eob, 0xe);
- eob = _mm_max_epi16(eob, eob_shuffled);
- eob_shuffled = _mm_shufflelo_epi16(eob, 0x1);
- eob = _mm_max_epi16(eob, eob_shuffled);
- *eob_ptr = _mm_extract_epi16(eob, 1);
- }
- } else {
- do {
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, zero);
- n_coeffs += 8 * 2;
- } while (n_coeffs < 0);
- *eob_ptr = 0;
- }
-}
-
-// load 8x8 array
-static INLINE void load_buffer_8x8(const int16_t *input, __m128i *in,
- int stride) {
- in[0] = _mm_load_si128((const __m128i *)(input + 0 * stride));
- in[1] = _mm_load_si128((const __m128i *)(input + 1 * stride));
- in[2] = _mm_load_si128((const __m128i *)(input + 2 * stride));
- in[3] = _mm_load_si128((const __m128i *)(input + 3 * stride));
- in[4] = _mm_load_si128((const __m128i *)(input + 4 * stride));
- in[5] = _mm_load_si128((const __m128i *)(input + 5 * stride));
- in[6] = _mm_load_si128((const __m128i *)(input + 6 * stride));
- in[7] = _mm_load_si128((const __m128i *)(input + 7 * stride));
-
- in[0] = _mm_slli_epi16(in[0], 2);
- in[1] = _mm_slli_epi16(in[1], 2);
- in[2] = _mm_slli_epi16(in[2], 2);
- in[3] = _mm_slli_epi16(in[3], 2);
- in[4] = _mm_slli_epi16(in[4], 2);
- in[5] = _mm_slli_epi16(in[5], 2);
- in[6] = _mm_slli_epi16(in[6], 2);
- in[7] = _mm_slli_epi16(in[7], 2);
-}
-
-// right shift and rounding
-static INLINE void right_shift_8x8(__m128i *res, const int bit) {
- __m128i sign0 = _mm_srai_epi16(res[0], 15);
- __m128i sign1 = _mm_srai_epi16(res[1], 15);
- __m128i sign2 = _mm_srai_epi16(res[2], 15);
- __m128i sign3 = _mm_srai_epi16(res[3], 15);
- __m128i sign4 = _mm_srai_epi16(res[4], 15);
- __m128i sign5 = _mm_srai_epi16(res[5], 15);
- __m128i sign6 = _mm_srai_epi16(res[6], 15);
- __m128i sign7 = _mm_srai_epi16(res[7], 15);
-
- if (bit == 2) {
- const __m128i const_rounding = _mm_set1_epi16(1);
- res[0] = _mm_add_epi16(res[0], const_rounding);
- res[1] = _mm_add_epi16(res[1], const_rounding);
- res[2] = _mm_add_epi16(res[2], const_rounding);
- res[3] = _mm_add_epi16(res[3], const_rounding);
- res[4] = _mm_add_epi16(res[4], const_rounding);
- res[5] = _mm_add_epi16(res[5], const_rounding);
- res[6] = _mm_add_epi16(res[6], const_rounding);
- res[7] = _mm_add_epi16(res[7], const_rounding);
- }
-
- res[0] = _mm_sub_epi16(res[0], sign0);
- res[1] = _mm_sub_epi16(res[1], sign1);
- res[2] = _mm_sub_epi16(res[2], sign2);
- res[3] = _mm_sub_epi16(res[3], sign3);
- res[4] = _mm_sub_epi16(res[4], sign4);
- res[5] = _mm_sub_epi16(res[5], sign5);
- res[6] = _mm_sub_epi16(res[6], sign6);
- res[7] = _mm_sub_epi16(res[7], sign7);
-
- if (bit == 1) {
- res[0] = _mm_srai_epi16(res[0], 1);
- res[1] = _mm_srai_epi16(res[1], 1);
- res[2] = _mm_srai_epi16(res[2], 1);
- res[3] = _mm_srai_epi16(res[3], 1);
- res[4] = _mm_srai_epi16(res[4], 1);
- res[5] = _mm_srai_epi16(res[5], 1);
- res[6] = _mm_srai_epi16(res[6], 1);
- res[7] = _mm_srai_epi16(res[7], 1);
- } else {
- res[0] = _mm_srai_epi16(res[0], 2);
- res[1] = _mm_srai_epi16(res[1], 2);
- res[2] = _mm_srai_epi16(res[2], 2);
- res[3] = _mm_srai_epi16(res[3], 2);
- res[4] = _mm_srai_epi16(res[4], 2);
- res[5] = _mm_srai_epi16(res[5], 2);
- res[6] = _mm_srai_epi16(res[6], 2);
- res[7] = _mm_srai_epi16(res[7], 2);
- }
-}
-
-// write 8x8 array
-static INLINE void write_buffer_8x8(tran_low_t *output, __m128i *res,
- int stride) {
- store_output(&res[0], (output + 0 * stride));
- store_output(&res[1], (output + 1 * stride));
- store_output(&res[2], (output + 2 * stride));
- store_output(&res[3], (output + 3 * stride));
- store_output(&res[4], (output + 4 * stride));
- store_output(&res[5], (output + 5 * stride));
- store_output(&res[6], (output + 6 * stride));
- store_output(&res[7], (output + 7 * stride));
-}
-
-// perform in-place transpose
-static INLINE void array_transpose_8x8(__m128i *in, __m128i *res) {
- const __m128i tr0_0 = _mm_unpacklo_epi16(in[0], in[1]);
- const __m128i tr0_1 = _mm_unpacklo_epi16(in[2], in[3]);
- const __m128i tr0_2 = _mm_unpackhi_epi16(in[0], in[1]);
- const __m128i tr0_3 = _mm_unpackhi_epi16(in[2], in[3]);
- const __m128i tr0_4 = _mm_unpacklo_epi16(in[4], in[5]);
- const __m128i tr0_5 = _mm_unpacklo_epi16(in[6], in[7]);
- const __m128i tr0_6 = _mm_unpackhi_epi16(in[4], in[5]);
- const __m128i tr0_7 = _mm_unpackhi_epi16(in[6], in[7]);
- // 00 10 01 11 02 12 03 13
- // 20 30 21 31 22 32 23 33
- // 04 14 05 15 06 16 07 17
- // 24 34 25 35 26 36 27 37
- // 40 50 41 51 42 52 43 53
- // 60 70 61 71 62 72 63 73
- // 44 54 45 55 46 56 47 57
- // 64 74 65 75 66 76 67 77
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_4, tr0_5);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_2, tr0_3);
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_2, tr0_3);
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7);
- // 00 10 20 30 01 11 21 31
- // 40 50 60 70 41 51 61 71
- // 02 12 22 32 03 13 23 33
- // 42 52 62 72 43 53 63 73
- // 04 14 24 34 05 15 25 35
- // 44 54 64 74 45 55 65 75
- // 06 16 26 36 07 17 27 37
- // 46 56 66 76 47 57 67 77
- res[0] = _mm_unpacklo_epi64(tr1_0, tr1_1);
- res[1] = _mm_unpackhi_epi64(tr1_0, tr1_1);
- res[2] = _mm_unpacklo_epi64(tr1_2, tr1_3);
- res[3] = _mm_unpackhi_epi64(tr1_2, tr1_3);
- res[4] = _mm_unpacklo_epi64(tr1_4, tr1_5);
- res[5] = _mm_unpackhi_epi64(tr1_4, tr1_5);
- res[6] = _mm_unpacklo_epi64(tr1_6, tr1_7);
- res[7] = _mm_unpackhi_epi64(tr1_6, tr1_7);
- // 00 10 20 30 40 50 60 70
- // 01 11 21 31 41 51 61 71
- // 02 12 22 32 42 52 62 72
- // 03 13 23 33 43 53 63 73
- // 04 14 24 34 44 54 64 74
- // 05 15 25 35 45 55 65 75
- // 06 16 26 36 46 56 66 76
- // 07 17 27 37 47 57 67 77
-}
-
-static void fdct8_sse2(__m128i *in) {
- // constants
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- __m128i u0, u1, u2, u3, u4, u5, u6, u7;
- __m128i v0, v1, v2, v3, v4, v5, v6, v7;
- __m128i s0, s1, s2, s3, s4, s5, s6, s7;
-
- // stage 1
- s0 = _mm_add_epi16(in[0], in[7]);
- s1 = _mm_add_epi16(in[1], in[6]);
- s2 = _mm_add_epi16(in[2], in[5]);
- s3 = _mm_add_epi16(in[3], in[4]);
- s4 = _mm_sub_epi16(in[3], in[4]);
- s5 = _mm_sub_epi16(in[2], in[5]);
- s6 = _mm_sub_epi16(in[1], in[6]);
- s7 = _mm_sub_epi16(in[0], in[7]);
-
- u0 = _mm_add_epi16(s0, s3);
- u1 = _mm_add_epi16(s1, s2);
- u2 = _mm_sub_epi16(s1, s2);
- u3 = _mm_sub_epi16(s0, s3);
- // interleave and perform butterfly multiplication/addition
- v0 = _mm_unpacklo_epi16(u0, u1);
- v1 = _mm_unpackhi_epi16(u0, u1);
- v2 = _mm_unpacklo_epi16(u2, u3);
- v3 = _mm_unpackhi_epi16(u2, u3);
-
- u0 = _mm_madd_epi16(v0, k__cospi_p16_p16);
- u1 = _mm_madd_epi16(v1, k__cospi_p16_p16);
- u2 = _mm_madd_epi16(v0, k__cospi_p16_m16);
- u3 = _mm_madd_epi16(v1, k__cospi_p16_m16);
- u4 = _mm_madd_epi16(v2, k__cospi_p24_p08);
- u5 = _mm_madd_epi16(v3, k__cospi_p24_p08);
- u6 = _mm_madd_epi16(v2, k__cospi_m08_p24);
- u7 = _mm_madd_epi16(v3, k__cospi_m08_p24);
-
- // shift and rounding
- v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
-
- u0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- u1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- u2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- u3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- u4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- u5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- u6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- u7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
-
- in[0] = _mm_packs_epi32(u0, u1);
- in[2] = _mm_packs_epi32(u4, u5);
- in[4] = _mm_packs_epi32(u2, u3);
- in[6] = _mm_packs_epi32(u6, u7);
-
- // stage 2
- // interleave and perform butterfly multiplication/addition
- u0 = _mm_unpacklo_epi16(s6, s5);
- u1 = _mm_unpackhi_epi16(s6, s5);
- v0 = _mm_madd_epi16(u0, k__cospi_p16_m16);
- v1 = _mm_madd_epi16(u1, k__cospi_p16_m16);
- v2 = _mm_madd_epi16(u0, k__cospi_p16_p16);
- v3 = _mm_madd_epi16(u1, k__cospi_p16_p16);
-
- // shift and rounding
- u0 = _mm_add_epi32(v0, k__DCT_CONST_ROUNDING);
- u1 = _mm_add_epi32(v1, k__DCT_CONST_ROUNDING);
- u2 = _mm_add_epi32(v2, k__DCT_CONST_ROUNDING);
- u3 = _mm_add_epi32(v3, k__DCT_CONST_ROUNDING);
-
- v0 = _mm_srai_epi32(u0, DCT_CONST_BITS);
- v1 = _mm_srai_epi32(u1, DCT_CONST_BITS);
- v2 = _mm_srai_epi32(u2, DCT_CONST_BITS);
- v3 = _mm_srai_epi32(u3, DCT_CONST_BITS);
-
- u0 = _mm_packs_epi32(v0, v1);
- u1 = _mm_packs_epi32(v2, v3);
-
- // stage 3
- s0 = _mm_add_epi16(s4, u0);
- s1 = _mm_sub_epi16(s4, u0);
- s2 = _mm_sub_epi16(s7, u1);
- s3 = _mm_add_epi16(s7, u1);
-
- // stage 4
- u0 = _mm_unpacklo_epi16(s0, s3);
- u1 = _mm_unpackhi_epi16(s0, s3);
- u2 = _mm_unpacklo_epi16(s1, s2);
- u3 = _mm_unpackhi_epi16(s1, s2);
-
- v0 = _mm_madd_epi16(u0, k__cospi_p28_p04);
- v1 = _mm_madd_epi16(u1, k__cospi_p28_p04);
- v2 = _mm_madd_epi16(u2, k__cospi_p12_p20);
- v3 = _mm_madd_epi16(u3, k__cospi_p12_p20);
- v4 = _mm_madd_epi16(u2, k__cospi_m20_p12);
- v5 = _mm_madd_epi16(u3, k__cospi_m20_p12);
- v6 = _mm_madd_epi16(u0, k__cospi_m04_p28);
- v7 = _mm_madd_epi16(u1, k__cospi_m04_p28);
-
- // shift and rounding
- u0 = _mm_add_epi32(v0, k__DCT_CONST_ROUNDING);
- u1 = _mm_add_epi32(v1, k__DCT_CONST_ROUNDING);
- u2 = _mm_add_epi32(v2, k__DCT_CONST_ROUNDING);
- u3 = _mm_add_epi32(v3, k__DCT_CONST_ROUNDING);
- u4 = _mm_add_epi32(v4, k__DCT_CONST_ROUNDING);
- u5 = _mm_add_epi32(v5, k__DCT_CONST_ROUNDING);
- u6 = _mm_add_epi32(v6, k__DCT_CONST_ROUNDING);
- u7 = _mm_add_epi32(v7, k__DCT_CONST_ROUNDING);
-
- v0 = _mm_srai_epi32(u0, DCT_CONST_BITS);
- v1 = _mm_srai_epi32(u1, DCT_CONST_BITS);
- v2 = _mm_srai_epi32(u2, DCT_CONST_BITS);
- v3 = _mm_srai_epi32(u3, DCT_CONST_BITS);
- v4 = _mm_srai_epi32(u4, DCT_CONST_BITS);
- v5 = _mm_srai_epi32(u5, DCT_CONST_BITS);
- v6 = _mm_srai_epi32(u6, DCT_CONST_BITS);
- v7 = _mm_srai_epi32(u7, DCT_CONST_BITS);
-
- in[1] = _mm_packs_epi32(v0, v1);
- in[3] = _mm_packs_epi32(v4, v5);
- in[5] = _mm_packs_epi32(v2, v3);
- in[7] = _mm_packs_epi32(v6, v7);
-
- // transpose
- array_transpose_8x8(in, in);
-}
-
-static void fadst8_sse2(__m128i *in) {
- // Constants
- const __m128i k__cospi_p02_p30 = pair_set_epi16(cospi_2_64, cospi_30_64);
- const __m128i k__cospi_p30_m02 = pair_set_epi16(cospi_30_64, -cospi_2_64);
- const __m128i k__cospi_p10_p22 = pair_set_epi16(cospi_10_64, cospi_22_64);
- const __m128i k__cospi_p22_m10 = pair_set_epi16(cospi_22_64, -cospi_10_64);
- const __m128i k__cospi_p18_p14 = pair_set_epi16(cospi_18_64, cospi_14_64);
- const __m128i k__cospi_p14_m18 = pair_set_epi16(cospi_14_64, -cospi_18_64);
- const __m128i k__cospi_p26_p06 = pair_set_epi16(cospi_26_64, cospi_6_64);
- const __m128i k__cospi_p06_m26 = pair_set_epi16(cospi_6_64, -cospi_26_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_m24_p08 = pair_set_epi16(-cospi_24_64, cospi_8_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__const_0 = _mm_set1_epi16(0);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
-
- __m128i u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, u10, u11, u12, u13, u14, u15;
- __m128i v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15;
- __m128i w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15;
- __m128i s0, s1, s2, s3, s4, s5, s6, s7;
- __m128i in0, in1, in2, in3, in4, in5, in6, in7;
-
- // properly aligned for butterfly input
- in0 = in[7];
- in1 = in[0];
- in2 = in[5];
- in3 = in[2];
- in4 = in[3];
- in5 = in[4];
- in6 = in[1];
- in7 = in[6];
-
- // column transformation
- // stage 1
- // interleave and multiply/add into 32-bit integer
- s0 = _mm_unpacklo_epi16(in0, in1);
- s1 = _mm_unpackhi_epi16(in0, in1);
- s2 = _mm_unpacklo_epi16(in2, in3);
- s3 = _mm_unpackhi_epi16(in2, in3);
- s4 = _mm_unpacklo_epi16(in4, in5);
- s5 = _mm_unpackhi_epi16(in4, in5);
- s6 = _mm_unpacklo_epi16(in6, in7);
- s7 = _mm_unpackhi_epi16(in6, in7);
-
- u0 = _mm_madd_epi16(s0, k__cospi_p02_p30);
- u1 = _mm_madd_epi16(s1, k__cospi_p02_p30);
- u2 = _mm_madd_epi16(s0, k__cospi_p30_m02);
- u3 = _mm_madd_epi16(s1, k__cospi_p30_m02);
- u4 = _mm_madd_epi16(s2, k__cospi_p10_p22);
- u5 = _mm_madd_epi16(s3, k__cospi_p10_p22);
- u6 = _mm_madd_epi16(s2, k__cospi_p22_m10);
- u7 = _mm_madd_epi16(s3, k__cospi_p22_m10);
- u8 = _mm_madd_epi16(s4, k__cospi_p18_p14);
- u9 = _mm_madd_epi16(s5, k__cospi_p18_p14);
- u10 = _mm_madd_epi16(s4, k__cospi_p14_m18);
- u11 = _mm_madd_epi16(s5, k__cospi_p14_m18);
- u12 = _mm_madd_epi16(s6, k__cospi_p26_p06);
- u13 = _mm_madd_epi16(s7, k__cospi_p26_p06);
- u14 = _mm_madd_epi16(s6, k__cospi_p06_m26);
- u15 = _mm_madd_epi16(s7, k__cospi_p06_m26);
-
- // addition
- w0 = _mm_add_epi32(u0, u8);
- w1 = _mm_add_epi32(u1, u9);
- w2 = _mm_add_epi32(u2, u10);
- w3 = _mm_add_epi32(u3, u11);
- w4 = _mm_add_epi32(u4, u12);
- w5 = _mm_add_epi32(u5, u13);
- w6 = _mm_add_epi32(u6, u14);
- w7 = _mm_add_epi32(u7, u15);
- w8 = _mm_sub_epi32(u0, u8);
- w9 = _mm_sub_epi32(u1, u9);
- w10 = _mm_sub_epi32(u2, u10);
- w11 = _mm_sub_epi32(u3, u11);
- w12 = _mm_sub_epi32(u4, u12);
- w13 = _mm_sub_epi32(u5, u13);
- w14 = _mm_sub_epi32(u6, u14);
- w15 = _mm_sub_epi32(u7, u15);
-
- // shift and rounding
- v0 = _mm_add_epi32(w0, k__DCT_CONST_ROUNDING);
- v1 = _mm_add_epi32(w1, k__DCT_CONST_ROUNDING);
- v2 = _mm_add_epi32(w2, k__DCT_CONST_ROUNDING);
- v3 = _mm_add_epi32(w3, k__DCT_CONST_ROUNDING);
- v4 = _mm_add_epi32(w4, k__DCT_CONST_ROUNDING);
- v5 = _mm_add_epi32(w5, k__DCT_CONST_ROUNDING);
- v6 = _mm_add_epi32(w6, k__DCT_CONST_ROUNDING);
- v7 = _mm_add_epi32(w7, k__DCT_CONST_ROUNDING);
- v8 = _mm_add_epi32(w8, k__DCT_CONST_ROUNDING);
- v9 = _mm_add_epi32(w9, k__DCT_CONST_ROUNDING);
- v10 = _mm_add_epi32(w10, k__DCT_CONST_ROUNDING);
- v11 = _mm_add_epi32(w11, k__DCT_CONST_ROUNDING);
- v12 = _mm_add_epi32(w12, k__DCT_CONST_ROUNDING);
- v13 = _mm_add_epi32(w13, k__DCT_CONST_ROUNDING);
- v14 = _mm_add_epi32(w14, k__DCT_CONST_ROUNDING);
- v15 = _mm_add_epi32(w15, k__DCT_CONST_ROUNDING);
-
- u0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- u1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- u2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- u3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- u4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- u5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- u6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- u7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- u8 = _mm_srai_epi32(v8, DCT_CONST_BITS);
- u9 = _mm_srai_epi32(v9, DCT_CONST_BITS);
- u10 = _mm_srai_epi32(v10, DCT_CONST_BITS);
- u11 = _mm_srai_epi32(v11, DCT_CONST_BITS);
- u12 = _mm_srai_epi32(v12, DCT_CONST_BITS);
- u13 = _mm_srai_epi32(v13, DCT_CONST_BITS);
- u14 = _mm_srai_epi32(v14, DCT_CONST_BITS);
- u15 = _mm_srai_epi32(v15, DCT_CONST_BITS);
-
- // back to 16-bit and pack 8 integers into __m128i
- in[0] = _mm_packs_epi32(u0, u1);
- in[1] = _mm_packs_epi32(u2, u3);
- in[2] = _mm_packs_epi32(u4, u5);
- in[3] = _mm_packs_epi32(u6, u7);
- in[4] = _mm_packs_epi32(u8, u9);
- in[5] = _mm_packs_epi32(u10, u11);
- in[6] = _mm_packs_epi32(u12, u13);
- in[7] = _mm_packs_epi32(u14, u15);
-
- // stage 2
- s0 = _mm_add_epi16(in[0], in[2]);
- s1 = _mm_add_epi16(in[1], in[3]);
- s2 = _mm_sub_epi16(in[0], in[2]);
- s3 = _mm_sub_epi16(in[1], in[3]);
- u0 = _mm_unpacklo_epi16(in[4], in[5]);
- u1 = _mm_unpackhi_epi16(in[4], in[5]);
- u2 = _mm_unpacklo_epi16(in[6], in[7]);
- u3 = _mm_unpackhi_epi16(in[6], in[7]);
-
- v0 = _mm_madd_epi16(u0, k__cospi_p08_p24);
- v1 = _mm_madd_epi16(u1, k__cospi_p08_p24);
- v2 = _mm_madd_epi16(u0, k__cospi_p24_m08);
- v3 = _mm_madd_epi16(u1, k__cospi_p24_m08);
- v4 = _mm_madd_epi16(u2, k__cospi_m24_p08);
- v5 = _mm_madd_epi16(u3, k__cospi_m24_p08);
- v6 = _mm_madd_epi16(u2, k__cospi_p08_p24);
- v7 = _mm_madd_epi16(u3, k__cospi_p08_p24);
-
- w0 = _mm_add_epi32(v0, v4);
- w1 = _mm_add_epi32(v1, v5);
- w2 = _mm_add_epi32(v2, v6);
- w3 = _mm_add_epi32(v3, v7);
- w4 = _mm_sub_epi32(v0, v4);
- w5 = _mm_sub_epi32(v1, v5);
- w6 = _mm_sub_epi32(v2, v6);
- w7 = _mm_sub_epi32(v3, v7);
-
- v0 = _mm_add_epi32(w0, k__DCT_CONST_ROUNDING);
- v1 = _mm_add_epi32(w1, k__DCT_CONST_ROUNDING);
- v2 = _mm_add_epi32(w2, k__DCT_CONST_ROUNDING);
- v3 = _mm_add_epi32(w3, k__DCT_CONST_ROUNDING);
- v4 = _mm_add_epi32(w4, k__DCT_CONST_ROUNDING);
- v5 = _mm_add_epi32(w5, k__DCT_CONST_ROUNDING);
- v6 = _mm_add_epi32(w6, k__DCT_CONST_ROUNDING);
- v7 = _mm_add_epi32(w7, k__DCT_CONST_ROUNDING);
-
- u0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- u1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- u2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- u3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- u4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- u5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- u6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- u7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
-
- // back to 16-bit intergers
- s4 = _mm_packs_epi32(u0, u1);
- s5 = _mm_packs_epi32(u2, u3);
- s6 = _mm_packs_epi32(u4, u5);
- s7 = _mm_packs_epi32(u6, u7);
-
- // stage 3
- u0 = _mm_unpacklo_epi16(s2, s3);
- u1 = _mm_unpackhi_epi16(s2, s3);
- u2 = _mm_unpacklo_epi16(s6, s7);
- u3 = _mm_unpackhi_epi16(s6, s7);
-
- v0 = _mm_madd_epi16(u0, k__cospi_p16_p16);
- v1 = _mm_madd_epi16(u1, k__cospi_p16_p16);
- v2 = _mm_madd_epi16(u0, k__cospi_p16_m16);
- v3 = _mm_madd_epi16(u1, k__cospi_p16_m16);
- v4 = _mm_madd_epi16(u2, k__cospi_p16_p16);
- v5 = _mm_madd_epi16(u3, k__cospi_p16_p16);
- v6 = _mm_madd_epi16(u2, k__cospi_p16_m16);
- v7 = _mm_madd_epi16(u3, k__cospi_p16_m16);
-
- u0 = _mm_add_epi32(v0, k__DCT_CONST_ROUNDING);
- u1 = _mm_add_epi32(v1, k__DCT_CONST_ROUNDING);
- u2 = _mm_add_epi32(v2, k__DCT_CONST_ROUNDING);
- u3 = _mm_add_epi32(v3, k__DCT_CONST_ROUNDING);
- u4 = _mm_add_epi32(v4, k__DCT_CONST_ROUNDING);
- u5 = _mm_add_epi32(v5, k__DCT_CONST_ROUNDING);
- u6 = _mm_add_epi32(v6, k__DCT_CONST_ROUNDING);
- u7 = _mm_add_epi32(v7, k__DCT_CONST_ROUNDING);
-
- v0 = _mm_srai_epi32(u0, DCT_CONST_BITS);
- v1 = _mm_srai_epi32(u1, DCT_CONST_BITS);
- v2 = _mm_srai_epi32(u2, DCT_CONST_BITS);
- v3 = _mm_srai_epi32(u3, DCT_CONST_BITS);
- v4 = _mm_srai_epi32(u4, DCT_CONST_BITS);
- v5 = _mm_srai_epi32(u5, DCT_CONST_BITS);
- v6 = _mm_srai_epi32(u6, DCT_CONST_BITS);
- v7 = _mm_srai_epi32(u7, DCT_CONST_BITS);
-
- s2 = _mm_packs_epi32(v0, v1);
- s3 = _mm_packs_epi32(v2, v3);
- s6 = _mm_packs_epi32(v4, v5);
- s7 = _mm_packs_epi32(v6, v7);
-
- // FIXME(jingning): do subtract using bit inversion?
- in[0] = s0;
- in[1] = _mm_sub_epi16(k__const_0, s4);
- in[2] = s6;
- in[3] = _mm_sub_epi16(k__const_0, s2);
- in[4] = s3;
- in[5] = _mm_sub_epi16(k__const_0, s7);
- in[6] = s5;
- in[7] = _mm_sub_epi16(k__const_0, s1);
-
- // transpose
- array_transpose_8x8(in, in);
-}
-
-void vp10_fht8x8_sse2(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- __m128i in[8];
-
- switch (tx_type) {
- case DCT_DCT:
- vpx_fdct8x8_sse2(input, output, stride);
- break;
- case ADST_DCT:
- load_buffer_8x8(input, in, stride);
- fadst8_sse2(in);
- fdct8_sse2(in);
- right_shift_8x8(in, 1);
- write_buffer_8x8(output, in, 8);
- break;
- case DCT_ADST:
- load_buffer_8x8(input, in, stride);
- fdct8_sse2(in);
- fadst8_sse2(in);
- right_shift_8x8(in, 1);
- write_buffer_8x8(output, in, 8);
- break;
- case ADST_ADST:
- load_buffer_8x8(input, in, stride);
- fadst8_sse2(in);
- fadst8_sse2(in);
- right_shift_8x8(in, 1);
- write_buffer_8x8(output, in, 8);
- break;
- default:
- assert(0);
- break;
- }
-}
-
-static INLINE void load_buffer_16x16(const int16_t* input, __m128i *in0,
- __m128i *in1, int stride) {
- // load first 8 columns
- load_buffer_8x8(input, in0, stride);
- load_buffer_8x8(input + 8 * stride, in0 + 8, stride);
-
- input += 8;
- // load second 8 columns
- load_buffer_8x8(input, in1, stride);
- load_buffer_8x8(input + 8 * stride, in1 + 8, stride);
-}
-
-static INLINE void write_buffer_16x16(tran_low_t *output, __m128i *in0,
- __m128i *in1, int stride) {
- // write first 8 columns
- write_buffer_8x8(output, in0, stride);
- write_buffer_8x8(output + 8 * stride, in0 + 8, stride);
- // write second 8 columns
- output += 8;
- write_buffer_8x8(output, in1, stride);
- write_buffer_8x8(output + 8 * stride, in1 + 8, stride);
-}
-
-static INLINE void array_transpose_16x16(__m128i *res0, __m128i *res1) {
- __m128i tbuf[8];
- array_transpose_8x8(res0, res0);
- array_transpose_8x8(res1, tbuf);
- array_transpose_8x8(res0 + 8, res1);
- array_transpose_8x8(res1 + 8, res1 + 8);
-
- res0[8] = tbuf[0];
- res0[9] = tbuf[1];
- res0[10] = tbuf[2];
- res0[11] = tbuf[3];
- res0[12] = tbuf[4];
- res0[13] = tbuf[5];
- res0[14] = tbuf[6];
- res0[15] = tbuf[7];
-}
-
-static INLINE void right_shift_16x16(__m128i *res0, __m128i *res1) {
- // perform rounding operations
- right_shift_8x8(res0, 2);
- right_shift_8x8(res0 + 8, 2);
- right_shift_8x8(res1, 2);
- right_shift_8x8(res1 + 8, 2);
-}
-
-static void fdct16_8col(__m128i *in) {
- // perform 16x16 1-D DCT for 8 columns
- __m128i i[8], s[8], p[8], t[8], u[16], v[16];
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_m16_p16 = pair_set_epi16(-cospi_16_64, cospi_16_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_p08_m24 = pair_set_epi16(cospi_8_64, -cospi_24_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__cospi_p30_p02 = pair_set_epi16(cospi_30_64, cospi_2_64);
- const __m128i k__cospi_p14_p18 = pair_set_epi16(cospi_14_64, cospi_18_64);
- const __m128i k__cospi_m02_p30 = pair_set_epi16(-cospi_2_64, cospi_30_64);
- const __m128i k__cospi_m18_p14 = pair_set_epi16(-cospi_18_64, cospi_14_64);
- const __m128i k__cospi_p22_p10 = pair_set_epi16(cospi_22_64, cospi_10_64);
- const __m128i k__cospi_p06_p26 = pair_set_epi16(cospi_6_64, cospi_26_64);
- const __m128i k__cospi_m10_p22 = pair_set_epi16(-cospi_10_64, cospi_22_64);
- const __m128i k__cospi_m26_p06 = pair_set_epi16(-cospi_26_64, cospi_6_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
-
- // stage 1
- i[0] = _mm_add_epi16(in[0], in[15]);
- i[1] = _mm_add_epi16(in[1], in[14]);
- i[2] = _mm_add_epi16(in[2], in[13]);
- i[3] = _mm_add_epi16(in[3], in[12]);
- i[4] = _mm_add_epi16(in[4], in[11]);
- i[5] = _mm_add_epi16(in[5], in[10]);
- i[6] = _mm_add_epi16(in[6], in[9]);
- i[7] = _mm_add_epi16(in[7], in[8]);
-
- s[0] = _mm_sub_epi16(in[7], in[8]);
- s[1] = _mm_sub_epi16(in[6], in[9]);
- s[2] = _mm_sub_epi16(in[5], in[10]);
- s[3] = _mm_sub_epi16(in[4], in[11]);
- s[4] = _mm_sub_epi16(in[3], in[12]);
- s[5] = _mm_sub_epi16(in[2], in[13]);
- s[6] = _mm_sub_epi16(in[1], in[14]);
- s[7] = _mm_sub_epi16(in[0], in[15]);
-
- p[0] = _mm_add_epi16(i[0], i[7]);
- p[1] = _mm_add_epi16(i[1], i[6]);
- p[2] = _mm_add_epi16(i[2], i[5]);
- p[3] = _mm_add_epi16(i[3], i[4]);
- p[4] = _mm_sub_epi16(i[3], i[4]);
- p[5] = _mm_sub_epi16(i[2], i[5]);
- p[6] = _mm_sub_epi16(i[1], i[6]);
- p[7] = _mm_sub_epi16(i[0], i[7]);
-
- u[0] = _mm_add_epi16(p[0], p[3]);
- u[1] = _mm_add_epi16(p[1], p[2]);
- u[2] = _mm_sub_epi16(p[1], p[2]);
- u[3] = _mm_sub_epi16(p[0], p[3]);
-
- v[0] = _mm_unpacklo_epi16(u[0], u[1]);
- v[1] = _mm_unpackhi_epi16(u[0], u[1]);
- v[2] = _mm_unpacklo_epi16(u[2], u[3]);
- v[3] = _mm_unpackhi_epi16(u[2], u[3]);
-
- u[0] = _mm_madd_epi16(v[0], k__cospi_p16_p16);
- u[1] = _mm_madd_epi16(v[1], k__cospi_p16_p16);
- u[2] = _mm_madd_epi16(v[0], k__cospi_p16_m16);
- u[3] = _mm_madd_epi16(v[1], k__cospi_p16_m16);
- u[4] = _mm_madd_epi16(v[2], k__cospi_p24_p08);
- u[5] = _mm_madd_epi16(v[3], k__cospi_p24_p08);
- u[6] = _mm_madd_epi16(v[2], k__cospi_m08_p24);
- u[7] = _mm_madd_epi16(v[3], k__cospi_m08_p24);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
-
- in[0] = _mm_packs_epi32(u[0], u[1]);
- in[4] = _mm_packs_epi32(u[4], u[5]);
- in[8] = _mm_packs_epi32(u[2], u[3]);
- in[12] = _mm_packs_epi32(u[6], u[7]);
-
- u[0] = _mm_unpacklo_epi16(p[5], p[6]);
- u[1] = _mm_unpackhi_epi16(p[5], p[6]);
- v[0] = _mm_madd_epi16(u[0], k__cospi_m16_p16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m16_p16);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p16_p16);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p16_p16);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
-
- u[0] = _mm_packs_epi32(v[0], v[1]);
- u[1] = _mm_packs_epi32(v[2], v[3]);
-
- t[0] = _mm_add_epi16(p[4], u[0]);
- t[1] = _mm_sub_epi16(p[4], u[0]);
- t[2] = _mm_sub_epi16(p[7], u[1]);
- t[3] = _mm_add_epi16(p[7], u[1]);
-
- u[0] = _mm_unpacklo_epi16(t[0], t[3]);
- u[1] = _mm_unpackhi_epi16(t[0], t[3]);
- u[2] = _mm_unpacklo_epi16(t[1], t[2]);
- u[3] = _mm_unpackhi_epi16(t[1], t[2]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p28_p04);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p28_p04);
- v[2] = _mm_madd_epi16(u[2], k__cospi_p12_p20);
- v[3] = _mm_madd_epi16(u[3], k__cospi_p12_p20);
- v[4] = _mm_madd_epi16(u[2], k__cospi_m20_p12);
- v[5] = _mm_madd_epi16(u[3], k__cospi_m20_p12);
- v[6] = _mm_madd_epi16(u[0], k__cospi_m04_p28);
- v[7] = _mm_madd_epi16(u[1], k__cospi_m04_p28);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
-
- in[2] = _mm_packs_epi32(v[0], v[1]);
- in[6] = _mm_packs_epi32(v[4], v[5]);
- in[10] = _mm_packs_epi32(v[2], v[3]);
- in[14] = _mm_packs_epi32(v[6], v[7]);
-
- // stage 2
- u[0] = _mm_unpacklo_epi16(s[2], s[5]);
- u[1] = _mm_unpackhi_epi16(s[2], s[5]);
- u[2] = _mm_unpacklo_epi16(s[3], s[4]);
- u[3] = _mm_unpackhi_epi16(s[3], s[4]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_m16_p16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m16_p16);
- v[2] = _mm_madd_epi16(u[2], k__cospi_m16_p16);
- v[3] = _mm_madd_epi16(u[3], k__cospi_m16_p16);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p16_p16);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p16_p16);
- v[6] = _mm_madd_epi16(u[0], k__cospi_p16_p16);
- v[7] = _mm_madd_epi16(u[1], k__cospi_p16_p16);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
-
- t[2] = _mm_packs_epi32(v[0], v[1]);
- t[3] = _mm_packs_epi32(v[2], v[3]);
- t[4] = _mm_packs_epi32(v[4], v[5]);
- t[5] = _mm_packs_epi32(v[6], v[7]);
-
- // stage 3
- p[0] = _mm_add_epi16(s[0], t[3]);
- p[1] = _mm_add_epi16(s[1], t[2]);
- p[2] = _mm_sub_epi16(s[1], t[2]);
- p[3] = _mm_sub_epi16(s[0], t[3]);
- p[4] = _mm_sub_epi16(s[7], t[4]);
- p[5] = _mm_sub_epi16(s[6], t[5]);
- p[6] = _mm_add_epi16(s[6], t[5]);
- p[7] = _mm_add_epi16(s[7], t[4]);
-
- // stage 4
- u[0] = _mm_unpacklo_epi16(p[1], p[6]);
- u[1] = _mm_unpackhi_epi16(p[1], p[6]);
- u[2] = _mm_unpacklo_epi16(p[2], p[5]);
- u[3] = _mm_unpackhi_epi16(p[2], p[5]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_m08_p24);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m08_p24);
- v[2] = _mm_madd_epi16(u[2], k__cospi_p24_p08);
- v[3] = _mm_madd_epi16(u[3], k__cospi_p24_p08);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p08_m24);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p08_m24);
- v[6] = _mm_madd_epi16(u[0], k__cospi_p24_p08);
- v[7] = _mm_madd_epi16(u[1], k__cospi_p24_p08);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
-
- t[1] = _mm_packs_epi32(v[0], v[1]);
- t[2] = _mm_packs_epi32(v[2], v[3]);
- t[5] = _mm_packs_epi32(v[4], v[5]);
- t[6] = _mm_packs_epi32(v[6], v[7]);
-
- // stage 5
- s[0] = _mm_add_epi16(p[0], t[1]);
- s[1] = _mm_sub_epi16(p[0], t[1]);
- s[2] = _mm_add_epi16(p[3], t[2]);
- s[3] = _mm_sub_epi16(p[3], t[2]);
- s[4] = _mm_sub_epi16(p[4], t[5]);
- s[5] = _mm_add_epi16(p[4], t[5]);
- s[6] = _mm_sub_epi16(p[7], t[6]);
- s[7] = _mm_add_epi16(p[7], t[6]);
-
- // stage 6
- u[0] = _mm_unpacklo_epi16(s[0], s[7]);
- u[1] = _mm_unpackhi_epi16(s[0], s[7]);
- u[2] = _mm_unpacklo_epi16(s[1], s[6]);
- u[3] = _mm_unpackhi_epi16(s[1], s[6]);
- u[4] = _mm_unpacklo_epi16(s[2], s[5]);
- u[5] = _mm_unpackhi_epi16(s[2], s[5]);
- u[6] = _mm_unpacklo_epi16(s[3], s[4]);
- u[7] = _mm_unpackhi_epi16(s[3], s[4]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p30_p02);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p30_p02);
- v[2] = _mm_madd_epi16(u[2], k__cospi_p14_p18);
- v[3] = _mm_madd_epi16(u[3], k__cospi_p14_p18);
- v[4] = _mm_madd_epi16(u[4], k__cospi_p22_p10);
- v[5] = _mm_madd_epi16(u[5], k__cospi_p22_p10);
- v[6] = _mm_madd_epi16(u[6], k__cospi_p06_p26);
- v[7] = _mm_madd_epi16(u[7], k__cospi_p06_p26);
- v[8] = _mm_madd_epi16(u[6], k__cospi_m26_p06);
- v[9] = _mm_madd_epi16(u[7], k__cospi_m26_p06);
- v[10] = _mm_madd_epi16(u[4], k__cospi_m10_p22);
- v[11] = _mm_madd_epi16(u[5], k__cospi_m10_p22);
- v[12] = _mm_madd_epi16(u[2], k__cospi_m18_p14);
- v[13] = _mm_madd_epi16(u[3], k__cospi_m18_p14);
- v[14] = _mm_madd_epi16(u[0], k__cospi_m02_p30);
- v[15] = _mm_madd_epi16(u[1], k__cospi_m02_p30);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(v[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(v[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(v[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(v[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(v[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(v[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(v[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(v[15], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- v[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- v[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- v[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- v[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- v[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- v[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- v[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- v[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- in[1] = _mm_packs_epi32(v[0], v[1]);
- in[9] = _mm_packs_epi32(v[2], v[3]);
- in[5] = _mm_packs_epi32(v[4], v[5]);
- in[13] = _mm_packs_epi32(v[6], v[7]);
- in[3] = _mm_packs_epi32(v[8], v[9]);
- in[11] = _mm_packs_epi32(v[10], v[11]);
- in[7] = _mm_packs_epi32(v[12], v[13]);
- in[15] = _mm_packs_epi32(v[14], v[15]);
-}
-
-static void fadst16_8col(__m128i *in) {
- // perform 16x16 1-D ADST for 8 columns
- __m128i s[16], x[16], u[32], v[32];
- const __m128i k__cospi_p01_p31 = pair_set_epi16(cospi_1_64, cospi_31_64);
- const __m128i k__cospi_p31_m01 = pair_set_epi16(cospi_31_64, -cospi_1_64);
- const __m128i k__cospi_p05_p27 = pair_set_epi16(cospi_5_64, cospi_27_64);
- const __m128i k__cospi_p27_m05 = pair_set_epi16(cospi_27_64, -cospi_5_64);
- const __m128i k__cospi_p09_p23 = pair_set_epi16(cospi_9_64, cospi_23_64);
- const __m128i k__cospi_p23_m09 = pair_set_epi16(cospi_23_64, -cospi_9_64);
- const __m128i k__cospi_p13_p19 = pair_set_epi16(cospi_13_64, cospi_19_64);
- const __m128i k__cospi_p19_m13 = pair_set_epi16(cospi_19_64, -cospi_13_64);
- const __m128i k__cospi_p17_p15 = pair_set_epi16(cospi_17_64, cospi_15_64);
- const __m128i k__cospi_p15_m17 = pair_set_epi16(cospi_15_64, -cospi_17_64);
- const __m128i k__cospi_p21_p11 = pair_set_epi16(cospi_21_64, cospi_11_64);
- const __m128i k__cospi_p11_m21 = pair_set_epi16(cospi_11_64, -cospi_21_64);
- const __m128i k__cospi_p25_p07 = pair_set_epi16(cospi_25_64, cospi_7_64);
- const __m128i k__cospi_p07_m25 = pair_set_epi16(cospi_7_64, -cospi_25_64);
- const __m128i k__cospi_p29_p03 = pair_set_epi16(cospi_29_64, cospi_3_64);
- const __m128i k__cospi_p03_m29 = pair_set_epi16(cospi_3_64, -cospi_29_64);
- const __m128i k__cospi_p04_p28 = pair_set_epi16(cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p28_m04 = pair_set_epi16(cospi_28_64, -cospi_4_64);
- const __m128i k__cospi_p20_p12 = pair_set_epi16(cospi_20_64, cospi_12_64);
- const __m128i k__cospi_p12_m20 = pair_set_epi16(cospi_12_64, -cospi_20_64);
- const __m128i k__cospi_m28_p04 = pair_set_epi16(-cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m12_p20 = pair_set_epi16(-cospi_12_64, cospi_20_64);
- const __m128i k__cospi_p08_p24 = pair_set_epi16(cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p24_m08 = pair_set_epi16(cospi_24_64, -cospi_8_64);
- const __m128i k__cospi_m24_p08 = pair_set_epi16(-cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m16_m16 = _mm_set1_epi16((int16_t)-cospi_16_64);
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_m16_p16 = pair_set_epi16(-cospi_16_64, cospi_16_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- const __m128i kZero = _mm_set1_epi16(0);
-
- u[0] = _mm_unpacklo_epi16(in[15], in[0]);
- u[1] = _mm_unpackhi_epi16(in[15], in[0]);
- u[2] = _mm_unpacklo_epi16(in[13], in[2]);
- u[3] = _mm_unpackhi_epi16(in[13], in[2]);
- u[4] = _mm_unpacklo_epi16(in[11], in[4]);
- u[5] = _mm_unpackhi_epi16(in[11], in[4]);
- u[6] = _mm_unpacklo_epi16(in[9], in[6]);
- u[7] = _mm_unpackhi_epi16(in[9], in[6]);
- u[8] = _mm_unpacklo_epi16(in[7], in[8]);
- u[9] = _mm_unpackhi_epi16(in[7], in[8]);
- u[10] = _mm_unpacklo_epi16(in[5], in[10]);
- u[11] = _mm_unpackhi_epi16(in[5], in[10]);
- u[12] = _mm_unpacklo_epi16(in[3], in[12]);
- u[13] = _mm_unpackhi_epi16(in[3], in[12]);
- u[14] = _mm_unpacklo_epi16(in[1], in[14]);
- u[15] = _mm_unpackhi_epi16(in[1], in[14]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p01_p31);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p01_p31);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p31_m01);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p31_m01);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p05_p27);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p05_p27);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p27_m05);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p27_m05);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p09_p23);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p09_p23);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p23_m09);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p23_m09);
- v[12] = _mm_madd_epi16(u[6], k__cospi_p13_p19);
- v[13] = _mm_madd_epi16(u[7], k__cospi_p13_p19);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p19_m13);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p19_m13);
- v[16] = _mm_madd_epi16(u[8], k__cospi_p17_p15);
- v[17] = _mm_madd_epi16(u[9], k__cospi_p17_p15);
- v[18] = _mm_madd_epi16(u[8], k__cospi_p15_m17);
- v[19] = _mm_madd_epi16(u[9], k__cospi_p15_m17);
- v[20] = _mm_madd_epi16(u[10], k__cospi_p21_p11);
- v[21] = _mm_madd_epi16(u[11], k__cospi_p21_p11);
- v[22] = _mm_madd_epi16(u[10], k__cospi_p11_m21);
- v[23] = _mm_madd_epi16(u[11], k__cospi_p11_m21);
- v[24] = _mm_madd_epi16(u[12], k__cospi_p25_p07);
- v[25] = _mm_madd_epi16(u[13], k__cospi_p25_p07);
- v[26] = _mm_madd_epi16(u[12], k__cospi_p07_m25);
- v[27] = _mm_madd_epi16(u[13], k__cospi_p07_m25);
- v[28] = _mm_madd_epi16(u[14], k__cospi_p29_p03);
- v[29] = _mm_madd_epi16(u[15], k__cospi_p29_p03);
- v[30] = _mm_madd_epi16(u[14], k__cospi_p03_m29);
- v[31] = _mm_madd_epi16(u[15], k__cospi_p03_m29);
-
- u[0] = _mm_add_epi32(v[0], v[16]);
- u[1] = _mm_add_epi32(v[1], v[17]);
- u[2] = _mm_add_epi32(v[2], v[18]);
- u[3] = _mm_add_epi32(v[3], v[19]);
- u[4] = _mm_add_epi32(v[4], v[20]);
- u[5] = _mm_add_epi32(v[5], v[21]);
- u[6] = _mm_add_epi32(v[6], v[22]);
- u[7] = _mm_add_epi32(v[7], v[23]);
- u[8] = _mm_add_epi32(v[8], v[24]);
- u[9] = _mm_add_epi32(v[9], v[25]);
- u[10] = _mm_add_epi32(v[10], v[26]);
- u[11] = _mm_add_epi32(v[11], v[27]);
- u[12] = _mm_add_epi32(v[12], v[28]);
- u[13] = _mm_add_epi32(v[13], v[29]);
- u[14] = _mm_add_epi32(v[14], v[30]);
- u[15] = _mm_add_epi32(v[15], v[31]);
- u[16] = _mm_sub_epi32(v[0], v[16]);
- u[17] = _mm_sub_epi32(v[1], v[17]);
- u[18] = _mm_sub_epi32(v[2], v[18]);
- u[19] = _mm_sub_epi32(v[3], v[19]);
- u[20] = _mm_sub_epi32(v[4], v[20]);
- u[21] = _mm_sub_epi32(v[5], v[21]);
- u[22] = _mm_sub_epi32(v[6], v[22]);
- u[23] = _mm_sub_epi32(v[7], v[23]);
- u[24] = _mm_sub_epi32(v[8], v[24]);
- u[25] = _mm_sub_epi32(v[9], v[25]);
- u[26] = _mm_sub_epi32(v[10], v[26]);
- u[27] = _mm_sub_epi32(v[11], v[27]);
- u[28] = _mm_sub_epi32(v[12], v[28]);
- u[29] = _mm_sub_epi32(v[13], v[29]);
- u[30] = _mm_sub_epi32(v[14], v[30]);
- u[31] = _mm_sub_epi32(v[15], v[31]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
- v[8] = _mm_add_epi32(u[8], k__DCT_CONST_ROUNDING);
- v[9] = _mm_add_epi32(u[9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
- v[16] = _mm_add_epi32(u[16], k__DCT_CONST_ROUNDING);
- v[17] = _mm_add_epi32(u[17], k__DCT_CONST_ROUNDING);
- v[18] = _mm_add_epi32(u[18], k__DCT_CONST_ROUNDING);
- v[19] = _mm_add_epi32(u[19], k__DCT_CONST_ROUNDING);
- v[20] = _mm_add_epi32(u[20], k__DCT_CONST_ROUNDING);
- v[21] = _mm_add_epi32(u[21], k__DCT_CONST_ROUNDING);
- v[22] = _mm_add_epi32(u[22], k__DCT_CONST_ROUNDING);
- v[23] = _mm_add_epi32(u[23], k__DCT_CONST_ROUNDING);
- v[24] = _mm_add_epi32(u[24], k__DCT_CONST_ROUNDING);
- v[25] = _mm_add_epi32(u[25], k__DCT_CONST_ROUNDING);
- v[26] = _mm_add_epi32(u[26], k__DCT_CONST_ROUNDING);
- v[27] = _mm_add_epi32(u[27], k__DCT_CONST_ROUNDING);
- v[28] = _mm_add_epi32(u[28], k__DCT_CONST_ROUNDING);
- v[29] = _mm_add_epi32(u[29], k__DCT_CONST_ROUNDING);
- v[30] = _mm_add_epi32(u[30], k__DCT_CONST_ROUNDING);
- v[31] = _mm_add_epi32(u[31], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
- u[8] = _mm_srai_epi32(v[8], DCT_CONST_BITS);
- u[9] = _mm_srai_epi32(v[9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
- u[16] = _mm_srai_epi32(v[16], DCT_CONST_BITS);
- u[17] = _mm_srai_epi32(v[17], DCT_CONST_BITS);
- u[18] = _mm_srai_epi32(v[18], DCT_CONST_BITS);
- u[19] = _mm_srai_epi32(v[19], DCT_CONST_BITS);
- u[20] = _mm_srai_epi32(v[20], DCT_CONST_BITS);
- u[21] = _mm_srai_epi32(v[21], DCT_CONST_BITS);
- u[22] = _mm_srai_epi32(v[22], DCT_CONST_BITS);
- u[23] = _mm_srai_epi32(v[23], DCT_CONST_BITS);
- u[24] = _mm_srai_epi32(v[24], DCT_CONST_BITS);
- u[25] = _mm_srai_epi32(v[25], DCT_CONST_BITS);
- u[26] = _mm_srai_epi32(v[26], DCT_CONST_BITS);
- u[27] = _mm_srai_epi32(v[27], DCT_CONST_BITS);
- u[28] = _mm_srai_epi32(v[28], DCT_CONST_BITS);
- u[29] = _mm_srai_epi32(v[29], DCT_CONST_BITS);
- u[30] = _mm_srai_epi32(v[30], DCT_CONST_BITS);
- u[31] = _mm_srai_epi32(v[31], DCT_CONST_BITS);
-
- s[0] = _mm_packs_epi32(u[0], u[1]);
- s[1] = _mm_packs_epi32(u[2], u[3]);
- s[2] = _mm_packs_epi32(u[4], u[5]);
- s[3] = _mm_packs_epi32(u[6], u[7]);
- s[4] = _mm_packs_epi32(u[8], u[9]);
- s[5] = _mm_packs_epi32(u[10], u[11]);
- s[6] = _mm_packs_epi32(u[12], u[13]);
- s[7] = _mm_packs_epi32(u[14], u[15]);
- s[8] = _mm_packs_epi32(u[16], u[17]);
- s[9] = _mm_packs_epi32(u[18], u[19]);
- s[10] = _mm_packs_epi32(u[20], u[21]);
- s[11] = _mm_packs_epi32(u[22], u[23]);
- s[12] = _mm_packs_epi32(u[24], u[25]);
- s[13] = _mm_packs_epi32(u[26], u[27]);
- s[14] = _mm_packs_epi32(u[28], u[29]);
- s[15] = _mm_packs_epi32(u[30], u[31]);
-
- // stage 2
- u[0] = _mm_unpacklo_epi16(s[8], s[9]);
- u[1] = _mm_unpackhi_epi16(s[8], s[9]);
- u[2] = _mm_unpacklo_epi16(s[10], s[11]);
- u[3] = _mm_unpackhi_epi16(s[10], s[11]);
- u[4] = _mm_unpacklo_epi16(s[12], s[13]);
- u[5] = _mm_unpackhi_epi16(s[12], s[13]);
- u[6] = _mm_unpacklo_epi16(s[14], s[15]);
- u[7] = _mm_unpackhi_epi16(s[14], s[15]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p04_p28);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p04_p28);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p28_m04);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p28_m04);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p20_p12);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p20_p12);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p12_m20);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p12_m20);
- v[8] = _mm_madd_epi16(u[4], k__cospi_m28_p04);
- v[9] = _mm_madd_epi16(u[5], k__cospi_m28_p04);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p04_p28);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p04_p28);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m12_p20);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m12_p20);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p20_p12);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p20_p12);
-
- u[0] = _mm_add_epi32(v[0], v[8]);
- u[1] = _mm_add_epi32(v[1], v[9]);
- u[2] = _mm_add_epi32(v[2], v[10]);
- u[3] = _mm_add_epi32(v[3], v[11]);
- u[4] = _mm_add_epi32(v[4], v[12]);
- u[5] = _mm_add_epi32(v[5], v[13]);
- u[6] = _mm_add_epi32(v[6], v[14]);
- u[7] = _mm_add_epi32(v[7], v[15]);
- u[8] = _mm_sub_epi32(v[0], v[8]);
- u[9] = _mm_sub_epi32(v[1], v[9]);
- u[10] = _mm_sub_epi32(v[2], v[10]);
- u[11] = _mm_sub_epi32(v[3], v[11]);
- u[12] = _mm_sub_epi32(v[4], v[12]);
- u[13] = _mm_sub_epi32(v[5], v[13]);
- u[14] = _mm_sub_epi32(v[6], v[14]);
- u[15] = _mm_sub_epi32(v[7], v[15]);
-
- v[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- v[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- v[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- v[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- v[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- v[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- v[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- v[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
- v[8] = _mm_add_epi32(u[8], k__DCT_CONST_ROUNDING);
- v[9] = _mm_add_epi32(u[9], k__DCT_CONST_ROUNDING);
- v[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- v[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- v[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- v[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- v[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- v[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- u[0] = _mm_srai_epi32(v[0], DCT_CONST_BITS);
- u[1] = _mm_srai_epi32(v[1], DCT_CONST_BITS);
- u[2] = _mm_srai_epi32(v[2], DCT_CONST_BITS);
- u[3] = _mm_srai_epi32(v[3], DCT_CONST_BITS);
- u[4] = _mm_srai_epi32(v[4], DCT_CONST_BITS);
- u[5] = _mm_srai_epi32(v[5], DCT_CONST_BITS);
- u[6] = _mm_srai_epi32(v[6], DCT_CONST_BITS);
- u[7] = _mm_srai_epi32(v[7], DCT_CONST_BITS);
- u[8] = _mm_srai_epi32(v[8], DCT_CONST_BITS);
- u[9] = _mm_srai_epi32(v[9], DCT_CONST_BITS);
- u[10] = _mm_srai_epi32(v[10], DCT_CONST_BITS);
- u[11] = _mm_srai_epi32(v[11], DCT_CONST_BITS);
- u[12] = _mm_srai_epi32(v[12], DCT_CONST_BITS);
- u[13] = _mm_srai_epi32(v[13], DCT_CONST_BITS);
- u[14] = _mm_srai_epi32(v[14], DCT_CONST_BITS);
- u[15] = _mm_srai_epi32(v[15], DCT_CONST_BITS);
-
- x[0] = _mm_add_epi16(s[0], s[4]);
- x[1] = _mm_add_epi16(s[1], s[5]);
- x[2] = _mm_add_epi16(s[2], s[6]);
- x[3] = _mm_add_epi16(s[3], s[7]);
- x[4] = _mm_sub_epi16(s[0], s[4]);
- x[5] = _mm_sub_epi16(s[1], s[5]);
- x[6] = _mm_sub_epi16(s[2], s[6]);
- x[7] = _mm_sub_epi16(s[3], s[7]);
- x[8] = _mm_packs_epi32(u[0], u[1]);
- x[9] = _mm_packs_epi32(u[2], u[3]);
- x[10] = _mm_packs_epi32(u[4], u[5]);
- x[11] = _mm_packs_epi32(u[6], u[7]);
- x[12] = _mm_packs_epi32(u[8], u[9]);
- x[13] = _mm_packs_epi32(u[10], u[11]);
- x[14] = _mm_packs_epi32(u[12], u[13]);
- x[15] = _mm_packs_epi32(u[14], u[15]);
-
- // stage 3
- u[0] = _mm_unpacklo_epi16(x[4], x[5]);
- u[1] = _mm_unpackhi_epi16(x[4], x[5]);
- u[2] = _mm_unpacklo_epi16(x[6], x[7]);
- u[3] = _mm_unpackhi_epi16(x[6], x[7]);
- u[4] = _mm_unpacklo_epi16(x[12], x[13]);
- u[5] = _mm_unpackhi_epi16(x[12], x[13]);
- u[6] = _mm_unpacklo_epi16(x[14], x[15]);
- u[7] = _mm_unpackhi_epi16(x[14], x[15]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_p08_p24);
- v[1] = _mm_madd_epi16(u[1], k__cospi_p08_p24);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p24_m08);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p24_m08);
- v[4] = _mm_madd_epi16(u[2], k__cospi_m24_p08);
- v[5] = _mm_madd_epi16(u[3], k__cospi_m24_p08);
- v[6] = _mm_madd_epi16(u[2], k__cospi_p08_p24);
- v[7] = _mm_madd_epi16(u[3], k__cospi_p08_p24);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p08_p24);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p08_p24);
- v[10] = _mm_madd_epi16(u[4], k__cospi_p24_m08);
- v[11] = _mm_madd_epi16(u[5], k__cospi_p24_m08);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m24_p08);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m24_p08);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p08_p24);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p08_p24);
-
- u[0] = _mm_add_epi32(v[0], v[4]);
- u[1] = _mm_add_epi32(v[1], v[5]);
- u[2] = _mm_add_epi32(v[2], v[6]);
- u[3] = _mm_add_epi32(v[3], v[7]);
- u[4] = _mm_sub_epi32(v[0], v[4]);
- u[5] = _mm_sub_epi32(v[1], v[5]);
- u[6] = _mm_sub_epi32(v[2], v[6]);
- u[7] = _mm_sub_epi32(v[3], v[7]);
- u[8] = _mm_add_epi32(v[8], v[12]);
- u[9] = _mm_add_epi32(v[9], v[13]);
- u[10] = _mm_add_epi32(v[10], v[14]);
- u[11] = _mm_add_epi32(v[11], v[15]);
- u[12] = _mm_sub_epi32(v[8], v[12]);
- u[13] = _mm_sub_epi32(v[9], v[13]);
- u[14] = _mm_sub_epi32(v[10], v[14]);
- u[15] = _mm_sub_epi32(v[11], v[15]);
-
- u[0] = _mm_add_epi32(u[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(u[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(u[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(u[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(u[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(u[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(u[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(u[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(u[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(u[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(u[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(u[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(u[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(u[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(u[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(u[15], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- v[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- v[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- v[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- v[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- v[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- v[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- v[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- v[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- s[0] = _mm_add_epi16(x[0], x[2]);
- s[1] = _mm_add_epi16(x[1], x[3]);
- s[2] = _mm_sub_epi16(x[0], x[2]);
- s[3] = _mm_sub_epi16(x[1], x[3]);
- s[4] = _mm_packs_epi32(v[0], v[1]);
- s[5] = _mm_packs_epi32(v[2], v[3]);
- s[6] = _mm_packs_epi32(v[4], v[5]);
- s[7] = _mm_packs_epi32(v[6], v[7]);
- s[8] = _mm_add_epi16(x[8], x[10]);
- s[9] = _mm_add_epi16(x[9], x[11]);
- s[10] = _mm_sub_epi16(x[8], x[10]);
- s[11] = _mm_sub_epi16(x[9], x[11]);
- s[12] = _mm_packs_epi32(v[8], v[9]);
- s[13] = _mm_packs_epi32(v[10], v[11]);
- s[14] = _mm_packs_epi32(v[12], v[13]);
- s[15] = _mm_packs_epi32(v[14], v[15]);
-
- // stage 4
- u[0] = _mm_unpacklo_epi16(s[2], s[3]);
- u[1] = _mm_unpackhi_epi16(s[2], s[3]);
- u[2] = _mm_unpacklo_epi16(s[6], s[7]);
- u[3] = _mm_unpackhi_epi16(s[6], s[7]);
- u[4] = _mm_unpacklo_epi16(s[10], s[11]);
- u[5] = _mm_unpackhi_epi16(s[10], s[11]);
- u[6] = _mm_unpacklo_epi16(s[14], s[15]);
- u[7] = _mm_unpackhi_epi16(s[14], s[15]);
-
- v[0] = _mm_madd_epi16(u[0], k__cospi_m16_m16);
- v[1] = _mm_madd_epi16(u[1], k__cospi_m16_m16);
- v[2] = _mm_madd_epi16(u[0], k__cospi_p16_m16);
- v[3] = _mm_madd_epi16(u[1], k__cospi_p16_m16);
- v[4] = _mm_madd_epi16(u[2], k__cospi_p16_p16);
- v[5] = _mm_madd_epi16(u[3], k__cospi_p16_p16);
- v[6] = _mm_madd_epi16(u[2], k__cospi_m16_p16);
- v[7] = _mm_madd_epi16(u[3], k__cospi_m16_p16);
- v[8] = _mm_madd_epi16(u[4], k__cospi_p16_p16);
- v[9] = _mm_madd_epi16(u[5], k__cospi_p16_p16);
- v[10] = _mm_madd_epi16(u[4], k__cospi_m16_p16);
- v[11] = _mm_madd_epi16(u[5], k__cospi_m16_p16);
- v[12] = _mm_madd_epi16(u[6], k__cospi_m16_m16);
- v[13] = _mm_madd_epi16(u[7], k__cospi_m16_m16);
- v[14] = _mm_madd_epi16(u[6], k__cospi_p16_m16);
- v[15] = _mm_madd_epi16(u[7], k__cospi_p16_m16);
-
- u[0] = _mm_add_epi32(v[0], k__DCT_CONST_ROUNDING);
- u[1] = _mm_add_epi32(v[1], k__DCT_CONST_ROUNDING);
- u[2] = _mm_add_epi32(v[2], k__DCT_CONST_ROUNDING);
- u[3] = _mm_add_epi32(v[3], k__DCT_CONST_ROUNDING);
- u[4] = _mm_add_epi32(v[4], k__DCT_CONST_ROUNDING);
- u[5] = _mm_add_epi32(v[5], k__DCT_CONST_ROUNDING);
- u[6] = _mm_add_epi32(v[6], k__DCT_CONST_ROUNDING);
- u[7] = _mm_add_epi32(v[7], k__DCT_CONST_ROUNDING);
- u[8] = _mm_add_epi32(v[8], k__DCT_CONST_ROUNDING);
- u[9] = _mm_add_epi32(v[9], k__DCT_CONST_ROUNDING);
- u[10] = _mm_add_epi32(v[10], k__DCT_CONST_ROUNDING);
- u[11] = _mm_add_epi32(v[11], k__DCT_CONST_ROUNDING);
- u[12] = _mm_add_epi32(v[12], k__DCT_CONST_ROUNDING);
- u[13] = _mm_add_epi32(v[13], k__DCT_CONST_ROUNDING);
- u[14] = _mm_add_epi32(v[14], k__DCT_CONST_ROUNDING);
- u[15] = _mm_add_epi32(v[15], k__DCT_CONST_ROUNDING);
-
- v[0] = _mm_srai_epi32(u[0], DCT_CONST_BITS);
- v[1] = _mm_srai_epi32(u[1], DCT_CONST_BITS);
- v[2] = _mm_srai_epi32(u[2], DCT_CONST_BITS);
- v[3] = _mm_srai_epi32(u[3], DCT_CONST_BITS);
- v[4] = _mm_srai_epi32(u[4], DCT_CONST_BITS);
- v[5] = _mm_srai_epi32(u[5], DCT_CONST_BITS);
- v[6] = _mm_srai_epi32(u[6], DCT_CONST_BITS);
- v[7] = _mm_srai_epi32(u[7], DCT_CONST_BITS);
- v[8] = _mm_srai_epi32(u[8], DCT_CONST_BITS);
- v[9] = _mm_srai_epi32(u[9], DCT_CONST_BITS);
- v[10] = _mm_srai_epi32(u[10], DCT_CONST_BITS);
- v[11] = _mm_srai_epi32(u[11], DCT_CONST_BITS);
- v[12] = _mm_srai_epi32(u[12], DCT_CONST_BITS);
- v[13] = _mm_srai_epi32(u[13], DCT_CONST_BITS);
- v[14] = _mm_srai_epi32(u[14], DCT_CONST_BITS);
- v[15] = _mm_srai_epi32(u[15], DCT_CONST_BITS);
-
- in[0] = s[0];
- in[1] = _mm_sub_epi16(kZero, s[8]);
- in[2] = s[12];
- in[3] = _mm_sub_epi16(kZero, s[4]);
- in[4] = _mm_packs_epi32(v[4], v[5]);
- in[5] = _mm_packs_epi32(v[12], v[13]);
- in[6] = _mm_packs_epi32(v[8], v[9]);
- in[7] = _mm_packs_epi32(v[0], v[1]);
- in[8] = _mm_packs_epi32(v[2], v[3]);
- in[9] = _mm_packs_epi32(v[10], v[11]);
- in[10] = _mm_packs_epi32(v[14], v[15]);
- in[11] = _mm_packs_epi32(v[6], v[7]);
- in[12] = s[5];
- in[13] = _mm_sub_epi16(kZero, s[13]);
- in[14] = s[9];
- in[15] = _mm_sub_epi16(kZero, s[1]);
-}
-
-static void fdct16_sse2(__m128i *in0, __m128i *in1) {
- fdct16_8col(in0);
- fdct16_8col(in1);
- array_transpose_16x16(in0, in1);
-}
-
-static void fadst16_sse2(__m128i *in0, __m128i *in1) {
- fadst16_8col(in0);
- fadst16_8col(in1);
- array_transpose_16x16(in0, in1);
-}
-
-void vp10_fht16x16_sse2(const int16_t *input, tran_low_t *output,
- int stride, int tx_type) {
- __m128i in0[16], in1[16];
-
- switch (tx_type) {
- case DCT_DCT:
- vpx_fdct16x16_sse2(input, output, stride);
- break;
- case ADST_DCT:
- load_buffer_16x16(input, in0, in1, stride);
- fadst16_sse2(in0, in1);
- right_shift_16x16(in0, in1);
- fdct16_sse2(in0, in1);
- write_buffer_16x16(output, in0, in1, 16);
- break;
- case DCT_ADST:
- load_buffer_16x16(input, in0, in1, stride);
- fdct16_sse2(in0, in1);
- right_shift_16x16(in0, in1);
- fadst16_sse2(in0, in1);
- write_buffer_16x16(output, in0, in1, 16);
- break;
- case ADST_ADST:
- load_buffer_16x16(input, in0, in1, stride);
- fadst16_sse2(in0, in1);
- right_shift_16x16(in0, in1);
- fadst16_sse2(in0, in1);
- write_buffer_16x16(output, in0, in1, 16);
- break;
- default:
- assert(0);
- break;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <assert.h>
-#if defined(_MSC_VER) && _MSC_VER <= 1500
-// Need to include math.h before calling tmmintrin.h/intrin.h
-// in certain versions of MSVS.
-#include <math.h>
-#endif
-#include <tmmintrin.h> // SSSE3
-
-#include "./vp10_rtcd.h"
-#include "vpx_dsp/x86/inv_txfm_sse2.h"
-#include "vpx_dsp/x86/txfm_common_sse2.h"
-
-void vp10_fdct8x8_quant_ssse3(const int16_t *input, int stride,
- int16_t* coeff_ptr, intptr_t n_coeffs,
- int skip_block, const int16_t* zbin_ptr,
- const int16_t* round_ptr, const int16_t* quant_ptr,
- const int16_t* quant_shift_ptr,
- int16_t* qcoeff_ptr,
- int16_t* dqcoeff_ptr, const int16_t* dequant_ptr,
- uint16_t* eob_ptr,
- const int16_t* scan_ptr,
- const int16_t* iscan_ptr) {
- __m128i zero;
- int pass;
- // Constants
- // When we use them, in one case, they are all the same. In all others
- // it's a pair of them that we need to repeat four times. This is done
- // by constructing the 32 bit constant corresponding to that pair.
- const __m128i k__dual_p16_p16 = dual_set_epi16(23170, 23170);
- const __m128i k__cospi_p16_p16 = _mm_set1_epi16((int16_t)cospi_16_64);
- const __m128i k__cospi_p16_m16 = pair_set_epi16(cospi_16_64, -cospi_16_64);
- const __m128i k__cospi_p24_p08 = pair_set_epi16(cospi_24_64, cospi_8_64);
- const __m128i k__cospi_m08_p24 = pair_set_epi16(-cospi_8_64, cospi_24_64);
- const __m128i k__cospi_p28_p04 = pair_set_epi16(cospi_28_64, cospi_4_64);
- const __m128i k__cospi_m04_p28 = pair_set_epi16(-cospi_4_64, cospi_28_64);
- const __m128i k__cospi_p12_p20 = pair_set_epi16(cospi_12_64, cospi_20_64);
- const __m128i k__cospi_m20_p12 = pair_set_epi16(-cospi_20_64, cospi_12_64);
- const __m128i k__DCT_CONST_ROUNDING = _mm_set1_epi32(DCT_CONST_ROUNDING);
- // Load input
- __m128i in0 = _mm_load_si128((const __m128i *)(input + 0 * stride));
- __m128i in1 = _mm_load_si128((const __m128i *)(input + 1 * stride));
- __m128i in2 = _mm_load_si128((const __m128i *)(input + 2 * stride));
- __m128i in3 = _mm_load_si128((const __m128i *)(input + 3 * stride));
- __m128i in4 = _mm_load_si128((const __m128i *)(input + 4 * stride));
- __m128i in5 = _mm_load_si128((const __m128i *)(input + 5 * stride));
- __m128i in6 = _mm_load_si128((const __m128i *)(input + 6 * stride));
- __m128i in7 = _mm_load_si128((const __m128i *)(input + 7 * stride));
- __m128i *in[8];
- int index = 0;
-
- (void)scan_ptr;
- (void)zbin_ptr;
- (void)quant_shift_ptr;
- (void)coeff_ptr;
-
- // Pre-condition input (shift by two)
- in0 = _mm_slli_epi16(in0, 2);
- in1 = _mm_slli_epi16(in1, 2);
- in2 = _mm_slli_epi16(in2, 2);
- in3 = _mm_slli_epi16(in3, 2);
- in4 = _mm_slli_epi16(in4, 2);
- in5 = _mm_slli_epi16(in5, 2);
- in6 = _mm_slli_epi16(in6, 2);
- in7 = _mm_slli_epi16(in7, 2);
-
- in[0] = &in0;
- in[1] = &in1;
- in[2] = &in2;
- in[3] = &in3;
- in[4] = &in4;
- in[5] = &in5;
- in[6] = &in6;
- in[7] = &in7;
-
- // We do two passes, first the columns, then the rows. The results of the
- // first pass are transposed so that the same column code can be reused. The
- // results of the second pass are also transposed so that the rows (processed
- // as columns) are put back in row positions.
- for (pass = 0; pass < 2; pass++) {
- // To store results of each pass before the transpose.
- __m128i res0, res1, res2, res3, res4, res5, res6, res7;
- // Add/subtract
- const __m128i q0 = _mm_add_epi16(in0, in7);
- const __m128i q1 = _mm_add_epi16(in1, in6);
- const __m128i q2 = _mm_add_epi16(in2, in5);
- const __m128i q3 = _mm_add_epi16(in3, in4);
- const __m128i q4 = _mm_sub_epi16(in3, in4);
- const __m128i q5 = _mm_sub_epi16(in2, in5);
- const __m128i q6 = _mm_sub_epi16(in1, in6);
- const __m128i q7 = _mm_sub_epi16(in0, in7);
- // Work on first four results
- {
- // Add/subtract
- const __m128i r0 = _mm_add_epi16(q0, q3);
- const __m128i r1 = _mm_add_epi16(q1, q2);
- const __m128i r2 = _mm_sub_epi16(q1, q2);
- const __m128i r3 = _mm_sub_epi16(q0, q3);
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i t0 = _mm_unpacklo_epi16(r0, r1);
- const __m128i t1 = _mm_unpackhi_epi16(r0, r1);
- const __m128i t2 = _mm_unpacklo_epi16(r2, r3);
- const __m128i t3 = _mm_unpackhi_epi16(r2, r3);
-
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_p16_p16);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_p16_p16);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_p16_m16);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_p16_m16);
-
- const __m128i u4 = _mm_madd_epi16(t2, k__cospi_p24_p08);
- const __m128i u5 = _mm_madd_epi16(t3, k__cospi_p24_p08);
- const __m128i u6 = _mm_madd_epi16(t2, k__cospi_m08_p24);
- const __m128i u7 = _mm_madd_epi16(t3, k__cospi_m08_p24);
- // dct_const_round_shift
-
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
-
- const __m128i v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- const __m128i v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- const __m128i v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- const __m128i v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
-
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
-
- const __m128i w4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- const __m128i w5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- const __m128i w6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- const __m128i w7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- // Combine
-
- res0 = _mm_packs_epi32(w0, w1);
- res4 = _mm_packs_epi32(w2, w3);
- res2 = _mm_packs_epi32(w4, w5);
- res6 = _mm_packs_epi32(w6, w7);
- }
- // Work on next four results
- {
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i d0 = _mm_sub_epi16(q6, q5);
- const __m128i d1 = _mm_add_epi16(q6, q5);
- const __m128i r0 = _mm_mulhrs_epi16(d0, k__dual_p16_p16);
- const __m128i r1 = _mm_mulhrs_epi16(d1, k__dual_p16_p16);
-
- // Add/subtract
- const __m128i x0 = _mm_add_epi16(q4, r0);
- const __m128i x1 = _mm_sub_epi16(q4, r0);
- const __m128i x2 = _mm_sub_epi16(q7, r1);
- const __m128i x3 = _mm_add_epi16(q7, r1);
- // Interleave to do the multiply by constants which gets us into 32bits
- const __m128i t0 = _mm_unpacklo_epi16(x0, x3);
- const __m128i t1 = _mm_unpackhi_epi16(x0, x3);
- const __m128i t2 = _mm_unpacklo_epi16(x1, x2);
- const __m128i t3 = _mm_unpackhi_epi16(x1, x2);
- const __m128i u0 = _mm_madd_epi16(t0, k__cospi_p28_p04);
- const __m128i u1 = _mm_madd_epi16(t1, k__cospi_p28_p04);
- const __m128i u2 = _mm_madd_epi16(t0, k__cospi_m04_p28);
- const __m128i u3 = _mm_madd_epi16(t1, k__cospi_m04_p28);
- const __m128i u4 = _mm_madd_epi16(t2, k__cospi_p12_p20);
- const __m128i u5 = _mm_madd_epi16(t3, k__cospi_p12_p20);
- const __m128i u6 = _mm_madd_epi16(t2, k__cospi_m20_p12);
- const __m128i u7 = _mm_madd_epi16(t3, k__cospi_m20_p12);
- // dct_const_round_shift
- const __m128i v0 = _mm_add_epi32(u0, k__DCT_CONST_ROUNDING);
- const __m128i v1 = _mm_add_epi32(u1, k__DCT_CONST_ROUNDING);
- const __m128i v2 = _mm_add_epi32(u2, k__DCT_CONST_ROUNDING);
- const __m128i v3 = _mm_add_epi32(u3, k__DCT_CONST_ROUNDING);
- const __m128i v4 = _mm_add_epi32(u4, k__DCT_CONST_ROUNDING);
- const __m128i v5 = _mm_add_epi32(u5, k__DCT_CONST_ROUNDING);
- const __m128i v6 = _mm_add_epi32(u6, k__DCT_CONST_ROUNDING);
- const __m128i v7 = _mm_add_epi32(u7, k__DCT_CONST_ROUNDING);
- const __m128i w0 = _mm_srai_epi32(v0, DCT_CONST_BITS);
- const __m128i w1 = _mm_srai_epi32(v1, DCT_CONST_BITS);
- const __m128i w2 = _mm_srai_epi32(v2, DCT_CONST_BITS);
- const __m128i w3 = _mm_srai_epi32(v3, DCT_CONST_BITS);
- const __m128i w4 = _mm_srai_epi32(v4, DCT_CONST_BITS);
- const __m128i w5 = _mm_srai_epi32(v5, DCT_CONST_BITS);
- const __m128i w6 = _mm_srai_epi32(v6, DCT_CONST_BITS);
- const __m128i w7 = _mm_srai_epi32(v7, DCT_CONST_BITS);
- // Combine
- res1 = _mm_packs_epi32(w0, w1);
- res7 = _mm_packs_epi32(w2, w3);
- res5 = _mm_packs_epi32(w4, w5);
- res3 = _mm_packs_epi32(w6, w7);
- }
- // Transpose the 8x8.
- {
- // 00 01 02 03 04 05 06 07
- // 10 11 12 13 14 15 16 17
- // 20 21 22 23 24 25 26 27
- // 30 31 32 33 34 35 36 37
- // 40 41 42 43 44 45 46 47
- // 50 51 52 53 54 55 56 57
- // 60 61 62 63 64 65 66 67
- // 70 71 72 73 74 75 76 77
- const __m128i tr0_0 = _mm_unpacklo_epi16(res0, res1);
- const __m128i tr0_1 = _mm_unpacklo_epi16(res2, res3);
- const __m128i tr0_2 = _mm_unpackhi_epi16(res0, res1);
- const __m128i tr0_3 = _mm_unpackhi_epi16(res2, res3);
- const __m128i tr0_4 = _mm_unpacklo_epi16(res4, res5);
- const __m128i tr0_5 = _mm_unpacklo_epi16(res6, res7);
- const __m128i tr0_6 = _mm_unpackhi_epi16(res4, res5);
- const __m128i tr0_7 = _mm_unpackhi_epi16(res6, res7);
- // 00 10 01 11 02 12 03 13
- // 20 30 21 31 22 32 23 33
- // 04 14 05 15 06 16 07 17
- // 24 34 25 35 26 36 27 37
- // 40 50 41 51 42 52 43 53
- // 60 70 61 71 62 72 63 73
- // 54 54 55 55 56 56 57 57
- // 64 74 65 75 66 76 67 77
- const __m128i tr1_0 = _mm_unpacklo_epi32(tr0_0, tr0_1);
- const __m128i tr1_1 = _mm_unpacklo_epi32(tr0_2, tr0_3);
- const __m128i tr1_2 = _mm_unpackhi_epi32(tr0_0, tr0_1);
- const __m128i tr1_3 = _mm_unpackhi_epi32(tr0_2, tr0_3);
- const __m128i tr1_4 = _mm_unpacklo_epi32(tr0_4, tr0_5);
- const __m128i tr1_5 = _mm_unpacklo_epi32(tr0_6, tr0_7);
- const __m128i tr1_6 = _mm_unpackhi_epi32(tr0_4, tr0_5);
- const __m128i tr1_7 = _mm_unpackhi_epi32(tr0_6, tr0_7);
- // 00 10 20 30 01 11 21 31
- // 40 50 60 70 41 51 61 71
- // 02 12 22 32 03 13 23 33
- // 42 52 62 72 43 53 63 73
- // 04 14 24 34 05 15 21 36
- // 44 54 64 74 45 55 61 76
- // 06 16 26 36 07 17 27 37
- // 46 56 66 76 47 57 67 77
- in0 = _mm_unpacklo_epi64(tr1_0, tr1_4);
- in1 = _mm_unpackhi_epi64(tr1_0, tr1_4);
- in2 = _mm_unpacklo_epi64(tr1_2, tr1_6);
- in3 = _mm_unpackhi_epi64(tr1_2, tr1_6);
- in4 = _mm_unpacklo_epi64(tr1_1, tr1_5);
- in5 = _mm_unpackhi_epi64(tr1_1, tr1_5);
- in6 = _mm_unpacklo_epi64(tr1_3, tr1_7);
- in7 = _mm_unpackhi_epi64(tr1_3, tr1_7);
- // 00 10 20 30 40 50 60 70
- // 01 11 21 31 41 51 61 71
- // 02 12 22 32 42 52 62 72
- // 03 13 23 33 43 53 63 73
- // 04 14 24 34 44 54 64 74
- // 05 15 25 35 45 55 65 75
- // 06 16 26 36 46 56 66 76
- // 07 17 27 37 47 57 67 77
- }
- }
- // Post-condition output and store it
- {
- // Post-condition (division by two)
- // division of two 16 bits signed numbers using shifts
- // n / 2 = (n - (n >> 15)) >> 1
- const __m128i sign_in0 = _mm_srai_epi16(in0, 15);
- const __m128i sign_in1 = _mm_srai_epi16(in1, 15);
- const __m128i sign_in2 = _mm_srai_epi16(in2, 15);
- const __m128i sign_in3 = _mm_srai_epi16(in3, 15);
- const __m128i sign_in4 = _mm_srai_epi16(in4, 15);
- const __m128i sign_in5 = _mm_srai_epi16(in5, 15);
- const __m128i sign_in6 = _mm_srai_epi16(in6, 15);
- const __m128i sign_in7 = _mm_srai_epi16(in7, 15);
- in0 = _mm_sub_epi16(in0, sign_in0);
- in1 = _mm_sub_epi16(in1, sign_in1);
- in2 = _mm_sub_epi16(in2, sign_in2);
- in3 = _mm_sub_epi16(in3, sign_in3);
- in4 = _mm_sub_epi16(in4, sign_in4);
- in5 = _mm_sub_epi16(in5, sign_in5);
- in6 = _mm_sub_epi16(in6, sign_in6);
- in7 = _mm_sub_epi16(in7, sign_in7);
- in0 = _mm_srai_epi16(in0, 1);
- in1 = _mm_srai_epi16(in1, 1);
- in2 = _mm_srai_epi16(in2, 1);
- in3 = _mm_srai_epi16(in3, 1);
- in4 = _mm_srai_epi16(in4, 1);
- in5 = _mm_srai_epi16(in5, 1);
- in6 = _mm_srai_epi16(in6, 1);
- in7 = _mm_srai_epi16(in7, 1);
- }
-
- iscan_ptr += n_coeffs;
- qcoeff_ptr += n_coeffs;
- dqcoeff_ptr += n_coeffs;
- n_coeffs = -n_coeffs;
- zero = _mm_setzero_si128();
-
- if (!skip_block) {
- __m128i eob;
- __m128i round, quant, dequant, thr;
- int16_t nzflag;
- {
- __m128i coeff0, coeff1;
-
- // Setup global values
- {
- round = _mm_load_si128((const __m128i*)round_ptr);
- quant = _mm_load_si128((const __m128i*)quant_ptr);
- dequant = _mm_load_si128((const __m128i*)dequant_ptr);
- }
-
- {
- __m128i coeff0_sign, coeff1_sign;
- __m128i qcoeff0, qcoeff1;
- __m128i qtmp0, qtmp1;
- // Do DC and first 15 AC
- coeff0 = *in[0];
- coeff1 = *in[1];
-
- // Poor man's sign extract
- coeff0_sign = _mm_srai_epi16(coeff0, 15);
- coeff1_sign = _mm_srai_epi16(coeff1, 15);
- qcoeff0 = _mm_xor_si128(coeff0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(coeff1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- qcoeff0 = _mm_adds_epi16(qcoeff0, round);
- round = _mm_unpackhi_epi64(round, round);
- qcoeff1 = _mm_adds_epi16(qcoeff1, round);
- qtmp0 = _mm_mulhi_epi16(qcoeff0, quant);
- quant = _mm_unpackhi_epi64(quant, quant);
- qtmp1 = _mm_mulhi_epi16(qcoeff1, quant);
-
- // Reinsert signs
- qcoeff0 = _mm_xor_si128(qtmp0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(qtmp1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), qcoeff0);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, qcoeff1);
-
- coeff0 = _mm_mullo_epi16(qcoeff0, dequant);
- dequant = _mm_unpackhi_epi64(dequant, dequant);
- coeff1 = _mm_mullo_epi16(qcoeff1, dequant);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), coeff0);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, coeff1);
- }
-
- {
- // Scan for eob
- __m128i zero_coeff0, zero_coeff1;
- __m128i nzero_coeff0, nzero_coeff1;
- __m128i iscan0, iscan1;
- __m128i eob1;
- zero_coeff0 = _mm_cmpeq_epi16(coeff0, zero);
- zero_coeff1 = _mm_cmpeq_epi16(coeff1, zero);
- nzero_coeff0 = _mm_cmpeq_epi16(zero_coeff0, zero);
- nzero_coeff1 = _mm_cmpeq_epi16(zero_coeff1, zero);
- iscan0 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs));
- iscan1 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs) + 1);
- // Add one to convert from indices to counts
- iscan0 = _mm_sub_epi16(iscan0, nzero_coeff0);
- iscan1 = _mm_sub_epi16(iscan1, nzero_coeff1);
- eob = _mm_and_si128(iscan0, nzero_coeff0);
- eob1 = _mm_and_si128(iscan1, nzero_coeff1);
- eob = _mm_max_epi16(eob, eob1);
- }
- n_coeffs += 8 * 2;
- }
-
- // AC only loop
- index = 2;
- thr = _mm_srai_epi16(dequant, 1);
- while (n_coeffs < 0) {
- __m128i coeff0, coeff1;
- {
- __m128i coeff0_sign, coeff1_sign;
- __m128i qcoeff0, qcoeff1;
- __m128i qtmp0, qtmp1;
-
- assert(index < (int)(sizeof(in) / sizeof(in[0])) - 1);
- coeff0 = *in[index];
- coeff1 = *in[index + 1];
-
- // Poor man's sign extract
- coeff0_sign = _mm_srai_epi16(coeff0, 15);
- coeff1_sign = _mm_srai_epi16(coeff1, 15);
- qcoeff0 = _mm_xor_si128(coeff0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(coeff1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- nzflag = _mm_movemask_epi8(_mm_cmpgt_epi16(qcoeff0, thr)) |
- _mm_movemask_epi8(_mm_cmpgt_epi16(qcoeff1, thr));
-
- if (nzflag) {
- qcoeff0 = _mm_adds_epi16(qcoeff0, round);
- qcoeff1 = _mm_adds_epi16(qcoeff1, round);
- qtmp0 = _mm_mulhi_epi16(qcoeff0, quant);
- qtmp1 = _mm_mulhi_epi16(qcoeff1, quant);
-
- // Reinsert signs
- qcoeff0 = _mm_xor_si128(qtmp0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(qtmp1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), qcoeff0);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, qcoeff1);
-
- coeff0 = _mm_mullo_epi16(qcoeff0, dequant);
- coeff1 = _mm_mullo_epi16(qcoeff1, dequant);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), coeff0);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, coeff1);
- } else {
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, zero);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, zero);
- }
- }
-
- if (nzflag) {
- // Scan for eob
- __m128i zero_coeff0, zero_coeff1;
- __m128i nzero_coeff0, nzero_coeff1;
- __m128i iscan0, iscan1;
- __m128i eob0, eob1;
- zero_coeff0 = _mm_cmpeq_epi16(coeff0, zero);
- zero_coeff1 = _mm_cmpeq_epi16(coeff1, zero);
- nzero_coeff0 = _mm_cmpeq_epi16(zero_coeff0, zero);
- nzero_coeff1 = _mm_cmpeq_epi16(zero_coeff1, zero);
- iscan0 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs));
- iscan1 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs) + 1);
- // Add one to convert from indices to counts
- iscan0 = _mm_sub_epi16(iscan0, nzero_coeff0);
- iscan1 = _mm_sub_epi16(iscan1, nzero_coeff1);
- eob0 = _mm_and_si128(iscan0, nzero_coeff0);
- eob1 = _mm_and_si128(iscan1, nzero_coeff1);
- eob0 = _mm_max_epi16(eob0, eob1);
- eob = _mm_max_epi16(eob, eob0);
- }
- n_coeffs += 8 * 2;
- index += 2;
- }
-
- // Accumulate EOB
- {
- __m128i eob_shuffled;
- eob_shuffled = _mm_shuffle_epi32(eob, 0xe);
- eob = _mm_max_epi16(eob, eob_shuffled);
- eob_shuffled = _mm_shufflelo_epi16(eob, 0xe);
- eob = _mm_max_epi16(eob, eob_shuffled);
- eob_shuffled = _mm_shufflelo_epi16(eob, 0x1);
- eob = _mm_max_epi16(eob, eob_shuffled);
- *eob_ptr = _mm_extract_epi16(eob, 1);
- }
- } else {
- do {
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, zero);
- n_coeffs += 8 * 2;
- } while (n_coeffs < 0);
- *eob_ptr = 0;
- }
-}
+++ /dev/null
-;
-; Copyright (c) 2014 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-%define private_prefix vp10
-
-%include "third_party/x86inc/x86inc.asm"
-
-; This file provides SSSE3 version of the forward transformation. Part
-; of the macro definitions are originally derived from the ffmpeg project.
-; The current version applies to x86 64-bit only.
-
-SECTION .text
-
-%if ARCH_X86_64
-; matrix transpose
-%macro INTERLEAVE_2X 4
- punpckh%1 m%4, m%2, m%3
- punpckl%1 m%2, m%3
- SWAP %3, %4
-%endmacro
-
-%macro TRANSPOSE8X8 9
- INTERLEAVE_2X wd, %1, %2, %9
- INTERLEAVE_2X wd, %3, %4, %9
- INTERLEAVE_2X wd, %5, %6, %9
- INTERLEAVE_2X wd, %7, %8, %9
-
- INTERLEAVE_2X dq, %1, %3, %9
- INTERLEAVE_2X dq, %2, %4, %9
- INTERLEAVE_2X dq, %5, %7, %9
- INTERLEAVE_2X dq, %6, %8, %9
-
- INTERLEAVE_2X qdq, %1, %5, %9
- INTERLEAVE_2X qdq, %3, %7, %9
- INTERLEAVE_2X qdq, %2, %6, %9
- INTERLEAVE_2X qdq, %4, %8, %9
-
- SWAP %2, %5
- SWAP %4, %7
-%endmacro
-
-%macro HMD8_1D 0
- psubw m8, m0, m1
- psubw m9, m2, m3
- paddw m0, m1
- paddw m2, m3
- SWAP 1, 8
- SWAP 3, 9
- psubw m8, m4, m5
- psubw m9, m6, m7
- paddw m4, m5
- paddw m6, m7
- SWAP 5, 8
- SWAP 7, 9
-
- psubw m8, m0, m2
- psubw m9, m1, m3
- paddw m0, m2
- paddw m1, m3
- SWAP 2, 8
- SWAP 3, 9
- psubw m8, m4, m6
- psubw m9, m5, m7
- paddw m4, m6
- paddw m5, m7
- SWAP 6, 8
- SWAP 7, 9
-
- psubw m8, m0, m4
- psubw m9, m1, m5
- paddw m0, m4
- paddw m1, m5
- SWAP 4, 8
- SWAP 5, 9
- psubw m8, m2, m6
- psubw m9, m3, m7
- paddw m2, m6
- paddw m3, m7
- SWAP 6, 8
- SWAP 7, 9
-%endmacro
-
-INIT_XMM ssse3
-cglobal hadamard_8x8, 3, 5, 10, input, stride, output
- lea r3, [2 * strideq]
- lea r4, [4 * strideq]
-
- mova m0, [inputq]
- mova m1, [inputq + r3]
- lea inputq, [inputq + r4]
- mova m2, [inputq]
- mova m3, [inputq + r3]
- lea inputq, [inputq + r4]
- mova m4, [inputq]
- mova m5, [inputq + r3]
- lea inputq, [inputq + r4]
- mova m6, [inputq]
- mova m7, [inputq + r3]
-
- HMD8_1D
- TRANSPOSE8X8 0, 1, 2, 3, 4, 5, 6, 7, 9
- HMD8_1D
-
- mova [outputq + 0], m0
- mova [outputq + 16], m1
- mova [outputq + 32], m2
- mova [outputq + 48], m3
- mova [outputq + 64], m4
- mova [outputq + 80], m5
- mova [outputq + 96], m6
- mova [outputq + 112], m7
-
- RET
-%endif
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h>
-
-#include "./vpx_config.h"
-#include "./vp10_rtcd.h"
-
-#include "vpx_ports/emmintrin_compat.h"
-#include "vpx/vpx_integer.h"
-#include "vp10/common/reconinter.h"
-#include "vp10/encoder/context_tree.h"
-#include "vp10/encoder/denoiser.h"
-#include "vpx_mem/vpx_mem.h"
-
-// Compute the sum of all pixel differences of this MB.
-static INLINE int sum_diff_16x1(__m128i acc_diff) {
- const __m128i k_1 = _mm_set1_epi16(1);
- const __m128i acc_diff_lo =
- _mm_srai_epi16(_mm_unpacklo_epi8(acc_diff, acc_diff), 8);
- const __m128i acc_diff_hi =
- _mm_srai_epi16(_mm_unpackhi_epi8(acc_diff, acc_diff), 8);
- const __m128i acc_diff_16 = _mm_add_epi16(acc_diff_lo, acc_diff_hi);
- const __m128i hg_fe_dc_ba = _mm_madd_epi16(acc_diff_16, k_1);
- const __m128i hgfe_dcba =
- _mm_add_epi32(hg_fe_dc_ba, _mm_srli_si128(hg_fe_dc_ba, 8));
- const __m128i hgfedcba =
- _mm_add_epi32(hgfe_dcba, _mm_srli_si128(hgfe_dcba, 4));
- return _mm_cvtsi128_si32(hgfedcba);
-}
-
-// Denoise a 16x1 vector.
-static INLINE __m128i vp10_denoiser_16x1_sse2(const uint8_t *sig,
- const uint8_t *mc_running_avg_y,
- uint8_t *running_avg_y,
- const __m128i *k_0,
- const __m128i *k_4,
- const __m128i *k_8,
- const __m128i *k_16,
- const __m128i *l3,
- const __m128i *l32,
- const __m128i *l21,
- __m128i acc_diff) {
- // Calculate differences
- const __m128i v_sig = _mm_loadu_si128((const __m128i *)(&sig[0]));
- const __m128i v_mc_running_avg_y =
- _mm_loadu_si128((const __m128i *)(&mc_running_avg_y[0]));
- __m128i v_running_avg_y;
- const __m128i pdiff = _mm_subs_epu8(v_mc_running_avg_y, v_sig);
- const __m128i ndiff = _mm_subs_epu8(v_sig, v_mc_running_avg_y);
- // Obtain the sign. FF if diff is negative.
- const __m128i diff_sign = _mm_cmpeq_epi8(pdiff, *k_0);
- // Clamp absolute difference to 16 to be used to get mask. Doing this
- // allows us to use _mm_cmpgt_epi8, which operates on signed byte.
- const __m128i clamped_absdiff =
- _mm_min_epu8(_mm_or_si128(pdiff, ndiff), *k_16);
- // Get masks for l2 l1 and l0 adjustments.
- const __m128i mask2 = _mm_cmpgt_epi8(*k_16, clamped_absdiff);
- const __m128i mask1 = _mm_cmpgt_epi8(*k_8, clamped_absdiff);
- const __m128i mask0 = _mm_cmpgt_epi8(*k_4, clamped_absdiff);
- // Get adjustments for l2, l1, and l0.
- __m128i adj2 = _mm_and_si128(mask2, *l32);
- const __m128i adj1 = _mm_and_si128(mask1, *l21);
- const __m128i adj0 = _mm_and_si128(mask0, clamped_absdiff);
- __m128i adj, padj, nadj;
-
- // Combine the adjustments and get absolute adjustments.
- adj2 = _mm_add_epi8(adj2, adj1);
- adj = _mm_sub_epi8(*l3, adj2);
- adj = _mm_andnot_si128(mask0, adj);
- adj = _mm_or_si128(adj, adj0);
-
- // Restore the sign and get positive and negative adjustments.
- padj = _mm_andnot_si128(diff_sign, adj);
- nadj = _mm_and_si128(diff_sign, adj);
-
- // Calculate filtered value.
- v_running_avg_y = _mm_adds_epu8(v_sig, padj);
- v_running_avg_y = _mm_subs_epu8(v_running_avg_y, nadj);
- _mm_storeu_si128((__m128i *)running_avg_y, v_running_avg_y);
-
- // Adjustments <=7, and each element in acc_diff can fit in signed
- // char.
- acc_diff = _mm_adds_epi8(acc_diff, padj);
- acc_diff = _mm_subs_epi8(acc_diff, nadj);
- return acc_diff;
-}
-
-// Denoise a 16x1 vector with a weaker filter.
-static INLINE __m128i vp10_denoiser_adj_16x1_sse2(
- const uint8_t *sig, const uint8_t *mc_running_avg_y,
- uint8_t *running_avg_y, const __m128i k_0,
- const __m128i k_delta, __m128i acc_diff) {
- __m128i v_running_avg_y = _mm_loadu_si128((__m128i *)(&running_avg_y[0]));
- // Calculate differences.
- const __m128i v_sig = _mm_loadu_si128((const __m128i *)(&sig[0]));
- const __m128i v_mc_running_avg_y =
- _mm_loadu_si128((const __m128i *)(&mc_running_avg_y[0]));
- const __m128i pdiff = _mm_subs_epu8(v_mc_running_avg_y, v_sig);
- const __m128i ndiff = _mm_subs_epu8(v_sig, v_mc_running_avg_y);
- // Obtain the sign. FF if diff is negative.
- const __m128i diff_sign = _mm_cmpeq_epi8(pdiff, k_0);
- // Clamp absolute difference to delta to get the adjustment.
- const __m128i adj =
- _mm_min_epu8(_mm_or_si128(pdiff, ndiff), k_delta);
- // Restore the sign and get positive and negative adjustments.
- __m128i padj, nadj;
- padj = _mm_andnot_si128(diff_sign, adj);
- nadj = _mm_and_si128(diff_sign, adj);
- // Calculate filtered value.
- v_running_avg_y = _mm_subs_epu8(v_running_avg_y, padj);
- v_running_avg_y = _mm_adds_epu8(v_running_avg_y, nadj);
- _mm_storeu_si128((__m128i *)running_avg_y, v_running_avg_y);
-
- // Accumulate the adjustments.
- acc_diff = _mm_subs_epi8(acc_diff, padj);
- acc_diff = _mm_adds_epi8(acc_diff, nadj);
- return acc_diff;
-}
-
-// Denoiser for 4xM and 8xM blocks.
-static int vp10_denoiser_NxM_sse2_small(
- const uint8_t *sig, int sig_stride, const uint8_t *mc_running_avg_y,
- int mc_avg_y_stride, uint8_t *running_avg_y, int avg_y_stride,
- int increase_denoising, BLOCK_SIZE bs, int motion_magnitude, int width) {
- int sum_diff_thresh, r, sum_diff = 0;
- const int shift_inc = (increase_denoising &&
- motion_magnitude <= MOTION_MAGNITUDE_THRESHOLD) ?
- 1 : 0;
- uint8_t sig_buffer[8][16], mc_running_buffer[8][16], running_buffer[8][16];
- __m128i acc_diff = _mm_setzero_si128();
- const __m128i k_0 = _mm_setzero_si128();
- const __m128i k_4 = _mm_set1_epi8(4 + shift_inc);
- const __m128i k_8 = _mm_set1_epi8(8);
- const __m128i k_16 = _mm_set1_epi8(16);
- // Modify each level's adjustment according to motion_magnitude.
- const __m128i l3 = _mm_set1_epi8(
- (motion_magnitude <= MOTION_MAGNITUDE_THRESHOLD) ? 7 + shift_inc : 6);
- // Difference between level 3 and level 2 is 2.
- const __m128i l32 = _mm_set1_epi8(2);
- // Difference between level 2 and level 1 is 1.
- const __m128i l21 = _mm_set1_epi8(1);
- const uint8_t shift = (width == 4) ? 2 : 1;
-
- for (r = 0; r < ((4 << b_height_log2_lookup[bs]) >> shift); ++r) {
- memcpy(sig_buffer[r], sig, width);
- memcpy(sig_buffer[r] + width, sig + sig_stride, width);
- memcpy(mc_running_buffer[r], mc_running_avg_y, width);
- memcpy(mc_running_buffer[r] + width,
- mc_running_avg_y + mc_avg_y_stride, width);
- memcpy(running_buffer[r], running_avg_y, width);
- memcpy(running_buffer[r] + width, running_avg_y + avg_y_stride, width);
- if (width == 4) {
- memcpy(sig_buffer[r] + width * 2, sig + sig_stride * 2, width);
- memcpy(sig_buffer[r] + width * 3, sig + sig_stride * 3, width);
- memcpy(mc_running_buffer[r] + width * 2,
- mc_running_avg_y + mc_avg_y_stride * 2, width);
- memcpy(mc_running_buffer[r] + width * 3,
- mc_running_avg_y + mc_avg_y_stride * 3, width);
- memcpy(running_buffer[r] + width * 2,
- running_avg_y + avg_y_stride * 2, width);
- memcpy(running_buffer[r] + width * 3,
- running_avg_y + avg_y_stride * 3, width);
- }
- acc_diff = vp10_denoiser_16x1_sse2(sig_buffer[r],
- mc_running_buffer[r],
- running_buffer[r],
- &k_0, &k_4, &k_8, &k_16,
- &l3, &l32, &l21, acc_diff);
- memcpy(running_avg_y, running_buffer[r], width);
- memcpy(running_avg_y + avg_y_stride, running_buffer[r] + width, width);
- if (width == 4) {
- memcpy(running_avg_y + avg_y_stride * 2,
- running_buffer[r] + width * 2, width);
- memcpy(running_avg_y + avg_y_stride * 3,
- running_buffer[r] + width * 3, width);
- }
- // Update pointers for next iteration.
- sig += (sig_stride << shift);
- mc_running_avg_y += (mc_avg_y_stride << shift);
- running_avg_y += (avg_y_stride << shift);
- }
-
- {
- sum_diff = sum_diff_16x1(acc_diff);
- sum_diff_thresh = total_adj_strong_thresh(bs, increase_denoising);
- if (abs(sum_diff) > sum_diff_thresh) {
- // Before returning to copy the block (i.e., apply no denoising),
- // check if we can still apply some (weaker) temporal filtering to
- // this block, that would otherwise not be denoised at all. Simplest
- // is to apply an additional adjustment to running_avg_y to bring it
- // closer to sig. The adjustment is capped by a maximum delta, and
- // chosen such that in most cases the resulting sum_diff will be
- // within the acceptable range given by sum_diff_thresh.
-
- // The delta is set by the excess of absolute pixel diff over the
- // threshold.
- const int delta = ((abs(sum_diff) - sum_diff_thresh) >>
- num_pels_log2_lookup[bs]) + 1;
- // Only apply the adjustment for max delta up to 3.
- if (delta < 4) {
- const __m128i k_delta = _mm_set1_epi8(delta);
- running_avg_y -= avg_y_stride * (4 << b_height_log2_lookup[bs]);
- for (r = 0; r < ((4 << b_height_log2_lookup[bs]) >> shift); ++r) {
- acc_diff = vp10_denoiser_adj_16x1_sse2(
- sig_buffer[r], mc_running_buffer[r], running_buffer[r],
- k_0, k_delta, acc_diff);
- memcpy(running_avg_y, running_buffer[r], width);
- memcpy(running_avg_y + avg_y_stride,
- running_buffer[r] + width, width);
- if (width == 4) {
- memcpy(running_avg_y + avg_y_stride * 2,
- running_buffer[r] + width * 2, width);
- memcpy(running_avg_y + avg_y_stride * 3,
- running_buffer[r] + width * 3, width);
- }
- // Update pointers for next iteration.
- running_avg_y += (avg_y_stride << shift);
- }
- sum_diff = sum_diff_16x1(acc_diff);
- if (abs(sum_diff) > sum_diff_thresh) {
- return COPY_BLOCK;
- }
- } else {
- return COPY_BLOCK;
- }
- }
- }
- return FILTER_BLOCK;
-}
-
-// Denoiser for 16xM, 32xM and 64xM blocks
-static int vp10_denoiser_NxM_sse2_big(const uint8_t *sig, int sig_stride,
- const uint8_t *mc_running_avg_y,
- int mc_avg_y_stride,
- uint8_t *running_avg_y,
- int avg_y_stride,
- int increase_denoising, BLOCK_SIZE bs,
- int motion_magnitude) {
- int sum_diff_thresh, r, c, sum_diff = 0;
- const int shift_inc = (increase_denoising &&
- motion_magnitude <= MOTION_MAGNITUDE_THRESHOLD) ?
- 1 : 0;
- __m128i acc_diff[4][4];
- const __m128i k_0 = _mm_setzero_si128();
- const __m128i k_4 = _mm_set1_epi8(4 + shift_inc);
- const __m128i k_8 = _mm_set1_epi8(8);
- const __m128i k_16 = _mm_set1_epi8(16);
- // Modify each level's adjustment according to motion_magnitude.
- const __m128i l3 = _mm_set1_epi8(
- (motion_magnitude <= MOTION_MAGNITUDE_THRESHOLD) ? 7 + shift_inc : 6);
- // Difference between level 3 and level 2 is 2.
- const __m128i l32 = _mm_set1_epi8(2);
- // Difference between level 2 and level 1 is 1.
- const __m128i l21 = _mm_set1_epi8(1);
-
- for (c = 0; c < 4; ++c) {
- for (r = 0; r < 4; ++r) {
- acc_diff[c][r] = _mm_setzero_si128();
- }
- }
-
- for (r = 0; r < (4 << b_height_log2_lookup[bs]); ++r) {
- for (c = 0; c < (4 << b_width_log2_lookup[bs]); c += 16) {
- acc_diff[c>>4][r>>4] = vp10_denoiser_16x1_sse2(
- sig, mc_running_avg_y, running_avg_y, &k_0, &k_4,
- &k_8, &k_16, &l3, &l32, &l21, acc_diff[c>>4][r>>4]);
- // Update pointers for next iteration.
- sig += 16;
- mc_running_avg_y += 16;
- running_avg_y += 16;
- }
-
- if ((r + 1) % 16 == 0 || (bs == BLOCK_16X8 && r == 7)) {
- for (c = 0; c < (4 << b_width_log2_lookup[bs]); c += 16) {
- sum_diff += sum_diff_16x1(acc_diff[c>>4][r>>4]);
- }
- }
-
- // Update pointers for next iteration.
- sig = sig - 16 * ((4 << b_width_log2_lookup[bs]) >> 4) + sig_stride;
- mc_running_avg_y = mc_running_avg_y -
- 16 * ((4 << b_width_log2_lookup[bs]) >> 4) +
- mc_avg_y_stride;
- running_avg_y = running_avg_y -
- 16 * ((4 << b_width_log2_lookup[bs]) >> 4) +
- avg_y_stride;
- }
-
- {
- sum_diff_thresh = total_adj_strong_thresh(bs, increase_denoising);
- if (abs(sum_diff) > sum_diff_thresh) {
- const int delta = ((abs(sum_diff) - sum_diff_thresh) >>
- num_pels_log2_lookup[bs]) + 1;
-
- // Only apply the adjustment for max delta up to 3.
- if (delta < 4) {
- const __m128i k_delta = _mm_set1_epi8(delta);
- sig -= sig_stride * (4 << b_height_log2_lookup[bs]);
- mc_running_avg_y -= mc_avg_y_stride * (4 << b_height_log2_lookup[bs]);
- running_avg_y -= avg_y_stride * (4 << b_height_log2_lookup[bs]);
- sum_diff = 0;
- for (r = 0; r < (4 << b_height_log2_lookup[bs]); ++r) {
- for (c = 0; c < (4 << b_width_log2_lookup[bs]); c += 16) {
- acc_diff[c>>4][r>>4] = vp10_denoiser_adj_16x1_sse2(
- sig, mc_running_avg_y, running_avg_y, k_0,
- k_delta, acc_diff[c>>4][r>>4]);
- // Update pointers for next iteration.
- sig += 16;
- mc_running_avg_y += 16;
- running_avg_y += 16;
- }
-
- if ((r + 1) % 16 == 0 || (bs == BLOCK_16X8 && r == 7)) {
- for (c = 0; c < (4 << b_width_log2_lookup[bs]); c += 16) {
- sum_diff += sum_diff_16x1(acc_diff[c>>4][r>>4]);
- }
- }
- sig = sig - 16 * ((4 << b_width_log2_lookup[bs]) >> 4) + sig_stride;
- mc_running_avg_y = mc_running_avg_y -
- 16 * ((4 << b_width_log2_lookup[bs]) >> 4) +
- mc_avg_y_stride;
- running_avg_y = running_avg_y -
- 16 * ((4 << b_width_log2_lookup[bs]) >> 4) +
- avg_y_stride;
- }
- if (abs(sum_diff) > sum_diff_thresh) {
- return COPY_BLOCK;
- }
- } else {
- return COPY_BLOCK;
- }
- }
- }
- return FILTER_BLOCK;
-}
-
-int vp10_denoiser_filter_sse2(const uint8_t *sig, int sig_stride,
- const uint8_t *mc_avg,
- int mc_avg_stride,
- uint8_t *avg, int avg_stride,
- int increase_denoising,
- BLOCK_SIZE bs,
- int motion_magnitude) {
- if (bs == BLOCK_4X4 || bs == BLOCK_4X8) {
- return vp10_denoiser_NxM_sse2_small(sig, sig_stride,
- mc_avg, mc_avg_stride,
- avg, avg_stride,
- increase_denoising,
- bs, motion_magnitude, 4);
- } else if (bs == BLOCK_8X4 || bs == BLOCK_8X8 || bs == BLOCK_8X16) {
- return vp10_denoiser_NxM_sse2_small(sig, sig_stride,
- mc_avg, mc_avg_stride,
- avg, avg_stride,
- increase_denoising,
- bs, motion_magnitude, 8);
- } else if (bs == BLOCK_16X8 || bs == BLOCK_16X16 || bs == BLOCK_16X32 ||
- bs == BLOCK_32X16|| bs == BLOCK_32X32 || bs == BLOCK_32X64 ||
- bs == BLOCK_64X32 || bs == BLOCK_64X64) {
- return vp10_denoiser_NxM_sse2_big(sig, sig_stride,
- mc_avg, mc_avg_stride,
- avg, avg_stride,
- increase_denoising,
- bs, motion_magnitude);
- } else {
- return COPY_BLOCK;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Usee of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <immintrin.h> // AVX2
-
-#include "./vp10_rtcd.h"
-#include "vpx/vpx_integer.h"
-
-int64_t vp10_block_error_avx2(const int16_t *coeff,
- const int16_t *dqcoeff,
- intptr_t block_size,
- int64_t *ssz) {
- __m256i sse_reg, ssz_reg, coeff_reg, dqcoeff_reg;
- __m256i exp_dqcoeff_lo, exp_dqcoeff_hi, exp_coeff_lo, exp_coeff_hi;
- __m256i sse_reg_64hi, ssz_reg_64hi;
- __m128i sse_reg128, ssz_reg128;
- int64_t sse;
- int i;
- const __m256i zero_reg = _mm256_set1_epi16(0);
-
- // init sse and ssz registerd to zero
- sse_reg = _mm256_set1_epi16(0);
- ssz_reg = _mm256_set1_epi16(0);
-
- for (i = 0 ; i < block_size ; i+= 16) {
- // load 32 bytes from coeff and dqcoeff
- coeff_reg = _mm256_loadu_si256((const __m256i *)(coeff + i));
- dqcoeff_reg = _mm256_loadu_si256((const __m256i *)(dqcoeff + i));
- // dqcoeff - coeff
- dqcoeff_reg = _mm256_sub_epi16(dqcoeff_reg, coeff_reg);
- // madd (dqcoeff - coeff)
- dqcoeff_reg = _mm256_madd_epi16(dqcoeff_reg, dqcoeff_reg);
- // madd coeff
- coeff_reg = _mm256_madd_epi16(coeff_reg, coeff_reg);
- // expand each double word of madd (dqcoeff - coeff) to quad word
- exp_dqcoeff_lo = _mm256_unpacklo_epi32(dqcoeff_reg, zero_reg);
- exp_dqcoeff_hi = _mm256_unpackhi_epi32(dqcoeff_reg, zero_reg);
- // expand each double word of madd (coeff) to quad word
- exp_coeff_lo = _mm256_unpacklo_epi32(coeff_reg, zero_reg);
- exp_coeff_hi = _mm256_unpackhi_epi32(coeff_reg, zero_reg);
- // add each quad word of madd (dqcoeff - coeff) and madd (coeff)
- sse_reg = _mm256_add_epi64(sse_reg, exp_dqcoeff_lo);
- ssz_reg = _mm256_add_epi64(ssz_reg, exp_coeff_lo);
- sse_reg = _mm256_add_epi64(sse_reg, exp_dqcoeff_hi);
- ssz_reg = _mm256_add_epi64(ssz_reg, exp_coeff_hi);
- }
- // save the higher 64 bit of each 128 bit lane
- sse_reg_64hi = _mm256_srli_si256(sse_reg, 8);
- ssz_reg_64hi = _mm256_srli_si256(ssz_reg, 8);
- // add the higher 64 bit to the low 64 bit
- sse_reg = _mm256_add_epi64(sse_reg, sse_reg_64hi);
- ssz_reg = _mm256_add_epi64(ssz_reg, ssz_reg_64hi);
-
- // add each 64 bit from each of the 128 bit lane of the 256 bit
- sse_reg128 = _mm_add_epi64(_mm256_castsi256_si128(sse_reg),
- _mm256_extractf128_si256(sse_reg, 1));
-
- ssz_reg128 = _mm_add_epi64(_mm256_castsi256_si128(ssz_reg),
- _mm256_extractf128_si256(ssz_reg, 1));
-
- // store the results
- _mm_storel_epi64((__m128i*)(&sse), sse_reg128);
-
- _mm_storel_epi64((__m128i*)(ssz), ssz_reg128);
- return sse;
-}
+++ /dev/null
-;
-; Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-%define private_prefix vp10
-
-%include "third_party/x86inc/x86inc.asm"
-
-SECTION .text
-
-; int64_t vp10_block_error(int16_t *coeff, int16_t *dqcoeff, intptr_t block_size,
-; int64_t *ssz)
-
-INIT_XMM sse2
-cglobal block_error, 3, 3, 8, uqc, dqc, size, ssz
- pxor m4, m4 ; sse accumulator
- pxor m6, m6 ; ssz accumulator
- pxor m5, m5 ; dedicated zero register
- lea uqcq, [uqcq+sizeq*2]
- lea dqcq, [dqcq+sizeq*2]
- neg sizeq
-.loop:
- mova m2, [uqcq+sizeq*2]
- mova m0, [dqcq+sizeq*2]
- mova m3, [uqcq+sizeq*2+mmsize]
- mova m1, [dqcq+sizeq*2+mmsize]
- psubw m0, m2
- psubw m1, m3
- ; individual errors are max. 15bit+sign, so squares are 30bit, and
- ; thus the sum of 2 should fit in a 31bit integer (+ unused sign bit)
- pmaddwd m0, m0
- pmaddwd m1, m1
- pmaddwd m2, m2
- pmaddwd m3, m3
- ; accumulate in 64bit
- punpckldq m7, m0, m5
- punpckhdq m0, m5
- paddq m4, m7
- punpckldq m7, m1, m5
- paddq m4, m0
- punpckhdq m1, m5
- paddq m4, m7
- punpckldq m7, m2, m5
- paddq m4, m1
- punpckhdq m2, m5
- paddq m6, m7
- punpckldq m7, m3, m5
- paddq m6, m2
- punpckhdq m3, m5
- paddq m6, m7
- paddq m6, m3
- add sizeq, mmsize
- jl .loop
-
- ; accumulate horizontally and store in return value
- movhlps m5, m4
- movhlps m7, m6
- paddq m4, m5
- paddq m6, m7
-%if ARCH_X86_64
- movq rax, m4
- movq [sszq], m6
-%else
- mov eax, sszm
- pshufd m5, m4, 0x1
- movq [eax], m6
- movd eax, m4
- movd edx, m5
-%endif
- RET
-
-; Compute the sum of squared difference between two int16_t vectors.
-; int64_t vp10_block_error_fp(int16_t *coeff, int16_t *dqcoeff,
-; intptr_t block_size)
-
-INIT_XMM sse2
-cglobal block_error_fp, 3, 3, 6, uqc, dqc, size
- pxor m4, m4 ; sse accumulator
- pxor m5, m5 ; dedicated zero register
- lea uqcq, [uqcq+sizeq*2]
- lea dqcq, [dqcq+sizeq*2]
- neg sizeq
-.loop:
- mova m2, [uqcq+sizeq*2]
- mova m0, [dqcq+sizeq*2]
- mova m3, [uqcq+sizeq*2+mmsize]
- mova m1, [dqcq+sizeq*2+mmsize]
- psubw m0, m2
- psubw m1, m3
- ; individual errors are max. 15bit+sign, so squares are 30bit, and
- ; thus the sum of 2 should fit in a 31bit integer (+ unused sign bit)
- pmaddwd m0, m0
- pmaddwd m1, m1
- ; accumulate in 64bit
- punpckldq m3, m0, m5
- punpckhdq m0, m5
- paddq m4, m3
- punpckldq m3, m1, m5
- paddq m4, m0
- punpckhdq m1, m5
- paddq m4, m3
- paddq m4, m1
- add sizeq, mmsize
- jl .loop
-
- ; accumulate horizontally and store in return value
- movhlps m5, m4
- paddq m4, m5
-%if ARCH_X86_64
- movq rax, m4
-%else
- pshufd m5, m4, 0x1
- movd eax, m4
- movd edx, m5
-%endif
- RET
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h>
-#include <stdio.h>
-
-#include "vp10/common/common.h"
-
-int64_t vp10_highbd_block_error_sse2(tran_low_t *coeff, tran_low_t *dqcoeff,
- intptr_t block_size, int64_t *ssz,
- int bps) {
- int i, j, test;
- uint32_t temp[4];
- __m128i max, min, cmp0, cmp1, cmp2, cmp3;
- int64_t error = 0, sqcoeff = 0;
- const int shift = 2 * (bps - 8);
- const int rounding = shift > 0 ? 1 << (shift - 1) : 0;
-
- for (i = 0; i < block_size; i+=8) {
- // Load the data into xmm registers
- __m128i mm_coeff = _mm_load_si128((__m128i*) (coeff + i));
- __m128i mm_coeff2 = _mm_load_si128((__m128i*) (coeff + i + 4));
- __m128i mm_dqcoeff = _mm_load_si128((__m128i*) (dqcoeff + i));
- __m128i mm_dqcoeff2 = _mm_load_si128((__m128i*) (dqcoeff + i + 4));
- // Check if any values require more than 15 bit
- max = _mm_set1_epi32(0x3fff);
- min = _mm_set1_epi32(0xffffc000);
- cmp0 = _mm_xor_si128(_mm_cmpgt_epi32(mm_coeff, max),
- _mm_cmplt_epi32(mm_coeff, min));
- cmp1 = _mm_xor_si128(_mm_cmpgt_epi32(mm_coeff2, max),
- _mm_cmplt_epi32(mm_coeff2, min));
- cmp2 = _mm_xor_si128(_mm_cmpgt_epi32(mm_dqcoeff, max),
- _mm_cmplt_epi32(mm_dqcoeff, min));
- cmp3 = _mm_xor_si128(_mm_cmpgt_epi32(mm_dqcoeff2, max),
- _mm_cmplt_epi32(mm_dqcoeff2, min));
- test = _mm_movemask_epi8(_mm_or_si128(_mm_or_si128(cmp0, cmp1),
- _mm_or_si128(cmp2, cmp3)));
-
- if (!test) {
- __m128i mm_diff, error_sse2, sqcoeff_sse2;;
- mm_coeff = _mm_packs_epi32(mm_coeff, mm_coeff2);
- mm_dqcoeff = _mm_packs_epi32(mm_dqcoeff, mm_dqcoeff2);
- mm_diff = _mm_sub_epi16(mm_coeff, mm_dqcoeff);
- error_sse2 = _mm_madd_epi16(mm_diff, mm_diff);
- sqcoeff_sse2 = _mm_madd_epi16(mm_coeff, mm_coeff);
- _mm_storeu_si128((__m128i*)temp, error_sse2);
- error = error + temp[0] + temp[1] + temp[2] + temp[3];
- _mm_storeu_si128((__m128i*)temp, sqcoeff_sse2);
- sqcoeff += temp[0] + temp[1] + temp[2] + temp[3];
- } else {
- for (j = 0; j < 8; j++) {
- const int64_t diff = coeff[i + j] - dqcoeff[i + j];
- error += diff * diff;
- sqcoeff += (int64_t)coeff[i + j] * (int64_t)coeff[i + j];
- }
- }
- }
- assert(error >= 0 && sqcoeff >= 0);
- error = (error + rounding) >> shift;
- sqcoeff = (sqcoeff + rounding) >> shift;
-
- *ssz = sqcoeff;
- return error;
-}
+++ /dev/null
-/*
- * Copyright (c) 2014 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <emmintrin.h>
-#include <xmmintrin.h>
-
-#include "./vp10_rtcd.h"
-#include "vpx/vpx_integer.h"
-
-void vp10_quantize_fp_sse2(const int16_t* coeff_ptr, intptr_t n_coeffs,
- int skip_block, const int16_t* zbin_ptr,
- const int16_t* round_ptr, const int16_t* quant_ptr,
- const int16_t* quant_shift_ptr, int16_t* qcoeff_ptr,
- int16_t* dqcoeff_ptr, const int16_t* dequant_ptr,
- uint16_t* eob_ptr,
- const int16_t* scan_ptr,
- const int16_t* iscan_ptr) {
- __m128i zero;
- __m128i thr;
- int16_t nzflag;
- (void)scan_ptr;
- (void)zbin_ptr;
- (void)quant_shift_ptr;
-
- coeff_ptr += n_coeffs;
- iscan_ptr += n_coeffs;
- qcoeff_ptr += n_coeffs;
- dqcoeff_ptr += n_coeffs;
- n_coeffs = -n_coeffs;
- zero = _mm_setzero_si128();
-
- if (!skip_block) {
- __m128i eob;
- __m128i round, quant, dequant;
- {
- __m128i coeff0, coeff1;
-
- // Setup global values
- {
- round = _mm_load_si128((const __m128i*)round_ptr);
- quant = _mm_load_si128((const __m128i*)quant_ptr);
- dequant = _mm_load_si128((const __m128i*)dequant_ptr);
- }
-
- {
- __m128i coeff0_sign, coeff1_sign;
- __m128i qcoeff0, qcoeff1;
- __m128i qtmp0, qtmp1;
- // Do DC and first 15 AC
- coeff0 = _mm_load_si128((const __m128i*)(coeff_ptr + n_coeffs));
- coeff1 = _mm_load_si128((const __m128i*)(coeff_ptr + n_coeffs) + 1);
-
- // Poor man's sign extract
- coeff0_sign = _mm_srai_epi16(coeff0, 15);
- coeff1_sign = _mm_srai_epi16(coeff1, 15);
- qcoeff0 = _mm_xor_si128(coeff0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(coeff1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- qcoeff0 = _mm_adds_epi16(qcoeff0, round);
- round = _mm_unpackhi_epi64(round, round);
- qcoeff1 = _mm_adds_epi16(qcoeff1, round);
- qtmp0 = _mm_mulhi_epi16(qcoeff0, quant);
- quant = _mm_unpackhi_epi64(quant, quant);
- qtmp1 = _mm_mulhi_epi16(qcoeff1, quant);
-
- // Reinsert signs
- qcoeff0 = _mm_xor_si128(qtmp0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(qtmp1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), qcoeff0);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, qcoeff1);
-
- coeff0 = _mm_mullo_epi16(qcoeff0, dequant);
- dequant = _mm_unpackhi_epi64(dequant, dequant);
- coeff1 = _mm_mullo_epi16(qcoeff1, dequant);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), coeff0);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, coeff1);
- }
-
- {
- // Scan for eob
- __m128i zero_coeff0, zero_coeff1;
- __m128i nzero_coeff0, nzero_coeff1;
- __m128i iscan0, iscan1;
- __m128i eob1;
- zero_coeff0 = _mm_cmpeq_epi16(coeff0, zero);
- zero_coeff1 = _mm_cmpeq_epi16(coeff1, zero);
- nzero_coeff0 = _mm_cmpeq_epi16(zero_coeff0, zero);
- nzero_coeff1 = _mm_cmpeq_epi16(zero_coeff1, zero);
- iscan0 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs));
- iscan1 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs) + 1);
- // Add one to convert from indices to counts
- iscan0 = _mm_sub_epi16(iscan0, nzero_coeff0);
- iscan1 = _mm_sub_epi16(iscan1, nzero_coeff1);
- eob = _mm_and_si128(iscan0, nzero_coeff0);
- eob1 = _mm_and_si128(iscan1, nzero_coeff1);
- eob = _mm_max_epi16(eob, eob1);
- }
- n_coeffs += 8 * 2;
- }
-
- thr = _mm_srai_epi16(dequant, 1);
-
- // AC only loop
- while (n_coeffs < 0) {
- __m128i coeff0, coeff1;
- {
- __m128i coeff0_sign, coeff1_sign;
- __m128i qcoeff0, qcoeff1;
- __m128i qtmp0, qtmp1;
-
- coeff0 = _mm_load_si128((const __m128i*)(coeff_ptr + n_coeffs));
- coeff1 = _mm_load_si128((const __m128i*)(coeff_ptr + n_coeffs) + 1);
-
- // Poor man's sign extract
- coeff0_sign = _mm_srai_epi16(coeff0, 15);
- coeff1_sign = _mm_srai_epi16(coeff1, 15);
- qcoeff0 = _mm_xor_si128(coeff0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(coeff1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- nzflag = _mm_movemask_epi8(_mm_cmpgt_epi16(qcoeff0, thr)) |
- _mm_movemask_epi8(_mm_cmpgt_epi16(qcoeff1, thr));
-
- if (nzflag) {
- qcoeff0 = _mm_adds_epi16(qcoeff0, round);
- qcoeff1 = _mm_adds_epi16(qcoeff1, round);
- qtmp0 = _mm_mulhi_epi16(qcoeff0, quant);
- qtmp1 = _mm_mulhi_epi16(qcoeff1, quant);
-
- // Reinsert signs
- qcoeff0 = _mm_xor_si128(qtmp0, coeff0_sign);
- qcoeff1 = _mm_xor_si128(qtmp1, coeff1_sign);
- qcoeff0 = _mm_sub_epi16(qcoeff0, coeff0_sign);
- qcoeff1 = _mm_sub_epi16(qcoeff1, coeff1_sign);
-
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), qcoeff0);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, qcoeff1);
-
- coeff0 = _mm_mullo_epi16(qcoeff0, dequant);
- coeff1 = _mm_mullo_epi16(qcoeff1, dequant);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), coeff0);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, coeff1);
- } else {
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, zero);
-
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, zero);
- }
- }
-
- if (nzflag) {
- // Scan for eob
- __m128i zero_coeff0, zero_coeff1;
- __m128i nzero_coeff0, nzero_coeff1;
- __m128i iscan0, iscan1;
- __m128i eob0, eob1;
- zero_coeff0 = _mm_cmpeq_epi16(coeff0, zero);
- zero_coeff1 = _mm_cmpeq_epi16(coeff1, zero);
- nzero_coeff0 = _mm_cmpeq_epi16(zero_coeff0, zero);
- nzero_coeff1 = _mm_cmpeq_epi16(zero_coeff1, zero);
- iscan0 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs));
- iscan1 = _mm_load_si128((const __m128i*)(iscan_ptr + n_coeffs) + 1);
- // Add one to convert from indices to counts
- iscan0 = _mm_sub_epi16(iscan0, nzero_coeff0);
- iscan1 = _mm_sub_epi16(iscan1, nzero_coeff1);
- eob0 = _mm_and_si128(iscan0, nzero_coeff0);
- eob1 = _mm_and_si128(iscan1, nzero_coeff1);
- eob0 = _mm_max_epi16(eob0, eob1);
- eob = _mm_max_epi16(eob, eob0);
- }
- n_coeffs += 8 * 2;
- }
-
- // Accumulate EOB
- {
- __m128i eob_shuffled;
- eob_shuffled = _mm_shuffle_epi32(eob, 0xe);
- eob = _mm_max_epi16(eob, eob_shuffled);
- eob_shuffled = _mm_shufflelo_epi16(eob, 0xe);
- eob = _mm_max_epi16(eob, eob_shuffled);
- eob_shuffled = _mm_shufflelo_epi16(eob, 0x1);
- eob = _mm_max_epi16(eob, eob_shuffled);
- *eob_ptr = _mm_extract_epi16(eob, 1);
- }
- } else {
- do {
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(dqcoeff_ptr + n_coeffs) + 1, zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs), zero);
- _mm_store_si128((__m128i*)(qcoeff_ptr + n_coeffs) + 1, zero);
- n_coeffs += 8 * 2;
- } while (n_coeffs < 0);
- *eob_ptr = 0;
- }
-}
+++ /dev/null
-;
-; Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-%define private_prefix vp10
-
-%include "third_party/x86inc/x86inc.asm"
-
-SECTION_RODATA
-pw_1: times 8 dw 1
-
-SECTION .text
-
-%macro QUANTIZE_FP 2
-cglobal quantize_%1, 0, %2, 15, coeff, ncoeff, skip, zbin, round, quant, \
- shift, qcoeff, dqcoeff, dequant, \
- eob, scan, iscan
- cmp dword skipm, 0
- jne .blank
-
- ; actual quantize loop - setup pointers, rounders, etc.
- movifnidn coeffq, coeffmp
- movifnidn ncoeffq, ncoeffmp
- mov r2, dequantmp
- movifnidn zbinq, zbinmp
- movifnidn roundq, roundmp
- movifnidn quantq, quantmp
- mova m1, [roundq] ; m1 = round
- mova m2, [quantq] ; m2 = quant
-%ifidn %1, fp_32x32
- pcmpeqw m5, m5
- psrlw m5, 15
- paddw m1, m5
- psrlw m1, 1 ; m1 = (m1 + 1) / 2
-%endif
- mova m3, [r2q] ; m3 = dequant
- mov r3, qcoeffmp
- mov r4, dqcoeffmp
- mov r5, iscanmp
-%ifidn %1, fp_32x32
- psllw m2, 1
-%endif
- pxor m5, m5 ; m5 = dedicated zero
-
- lea coeffq, [ coeffq+ncoeffq*2]
- lea r5q, [ r5q+ncoeffq*2]
- lea r3q, [ r3q+ncoeffq*2]
- lea r4q, [r4q+ncoeffq*2]
- neg ncoeffq
-
- ; get DC and first 15 AC coeffs
- mova m9, [ coeffq+ncoeffq*2+ 0] ; m9 = c[i]
- mova m10, [ coeffq+ncoeffq*2+16] ; m10 = c[i]
- pabsw m6, m9 ; m6 = abs(m9)
- pabsw m11, m10 ; m11 = abs(m10)
- pcmpeqw m7, m7
-
- paddsw m6, m1 ; m6 += round
- punpckhqdq m1, m1
- paddsw m11, m1 ; m11 += round
- pmulhw m8, m6, m2 ; m8 = m6*q>>16
- punpckhqdq m2, m2
- pmulhw m13, m11, m2 ; m13 = m11*q>>16
- psignw m8, m9 ; m8 = reinsert sign
- psignw m13, m10 ; m13 = reinsert sign
- mova [r3q+ncoeffq*2+ 0], m8
- mova [r3q+ncoeffq*2+16], m13
-%ifidn %1, fp_32x32
- pabsw m8, m8
- pabsw m13, m13
-%endif
- pmullw m8, m3 ; r4[i] = r3[i] * q
- punpckhqdq m3, m3
- pmullw m13, m3 ; r4[i] = r3[i] * q
-%ifidn %1, fp_32x32
- psrlw m8, 1
- psrlw m13, 1
- psignw m8, m9
- psignw m13, m10
- psrlw m0, m3, 2
-%else
- psrlw m0, m3, 1
-%endif
- mova [r4q+ncoeffq*2+ 0], m8
- mova [r4q+ncoeffq*2+16], m13
- pcmpeqw m8, m5 ; m8 = c[i] == 0
- pcmpeqw m13, m5 ; m13 = c[i] == 0
- mova m6, [ r5q+ncoeffq*2+ 0] ; m6 = scan[i]
- mova m11, [ r5q+ncoeffq*2+16] ; m11 = scan[i]
- psubw m6, m7 ; m6 = scan[i] + 1
- psubw m11, m7 ; m11 = scan[i] + 1
- pandn m8, m6 ; m8 = max(eob)
- pandn m13, m11 ; m13 = max(eob)
- pmaxsw m8, m13
- add ncoeffq, mmsize
- jz .accumulate_eob
-
-.ac_only_loop:
- mova m9, [ coeffq+ncoeffq*2+ 0] ; m9 = c[i]
- mova m10, [ coeffq+ncoeffq*2+16] ; m10 = c[i]
- pabsw m6, m9 ; m6 = abs(m9)
- pabsw m11, m10 ; m11 = abs(m10)
-
- pcmpgtw m7, m6, m0
- pcmpgtw m12, m11, m0
- pmovmskb r6d, m7
- pmovmskb r2d, m12
-
- or r6, r2
- jz .skip_iter
-
- pcmpeqw m7, m7
-
- paddsw m6, m1 ; m6 += round
- paddsw m11, m1 ; m11 += round
- pmulhw m14, m6, m2 ; m14 = m6*q>>16
- pmulhw m13, m11, m2 ; m13 = m11*q>>16
- psignw m14, m9 ; m14 = reinsert sign
- psignw m13, m10 ; m13 = reinsert sign
- mova [r3q+ncoeffq*2+ 0], m14
- mova [r3q+ncoeffq*2+16], m13
-%ifidn %1, fp_32x32
- pabsw m14, m14
- pabsw m13, m13
-%endif
- pmullw m14, m3 ; r4[i] = r3[i] * q
- pmullw m13, m3 ; r4[i] = r3[i] * q
-%ifidn %1, fp_32x32
- psrlw m14, 1
- psrlw m13, 1
- psignw m14, m9
- psignw m13, m10
-%endif
- mova [r4q+ncoeffq*2+ 0], m14
- mova [r4q+ncoeffq*2+16], m13
- pcmpeqw m14, m5 ; m14 = c[i] == 0
- pcmpeqw m13, m5 ; m13 = c[i] == 0
- mova m6, [ r5q+ncoeffq*2+ 0] ; m6 = scan[i]
- mova m11, [ r5q+ncoeffq*2+16] ; m11 = scan[i]
- psubw m6, m7 ; m6 = scan[i] + 1
- psubw m11, m7 ; m11 = scan[i] + 1
- pandn m14, m6 ; m14 = max(eob)
- pandn m13, m11 ; m13 = max(eob)
- pmaxsw m8, m14
- pmaxsw m8, m13
- add ncoeffq, mmsize
- jl .ac_only_loop
-
- jmp .accumulate_eob
-.skip_iter:
- mova [r3q+ncoeffq*2+ 0], m5
- mova [r3q+ncoeffq*2+16], m5
- mova [r4q+ncoeffq*2+ 0], m5
- mova [r4q+ncoeffq*2+16], m5
- add ncoeffq, mmsize
- jl .ac_only_loop
-
-.accumulate_eob:
- ; horizontally accumulate/max eobs and write into [eob] memory pointer
- mov r2, eobmp
- pshufd m7, m8, 0xe
- pmaxsw m8, m7
- pshuflw m7, m8, 0xe
- pmaxsw m8, m7
- pshuflw m7, m8, 0x1
- pmaxsw m8, m7
- pextrw r6, m8, 0
- mov [r2], r6
- RET
-
- ; skip-block, i.e. just write all zeroes
-.blank:
- mov r0, dqcoeffmp
- movifnidn ncoeffq, ncoeffmp
- mov r2, qcoeffmp
- mov r3, eobmp
-
- lea r0q, [r0q+ncoeffq*2]
- lea r2q, [r2q+ncoeffq*2]
- neg ncoeffq
- pxor m7, m7
-.blank_loop:
- mova [r0q+ncoeffq*2+ 0], m7
- mova [r0q+ncoeffq*2+16], m7
- mova [r2q+ncoeffq*2+ 0], m7
- mova [r2q+ncoeffq*2+16], m7
- add ncoeffq, mmsize
- jl .blank_loop
- mov word [r3q], 0
- RET
-%endmacro
-
-INIT_XMM ssse3
-QUANTIZE_FP fp, 7
-QUANTIZE_FP fp_32x32, 7
+++ /dev/null
-;
-; Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-%include "vpx_ports/x86_abi_support.asm"
-
-; tabulate_ssim - sums sum_s,sum_r,sum_sq_s,sum_sq_r, sum_sxr
-%macro TABULATE_SSIM 0
- paddusw xmm15, xmm3 ; sum_s
- paddusw xmm14, xmm4 ; sum_r
- movdqa xmm1, xmm3
- pmaddwd xmm1, xmm1
- paddd xmm13, xmm1 ; sum_sq_s
- movdqa xmm2, xmm4
- pmaddwd xmm2, xmm2
- paddd xmm12, xmm2 ; sum_sq_r
- pmaddwd xmm3, xmm4
- paddd xmm11, xmm3 ; sum_sxr
-%endmacro
-
-; Sum across the register %1 starting with q words
-%macro SUM_ACROSS_Q 1
- movdqa xmm2,%1
- punpckldq %1,xmm0
- punpckhdq xmm2,xmm0
- paddq %1,xmm2
- movdqa xmm2,%1
- punpcklqdq %1,xmm0
- punpckhqdq xmm2,xmm0
- paddq %1,xmm2
-%endmacro
-
-; Sum across the register %1 starting with q words
-%macro SUM_ACROSS_W 1
- movdqa xmm1, %1
- punpcklwd %1,xmm0
- punpckhwd xmm1,xmm0
- paddd %1, xmm1
- SUM_ACROSS_Q %1
-%endmacro
-;void ssim_parms_sse2(
-; unsigned char *s,
-; int sp,
-; unsigned char *r,
-; int rp
-; unsigned long *sum_s,
-; unsigned long *sum_r,
-; unsigned long *sum_sq_s,
-; unsigned long *sum_sq_r,
-; unsigned long *sum_sxr);
-;
-; TODO: Use parm passing through structure, probably don't need the pxors
-; ( calling app will initialize to 0 ) could easily fit everything in sse2
-; without too much hastle, and can probably do better estimates with psadw
-; or pavgb At this point this is just meant to be first pass for calculating
-; all the parms needed for 16x16 ssim so we can play with dssim as distortion
-; in mode selection code.
-global sym(vp10_ssim_parms_16x16_sse2) PRIVATE
-sym(vp10_ssim_parms_16x16_sse2):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 9
- SAVE_XMM 15
- push rsi
- push rdi
- ; end prolog
-
- mov rsi, arg(0) ;s
- mov rcx, arg(1) ;sp
- mov rdi, arg(2) ;r
- mov rax, arg(3) ;rp
-
- pxor xmm0, xmm0
- pxor xmm15,xmm15 ;sum_s
- pxor xmm14,xmm14 ;sum_r
- pxor xmm13,xmm13 ;sum_sq_s
- pxor xmm12,xmm12 ;sum_sq_r
- pxor xmm11,xmm11 ;sum_sxr
-
- mov rdx, 16 ;row counter
-.NextRow:
-
- ;grab source and reference pixels
- movdqu xmm5, [rsi]
- movdqu xmm6, [rdi]
- movdqa xmm3, xmm5
- movdqa xmm4, xmm6
- punpckhbw xmm3, xmm0 ; high_s
- punpckhbw xmm4, xmm0 ; high_r
-
- TABULATE_SSIM
-
- movdqa xmm3, xmm5
- movdqa xmm4, xmm6
- punpcklbw xmm3, xmm0 ; low_s
- punpcklbw xmm4, xmm0 ; low_r
-
- TABULATE_SSIM
-
- add rsi, rcx ; next s row
- add rdi, rax ; next r row
-
- dec rdx ; counter
- jnz .NextRow
-
- SUM_ACROSS_W xmm15
- SUM_ACROSS_W xmm14
- SUM_ACROSS_Q xmm13
- SUM_ACROSS_Q xmm12
- SUM_ACROSS_Q xmm11
-
- mov rdi,arg(4)
- movd [rdi], xmm15;
- mov rdi,arg(5)
- movd [rdi], xmm14;
- mov rdi,arg(6)
- movd [rdi], xmm13;
- mov rdi,arg(7)
- movd [rdi], xmm12;
- mov rdi,arg(8)
- movd [rdi], xmm11;
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
- ret
-
-;void ssim_parms_sse2(
-; unsigned char *s,
-; int sp,
-; unsigned char *r,
-; int rp
-; unsigned long *sum_s,
-; unsigned long *sum_r,
-; unsigned long *sum_sq_s,
-; unsigned long *sum_sq_r,
-; unsigned long *sum_sxr);
-;
-; TODO: Use parm passing through structure, probably don't need the pxors
-; ( calling app will initialize to 0 ) could easily fit everything in sse2
-; without too much hastle, and can probably do better estimates with psadw
-; or pavgb At this point this is just meant to be first pass for calculating
-; all the parms needed for 16x16 ssim so we can play with dssim as distortion
-; in mode selection code.
-global sym(vp10_ssim_parms_8x8_sse2) PRIVATE
-sym(vp10_ssim_parms_8x8_sse2):
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 9
- SAVE_XMM 15
- push rsi
- push rdi
- ; end prolog
-
- mov rsi, arg(0) ;s
- mov rcx, arg(1) ;sp
- mov rdi, arg(2) ;r
- mov rax, arg(3) ;rp
-
- pxor xmm0, xmm0
- pxor xmm15,xmm15 ;sum_s
- pxor xmm14,xmm14 ;sum_r
- pxor xmm13,xmm13 ;sum_sq_s
- pxor xmm12,xmm12 ;sum_sq_r
- pxor xmm11,xmm11 ;sum_sxr
-
- mov rdx, 8 ;row counter
-.NextRow:
-
- ;grab source and reference pixels
- movq xmm3, [rsi]
- movq xmm4, [rdi]
- punpcklbw xmm3, xmm0 ; low_s
- punpcklbw xmm4, xmm0 ; low_r
-
- TABULATE_SSIM
-
- add rsi, rcx ; next s row
- add rdi, rax ; next r row
-
- dec rdx ; counter
- jnz .NextRow
-
- SUM_ACROSS_W xmm15
- SUM_ACROSS_W xmm14
- SUM_ACROSS_Q xmm13
- SUM_ACROSS_Q xmm12
- SUM_ACROSS_Q xmm11
-
- mov rdi,arg(4)
- movd [rdi], xmm15;
- mov rdi,arg(5)
- movd [rdi], xmm14;
- mov rdi,arg(6)
- movd [rdi], xmm13;
- mov rdi,arg(7)
- movd [rdi], xmm12;
- mov rdi,arg(8)
- movd [rdi], xmm11;
-
- ; begin epilog
- pop rdi
- pop rsi
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
- ret
+++ /dev/null
-;
-; Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-;
-; Use of this source code is governed by a BSD-style license
-; that can be found in the LICENSE file in the root of the source
-; tree. An additional intellectual property rights grant can be found
-; in the file PATENTS. All contributing project authors may
-; be found in the AUTHORS file in the root of the source tree.
-;
-
-
-%include "vpx_ports/x86_abi_support.asm"
-
-; void vp10_temporal_filter_apply_sse2 | arg
-; (unsigned char *frame1, | 0
-; unsigned int stride, | 1
-; unsigned char *frame2, | 2
-; unsigned int block_width, | 3
-; unsigned int block_height, | 4
-; int strength, | 5
-; int filter_weight, | 6
-; unsigned int *accumulator, | 7
-; unsigned short *count) | 8
-global sym(vp10_temporal_filter_apply_sse2) PRIVATE
-sym(vp10_temporal_filter_apply_sse2):
-
- push rbp
- mov rbp, rsp
- SHADOW_ARGS_TO_STACK 9
- SAVE_XMM 7
- GET_GOT rbx
- push rsi
- push rdi
- ALIGN_STACK 16, rax
- %define block_width 0
- %define block_height 16
- %define strength 32
- %define filter_weight 48
- %define rounding_bit 64
- %define rbp_backup 80
- %define stack_size 96
- sub rsp, stack_size
- mov [rsp + rbp_backup], rbp
- ; end prolog
-
- mov edx, arg(3)
- mov [rsp + block_width], rdx
- mov edx, arg(4)
- mov [rsp + block_height], rdx
- movd xmm6, arg(5)
- movdqa [rsp + strength], xmm6 ; where strength is used, all 16 bytes are read
-
- ; calculate the rounding bit outside the loop
- ; 0x8000 >> (16 - strength)
- mov rdx, 16
- sub rdx, arg(5) ; 16 - strength
- movq xmm4, rdx ; can't use rdx w/ shift
- movdqa xmm5, [GLOBAL(_const_top_bit)]
- psrlw xmm5, xmm4
- movdqa [rsp + rounding_bit], xmm5
-
- mov rsi, arg(0) ; src/frame1
- mov rdx, arg(2) ; predictor frame
- mov rdi, arg(7) ; accumulator
- mov rax, arg(8) ; count
-
- ; dup the filter weight and store for later
- movd xmm0, arg(6) ; filter_weight
- pshuflw xmm0, xmm0, 0
- punpcklwd xmm0, xmm0
- movdqa [rsp + filter_weight], xmm0
-
- mov rbp, arg(1) ; stride
- pxor xmm7, xmm7 ; zero for extraction
-
- mov rcx, [rsp + block_width]
- imul rcx, [rsp + block_height]
- add rcx, rdx
- cmp dword ptr [rsp + block_width], 8
- jne .temporal_filter_apply_load_16
-
-.temporal_filter_apply_load_8:
- movq xmm0, [rsi] ; first row
- lea rsi, [rsi + rbp] ; += stride
- punpcklbw xmm0, xmm7 ; src[ 0- 7]
- movq xmm1, [rsi] ; second row
- lea rsi, [rsi + rbp] ; += stride
- punpcklbw xmm1, xmm7 ; src[ 8-15]
- jmp .temporal_filter_apply_load_finished
-
-.temporal_filter_apply_load_16:
- movdqa xmm0, [rsi] ; src (frame1)
- lea rsi, [rsi + rbp] ; += stride
- movdqa xmm1, xmm0
- punpcklbw xmm0, xmm7 ; src[ 0- 7]
- punpckhbw xmm1, xmm7 ; src[ 8-15]
-
-.temporal_filter_apply_load_finished:
- movdqa xmm2, [rdx] ; predictor (frame2)
- movdqa xmm3, xmm2
- punpcklbw xmm2, xmm7 ; pred[ 0- 7]
- punpckhbw xmm3, xmm7 ; pred[ 8-15]
-
- ; modifier = src_byte - pixel_value
- psubw xmm0, xmm2 ; src - pred[ 0- 7]
- psubw xmm1, xmm3 ; src - pred[ 8-15]
-
- ; modifier *= modifier
- pmullw xmm0, xmm0 ; modifer[ 0- 7]^2
- pmullw xmm1, xmm1 ; modifer[ 8-15]^2
-
- ; modifier *= 3
- pmullw xmm0, [GLOBAL(_const_3w)]
- pmullw xmm1, [GLOBAL(_const_3w)]
-
- ; modifer += 0x8000 >> (16 - strength)
- paddw xmm0, [rsp + rounding_bit]
- paddw xmm1, [rsp + rounding_bit]
-
- ; modifier >>= strength
- psrlw xmm0, [rsp + strength]
- psrlw xmm1, [rsp + strength]
-
- ; modifier = 16 - modifier
- ; saturation takes care of modifier > 16
- movdqa xmm3, [GLOBAL(_const_16w)]
- movdqa xmm2, [GLOBAL(_const_16w)]
- psubusw xmm3, xmm1
- psubusw xmm2, xmm0
-
- ; modifier *= filter_weight
- pmullw xmm2, [rsp + filter_weight]
- pmullw xmm3, [rsp + filter_weight]
-
- ; count
- movdqa xmm4, [rax]
- movdqa xmm5, [rax+16]
- ; += modifier
- paddw xmm4, xmm2
- paddw xmm5, xmm3
- ; write back
- movdqa [rax], xmm4
- movdqa [rax+16], xmm5
- lea rax, [rax + 16*2] ; count += 16*(sizeof(short))
-
- ; load and extract the predictor up to shorts
- pxor xmm7, xmm7
- movdqa xmm0, [rdx]
- lea rdx, [rdx + 16*1] ; pred += 16*(sizeof(char))
- movdqa xmm1, xmm0
- punpcklbw xmm0, xmm7 ; pred[ 0- 7]
- punpckhbw xmm1, xmm7 ; pred[ 8-15]
-
- ; modifier *= pixel_value
- pmullw xmm0, xmm2
- pmullw xmm1, xmm3
-
- ; expand to double words
- movdqa xmm2, xmm0
- punpcklwd xmm0, xmm7 ; [ 0- 3]
- punpckhwd xmm2, xmm7 ; [ 4- 7]
- movdqa xmm3, xmm1
- punpcklwd xmm1, xmm7 ; [ 8-11]
- punpckhwd xmm3, xmm7 ; [12-15]
-
- ; accumulator
- movdqa xmm4, [rdi]
- movdqa xmm5, [rdi+16]
- movdqa xmm6, [rdi+32]
- movdqa xmm7, [rdi+48]
- ; += modifier
- paddd xmm4, xmm0
- paddd xmm5, xmm2
- paddd xmm6, xmm1
- paddd xmm7, xmm3
- ; write back
- movdqa [rdi], xmm4
- movdqa [rdi+16], xmm5
- movdqa [rdi+32], xmm6
- movdqa [rdi+48], xmm7
- lea rdi, [rdi + 16*4] ; accumulator += 16*(sizeof(int))
-
- cmp rdx, rcx
- je .temporal_filter_apply_epilog
- pxor xmm7, xmm7 ; zero for extraction
- cmp dword ptr [rsp + block_width], 16
- je .temporal_filter_apply_load_16
- jmp .temporal_filter_apply_load_8
-
-.temporal_filter_apply_epilog:
- ; begin epilog
- mov rbp, [rsp + rbp_backup]
- add rsp, stack_size
- pop rsp
- pop rdi
- pop rsi
- RESTORE_GOT
- RESTORE_XMM
- UNSHADOW_ARGS
- pop rbp
- ret
-
-SECTION_RODATA
-align 16
-_const_3w:
- times 8 dw 3
-align 16
-_const_top_bit:
- times 8 dw 1<<15
-align 16
-_const_16w
- times 8 dw 16
+++ /dev/null
-data vpx_codec_vp10_dx_algo
-text vpx_codec_vp10_dx
+++ /dev/null
-data vpx_codec_vp10_cx_algo
-text vpx_codec_vp10_cx
+++ /dev/null
-##
-## Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-##
-## Use of this source code is governed by a BSD-style license
-## that can be found in the LICENSE file in the root of the source
-## tree. An additional intellectual property rights grant can be found
-## in the file PATENTS. All contributing project authors may
-## be found in the AUTHORS file in the root of the source tree.
-##
-
-VP10_COMMON_SRCS-yes += vp10_common.mk
-VP10_COMMON_SRCS-yes += vp10_iface_common.h
-VP10_COMMON_SRCS-yes += common/ppflags.h
-VP10_COMMON_SRCS-yes += common/alloccommon.c
-VP10_COMMON_SRCS-yes += common/blockd.c
-VP10_COMMON_SRCS-yes += common/debugmodes.c
-VP10_COMMON_SRCS-yes += common/entropy.c
-VP10_COMMON_SRCS-yes += common/entropymode.c
-VP10_COMMON_SRCS-yes += common/entropymv.c
-VP10_COMMON_SRCS-yes += common/frame_buffers.c
-VP10_COMMON_SRCS-yes += common/frame_buffers.h
-VP10_COMMON_SRCS-yes += common/alloccommon.h
-VP10_COMMON_SRCS-yes += common/blockd.h
-VP10_COMMON_SRCS-yes += common/common.h
-VP10_COMMON_SRCS-yes += common/entropy.h
-VP10_COMMON_SRCS-yes += common/entropymode.h
-VP10_COMMON_SRCS-yes += common/entropymv.h
-VP10_COMMON_SRCS-yes += common/enums.h
-VP10_COMMON_SRCS-yes += common/filter.h
-VP10_COMMON_SRCS-yes += common/filter.c
-VP10_COMMON_SRCS-yes += common/idct.h
-VP10_COMMON_SRCS-yes += common/idct.c
-VP10_COMMON_SRCS-yes += common/vp10_inv_txfm.h
-VP10_COMMON_SRCS-yes += common/vp10_inv_txfm.c
-VP10_COMMON_SRCS-yes += common/loopfilter.h
-VP10_COMMON_SRCS-yes += common/thread_common.h
-VP10_COMMON_SRCS-yes += common/mv.h
-VP10_COMMON_SRCS-yes += common/onyxc_int.h
-VP10_COMMON_SRCS-yes += common/pred_common.h
-VP10_COMMON_SRCS-yes += common/pred_common.c
-VP10_COMMON_SRCS-yes += common/quant_common.h
-VP10_COMMON_SRCS-yes += common/reconinter.h
-VP10_COMMON_SRCS-yes += common/reconintra.h
-VP10_COMMON_SRCS-yes += common/vp10_rtcd.c
-VP10_COMMON_SRCS-yes += common/vp10_rtcd_defs.pl
-VP10_COMMON_SRCS-yes += common/scale.h
-VP10_COMMON_SRCS-yes += common/scale.c
-VP10_COMMON_SRCS-yes += common/seg_common.h
-VP10_COMMON_SRCS-yes += common/seg_common.c
-VP10_COMMON_SRCS-yes += common/textblit.h
-VP10_COMMON_SRCS-yes += common/tile_common.h
-VP10_COMMON_SRCS-yes += common/tile_common.c
-VP10_COMMON_SRCS-yes += common/loopfilter.c
-VP10_COMMON_SRCS-yes += common/thread_common.c
-VP10_COMMON_SRCS-yes += common/mvref_common.c
-VP10_COMMON_SRCS-yes += common/mvref_common.h
-VP10_COMMON_SRCS-yes += common/quant_common.c
-VP10_COMMON_SRCS-yes += common/reconinter.c
-VP10_COMMON_SRCS-yes += common/reconintra.c
-VP10_COMMON_SRCS-$(CONFIG_POSTPROC_VISUALIZER) += common/textblit.c
-VP10_COMMON_SRCS-yes += common/common_data.h
-VP10_COMMON_SRCS-yes += common/scan.c
-VP10_COMMON_SRCS-yes += common/scan.h
-VP10_COMMON_SRCS-yes += common/vp10_fwd_txfm.h
-VP10_COMMON_SRCS-yes += common/vp10_fwd_txfm.c
-
-VP10_COMMON_SRCS-$(CONFIG_VP9_POSTPROC) += common/postproc.h
-VP10_COMMON_SRCS-$(CONFIG_VP9_POSTPROC) += common/postproc.c
-VP10_COMMON_SRCS-$(CONFIG_VP9_POSTPROC) += common/mfqe.h
-VP10_COMMON_SRCS-$(CONFIG_VP9_POSTPROC) += common/mfqe.c
-ifeq ($(CONFIG_VP9_POSTPROC),yes)
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/mfqe_sse2.asm
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/postproc_sse2.asm
-endif
-
-ifneq ($(CONFIG_VP9_HIGHBITDEPTH),yes)
-VP10_COMMON_SRCS-$(HAVE_DSPR2) += common/mips/dspr2/itrans4_dspr2.c
-VP10_COMMON_SRCS-$(HAVE_DSPR2) += common/mips/dspr2/itrans8_dspr2.c
-VP10_COMMON_SRCS-$(HAVE_DSPR2) += common/mips/dspr2/itrans16_dspr2.c
-endif
-
-# common (msa)
-VP10_COMMON_SRCS-$(HAVE_MSA) += common/mips/msa/idct4x4_msa.c
-VP10_COMMON_SRCS-$(HAVE_MSA) += common/mips/msa/idct8x8_msa.c
-VP10_COMMON_SRCS-$(HAVE_MSA) += common/mips/msa/idct16x16_msa.c
-
-ifeq ($(CONFIG_VP9_POSTPROC),yes)
-VP10_COMMON_SRCS-$(HAVE_MSA) += common/mips/msa/mfqe_msa.c
-endif
-
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/idct_intrin_sse2.c
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/vp10_fwd_txfm_sse2.c
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/vp10_fwd_dct32x32_impl_sse2.h
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/vp10_fwd_txfm_impl_sse2.h
-
-ifneq ($(CONFIG_VP9_HIGHBITDEPTH),yes)
-VP10_COMMON_SRCS-$(HAVE_NEON) += common/arm/neon/iht4x4_add_neon.c
-VP10_COMMON_SRCS-$(HAVE_NEON) += common/arm/neon/iht8x8_add_neon.c
-endif
-
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/vp10_inv_txfm_sse2.c
-VP10_COMMON_SRCS-$(HAVE_SSE2) += common/x86/vp10_inv_txfm_sse2.h
-
-$(eval $(call rtcd_h_template,vp10_rtcd,vp10/common/vp10_rtcd_defs.pl))
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <stdlib.h>
-#include <string.h>
-
-#include "./vpx_config.h"
-#include "vpx/vpx_encoder.h"
-#include "vpx_ports/vpx_once.h"
-#include "vpx/internal/vpx_codec_internal.h"
-#include "./vpx_version.h"
-#include "vp10/encoder/encoder.h"
-#include "vpx/vp8cx.h"
-#include "vp10/encoder/firstpass.h"
-#include "vp10/vp10_iface_common.h"
-
-struct vp10_extracfg {
- int cpu_used; // available cpu percentage in 1/16
- unsigned int enable_auto_alt_ref;
- unsigned int noise_sensitivity;
- unsigned int sharpness;
- unsigned int static_thresh;
- unsigned int tile_columns;
- unsigned int tile_rows;
- unsigned int arnr_max_frames;
- unsigned int arnr_strength;
- unsigned int min_gf_interval;
- unsigned int max_gf_interval;
- vp8e_tuning tuning;
- unsigned int cq_level; // constrained quality level
- unsigned int rc_max_intra_bitrate_pct;
- unsigned int rc_max_inter_bitrate_pct;
- unsigned int gf_cbr_boost_pct;
- unsigned int lossless;
- unsigned int frame_parallel_decoding_mode;
- AQ_MODE aq_mode;
- unsigned int frame_periodic_boost;
- vpx_bit_depth_t bit_depth;
- vp9e_tune_content content;
- vpx_color_space_t color_space;
- int color_range;
- int render_width;
- int render_height;
-};
-
-static struct vp10_extracfg default_extra_cfg = {
- 0, // cpu_used
- 1, // enable_auto_alt_ref
- 0, // noise_sensitivity
- 0, // sharpness
- 0, // static_thresh
- 6, // tile_columns
- 0, // tile_rows
- 7, // arnr_max_frames
- 5, // arnr_strength
- 0, // min_gf_interval; 0 -> default decision
- 0, // max_gf_interval; 0 -> default decision
- VP8_TUNE_PSNR, // tuning
- 10, // cq_level
- 0, // rc_max_intra_bitrate_pct
- 0, // rc_max_inter_bitrate_pct
- 0, // gf_cbr_boost_pct
- 0, // lossless
- 1, // frame_parallel_decoding_mode
- NO_AQ, // aq_mode
- 0, // frame_periodic_delta_q
- VPX_BITS_8, // Bit depth
- VP9E_CONTENT_DEFAULT, // content
- VPX_CS_UNKNOWN, // color space
- 0, // color range
- 0, // render width
- 0, // render height
-};
-
-struct vpx_codec_alg_priv {
- vpx_codec_priv_t base;
- vpx_codec_enc_cfg_t cfg;
- struct vp10_extracfg extra_cfg;
- VP10EncoderConfig oxcf;
- VP10_COMP *cpi;
- unsigned char *cx_data;
- size_t cx_data_sz;
- unsigned char *pending_cx_data;
- size_t pending_cx_data_sz;
- int pending_frame_count;
- size_t pending_frame_sizes[8];
-#if !CONFIG_MISC_FIXES
- size_t pending_frame_magnitude;
-#endif
- vpx_image_t preview_img;
- vpx_enc_frame_flags_t next_frame_flags;
- vp8_postproc_cfg_t preview_ppcfg;
- vpx_codec_pkt_list_decl(256) pkt_list;
- unsigned int fixed_kf_cntr;
- vpx_codec_priv_output_cx_pkt_cb_pair_t output_cx_pkt_cb;
- // BufferPool that holds all reference frames.
- BufferPool *buffer_pool;
-};
-
-static VP9_REFFRAME ref_frame_to_vp10_reframe(vpx_ref_frame_type_t frame) {
- switch (frame) {
- case VP8_LAST_FRAME:
- return VP9_LAST_FLAG;
- case VP8_GOLD_FRAME:
- return VP9_GOLD_FLAG;
- case VP8_ALTR_FRAME:
- return VP9_ALT_FLAG;
- }
- assert(0 && "Invalid Reference Frame");
- return VP9_LAST_FLAG;
-}
-
-static vpx_codec_err_t update_error_state(vpx_codec_alg_priv_t *ctx,
- const struct vpx_internal_error_info *error) {
- const vpx_codec_err_t res = error->error_code;
-
- if (res != VPX_CODEC_OK)
- ctx->base.err_detail = error->has_detail ? error->detail : NULL;
-
- return res;
-}
-
-
-#undef ERROR
-#define ERROR(str) do {\
- ctx->base.err_detail = str;\
- return VPX_CODEC_INVALID_PARAM;\
- } while (0)
-
-#define RANGE_CHECK(p, memb, lo, hi) do {\
- if (!(((p)->memb == lo || (p)->memb > (lo)) && (p)->memb <= hi)) \
- ERROR(#memb " out of range ["#lo".."#hi"]");\
- } while (0)
-
-#define RANGE_CHECK_HI(p, memb, hi) do {\
- if (!((p)->memb <= (hi))) \
- ERROR(#memb " out of range [.."#hi"]");\
- } while (0)
-
-#define RANGE_CHECK_LO(p, memb, lo) do {\
- if (!((p)->memb >= (lo))) \
- ERROR(#memb " out of range ["#lo"..]");\
- } while (0)
-
-#define RANGE_CHECK_BOOL(p, memb) do {\
- if (!!((p)->memb) != (p)->memb) ERROR(#memb " expected boolean");\
- } while (0)
-
-static vpx_codec_err_t validate_config(vpx_codec_alg_priv_t *ctx,
- const vpx_codec_enc_cfg_t *cfg,
- const struct vp10_extracfg *extra_cfg) {
- RANGE_CHECK(cfg, g_w, 1, 65535); // 16 bits available
- RANGE_CHECK(cfg, g_h, 1, 65535); // 16 bits available
- RANGE_CHECK(cfg, g_timebase.den, 1, 1000000000);
- RANGE_CHECK(cfg, g_timebase.num, 1, cfg->g_timebase.den);
- RANGE_CHECK_HI(cfg, g_profile, 3);
-
- RANGE_CHECK_HI(cfg, rc_max_quantizer, 63);
- RANGE_CHECK_HI(cfg, rc_min_quantizer, cfg->rc_max_quantizer);
- RANGE_CHECK_BOOL(extra_cfg, lossless);
- RANGE_CHECK(extra_cfg, aq_mode, 0, AQ_MODE_COUNT - 1);
- RANGE_CHECK(extra_cfg, frame_periodic_boost, 0, 1);
- RANGE_CHECK_HI(cfg, g_threads, 64);
- RANGE_CHECK_HI(cfg, g_lag_in_frames, MAX_LAG_BUFFERS);
- RANGE_CHECK(cfg, rc_end_usage, VPX_VBR, VPX_Q);
- RANGE_CHECK_HI(cfg, rc_undershoot_pct, 100);
- RANGE_CHECK_HI(cfg, rc_overshoot_pct, 100);
- RANGE_CHECK_HI(cfg, rc_2pass_vbr_bias_pct, 100);
- RANGE_CHECK(cfg, kf_mode, VPX_KF_DISABLED, VPX_KF_AUTO);
- RANGE_CHECK_BOOL(cfg, rc_resize_allowed);
- RANGE_CHECK_HI(cfg, rc_dropframe_thresh, 100);
- RANGE_CHECK_HI(cfg, rc_resize_up_thresh, 100);
- RANGE_CHECK_HI(cfg, rc_resize_down_thresh, 100);
- RANGE_CHECK(cfg, g_pass, VPX_RC_ONE_PASS, VPX_RC_LAST_PASS);
- RANGE_CHECK(extra_cfg, min_gf_interval, 0, (MAX_LAG_BUFFERS - 1));
- RANGE_CHECK(extra_cfg, max_gf_interval, 0, (MAX_LAG_BUFFERS - 1));
- if (extra_cfg->max_gf_interval > 0) {
- RANGE_CHECK(extra_cfg, max_gf_interval, 2, (MAX_LAG_BUFFERS - 1));
- }
- if (extra_cfg->min_gf_interval > 0 && extra_cfg->max_gf_interval > 0) {
- RANGE_CHECK(extra_cfg, max_gf_interval, extra_cfg->min_gf_interval,
- (MAX_LAG_BUFFERS - 1));
- }
-
- if (cfg->rc_resize_allowed == 1) {
- RANGE_CHECK(cfg, rc_scaled_width, 0, cfg->g_w);
- RANGE_CHECK(cfg, rc_scaled_height, 0, cfg->g_h);
- }
-
- // Spatial/temporal scalability are not yet supported in VP10.
- // Only accept the default value for range checking.
- RANGE_CHECK(cfg, ss_number_layers, 1, 1);
- RANGE_CHECK(cfg, ts_number_layers, 1, 1);
- // VP9 does not support a lower bound on the keyframe interval in
- // automatic keyframe placement mode.
- if (cfg->kf_mode != VPX_KF_DISABLED &&
- cfg->kf_min_dist != cfg->kf_max_dist &&
- cfg->kf_min_dist > 0)
- ERROR("kf_min_dist not supported in auto mode, use 0 "
- "or kf_max_dist instead.");
-
- RANGE_CHECK(extra_cfg, enable_auto_alt_ref, 0, 2);
- RANGE_CHECK(extra_cfg, cpu_used, -8, 8);
- RANGE_CHECK_HI(extra_cfg, noise_sensitivity, 6);
- RANGE_CHECK(extra_cfg, tile_columns, 0, 6);
- RANGE_CHECK(extra_cfg, tile_rows, 0, 2);
- RANGE_CHECK_HI(extra_cfg, sharpness, 7);
- RANGE_CHECK(extra_cfg, arnr_max_frames, 0, 15);
- RANGE_CHECK_HI(extra_cfg, arnr_strength, 6);
- RANGE_CHECK(extra_cfg, cq_level, 0, 63);
- RANGE_CHECK(cfg, g_bit_depth, VPX_BITS_8, VPX_BITS_12);
- RANGE_CHECK(cfg, g_input_bit_depth, 8, 12);
- RANGE_CHECK(extra_cfg, content,
- VP9E_CONTENT_DEFAULT, VP9E_CONTENT_INVALID - 1);
-
- // TODO(yaowu): remove this when ssim tuning is implemented for vp9
- if (extra_cfg->tuning == VP8_TUNE_SSIM)
- ERROR("Option --tune=ssim is not currently supported in VP9.");
-
- if (cfg->g_pass == VPX_RC_LAST_PASS) {
- const size_t packet_sz = sizeof(FIRSTPASS_STATS);
- const int n_packets = (int)(cfg->rc_twopass_stats_in.sz / packet_sz);
- const FIRSTPASS_STATS *stats;
-
- if (cfg->rc_twopass_stats_in.buf == NULL)
- ERROR("rc_twopass_stats_in.buf not set.");
-
- if (cfg->rc_twopass_stats_in.sz % packet_sz)
- ERROR("rc_twopass_stats_in.sz indicates truncated packet.");
-
- if (cfg->rc_twopass_stats_in.sz < 2 * packet_sz)
- ERROR("rc_twopass_stats_in requires at least two packets.");
-
- stats =
- (const FIRSTPASS_STATS *)cfg->rc_twopass_stats_in.buf + n_packets - 1;
-
- if ((int)(stats->count + 0.5) != n_packets - 1)
- ERROR("rc_twopass_stats_in missing EOS stats packet");
- }
-
-#if !CONFIG_VP9_HIGHBITDEPTH
- if (cfg->g_profile > (unsigned int)PROFILE_1) {
- ERROR("Profile > 1 not supported in this build configuration");
- }
-#endif
- if (cfg->g_profile <= (unsigned int)PROFILE_1 &&
- cfg->g_bit_depth > VPX_BITS_8) {
- ERROR("Codec high bit-depth not supported in profile < 2");
- }
- if (cfg->g_profile <= (unsigned int)PROFILE_1 &&
- cfg->g_input_bit_depth > 8) {
- ERROR("Source high bit-depth not supported in profile < 2");
- }
- if (cfg->g_profile > (unsigned int)PROFILE_1 &&
- cfg->g_bit_depth == VPX_BITS_8) {
- ERROR("Codec bit-depth 8 not supported in profile > 1");
- }
- RANGE_CHECK(extra_cfg, color_space, VPX_CS_UNKNOWN, VPX_CS_SRGB);
- RANGE_CHECK(extra_cfg, color_range, 0, 1);
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t validate_img(vpx_codec_alg_priv_t *ctx,
- const vpx_image_t *img) {
- switch (img->fmt) {
- case VPX_IMG_FMT_YV12:
- case VPX_IMG_FMT_I420:
- case VPX_IMG_FMT_I42016:
- break;
- case VPX_IMG_FMT_I422:
- case VPX_IMG_FMT_I444:
- case VPX_IMG_FMT_I440:
- if (ctx->cfg.g_profile != (unsigned int)PROFILE_1) {
- ERROR("Invalid image format. I422, I444, I440 images are "
- "not supported in profile.");
- }
- break;
- case VPX_IMG_FMT_I42216:
- case VPX_IMG_FMT_I44416:
- case VPX_IMG_FMT_I44016:
- if (ctx->cfg.g_profile != (unsigned int)PROFILE_1 &&
- ctx->cfg.g_profile != (unsigned int)PROFILE_3) {
- ERROR("Invalid image format. 16-bit I422, I444, I440 images are "
- "not supported in profile.");
- }
- break;
- default:
- ERROR("Invalid image format. Only YV12, I420, I422, I444 images are "
- "supported.");
- break;
- }
-
- if (img->d_w != ctx->cfg.g_w || img->d_h != ctx->cfg.g_h)
- ERROR("Image size must match encoder init configuration size");
-
- return VPX_CODEC_OK;
-}
-
-static int get_image_bps(const vpx_image_t *img) {
- switch (img->fmt) {
- case VPX_IMG_FMT_YV12:
- case VPX_IMG_FMT_I420: return 12;
- case VPX_IMG_FMT_I422: return 16;
- case VPX_IMG_FMT_I444: return 24;
- case VPX_IMG_FMT_I440: return 16;
- case VPX_IMG_FMT_I42016: return 24;
- case VPX_IMG_FMT_I42216: return 32;
- case VPX_IMG_FMT_I44416: return 48;
- case VPX_IMG_FMT_I44016: return 32;
- default: assert(0 && "Invalid image format"); break;
- }
- return 0;
-}
-
-static vpx_codec_err_t set_encoder_config(
- VP10EncoderConfig *oxcf,
- const vpx_codec_enc_cfg_t *cfg,
- const struct vp10_extracfg *extra_cfg) {
- const int is_vbr = cfg->rc_end_usage == VPX_VBR;
- oxcf->profile = cfg->g_profile;
- oxcf->max_threads = (int)cfg->g_threads;
- oxcf->width = cfg->g_w;
- oxcf->height = cfg->g_h;
- oxcf->bit_depth = cfg->g_bit_depth;
- oxcf->input_bit_depth = cfg->g_input_bit_depth;
- // guess a frame rate if out of whack, use 30
- oxcf->init_framerate = (double)cfg->g_timebase.den / cfg->g_timebase.num;
- if (oxcf->init_framerate > 180)
- oxcf->init_framerate = 30;
-
- oxcf->mode = GOOD;
-
- switch (cfg->g_pass) {
- case VPX_RC_ONE_PASS:
- oxcf->pass = 0;
- break;
- case VPX_RC_FIRST_PASS:
- oxcf->pass = 1;
- break;
- case VPX_RC_LAST_PASS:
- oxcf->pass = 2;
- break;
- }
-
- oxcf->lag_in_frames = cfg->g_pass == VPX_RC_FIRST_PASS ? 0
- : cfg->g_lag_in_frames;
- oxcf->rc_mode = cfg->rc_end_usage;
-
- // Convert target bandwidth from Kbit/s to Bit/s
- oxcf->target_bandwidth = 1000 * cfg->rc_target_bitrate;
- oxcf->rc_max_intra_bitrate_pct = extra_cfg->rc_max_intra_bitrate_pct;
- oxcf->rc_max_inter_bitrate_pct = extra_cfg->rc_max_inter_bitrate_pct;
- oxcf->gf_cbr_boost_pct = extra_cfg->gf_cbr_boost_pct;
-
- oxcf->best_allowed_q =
- extra_cfg->lossless ? 0 : vp10_quantizer_to_qindex(cfg->rc_min_quantizer);
- oxcf->worst_allowed_q =
- extra_cfg->lossless ? 0 : vp10_quantizer_to_qindex(cfg->rc_max_quantizer);
- oxcf->cq_level = vp10_quantizer_to_qindex(extra_cfg->cq_level);
- oxcf->fixed_q = -1;
-
- oxcf->under_shoot_pct = cfg->rc_undershoot_pct;
- oxcf->over_shoot_pct = cfg->rc_overshoot_pct;
-
- oxcf->scaled_frame_width = cfg->rc_scaled_width;
- oxcf->scaled_frame_height = cfg->rc_scaled_height;
- if (cfg->rc_resize_allowed == 1) {
- oxcf->resize_mode =
- (oxcf->scaled_frame_width == 0 || oxcf->scaled_frame_height == 0) ?
- RESIZE_DYNAMIC : RESIZE_FIXED;
- } else {
- oxcf->resize_mode = RESIZE_NONE;
- }
-
- oxcf->maximum_buffer_size_ms = is_vbr ? 240000 : cfg->rc_buf_sz;
- oxcf->starting_buffer_level_ms = is_vbr ? 60000 : cfg->rc_buf_initial_sz;
- oxcf->optimal_buffer_level_ms = is_vbr ? 60000 : cfg->rc_buf_optimal_sz;
-
- oxcf->drop_frames_water_mark = cfg->rc_dropframe_thresh;
-
- oxcf->two_pass_vbrbias = cfg->rc_2pass_vbr_bias_pct;
- oxcf->two_pass_vbrmin_section = cfg->rc_2pass_vbr_minsection_pct;
- oxcf->two_pass_vbrmax_section = cfg->rc_2pass_vbr_maxsection_pct;
-
- oxcf->auto_key = cfg->kf_mode == VPX_KF_AUTO &&
- cfg->kf_min_dist != cfg->kf_max_dist;
-
- oxcf->key_freq = cfg->kf_max_dist;
-
- oxcf->speed = abs(extra_cfg->cpu_used);
- oxcf->encode_breakout = extra_cfg->static_thresh;
- oxcf->enable_auto_arf = extra_cfg->enable_auto_alt_ref;
- oxcf->noise_sensitivity = extra_cfg->noise_sensitivity;
- oxcf->sharpness = extra_cfg->sharpness;
-
- oxcf->two_pass_stats_in = cfg->rc_twopass_stats_in;
-
-#if CONFIG_FP_MB_STATS
- oxcf->firstpass_mb_stats_in = cfg->rc_firstpass_mb_stats_in;
-#endif
-
- oxcf->color_space = extra_cfg->color_space;
- oxcf->color_range = extra_cfg->color_range;
- oxcf->render_width = extra_cfg->render_width;
- oxcf->render_height = extra_cfg->render_height;
- oxcf->arnr_max_frames = extra_cfg->arnr_max_frames;
- oxcf->arnr_strength = extra_cfg->arnr_strength;
- oxcf->min_gf_interval = extra_cfg->min_gf_interval;
- oxcf->max_gf_interval = extra_cfg->max_gf_interval;
-
- oxcf->tuning = extra_cfg->tuning;
- oxcf->content = extra_cfg->content;
-
- oxcf->tile_columns = extra_cfg->tile_columns;
- oxcf->tile_rows = extra_cfg->tile_rows;
-
- oxcf->error_resilient_mode = cfg->g_error_resilient;
- oxcf->frame_parallel_decoding_mode = extra_cfg->frame_parallel_decoding_mode;
-
- oxcf->aq_mode = extra_cfg->aq_mode;
-
- oxcf->frame_periodic_boost = extra_cfg->frame_periodic_boost;
-
- /*
- printf("Current VP9 Settings: \n");
- printf("target_bandwidth: %d\n", oxcf->target_bandwidth);
- printf("noise_sensitivity: %d\n", oxcf->noise_sensitivity);
- printf("sharpness: %d\n", oxcf->sharpness);
- printf("cpu_used: %d\n", oxcf->cpu_used);
- printf("Mode: %d\n", oxcf->mode);
- printf("auto_key: %d\n", oxcf->auto_key);
- printf("key_freq: %d\n", oxcf->key_freq);
- printf("end_usage: %d\n", oxcf->end_usage);
- printf("under_shoot_pct: %d\n", oxcf->under_shoot_pct);
- printf("over_shoot_pct: %d\n", oxcf->over_shoot_pct);
- printf("starting_buffer_level: %d\n", oxcf->starting_buffer_level);
- printf("optimal_buffer_level: %d\n", oxcf->optimal_buffer_level);
- printf("maximum_buffer_size: %d\n", oxcf->maximum_buffer_size);
- printf("fixed_q: %d\n", oxcf->fixed_q);
- printf("worst_allowed_q: %d\n", oxcf->worst_allowed_q);
- printf("best_allowed_q: %d\n", oxcf->best_allowed_q);
- printf("allow_spatial_resampling: %d\n", oxcf->allow_spatial_resampling);
- printf("scaled_frame_width: %d\n", oxcf->scaled_frame_width);
- printf("scaled_frame_height: %d\n", oxcf->scaled_frame_height);
- printf("two_pass_vbrbias: %d\n", oxcf->two_pass_vbrbias);
- printf("two_pass_vbrmin_section: %d\n", oxcf->two_pass_vbrmin_section);
- printf("two_pass_vbrmax_section: %d\n", oxcf->two_pass_vbrmax_section);
- printf("lag_in_frames: %d\n", oxcf->lag_in_frames);
- printf("enable_auto_arf: %d\n", oxcf->enable_auto_arf);
- printf("Version: %d\n", oxcf->Version);
- printf("encode_breakout: %d\n", oxcf->encode_breakout);
- printf("error resilient: %d\n", oxcf->error_resilient_mode);
- printf("frame parallel detokenization: %d\n",
- oxcf->frame_parallel_decoding_mode);
- */
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t encoder_set_config(vpx_codec_alg_priv_t *ctx,
- const vpx_codec_enc_cfg_t *cfg) {
- vpx_codec_err_t res;
- int force_key = 0;
-
- if (cfg->g_w != ctx->cfg.g_w || cfg->g_h != ctx->cfg.g_h) {
- if (cfg->g_lag_in_frames > 1 || cfg->g_pass != VPX_RC_ONE_PASS)
- ERROR("Cannot change width or height after initialization");
- if (!valid_ref_frame_size(ctx->cfg.g_w, ctx->cfg.g_h, cfg->g_w, cfg->g_h) ||
- (ctx->cpi->initial_width && (int)cfg->g_w > ctx->cpi->initial_width) ||
- (ctx->cpi->initial_height && (int)cfg->g_h > ctx->cpi->initial_height))
- force_key = 1;
- }
-
- // Prevent increasing lag_in_frames. This check is stricter than it needs
- // to be -- the limit is not increasing past the first lag_in_frames
- // value, but we don't track the initial config, only the last successful
- // config.
- if (cfg->g_lag_in_frames > ctx->cfg.g_lag_in_frames)
- ERROR("Cannot increase lag_in_frames");
-
- res = validate_config(ctx, cfg, &ctx->extra_cfg);
-
- if (res == VPX_CODEC_OK) {
- ctx->cfg = *cfg;
- set_encoder_config(&ctx->oxcf, &ctx->cfg, &ctx->extra_cfg);
- // On profile change, request a key frame
- force_key |= ctx->cpi->common.profile != ctx->oxcf.profile;
- vp10_change_config(ctx->cpi, &ctx->oxcf);
- }
-
- if (force_key)
- ctx->next_frame_flags |= VPX_EFLAG_FORCE_KF;
-
- return res;
-}
-
-static vpx_codec_err_t ctrl_get_quantizer(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- int *const arg = va_arg(args, int *);
- if (arg == NULL)
- return VPX_CODEC_INVALID_PARAM;
- *arg = vp10_get_quantizer(ctx->cpi);
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t ctrl_get_quantizer64(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- int *const arg = va_arg(args, int *);
- if (arg == NULL)
- return VPX_CODEC_INVALID_PARAM;
- *arg = vp10_qindex_to_quantizer(vp10_get_quantizer(ctx->cpi));
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t update_extra_cfg(vpx_codec_alg_priv_t *ctx,
- const struct vp10_extracfg *extra_cfg) {
- const vpx_codec_err_t res = validate_config(ctx, &ctx->cfg, extra_cfg);
- if (res == VPX_CODEC_OK) {
- ctx->extra_cfg = *extra_cfg;
- set_encoder_config(&ctx->oxcf, &ctx->cfg, &ctx->extra_cfg);
- vp10_change_config(ctx->cpi, &ctx->oxcf);
- }
- return res;
-}
-
-static vpx_codec_err_t ctrl_set_cpuused(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.cpu_used = CAST(VP8E_SET_CPUUSED, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_enable_auto_alt_ref(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.enable_auto_alt_ref = CAST(VP8E_SET_ENABLEAUTOALTREF, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_noise_sensitivity(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.noise_sensitivity = CAST(VP9E_SET_NOISE_SENSITIVITY, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_sharpness(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.sharpness = CAST(VP8E_SET_SHARPNESS, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_static_thresh(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.static_thresh = CAST(VP8E_SET_STATIC_THRESHOLD, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_tile_columns(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.tile_columns = CAST(VP9E_SET_TILE_COLUMNS, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_tile_rows(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.tile_rows = CAST(VP9E_SET_TILE_ROWS, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_arnr_max_frames(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.arnr_max_frames = CAST(VP8E_SET_ARNR_MAXFRAMES, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_arnr_strength(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.arnr_strength = CAST(VP8E_SET_ARNR_STRENGTH, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_arnr_type(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- (void)ctx;
- (void)args;
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t ctrl_set_tuning(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.tuning = CAST(VP8E_SET_TUNING, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_cq_level(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.cq_level = CAST(VP8E_SET_CQ_LEVEL, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_rc_max_intra_bitrate_pct(
- vpx_codec_alg_priv_t *ctx, va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.rc_max_intra_bitrate_pct =
- CAST(VP8E_SET_MAX_INTRA_BITRATE_PCT, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_rc_max_inter_bitrate_pct(
- vpx_codec_alg_priv_t *ctx, va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.rc_max_inter_bitrate_pct =
- CAST(VP8E_SET_MAX_INTER_BITRATE_PCT, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_rc_gf_cbr_boost_pct(
- vpx_codec_alg_priv_t *ctx, va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.gf_cbr_boost_pct =
- CAST(VP9E_SET_GF_CBR_BOOST_PCT, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_lossless(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.lossless = CAST(VP9E_SET_LOSSLESS, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_frame_parallel_decoding_mode(
- vpx_codec_alg_priv_t *ctx, va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.frame_parallel_decoding_mode =
- CAST(VP9E_SET_FRAME_PARALLEL_DECODING, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_aq_mode(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.aq_mode = CAST(VP9E_SET_AQ_MODE, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_min_gf_interval(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.min_gf_interval = CAST(VP9E_SET_MIN_GF_INTERVAL, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_max_gf_interval(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.max_gf_interval = CAST(VP9E_SET_MAX_GF_INTERVAL, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_frame_periodic_boost(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.frame_periodic_boost = CAST(VP9E_SET_FRAME_PERIODIC_BOOST, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t encoder_init(vpx_codec_ctx_t *ctx,
- vpx_codec_priv_enc_mr_cfg_t *data) {
- vpx_codec_err_t res = VPX_CODEC_OK;
- (void)data;
-
- if (ctx->priv == NULL) {
- vpx_codec_alg_priv_t *const priv = vpx_calloc(1, sizeof(*priv));
- if (priv == NULL)
- return VPX_CODEC_MEM_ERROR;
-
- ctx->priv = (vpx_codec_priv_t *)priv;
- ctx->priv->init_flags = ctx->init_flags;
- ctx->priv->enc.total_encoders = 1;
- priv->buffer_pool =
- (BufferPool *)vpx_calloc(1, sizeof(BufferPool));
- if (priv->buffer_pool == NULL)
- return VPX_CODEC_MEM_ERROR;
-
-#if CONFIG_MULTITHREAD
- if (pthread_mutex_init(&priv->buffer_pool->pool_mutex, NULL)) {
- return VPX_CODEC_MEM_ERROR;
- }
-#endif
-
- if (ctx->config.enc) {
- // Update the reference to the config structure to an internal copy.
- priv->cfg = *ctx->config.enc;
- ctx->config.enc = &priv->cfg;
- }
-
- priv->extra_cfg = default_extra_cfg;
- once(vp10_initialize_enc);
-
- res = validate_config(priv, &priv->cfg, &priv->extra_cfg);
-
- if (res == VPX_CODEC_OK) {
- set_encoder_config(&priv->oxcf, &priv->cfg, &priv->extra_cfg);
-#if CONFIG_VP9_HIGHBITDEPTH
- priv->oxcf.use_highbitdepth =
- (ctx->init_flags & VPX_CODEC_USE_HIGHBITDEPTH) ? 1 : 0;
-#endif
- priv->cpi = vp10_create_compressor(&priv->oxcf, priv->buffer_pool);
- if (priv->cpi == NULL)
- res = VPX_CODEC_MEM_ERROR;
- else
- priv->cpi->output_pkt_list = &priv->pkt_list.head;
- }
- }
-
- return res;
-}
-
-static vpx_codec_err_t encoder_destroy(vpx_codec_alg_priv_t *ctx) {
- free(ctx->cx_data);
- vp10_remove_compressor(ctx->cpi);
-#if CONFIG_MULTITHREAD
- pthread_mutex_destroy(&ctx->buffer_pool->pool_mutex);
-#endif
- vpx_free(ctx->buffer_pool);
- vpx_free(ctx);
- return VPX_CODEC_OK;
-}
-
-static void pick_quickcompress_mode(vpx_codec_alg_priv_t *ctx,
- unsigned long duration,
- unsigned long deadline) {
- MODE new_mode = BEST;
-
- switch (ctx->cfg.g_pass) {
- case VPX_RC_ONE_PASS:
- if (deadline > 0) {
- const vpx_codec_enc_cfg_t *const cfg = &ctx->cfg;
-
- // Convert duration parameter from stream timebase to microseconds.
- const uint64_t duration_us = (uint64_t)duration * 1000000 *
- (uint64_t)cfg->g_timebase.num /(uint64_t)cfg->g_timebase.den;
-
- // If the deadline is more that the duration this frame is to be shown,
- // use good quality mode. Otherwise use realtime mode.
- new_mode = (deadline > duration_us) ? GOOD : REALTIME;
- } else {
- new_mode = BEST;
- }
- break;
- case VPX_RC_FIRST_PASS:
- break;
- case VPX_RC_LAST_PASS:
- new_mode = deadline > 0 ? GOOD : BEST;
- break;
- }
-
- if (ctx->oxcf.mode != new_mode) {
- ctx->oxcf.mode = new_mode;
- vp10_change_config(ctx->cpi, &ctx->oxcf);
- }
-}
-
-// Turn on to test if supplemental superframe data breaks decoding
-// #define TEST_SUPPLEMENTAL_SUPERFRAME_DATA
-static int write_superframe_index(vpx_codec_alg_priv_t *ctx) {
- uint8_t marker = 0xc0;
- unsigned int mask;
- int mag, index_sz;
-#if CONFIG_MISC_FIXES
- int i;
- size_t max_frame_sz = 0;
-#endif
-
- assert(ctx->pending_frame_count);
- assert(ctx->pending_frame_count <= 8);
-
- // Add the number of frames to the marker byte
- marker |= ctx->pending_frame_count - 1;
-#if CONFIG_MISC_FIXES
- for (i = 0; i < ctx->pending_frame_count - 1; i++) {
- const size_t frame_sz = (unsigned int) ctx->pending_frame_sizes[i] - 1;
- max_frame_sz = frame_sz > max_frame_sz ? frame_sz : max_frame_sz;
- }
-#endif
-
- // Choose the magnitude
- for (mag = 0, mask = 0xff; mag < 4; mag++) {
-#if CONFIG_MISC_FIXES
- if (max_frame_sz <= mask)
- break;
-#else
- if (ctx->pending_frame_magnitude < mask)
- break;
-#endif
- mask <<= 8;
- mask |= 0xff;
- }
- marker |= mag << 3;
-
- // Write the index
- index_sz = 2 + (mag + 1) * (ctx->pending_frame_count - CONFIG_MISC_FIXES);
- if (ctx->pending_cx_data_sz + index_sz < ctx->cx_data_sz) {
- uint8_t *x = ctx->pending_cx_data + ctx->pending_cx_data_sz;
- int i, j;
-#ifdef TEST_SUPPLEMENTAL_SUPERFRAME_DATA
- uint8_t marker_test = 0xc0;
- int mag_test = 2; // 1 - 4
- int frames_test = 4; // 1 - 8
- int index_sz_test = 2 + mag_test * frames_test;
- marker_test |= frames_test - 1;
- marker_test |= (mag_test - 1) << 3;
- *x++ = marker_test;
- for (i = 0; i < mag_test * frames_test; ++i)
- *x++ = 0; // fill up with arbitrary data
- *x++ = marker_test;
- ctx->pending_cx_data_sz += index_sz_test;
- printf("Added supplemental superframe data\n");
-#endif
-
- *x++ = marker;
- for (i = 0; i < ctx->pending_frame_count - CONFIG_MISC_FIXES; i++) {
- unsigned int this_sz;
-
- assert(ctx->pending_frame_sizes[i] > 0);
- this_sz = (unsigned int)ctx->pending_frame_sizes[i] - CONFIG_MISC_FIXES;
- for (j = 0; j <= mag; j++) {
- *x++ = this_sz & 0xff;
- this_sz >>= 8;
- }
- }
- *x++ = marker;
- ctx->pending_cx_data_sz += index_sz;
-#ifdef TEST_SUPPLEMENTAL_SUPERFRAME_DATA
- index_sz += index_sz_test;
-#endif
- }
- return index_sz;
-}
-
-// vp9 uses 10,000,000 ticks/second as time stamp
-#define TICKS_PER_SEC 10000000LL
-
-static int64_t timebase_units_to_ticks(const vpx_rational_t *timebase,
- int64_t n) {
- return n * TICKS_PER_SEC * timebase->num / timebase->den;
-}
-
-static int64_t ticks_to_timebase_units(const vpx_rational_t *timebase,
- int64_t n) {
- const int64_t round = TICKS_PER_SEC * timebase->num / 2 - 1;
- return (n * timebase->den + round) / timebase->num / TICKS_PER_SEC;
-}
-
-static vpx_codec_frame_flags_t get_frame_pkt_flags(const VP10_COMP *cpi,
- unsigned int lib_flags) {
- vpx_codec_frame_flags_t flags = lib_flags << 16;
-
- if (lib_flags & FRAMEFLAGS_KEY)
- flags |= VPX_FRAME_IS_KEY;
-
- if (cpi->droppable)
- flags |= VPX_FRAME_IS_DROPPABLE;
-
- return flags;
-}
-
-static vpx_codec_err_t encoder_encode(vpx_codec_alg_priv_t *ctx,
- const vpx_image_t *img,
- vpx_codec_pts_t pts,
- unsigned long duration,
- vpx_enc_frame_flags_t flags,
- unsigned long deadline) {
- vpx_codec_err_t res = VPX_CODEC_OK;
- VP10_COMP *const cpi = ctx->cpi;
- const vpx_rational_t *const timebase = &ctx->cfg.g_timebase;
- size_t data_sz;
-
- if (img != NULL) {
- res = validate_img(ctx, img);
- // TODO(jzern) the checks related to cpi's validity should be treated as a
- // failure condition, encoder setup is done fully in init() currently.
- if (res == VPX_CODEC_OK && cpi != NULL) {
- // There's no codec control for multiple alt-refs so check the encoder
- // instance for its status to determine the compressed data size.
- data_sz = ctx->cfg.g_w * ctx->cfg.g_h * get_image_bps(img) / 8 *
- (cpi->multi_arf_allowed ? 8 : 2);
- if (data_sz < 4096)
- data_sz = 4096;
- if (ctx->cx_data == NULL || ctx->cx_data_sz < data_sz) {
- ctx->cx_data_sz = data_sz;
- free(ctx->cx_data);
- ctx->cx_data = (unsigned char*)malloc(ctx->cx_data_sz);
- if (ctx->cx_data == NULL) {
- return VPX_CODEC_MEM_ERROR;
- }
- }
- }
- }
-
- pick_quickcompress_mode(ctx, duration, deadline);
- vpx_codec_pkt_list_init(&ctx->pkt_list);
-
- // Handle Flags
- if (((flags & VP8_EFLAG_NO_UPD_GF) && (flags & VP8_EFLAG_FORCE_GF)) ||
- ((flags & VP8_EFLAG_NO_UPD_ARF) && (flags & VP8_EFLAG_FORCE_ARF))) {
- ctx->base.err_detail = "Conflicting flags.";
- return VPX_CODEC_INVALID_PARAM;
- }
-
- vp10_apply_encoding_flags(cpi, flags);
-
- // Handle fixed keyframe intervals
- if (ctx->cfg.kf_mode == VPX_KF_AUTO &&
- ctx->cfg.kf_min_dist == ctx->cfg.kf_max_dist) {
- if (++ctx->fixed_kf_cntr > ctx->cfg.kf_min_dist) {
- flags |= VPX_EFLAG_FORCE_KF;
- ctx->fixed_kf_cntr = 1;
- }
- }
-
- // Initialize the encoder instance on the first frame.
- if (res == VPX_CODEC_OK && cpi != NULL) {
- unsigned int lib_flags = 0;
- YV12_BUFFER_CONFIG sd;
- int64_t dst_time_stamp = timebase_units_to_ticks(timebase, pts);
- int64_t dst_end_time_stamp =
- timebase_units_to_ticks(timebase, pts + duration);
- size_t size, cx_data_sz;
- unsigned char *cx_data;
-
- // Set up internal flags
- if (ctx->base.init_flags & VPX_CODEC_USE_PSNR)
- cpi->b_calculate_psnr = 1;
-
- if (img != NULL) {
- res = image2yuvconfig(img, &sd);
-
- // Store the original flags in to the frame buffer. Will extract the
- // key frame flag when we actually encode this frame.
- if (vp10_receive_raw_frame(cpi, flags | ctx->next_frame_flags,
- &sd, dst_time_stamp, dst_end_time_stamp)) {
- res = update_error_state(ctx, &cpi->common.error);
- }
- ctx->next_frame_flags = 0;
- }
-
- cx_data = ctx->cx_data;
- cx_data_sz = ctx->cx_data_sz;
-
- /* Any pending invisible frames? */
- if (ctx->pending_cx_data) {
- memmove(cx_data, ctx->pending_cx_data, ctx->pending_cx_data_sz);
- ctx->pending_cx_data = cx_data;
- cx_data += ctx->pending_cx_data_sz;
- cx_data_sz -= ctx->pending_cx_data_sz;
-
- /* TODO: this is a minimal check, the underlying codec doesn't respect
- * the buffer size anyway.
- */
- if (cx_data_sz < ctx->cx_data_sz / 2) {
- ctx->base.err_detail = "Compressed data buffer too small";
- return VPX_CODEC_ERROR;
- }
- }
-
- while (cx_data_sz >= ctx->cx_data_sz / 2 &&
- -1 != vp10_get_compressed_data(cpi, &lib_flags, &size,
- cx_data, &dst_time_stamp,
- &dst_end_time_stamp, !img)) {
- if (size) {
- vpx_codec_cx_pkt_t pkt;
-
- // Pack invisible frames with the next visible frame
- if (!cpi->common.show_frame) {
- if (ctx->pending_cx_data == 0)
- ctx->pending_cx_data = cx_data;
- ctx->pending_cx_data_sz += size;
- ctx->pending_frame_sizes[ctx->pending_frame_count++] = size;
-#if !CONFIG_MISC_FIXES
- ctx->pending_frame_magnitude |= size;
-#endif
- cx_data += size;
- cx_data_sz -= size;
-
- if (ctx->output_cx_pkt_cb.output_cx_pkt) {
- pkt.kind = VPX_CODEC_CX_FRAME_PKT;
- pkt.data.frame.pts = ticks_to_timebase_units(timebase,
- dst_time_stamp);
- pkt.data.frame.duration =
- (unsigned long)ticks_to_timebase_units(timebase,
- dst_end_time_stamp - dst_time_stamp);
- pkt.data.frame.flags = get_frame_pkt_flags(cpi, lib_flags);
- pkt.data.frame.buf = ctx->pending_cx_data;
- pkt.data.frame.sz = size;
- ctx->pending_cx_data = NULL;
- ctx->pending_cx_data_sz = 0;
- ctx->pending_frame_count = 0;
-#if !CONFIG_MISC_FIXES
- ctx->pending_frame_magnitude = 0;
-#endif
- ctx->output_cx_pkt_cb.output_cx_pkt(
- &pkt, ctx->output_cx_pkt_cb.user_priv);
- }
- continue;
- }
-
- // Add the frame packet to the list of returned packets.
- pkt.kind = VPX_CODEC_CX_FRAME_PKT;
- pkt.data.frame.pts = ticks_to_timebase_units(timebase, dst_time_stamp);
- pkt.data.frame.duration =
- (unsigned long)ticks_to_timebase_units(timebase,
- dst_end_time_stamp - dst_time_stamp);
- pkt.data.frame.flags = get_frame_pkt_flags(cpi, lib_flags);
-
- if (ctx->pending_cx_data) {
- ctx->pending_frame_sizes[ctx->pending_frame_count++] = size;
-#if !CONFIG_MISC_FIXES
- ctx->pending_frame_magnitude |= size;
-#endif
- ctx->pending_cx_data_sz += size;
- // write the superframe only for the case when
- if (!ctx->output_cx_pkt_cb.output_cx_pkt)
- size += write_superframe_index(ctx);
- pkt.data.frame.buf = ctx->pending_cx_data;
- pkt.data.frame.sz = ctx->pending_cx_data_sz;
- ctx->pending_cx_data = NULL;
- ctx->pending_cx_data_sz = 0;
- ctx->pending_frame_count = 0;
-#if !CONFIG_MISC_FIXES
- ctx->pending_frame_magnitude = 0;
-#endif
- } else {
- pkt.data.frame.buf = cx_data;
- pkt.data.frame.sz = size;
- }
- pkt.data.frame.partition_id = -1;
-
- if(ctx->output_cx_pkt_cb.output_cx_pkt)
- ctx->output_cx_pkt_cb.output_cx_pkt(&pkt,
- ctx->output_cx_pkt_cb.user_priv);
- else
- vpx_codec_pkt_list_add(&ctx->pkt_list.head, &pkt);
-
- cx_data += size;
- cx_data_sz -= size;
- }
- }
- }
-
- return res;
-}
-
-static const vpx_codec_cx_pkt_t *encoder_get_cxdata(vpx_codec_alg_priv_t *ctx,
- vpx_codec_iter_t *iter) {
- return vpx_codec_pkt_list_get(&ctx->pkt_list.head, iter);
-}
-
-static vpx_codec_err_t ctrl_set_reference(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_ref_frame_t *const frame = va_arg(args, vpx_ref_frame_t *);
-
- if (frame != NULL) {
- YV12_BUFFER_CONFIG sd;
-
- image2yuvconfig(&frame->img, &sd);
- vp10_set_reference_enc(ctx->cpi, ref_frame_to_vp10_reframe(frame->frame_type),
- &sd);
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_copy_reference(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_ref_frame_t *const frame = va_arg(args, vpx_ref_frame_t *);
-
- if (frame != NULL) {
- YV12_BUFFER_CONFIG sd;
-
- image2yuvconfig(&frame->img, &sd);
- vp10_copy_reference_enc(ctx->cpi,
- ref_frame_to_vp10_reframe(frame->frame_type), &sd);
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_get_reference(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vp9_ref_frame_t *const frame = va_arg(args, vp9_ref_frame_t *);
-
- if (frame != NULL) {
- YV12_BUFFER_CONFIG *fb = get_ref_frame(&ctx->cpi->common, frame->idx);
- if (fb == NULL) return VPX_CODEC_ERROR;
-
- yuvconfig2image(&frame->img, fb, NULL);
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_set_previewpp(vpx_codec_alg_priv_t *ctx,
- va_list args) {
-#if CONFIG_VP9_POSTPROC
- vp8_postproc_cfg_t *config = va_arg(args, vp8_postproc_cfg_t *);
- if (config != NULL) {
- ctx->preview_ppcfg = *config;
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-#else
- (void)ctx;
- (void)args;
- return VPX_CODEC_INCAPABLE;
-#endif
-}
-
-
-static vpx_image_t *encoder_get_preview(vpx_codec_alg_priv_t *ctx) {
- YV12_BUFFER_CONFIG sd;
- vp10_ppflags_t flags;
- vp10_zero(flags);
-
- if (ctx->preview_ppcfg.post_proc_flag) {
- flags.post_proc_flag = ctx->preview_ppcfg.post_proc_flag;
- flags.deblocking_level = ctx->preview_ppcfg.deblocking_level;
- flags.noise_level = ctx->preview_ppcfg.noise_level;
- }
-
- if (vp10_get_preview_raw_frame(ctx->cpi, &sd, &flags) == 0) {
- yuvconfig2image(&ctx->preview_img, &sd, NULL);
- return &ctx->preview_img;
- } else {
- return NULL;
- }
-}
-
-static vpx_codec_err_t ctrl_set_roi_map(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- (void)ctx;
- (void)args;
-
- // TODO(yaowu): Need to re-implement and test for VP9.
- return VPX_CODEC_INVALID_PARAM;
-}
-
-
-static vpx_codec_err_t ctrl_set_active_map(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_active_map_t *const map = va_arg(args, vpx_active_map_t *);
-
- if (map) {
- if (!vp10_set_active_map(ctx->cpi, map->active_map,
- (int)map->rows, (int)map->cols))
- return VPX_CODEC_OK;
- else
- return VPX_CODEC_INVALID_PARAM;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_get_active_map(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_active_map_t *const map = va_arg(args, vpx_active_map_t *);
-
- if (map) {
- if (!vp10_get_active_map(ctx->cpi, map->active_map,
- (int)map->rows, (int)map->cols))
- return VPX_CODEC_OK;
- else
- return VPX_CODEC_INVALID_PARAM;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_set_scale_mode(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_scaling_mode_t *const mode = va_arg(args, vpx_scaling_mode_t *);
-
- if (mode) {
- const int res = vp10_set_internal_size(ctx->cpi,
- (VPX_SCALING)mode->h_scaling_mode,
- (VPX_SCALING)mode->v_scaling_mode);
- return (res == 0) ? VPX_CODEC_OK : VPX_CODEC_INVALID_PARAM;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_register_cx_callback(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_codec_priv_output_cx_pkt_cb_pair_t *cbp =
- (vpx_codec_priv_output_cx_pkt_cb_pair_t *)va_arg(args, void *);
- ctx->output_cx_pkt_cb.output_cx_pkt = cbp->output_cx_pkt;
- ctx->output_cx_pkt_cb.user_priv = cbp->user_priv;
-
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t ctrl_set_tune_content(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.content = CAST(VP9E_SET_TUNE_CONTENT, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_color_space(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.color_space = CAST(VP9E_SET_COLOR_SPACE, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_color_range(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- extra_cfg.color_range = CAST(VP9E_SET_COLOR_RANGE, args);
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_err_t ctrl_set_render_size(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- struct vp10_extracfg extra_cfg = ctx->extra_cfg;
- int *const render_size = va_arg(args, int *);
- extra_cfg.render_width = render_size[0];
- extra_cfg.render_height = render_size[0];
- return update_extra_cfg(ctx, &extra_cfg);
-}
-
-static vpx_codec_ctrl_fn_map_t encoder_ctrl_maps[] = {
- {VP8_COPY_REFERENCE, ctrl_copy_reference},
-
- // Setters
- {VP8_SET_REFERENCE, ctrl_set_reference},
- {VP8_SET_POSTPROC, ctrl_set_previewpp},
- {VP8E_SET_ROI_MAP, ctrl_set_roi_map},
- {VP8E_SET_ACTIVEMAP, ctrl_set_active_map},
- {VP8E_SET_SCALEMODE, ctrl_set_scale_mode},
- {VP8E_SET_CPUUSED, ctrl_set_cpuused},
- {VP8E_SET_ENABLEAUTOALTREF, ctrl_set_enable_auto_alt_ref},
- {VP8E_SET_SHARPNESS, ctrl_set_sharpness},
- {VP8E_SET_STATIC_THRESHOLD, ctrl_set_static_thresh},
- {VP9E_SET_TILE_COLUMNS, ctrl_set_tile_columns},
- {VP9E_SET_TILE_ROWS, ctrl_set_tile_rows},
- {VP8E_SET_ARNR_MAXFRAMES, ctrl_set_arnr_max_frames},
- {VP8E_SET_ARNR_STRENGTH, ctrl_set_arnr_strength},
- {VP8E_SET_ARNR_TYPE, ctrl_set_arnr_type},
- {VP8E_SET_TUNING, ctrl_set_tuning},
- {VP8E_SET_CQ_LEVEL, ctrl_set_cq_level},
- {VP8E_SET_MAX_INTRA_BITRATE_PCT, ctrl_set_rc_max_intra_bitrate_pct},
- {VP9E_SET_MAX_INTER_BITRATE_PCT, ctrl_set_rc_max_inter_bitrate_pct},
- {VP9E_SET_GF_CBR_BOOST_PCT, ctrl_set_rc_gf_cbr_boost_pct},
- {VP9E_SET_LOSSLESS, ctrl_set_lossless},
- {VP9E_SET_FRAME_PARALLEL_DECODING, ctrl_set_frame_parallel_decoding_mode},
- {VP9E_SET_AQ_MODE, ctrl_set_aq_mode},
- {VP9E_SET_FRAME_PERIODIC_BOOST, ctrl_set_frame_periodic_boost},
- {VP9E_REGISTER_CX_CALLBACK, ctrl_register_cx_callback},
- {VP9E_SET_TUNE_CONTENT, ctrl_set_tune_content},
- {VP9E_SET_COLOR_SPACE, ctrl_set_color_space},
- {VP9E_SET_COLOR_RANGE, ctrl_set_color_range},
- {VP9E_SET_NOISE_SENSITIVITY, ctrl_set_noise_sensitivity},
- {VP9E_SET_MIN_GF_INTERVAL, ctrl_set_min_gf_interval},
- {VP9E_SET_MAX_GF_INTERVAL, ctrl_set_max_gf_interval},
- {VP9E_SET_RENDER_SIZE, ctrl_set_render_size},
-
- // Getters
- {VP8E_GET_LAST_QUANTIZER, ctrl_get_quantizer},
- {VP8E_GET_LAST_QUANTIZER_64, ctrl_get_quantizer64},
- {VP9_GET_REFERENCE, ctrl_get_reference},
- {VP9E_GET_ACTIVEMAP, ctrl_get_active_map},
-
- { -1, NULL},
-};
-
-static vpx_codec_enc_cfg_map_t encoder_usage_cfg_map[] = {
- {
- 0,
- { // NOLINT
- 0, // g_usage
- 8, // g_threads
- 0, // g_profile
-
- 320, // g_width
- 240, // g_height
- VPX_BITS_8, // g_bit_depth
- 8, // g_input_bit_depth
-
- {1, 30}, // g_timebase
-
- 0, // g_error_resilient
-
- VPX_RC_ONE_PASS, // g_pass
-
- 25, // g_lag_in_frames
-
- 0, // rc_dropframe_thresh
- 0, // rc_resize_allowed
- 0, // rc_scaled_width
- 0, // rc_scaled_height
- 60, // rc_resize_down_thresold
- 30, // rc_resize_up_thresold
-
- VPX_VBR, // rc_end_usage
- {NULL, 0}, // rc_twopass_stats_in
- {NULL, 0}, // rc_firstpass_mb_stats_in
- 256, // rc_target_bandwidth
- 0, // rc_min_quantizer
- 63, // rc_max_quantizer
- 25, // rc_undershoot_pct
- 25, // rc_overshoot_pct
-
- 6000, // rc_max_buffer_size
- 4000, // rc_buffer_initial_size
- 5000, // rc_buffer_optimal_size
-
- 50, // rc_two_pass_vbrbias
- 0, // rc_two_pass_vbrmin_section
- 2000, // rc_two_pass_vbrmax_section
-
- // keyframing settings (kf)
- VPX_KF_AUTO, // g_kfmode
- 0, // kf_min_dist
- 9999, // kf_max_dist
-
- // TODO(yunqingwang): Spatial/temporal scalability are not supported
- // in VP10. The following 10 parameters are not used, which should
- // be removed later.
- 1, // ss_number_layers
- {0},
- {0}, // ss_target_bitrate
- 1, // ts_number_layers
- {0}, // ts_target_bitrate
- {0}, // ts_rate_decimator
- 0, // ts_periodicity
- {0}, // ts_layer_id
- {0}, // layer_taget_bitrate
- 0 // temporal_layering_mode
- }
- },
-};
-
-#ifndef VERSION_STRING
-#define VERSION_STRING
-#endif
-CODEC_INTERFACE(vpx_codec_vp10_cx) = {
- "WebM Project VP10 Encoder" VERSION_STRING,
- VPX_CODEC_INTERNAL_ABI_VERSION,
-#if CONFIG_VP9_HIGHBITDEPTH
- VPX_CODEC_CAP_HIGHBITDEPTH |
-#endif
- VPX_CODEC_CAP_ENCODER | VPX_CODEC_CAP_PSNR, // vpx_codec_caps_t
- encoder_init, // vpx_codec_init_fn_t
- encoder_destroy, // vpx_codec_destroy_fn_t
- encoder_ctrl_maps, // vpx_codec_ctrl_fn_map_t
- { // NOLINT
- NULL, // vpx_codec_peek_si_fn_t
- NULL, // vpx_codec_get_si_fn_t
- NULL, // vpx_codec_decode_fn_t
- NULL, // vpx_codec_frame_get_fn_t
- NULL // vpx_codec_set_fb_fn_t
- },
- { // NOLINT
- 1, // 1 cfg map
- encoder_usage_cfg_map, // vpx_codec_enc_cfg_map_t
- encoder_encode, // vpx_codec_encode_fn_t
- encoder_get_cxdata, // vpx_codec_get_cx_data_fn_t
- encoder_set_config, // vpx_codec_enc_config_set_fn_t
- NULL, // vpx_codec_get_global_headers_fn_t
- encoder_get_preview, // vpx_codec_get_preview_frame_fn_t
- NULL // vpx_codec_enc_mr_get_mem_loc_fn_t
- }
-};
+++ /dev/null
-/*
- * Copyright (c) 2010 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-
-#include <stdlib.h>
-#include <string.h>
-
-#include "./vpx_config.h"
-#include "./vpx_version.h"
-
-#include "vpx/internal/vpx_codec_internal.h"
-#include "vpx/vp8dx.h"
-#include "vpx/vpx_decoder.h"
-#include "vpx_dsp/bitreader_buffer.h"
-#include "vpx_dsp/vpx_dsp_common.h"
-#include "vpx_util/vpx_thread.h"
-
-#include "vp10/common/alloccommon.h"
-#include "vp10/common/frame_buffers.h"
-
-#include "vp10/decoder/decoder.h"
-#include "vp10/decoder/decodeframe.h"
-
-#include "vp10/vp10_iface_common.h"
-
-#define VP9_CAP_POSTPROC (CONFIG_VP9_POSTPROC ? VPX_CODEC_CAP_POSTPROC : 0)
-
-typedef vpx_codec_stream_info_t vp10_stream_info_t;
-
-// This limit is due to framebuffer numbers.
-// TODO(hkuang): Remove this limit after implementing ondemand framebuffers.
-#define FRAME_CACHE_SIZE 6 // Cache maximum 6 decoded frames.
-
-typedef struct cache_frame {
- int fb_idx;
- vpx_image_t img;
-} cache_frame;
-
-struct vpx_codec_alg_priv {
- vpx_codec_priv_t base;
- vpx_codec_dec_cfg_t cfg;
- vp10_stream_info_t si;
- int postproc_cfg_set;
- vp8_postproc_cfg_t postproc_cfg;
- vpx_decrypt_cb decrypt_cb;
- void *decrypt_state;
- vpx_image_t img;
- int img_avail;
- int flushed;
- int invert_tile_order;
- int last_show_frame; // Index of last output frame.
- int byte_alignment;
- int skip_loop_filter;
-
- // Frame parallel related.
- int frame_parallel_decode; // frame-based threading.
- VPxWorker *frame_workers;
- int num_frame_workers;
- int next_submit_worker_id;
- int last_submit_worker_id;
- int next_output_worker_id;
- int available_threads;
- cache_frame frame_cache[FRAME_CACHE_SIZE];
- int frame_cache_write;
- int frame_cache_read;
- int num_cache_frames;
- int need_resync; // wait for key/intra-only frame
- // BufferPool that holds all reference frames. Shared by all the FrameWorkers.
- BufferPool *buffer_pool;
-
- // External frame buffer info to save for VP9 common.
- void *ext_priv; // Private data associated with the external frame buffers.
- vpx_get_frame_buffer_cb_fn_t get_ext_fb_cb;
- vpx_release_frame_buffer_cb_fn_t release_ext_fb_cb;
-};
-
-static vpx_codec_err_t decoder_init(vpx_codec_ctx_t *ctx,
- vpx_codec_priv_enc_mr_cfg_t *data) {
- // This function only allocates space for the vpx_codec_alg_priv_t
- // structure. More memory may be required at the time the stream
- // information becomes known.
- (void)data;
-
- if (!ctx->priv) {
- vpx_codec_alg_priv_t *const priv =
- (vpx_codec_alg_priv_t *)vpx_calloc(1, sizeof(*priv));
- if (priv == NULL)
- return VPX_CODEC_MEM_ERROR;
-
- ctx->priv = (vpx_codec_priv_t *)priv;
- ctx->priv->init_flags = ctx->init_flags;
- priv->si.sz = sizeof(priv->si);
- priv->flushed = 0;
- // Only do frame parallel decode when threads > 1.
- priv->frame_parallel_decode =
- (ctx->config.dec && (ctx->config.dec->threads > 1) &&
- (ctx->init_flags & VPX_CODEC_USE_FRAME_THREADING)) ? 1 : 0;
- if (ctx->config.dec) {
- priv->cfg = *ctx->config.dec;
- ctx->config.dec = &priv->cfg;
- }
- }
-
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t decoder_destroy(vpx_codec_alg_priv_t *ctx) {
- if (ctx->frame_workers != NULL) {
- int i;
- for (i = 0; i < ctx->num_frame_workers; ++i) {
- VPxWorker *const worker = &ctx->frame_workers[i];
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- vpx_get_worker_interface()->end(worker);
- vp10_remove_common(&frame_worker_data->pbi->common);
-#if CONFIG_VP9_POSTPROC
- vp10_free_postproc_buffers(&frame_worker_data->pbi->common);
-#endif
- vp10_decoder_remove(frame_worker_data->pbi);
- vpx_free(frame_worker_data->scratch_buffer);
-#if CONFIG_MULTITHREAD
- pthread_mutex_destroy(&frame_worker_data->stats_mutex);
- pthread_cond_destroy(&frame_worker_data->stats_cond);
-#endif
- vpx_free(frame_worker_data);
- }
-#if CONFIG_MULTITHREAD
- pthread_mutex_destroy(&ctx->buffer_pool->pool_mutex);
-#endif
- }
-
- if (ctx->buffer_pool) {
- vp10_free_ref_frame_buffers(ctx->buffer_pool);
- vp10_free_internal_frame_buffers(&ctx->buffer_pool->int_frame_buffers);
- }
-
- vpx_free(ctx->frame_workers);
- vpx_free(ctx->buffer_pool);
- vpx_free(ctx);
- return VPX_CODEC_OK;
-}
-
-static int parse_bitdepth_colorspace_sampling(
- BITSTREAM_PROFILE profile, struct vpx_read_bit_buffer *rb) {
- vpx_color_space_t color_space;
- if (profile >= PROFILE_2)
- rb->bit_offset += 1; // Bit-depth 10 or 12.
- color_space = (vpx_color_space_t)vpx_rb_read_literal(rb, 3);
- if (color_space != VPX_CS_SRGB) {
- rb->bit_offset += 1; // [16,235] (including xvycc) vs [0,255] range.
- if (profile == PROFILE_1 || profile == PROFILE_3) {
- rb->bit_offset += 2; // subsampling x/y.
- rb->bit_offset += 1; // unused.
- }
- } else {
- if (profile == PROFILE_1 || profile == PROFILE_3) {
- rb->bit_offset += 1; // unused
- } else {
- // RGB is only available in version 1.
- return 0;
- }
- }
- return 1;
-}
-
-static vpx_codec_err_t decoder_peek_si_internal(const uint8_t *data,
- unsigned int data_sz,
- vpx_codec_stream_info_t *si,
- int *is_intra_only,
- vpx_decrypt_cb decrypt_cb,
- void *decrypt_state) {
- int intra_only_flag = 0;
- uint8_t clear_buffer[9];
-
- if (data + data_sz <= data)
- return VPX_CODEC_INVALID_PARAM;
-
- si->is_kf = 0;
- si->w = si->h = 0;
-
- if (decrypt_cb) {
- data_sz = VPXMIN(sizeof(clear_buffer), data_sz);
- decrypt_cb(decrypt_state, data, clear_buffer, data_sz);
- data = clear_buffer;
- }
-
- {
- int show_frame;
- int error_resilient;
- struct vpx_read_bit_buffer rb = { data, data + data_sz, 0, NULL, NULL };
- const int frame_marker = vpx_rb_read_literal(&rb, 2);
- const BITSTREAM_PROFILE profile = vp10_read_profile(&rb);
-
- if (frame_marker != VP9_FRAME_MARKER)
- return VPX_CODEC_UNSUP_BITSTREAM;
-
- if (profile >= MAX_PROFILES)
- return VPX_CODEC_UNSUP_BITSTREAM;
-
- if ((profile >= 2 && data_sz <= 1) || data_sz < 1)
- return VPX_CODEC_UNSUP_BITSTREAM;
-
- if (vpx_rb_read_bit(&rb)) { // show an existing frame
- vpx_rb_read_literal(&rb, 3); // Frame buffer to show.
- return VPX_CODEC_OK;
- }
-
- if (data_sz <= 8)
- return VPX_CODEC_UNSUP_BITSTREAM;
-
- si->is_kf = !vpx_rb_read_bit(&rb);
- show_frame = vpx_rb_read_bit(&rb);
- error_resilient = vpx_rb_read_bit(&rb);
-
- if (si->is_kf) {
- if (!vp10_read_sync_code(&rb))
- return VPX_CODEC_UNSUP_BITSTREAM;
-
- if (!parse_bitdepth_colorspace_sampling(profile, &rb))
- return VPX_CODEC_UNSUP_BITSTREAM;
- vp10_read_frame_size(&rb, (int *)&si->w, (int *)&si->h);
- } else {
- intra_only_flag = show_frame ? 0 : vpx_rb_read_bit(&rb);
-
- rb.bit_offset += error_resilient ? 0 : 2; // reset_frame_context
-
- if (intra_only_flag) {
- if (!vp10_read_sync_code(&rb))
- return VPX_CODEC_UNSUP_BITSTREAM;
- if (profile > PROFILE_0) {
- if (!parse_bitdepth_colorspace_sampling(profile, &rb))
- return VPX_CODEC_UNSUP_BITSTREAM;
- }
- rb.bit_offset += REF_FRAMES; // refresh_frame_flags
- vp10_read_frame_size(&rb, (int *)&si->w, (int *)&si->h);
- }
- }
- }
- if (is_intra_only != NULL)
- *is_intra_only = intra_only_flag;
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t decoder_peek_si(const uint8_t *data,
- unsigned int data_sz,
- vpx_codec_stream_info_t *si) {
- return decoder_peek_si_internal(data, data_sz, si, NULL, NULL, NULL);
-}
-
-static vpx_codec_err_t decoder_get_si(vpx_codec_alg_priv_t *ctx,
- vpx_codec_stream_info_t *si) {
- const size_t sz = (si->sz >= sizeof(vp10_stream_info_t))
- ? sizeof(vp10_stream_info_t)
- : sizeof(vpx_codec_stream_info_t);
- memcpy(si, &ctx->si, sz);
- si->sz = (unsigned int)sz;
-
- return VPX_CODEC_OK;
-}
-
-static void set_error_detail(vpx_codec_alg_priv_t *ctx,
- const char *const error) {
- ctx->base.err_detail = error;
-}
-
-static vpx_codec_err_t update_error_state(vpx_codec_alg_priv_t *ctx,
- const struct vpx_internal_error_info *error) {
- if (error->error_code)
- set_error_detail(ctx, error->has_detail ? error->detail : NULL);
-
- return error->error_code;
-}
-
-static void init_buffer_callbacks(vpx_codec_alg_priv_t *ctx) {
- int i;
-
- for (i = 0; i < ctx->num_frame_workers; ++i) {
- VPxWorker *const worker = &ctx->frame_workers[i];
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- VP10_COMMON *const cm = &frame_worker_data->pbi->common;
- BufferPool *const pool = cm->buffer_pool;
-
- cm->new_fb_idx = INVALID_IDX;
- cm->byte_alignment = ctx->byte_alignment;
- cm->skip_loop_filter = ctx->skip_loop_filter;
-
- if (ctx->get_ext_fb_cb != NULL && ctx->release_ext_fb_cb != NULL) {
- pool->get_fb_cb = ctx->get_ext_fb_cb;
- pool->release_fb_cb = ctx->release_ext_fb_cb;
- pool->cb_priv = ctx->ext_priv;
- } else {
- pool->get_fb_cb = vp10_get_frame_buffer;
- pool->release_fb_cb = vp10_release_frame_buffer;
-
- if (vp10_alloc_internal_frame_buffers(&pool->int_frame_buffers))
- vpx_internal_error(&cm->error, VPX_CODEC_MEM_ERROR,
- "Failed to initialize internal frame buffers");
-
- pool->cb_priv = &pool->int_frame_buffers;
- }
- }
-}
-
-static void set_default_ppflags(vp8_postproc_cfg_t *cfg) {
- cfg->post_proc_flag = VP8_DEBLOCK | VP8_DEMACROBLOCK;
- cfg->deblocking_level = 4;
- cfg->noise_level = 0;
-}
-
-static void set_ppflags(const vpx_codec_alg_priv_t *ctx,
- vp10_ppflags_t *flags) {
- flags->post_proc_flag =
- ctx->postproc_cfg.post_proc_flag;
-
- flags->deblocking_level = ctx->postproc_cfg.deblocking_level;
- flags->noise_level = ctx->postproc_cfg.noise_level;
-}
-
-static int frame_worker_hook(void *arg1, void *arg2) {
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)arg1;
- const uint8_t *data = frame_worker_data->data;
- (void)arg2;
-
- frame_worker_data->result =
- vp10_receive_compressed_data(frame_worker_data->pbi,
- frame_worker_data->data_size,
- &data);
- frame_worker_data->data_end = data;
-
- if (frame_worker_data->pbi->common.frame_parallel_decode) {
- // In frame parallel decoding, a worker thread must successfully decode all
- // the compressed data.
- if (frame_worker_data->result != 0 ||
- frame_worker_data->data + frame_worker_data->data_size - 1 > data) {
- VPxWorker *const worker = frame_worker_data->pbi->frame_worker_owner;
- BufferPool *const pool = frame_worker_data->pbi->common.buffer_pool;
- // Signal all the other threads that are waiting for this frame.
- vp10_frameworker_lock_stats(worker);
- frame_worker_data->frame_context_ready = 1;
- lock_buffer_pool(pool);
- frame_worker_data->pbi->cur_buf->buf.corrupted = 1;
- unlock_buffer_pool(pool);
- frame_worker_data->pbi->need_resync = 1;
- vp10_frameworker_signal_stats(worker);
- vp10_frameworker_unlock_stats(worker);
- return 0;
- }
- } else if (frame_worker_data->result != 0) {
- // Check decode result in serial decode.
- frame_worker_data->pbi->cur_buf->buf.corrupted = 1;
- frame_worker_data->pbi->need_resync = 1;
- }
- return !frame_worker_data->result;
-}
-
-static vpx_codec_err_t init_decoder(vpx_codec_alg_priv_t *ctx) {
- int i;
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
-
- ctx->last_show_frame = -1;
- ctx->next_submit_worker_id = 0;
- ctx->last_submit_worker_id = 0;
- ctx->next_output_worker_id = 0;
- ctx->frame_cache_read = 0;
- ctx->frame_cache_write = 0;
- ctx->num_cache_frames = 0;
- ctx->need_resync = 1;
- ctx->num_frame_workers =
- (ctx->frame_parallel_decode == 1) ? ctx->cfg.threads: 1;
- if (ctx->num_frame_workers > MAX_DECODE_THREADS)
- ctx->num_frame_workers = MAX_DECODE_THREADS;
- ctx->available_threads = ctx->num_frame_workers;
- ctx->flushed = 0;
-
- ctx->buffer_pool = (BufferPool *)vpx_calloc(1, sizeof(BufferPool));
- if (ctx->buffer_pool == NULL)
- return VPX_CODEC_MEM_ERROR;
-
-#if CONFIG_MULTITHREAD
- if (pthread_mutex_init(&ctx->buffer_pool->pool_mutex, NULL)) {
- set_error_detail(ctx, "Failed to allocate buffer pool mutex");
- return VPX_CODEC_MEM_ERROR;
- }
-#endif
-
- ctx->frame_workers = (VPxWorker *)
- vpx_malloc(ctx->num_frame_workers * sizeof(*ctx->frame_workers));
- if (ctx->frame_workers == NULL) {
- set_error_detail(ctx, "Failed to allocate frame_workers");
- return VPX_CODEC_MEM_ERROR;
- }
-
- for (i = 0; i < ctx->num_frame_workers; ++i) {
- VPxWorker *const worker = &ctx->frame_workers[i];
- FrameWorkerData *frame_worker_data = NULL;
- winterface->init(worker);
- worker->data1 = vpx_memalign(32, sizeof(FrameWorkerData));
- if (worker->data1 == NULL) {
- set_error_detail(ctx, "Failed to allocate frame_worker_data");
- return VPX_CODEC_MEM_ERROR;
- }
- frame_worker_data = (FrameWorkerData *)worker->data1;
- frame_worker_data->pbi = vp10_decoder_create(ctx->buffer_pool);
- if (frame_worker_data->pbi == NULL) {
- set_error_detail(ctx, "Failed to allocate frame_worker_data");
- return VPX_CODEC_MEM_ERROR;
- }
- frame_worker_data->pbi->frame_worker_owner = worker;
- frame_worker_data->worker_id = i;
- frame_worker_data->scratch_buffer = NULL;
- frame_worker_data->scratch_buffer_size = 0;
- frame_worker_data->frame_context_ready = 0;
- frame_worker_data->received_frame = 0;
-#if CONFIG_MULTITHREAD
- if (pthread_mutex_init(&frame_worker_data->stats_mutex, NULL)) {
- set_error_detail(ctx, "Failed to allocate frame_worker_data mutex");
- return VPX_CODEC_MEM_ERROR;
- }
-
- if (pthread_cond_init(&frame_worker_data->stats_cond, NULL)) {
- set_error_detail(ctx, "Failed to allocate frame_worker_data cond");
- return VPX_CODEC_MEM_ERROR;
- }
-#endif
- // If decoding in serial mode, FrameWorker thread could create tile worker
- // thread or loopfilter thread.
- frame_worker_data->pbi->max_threads =
- (ctx->frame_parallel_decode == 0) ? ctx->cfg.threads : 0;
-
- frame_worker_data->pbi->inv_tile_order = ctx->invert_tile_order;
- frame_worker_data->pbi->common.frame_parallel_decode =
- ctx->frame_parallel_decode;
- worker->hook = (VPxWorkerHook)frame_worker_hook;
- if (!winterface->reset(worker)) {
- set_error_detail(ctx, "Frame Worker thread creation failed");
- return VPX_CODEC_MEM_ERROR;
- }
- }
-
- // If postprocessing was enabled by the application and a
- // configuration has not been provided, default it.
- if (!ctx->postproc_cfg_set &&
- (ctx->base.init_flags & VPX_CODEC_USE_POSTPROC))
- set_default_ppflags(&ctx->postproc_cfg);
-
- init_buffer_callbacks(ctx);
-
- return VPX_CODEC_OK;
-}
-
-static INLINE void check_resync(vpx_codec_alg_priv_t *const ctx,
- const VP10Decoder *const pbi) {
- // Clear resync flag if worker got a key frame or intra only frame.
- if (ctx->need_resync == 1 && pbi->need_resync == 0 &&
- (pbi->common.intra_only || pbi->common.frame_type == KEY_FRAME))
- ctx->need_resync = 0;
-}
-
-static vpx_codec_err_t decode_one(vpx_codec_alg_priv_t *ctx,
- const uint8_t **data, unsigned int data_sz,
- void *user_priv, int64_t deadline) {
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- (void)deadline;
-
- // Determine the stream parameters. Note that we rely on peek_si to
- // validate that we have a buffer that does not wrap around the top
- // of the heap.
- if (!ctx->si.h) {
- int is_intra_only = 0;
- const vpx_codec_err_t res =
- decoder_peek_si_internal(*data, data_sz, &ctx->si, &is_intra_only,
- ctx->decrypt_cb, ctx->decrypt_state);
- if (res != VPX_CODEC_OK)
- return res;
-
- if (!ctx->si.is_kf && !is_intra_only)
- return VPX_CODEC_ERROR;
- }
-
- if (!ctx->frame_parallel_decode) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- frame_worker_data->data = *data;
- frame_worker_data->data_size = data_sz;
- frame_worker_data->user_priv = user_priv;
- frame_worker_data->received_frame = 1;
-
- // Set these even if already initialized. The caller may have changed the
- // decrypt config between frames.
- frame_worker_data->pbi->decrypt_cb = ctx->decrypt_cb;
- frame_worker_data->pbi->decrypt_state = ctx->decrypt_state;
-
- worker->had_error = 0;
- winterface->execute(worker);
-
- // Update data pointer after decode.
- *data = frame_worker_data->data_end;
-
- if (worker->had_error)
- return update_error_state(ctx, &frame_worker_data->pbi->common.error);
-
- check_resync(ctx, frame_worker_data->pbi);
- } else {
- VPxWorker *const worker = &ctx->frame_workers[ctx->next_submit_worker_id];
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- // Copy context from last worker thread to next worker thread.
- if (ctx->next_submit_worker_id != ctx->last_submit_worker_id)
- vp10_frameworker_copy_context(
- &ctx->frame_workers[ctx->next_submit_worker_id],
- &ctx->frame_workers[ctx->last_submit_worker_id]);
-
- frame_worker_data->pbi->ready_for_new_data = 0;
- // Copy the compressed data into worker's internal buffer.
- // TODO(hkuang): Will all the workers allocate the same size
- // as the size of the first intra frame be better? This will
- // avoid too many deallocate and allocate.
- if (frame_worker_data->scratch_buffer_size < data_sz) {
- frame_worker_data->scratch_buffer =
- (uint8_t *)vpx_realloc(frame_worker_data->scratch_buffer, data_sz);
- if (frame_worker_data->scratch_buffer == NULL) {
- set_error_detail(ctx, "Failed to reallocate scratch buffer");
- return VPX_CODEC_MEM_ERROR;
- }
- frame_worker_data->scratch_buffer_size = data_sz;
- }
- frame_worker_data->data_size = data_sz;
- memcpy(frame_worker_data->scratch_buffer, *data, data_sz);
-
- frame_worker_data->frame_decoded = 0;
- frame_worker_data->frame_context_ready = 0;
- frame_worker_data->received_frame = 1;
- frame_worker_data->data = frame_worker_data->scratch_buffer;
- frame_worker_data->user_priv = user_priv;
-
- if (ctx->next_submit_worker_id != ctx->last_submit_worker_id)
- ctx->last_submit_worker_id =
- (ctx->last_submit_worker_id + 1) % ctx->num_frame_workers;
-
- ctx->next_submit_worker_id =
- (ctx->next_submit_worker_id + 1) % ctx->num_frame_workers;
- --ctx->available_threads;
- worker->had_error = 0;
- winterface->launch(worker);
- }
-
- return VPX_CODEC_OK;
-}
-
-static void wait_worker_and_cache_frame(vpx_codec_alg_priv_t *ctx) {
- YV12_BUFFER_CONFIG sd;
- vp10_ppflags_t flags = {0, 0, 0};
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- VPxWorker *const worker = &ctx->frame_workers[ctx->next_output_worker_id];
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- ctx->next_output_worker_id =
- (ctx->next_output_worker_id + 1) % ctx->num_frame_workers;
- // TODO(hkuang): Add worker error handling here.
- winterface->sync(worker);
- frame_worker_data->received_frame = 0;
- ++ctx->available_threads;
-
- check_resync(ctx, frame_worker_data->pbi);
-
- if (vp10_get_raw_frame(frame_worker_data->pbi, &sd, &flags) == 0) {
- VP10_COMMON *const cm = &frame_worker_data->pbi->common;
- RefCntBuffer *const frame_bufs = cm->buffer_pool->frame_bufs;
- ctx->frame_cache[ctx->frame_cache_write].fb_idx = cm->new_fb_idx;
- yuvconfig2image(&ctx->frame_cache[ctx->frame_cache_write].img, &sd,
- frame_worker_data->user_priv);
- ctx->frame_cache[ctx->frame_cache_write].img.fb_priv =
- frame_bufs[cm->new_fb_idx].raw_frame_buffer.priv;
- ctx->frame_cache_write =
- (ctx->frame_cache_write + 1) % FRAME_CACHE_SIZE;
- ++ctx->num_cache_frames;
- }
-}
-
-static vpx_codec_err_t decoder_decode(vpx_codec_alg_priv_t *ctx,
- const uint8_t *data, unsigned int data_sz,
- void *user_priv, long deadline) {
- const uint8_t *data_start = data;
- const uint8_t * const data_end = data + data_sz;
- vpx_codec_err_t res;
- uint32_t frame_sizes[8];
- int frame_count;
-
- if (data == NULL && data_sz == 0) {
- ctx->flushed = 1;
- return VPX_CODEC_OK;
- }
-
- // Reset flushed when receiving a valid frame.
- ctx->flushed = 0;
-
- // Initialize the decoder workers on the first frame.
- if (ctx->frame_workers == NULL) {
- const vpx_codec_err_t res = init_decoder(ctx);
- if (res != VPX_CODEC_OK)
- return res;
- }
-
- res = vp10_parse_superframe_index(data, data_sz, frame_sizes, &frame_count,
- ctx->decrypt_cb, ctx->decrypt_state);
- if (res != VPX_CODEC_OK)
- return res;
-
- if (ctx->frame_parallel_decode) {
- // Decode in frame parallel mode. When decoding in this mode, the frame
- // passed to the decoder must be either a normal frame or a superframe with
- // superframe index so the decoder could get each frame's start position
- // in the superframe.
- if (frame_count > 0) {
- int i;
-
- for (i = 0; i < frame_count; ++i) {
- const uint8_t *data_start_copy = data_start;
- const uint32_t frame_size = frame_sizes[i];
- if (data_start < data
- || frame_size > (uint32_t) (data_end - data_start)) {
- set_error_detail(ctx, "Invalid frame size in index");
- return VPX_CODEC_CORRUPT_FRAME;
- }
-
- if (ctx->available_threads == 0) {
- // No more threads for decoding. Wait until the next output worker
- // finishes decoding. Then copy the decoded frame into cache.
- if (ctx->num_cache_frames < FRAME_CACHE_SIZE) {
- wait_worker_and_cache_frame(ctx);
- } else {
- // TODO(hkuang): Add unit test to test this path.
- set_error_detail(ctx, "Frame output cache is full.");
- return VPX_CODEC_ERROR;
- }
- }
-
- res = decode_one(ctx, &data_start_copy, frame_size, user_priv,
- deadline);
- if (res != VPX_CODEC_OK)
- return res;
- data_start += frame_size;
- }
- } else {
- if (ctx->available_threads == 0) {
- // No more threads for decoding. Wait until the next output worker
- // finishes decoding. Then copy the decoded frame into cache.
- if (ctx->num_cache_frames < FRAME_CACHE_SIZE) {
- wait_worker_and_cache_frame(ctx);
- } else {
- // TODO(hkuang): Add unit test to test this path.
- set_error_detail(ctx, "Frame output cache is full.");
- return VPX_CODEC_ERROR;
- }
- }
-
- res = decode_one(ctx, &data, data_sz, user_priv, deadline);
- if (res != VPX_CODEC_OK)
- return res;
- }
- } else {
- // Decode in serial mode.
- if (frame_count > 0) {
- int i;
-
- for (i = 0; i < frame_count; ++i) {
- const uint8_t *data_start_copy = data_start;
- const uint32_t frame_size = frame_sizes[i];
- vpx_codec_err_t res;
- if (data_start < data
- || frame_size > (uint32_t) (data_end - data_start)) {
- set_error_detail(ctx, "Invalid frame size in index");
- return VPX_CODEC_CORRUPT_FRAME;
- }
-
- res = decode_one(ctx, &data_start_copy, frame_size, user_priv,
- deadline);
- if (res != VPX_CODEC_OK)
- return res;
-
- data_start += frame_size;
- }
- } else {
- while (data_start < data_end) {
- const uint32_t frame_size = (uint32_t) (data_end - data_start);
- const vpx_codec_err_t res = decode_one(ctx, &data_start, frame_size,
- user_priv, deadline);
- if (res != VPX_CODEC_OK)
- return res;
-
- // Account for suboptimal termination by the encoder.
- while (data_start < data_end) {
- const uint8_t marker = read_marker(ctx->decrypt_cb,
- ctx->decrypt_state, data_start);
- if (marker)
- break;
- ++data_start;
- }
- }
- }
- }
-
- return res;
-}
-
-static void release_last_output_frame(vpx_codec_alg_priv_t *ctx) {
- RefCntBuffer *const frame_bufs = ctx->buffer_pool->frame_bufs;
- // Decrease reference count of last output frame in frame parallel mode.
- if (ctx->frame_parallel_decode && ctx->last_show_frame >= 0) {
- BufferPool *const pool = ctx->buffer_pool;
- lock_buffer_pool(pool);
- decrease_ref_count(ctx->last_show_frame, frame_bufs, pool);
- unlock_buffer_pool(pool);
- }
-}
-
-static vpx_image_t *decoder_get_frame(vpx_codec_alg_priv_t *ctx,
- vpx_codec_iter_t *iter) {
- vpx_image_t *img = NULL;
-
- // Only return frame when all the cpu are busy or
- // application fluhsed the decoder in frame parallel decode.
- if (ctx->frame_parallel_decode && ctx->available_threads > 0 &&
- !ctx->flushed) {
- return NULL;
- }
-
- // Output the frames in the cache first.
- if (ctx->num_cache_frames > 0) {
- release_last_output_frame(ctx);
- ctx->last_show_frame = ctx->frame_cache[ctx->frame_cache_read].fb_idx;
- if (ctx->need_resync)
- return NULL;
- img = &ctx->frame_cache[ctx->frame_cache_read].img;
- ctx->frame_cache_read = (ctx->frame_cache_read + 1) % FRAME_CACHE_SIZE;
- --ctx->num_cache_frames;
- return img;
- }
-
- // iter acts as a flip flop, so an image is only returned on the first
- // call to get_frame.
- if (*iter == NULL && ctx->frame_workers != NULL) {
- do {
- YV12_BUFFER_CONFIG sd;
- vp10_ppflags_t flags = {0, 0, 0};
- const VPxWorkerInterface *const winterface = vpx_get_worker_interface();
- VPxWorker *const worker =
- &ctx->frame_workers[ctx->next_output_worker_id];
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- ctx->next_output_worker_id =
- (ctx->next_output_worker_id + 1) % ctx->num_frame_workers;
- if (ctx->base.init_flags & VPX_CODEC_USE_POSTPROC)
- set_ppflags(ctx, &flags);
- // Wait for the frame from worker thread.
- if (winterface->sync(worker)) {
- // Check if worker has received any frames.
- if (frame_worker_data->received_frame == 1) {
- ++ctx->available_threads;
- frame_worker_data->received_frame = 0;
- check_resync(ctx, frame_worker_data->pbi);
- }
- if (vp10_get_raw_frame(frame_worker_data->pbi, &sd, &flags) == 0) {
- VP10_COMMON *const cm = &frame_worker_data->pbi->common;
- RefCntBuffer *const frame_bufs = cm->buffer_pool->frame_bufs;
- release_last_output_frame(ctx);
- ctx->last_show_frame = frame_worker_data->pbi->common.new_fb_idx;
- if (ctx->need_resync)
- return NULL;
- yuvconfig2image(&ctx->img, &sd, frame_worker_data->user_priv);
- ctx->img.fb_priv = frame_bufs[cm->new_fb_idx].raw_frame_buffer.priv;
- img = &ctx->img;
- return img;
- }
- } else {
- // Decoding failed. Release the worker thread.
- frame_worker_data->received_frame = 0;
- ++ctx->available_threads;
- ctx->need_resync = 1;
- if (ctx->flushed != 1)
- return NULL;
- }
- } while (ctx->next_output_worker_id != ctx->next_submit_worker_id);
- }
- return NULL;
-}
-
-static vpx_codec_err_t decoder_set_fb_fn(
- vpx_codec_alg_priv_t *ctx,
- vpx_get_frame_buffer_cb_fn_t cb_get,
- vpx_release_frame_buffer_cb_fn_t cb_release, void *cb_priv) {
- if (cb_get == NULL || cb_release == NULL) {
- return VPX_CODEC_INVALID_PARAM;
- } else if (ctx->frame_workers == NULL) {
- // If the decoder has already been initialized, do not accept changes to
- // the frame buffer functions.
- ctx->get_ext_fb_cb = cb_get;
- ctx->release_ext_fb_cb = cb_release;
- ctx->ext_priv = cb_priv;
- return VPX_CODEC_OK;
- }
-
- return VPX_CODEC_ERROR;
-}
-
-static vpx_codec_err_t ctrl_set_reference(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_ref_frame_t *const data = va_arg(args, vpx_ref_frame_t *);
-
- // Only support this function in serial decode.
- if (ctx->frame_parallel_decode) {
- set_error_detail(ctx, "Not supported in frame parallel decode");
- return VPX_CODEC_INCAPABLE;
- }
-
- if (data) {
- vpx_ref_frame_t *const frame = (vpx_ref_frame_t *)data;
- YV12_BUFFER_CONFIG sd;
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- image2yuvconfig(&frame->img, &sd);
- return vp10_set_reference_dec(&frame_worker_data->pbi->common,
- (VP9_REFFRAME)frame->frame_type, &sd);
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_copy_reference(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_ref_frame_t *data = va_arg(args, vpx_ref_frame_t *);
-
- // Only support this function in serial decode.
- if (ctx->frame_parallel_decode) {
- set_error_detail(ctx, "Not supported in frame parallel decode");
- return VPX_CODEC_INCAPABLE;
- }
-
- if (data) {
- vpx_ref_frame_t *frame = (vpx_ref_frame_t *) data;
- YV12_BUFFER_CONFIG sd;
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- image2yuvconfig(&frame->img, &sd);
- return vp10_copy_reference_dec(frame_worker_data->pbi,
- (VP9_REFFRAME)frame->frame_type, &sd);
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_get_reference(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vp9_ref_frame_t *data = va_arg(args, vp9_ref_frame_t *);
-
- // Only support this function in serial decode.
- if (ctx->frame_parallel_decode) {
- set_error_detail(ctx, "Not supported in frame parallel decode");
- return VPX_CODEC_INCAPABLE;
- }
-
- if (data) {
- YV12_BUFFER_CONFIG* fb;
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- fb = get_ref_frame(&frame_worker_data->pbi->common, data->idx);
- if (fb == NULL) return VPX_CODEC_ERROR;
- yuvconfig2image(&data->img, fb, NULL);
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-}
-
-static vpx_codec_err_t ctrl_set_postproc(vpx_codec_alg_priv_t *ctx,
- va_list args) {
-#if CONFIG_VP9_POSTPROC
- vp8_postproc_cfg_t *data = va_arg(args, vp8_postproc_cfg_t *);
-
- if (data) {
- ctx->postproc_cfg_set = 1;
- ctx->postproc_cfg = *((vp8_postproc_cfg_t *)data);
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_INVALID_PARAM;
- }
-#else
- (void)ctx;
- (void)args;
- return VPX_CODEC_INCAPABLE;
-#endif
-}
-
-static vpx_codec_err_t ctrl_set_dbg_options(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- (void)ctx;
- (void)args;
- return VPX_CODEC_INCAPABLE;
-}
-
-static vpx_codec_err_t ctrl_get_last_ref_updates(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- int *const update_info = va_arg(args, int *);
-
- // Only support this function in serial decode.
- if (ctx->frame_parallel_decode) {
- set_error_detail(ctx, "Not supported in frame parallel decode");
- return VPX_CODEC_INCAPABLE;
- }
-
- if (update_info) {
- if (ctx->frame_workers) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- *update_info = frame_worker_data->pbi->refresh_frame_flags;
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_ERROR;
- }
- }
-
- return VPX_CODEC_INVALID_PARAM;
-}
-
-static vpx_codec_err_t ctrl_get_frame_corrupted(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- int *corrupted = va_arg(args, int *);
-
- if (corrupted) {
- if (ctx->frame_workers) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- RefCntBuffer *const frame_bufs =
- frame_worker_data->pbi->common.buffer_pool->frame_bufs;
- if (frame_worker_data->pbi->common.frame_to_show == NULL)
- return VPX_CODEC_ERROR;
- if (ctx->last_show_frame >= 0)
- *corrupted = frame_bufs[ctx->last_show_frame].buf.corrupted;
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_ERROR;
- }
- }
-
- return VPX_CODEC_INVALID_PARAM;
-}
-
-static vpx_codec_err_t ctrl_get_frame_size(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- int *const frame_size = va_arg(args, int *);
-
- // Only support this function in serial decode.
- if (ctx->frame_parallel_decode) {
- set_error_detail(ctx, "Not supported in frame parallel decode");
- return VPX_CODEC_INCAPABLE;
- }
-
- if (frame_size) {
- if (ctx->frame_workers) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- const VP10_COMMON *const cm = &frame_worker_data->pbi->common;
- frame_size[0] = cm->width;
- frame_size[1] = cm->height;
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_ERROR;
- }
- }
-
- return VPX_CODEC_INVALID_PARAM;
-}
-
-static vpx_codec_err_t ctrl_get_render_size(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- int *const render_size = va_arg(args, int *);
-
- // Only support this function in serial decode.
- if (ctx->frame_parallel_decode) {
- set_error_detail(ctx, "Not supported in frame parallel decode");
- return VPX_CODEC_INCAPABLE;
- }
-
- if (render_size) {
- if (ctx->frame_workers) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- const VP10_COMMON *const cm = &frame_worker_data->pbi->common;
- render_size[0] = cm->render_width;
- render_size[1] = cm->render_height;
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_ERROR;
- }
- }
-
- return VPX_CODEC_INVALID_PARAM;
-}
-
-static vpx_codec_err_t ctrl_get_bit_depth(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- unsigned int *const bit_depth = va_arg(args, unsigned int *);
- VPxWorker *const worker = &ctx->frame_workers[ctx->next_output_worker_id];
-
- if (bit_depth) {
- if (worker) {
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- const VP10_COMMON *const cm = &frame_worker_data->pbi->common;
- *bit_depth = cm->bit_depth;
- return VPX_CODEC_OK;
- } else {
- return VPX_CODEC_ERROR;
- }
- }
-
- return VPX_CODEC_INVALID_PARAM;
-}
-
-static vpx_codec_err_t ctrl_set_invert_tile_order(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- ctx->invert_tile_order = va_arg(args, int);
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t ctrl_set_decryptor(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- vpx_decrypt_init *init = va_arg(args, vpx_decrypt_init *);
- ctx->decrypt_cb = init ? init->decrypt_cb : NULL;
- ctx->decrypt_state = init ? init->decrypt_state : NULL;
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t ctrl_set_byte_alignment(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- const int legacy_byte_alignment = 0;
- const int min_byte_alignment = 32;
- const int max_byte_alignment = 1024;
- const int byte_alignment = va_arg(args, int);
-
- if (byte_alignment != legacy_byte_alignment &&
- (byte_alignment < min_byte_alignment ||
- byte_alignment > max_byte_alignment ||
- (byte_alignment & (byte_alignment - 1)) != 0))
- return VPX_CODEC_INVALID_PARAM;
-
- ctx->byte_alignment = byte_alignment;
- if (ctx->frame_workers) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data =
- (FrameWorkerData *)worker->data1;
- frame_worker_data->pbi->common.byte_alignment = byte_alignment;
- }
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_err_t ctrl_set_skip_loop_filter(vpx_codec_alg_priv_t *ctx,
- va_list args) {
- ctx->skip_loop_filter = va_arg(args, int);
-
- if (ctx->frame_workers) {
- VPxWorker *const worker = ctx->frame_workers;
- FrameWorkerData *const frame_worker_data = (FrameWorkerData *)worker->data1;
- frame_worker_data->pbi->common.skip_loop_filter = ctx->skip_loop_filter;
- }
-
- return VPX_CODEC_OK;
-}
-
-static vpx_codec_ctrl_fn_map_t decoder_ctrl_maps[] = {
- {VP8_COPY_REFERENCE, ctrl_copy_reference},
-
- // Setters
- {VP8_SET_REFERENCE, ctrl_set_reference},
- {VP8_SET_POSTPROC, ctrl_set_postproc},
- {VP8_SET_DBG_COLOR_REF_FRAME, ctrl_set_dbg_options},
- {VP8_SET_DBG_COLOR_MB_MODES, ctrl_set_dbg_options},
- {VP8_SET_DBG_COLOR_B_MODES, ctrl_set_dbg_options},
- {VP8_SET_DBG_DISPLAY_MV, ctrl_set_dbg_options},
- {VP9_INVERT_TILE_DECODE_ORDER, ctrl_set_invert_tile_order},
- {VPXD_SET_DECRYPTOR, ctrl_set_decryptor},
- {VP9_SET_BYTE_ALIGNMENT, ctrl_set_byte_alignment},
- {VP9_SET_SKIP_LOOP_FILTER, ctrl_set_skip_loop_filter},
-
- // Getters
- {VP8D_GET_LAST_REF_UPDATES, ctrl_get_last_ref_updates},
- {VP8D_GET_FRAME_CORRUPTED, ctrl_get_frame_corrupted},
- {VP9_GET_REFERENCE, ctrl_get_reference},
- {VP9D_GET_DISPLAY_SIZE, ctrl_get_render_size},
- {VP9D_GET_BIT_DEPTH, ctrl_get_bit_depth},
- {VP9D_GET_FRAME_SIZE, ctrl_get_frame_size},
-
- { -1, NULL},
-};
-
-#ifndef VERSION_STRING
-#define VERSION_STRING
-#endif
-CODEC_INTERFACE(vpx_codec_vp10_dx) = {
- "WebM Project VP10 Decoder" VERSION_STRING,
- VPX_CODEC_INTERNAL_ABI_VERSION,
- VPX_CODEC_CAP_DECODER | VP9_CAP_POSTPROC |
- VPX_CODEC_CAP_EXTERNAL_FRAME_BUFFER, // vpx_codec_caps_t
- decoder_init, // vpx_codec_init_fn_t
- decoder_destroy, // vpx_codec_destroy_fn_t
- decoder_ctrl_maps, // vpx_codec_ctrl_fn_map_t
- { // NOLINT
- decoder_peek_si, // vpx_codec_peek_si_fn_t
- decoder_get_si, // vpx_codec_get_si_fn_t
- decoder_decode, // vpx_codec_decode_fn_t
- decoder_get_frame, // vpx_codec_frame_get_fn_t
- decoder_set_fb_fn, // vpx_codec_set_fb_fn_t
- },
- { // NOLINT
- 0,
- NULL, // vpx_codec_enc_cfg_map_t
- NULL, // vpx_codec_encode_fn_t
- NULL, // vpx_codec_get_cx_data_fn_t
- NULL, // vpx_codec_enc_config_set_fn_t
- NULL, // vpx_codec_get_global_headers_fn_t
- NULL, // vpx_codec_get_preview_frame_fn_t
- NULL // vpx_codec_enc_mr_get_mem_loc_fn_t
- }
-};
+++ /dev/null
-/*
- * Copyright (c) 2013 The WebM project authors. All Rights Reserved.
- *
- * Use of this source code is governed by a BSD-style license
- * that can be found in the LICENSE file in the root of the source
- * tree. An additional intellectual property rights grant can be found
- * in the file PATENTS. All contributing project authors may
- * be found in the AUTHORS file in the root of the source tree.
- */
-#ifndef VP10_VP10_IFACE_COMMON_H_
-#define VP10_VP10_IFACE_COMMON_H_
-
-#include "vpx_ports/mem.h"
-
-static void yuvconfig2image(vpx_image_t *img, const YV12_BUFFER_CONFIG *yv12,
- void *user_priv) {
- /** vpx_img_wrap() doesn't allow specifying independent strides for
- * the Y, U, and V planes, nor other alignment adjustments that
- * might be representable by a YV12_BUFFER_CONFIG, so we just
- * initialize all the fields.*/
- int bps;
- if (!yv12->subsampling_y) {
- if (!yv12->subsampling_x) {
- img->fmt = VPX_IMG_FMT_I444;
- bps = 24;
- } else {
- img->fmt = VPX_IMG_FMT_I422;
- bps = 16;
- }
- } else {
- if (!yv12->subsampling_x) {
- img->fmt = VPX_IMG_FMT_I440;
- bps = 16;
- } else {
- img->fmt = VPX_IMG_FMT_I420;
- bps = 12;
- }
- }
- img->cs = yv12->color_space;
- img->range = yv12->color_range;
- img->bit_depth = 8;
- img->w = yv12->y_stride;
- img->h = ALIGN_POWER_OF_TWO(yv12->y_height + 2 * VP9_ENC_BORDER_IN_PIXELS, 3);
- img->d_w = yv12->y_crop_width;
- img->d_h = yv12->y_crop_height;
- img->r_w = yv12->render_width;
- img->r_h = yv12->render_height;
- img->x_chroma_shift = yv12->subsampling_x;
- img->y_chroma_shift = yv12->subsampling_y;
- img->planes[VPX_PLANE_Y] = yv12->y_buffer;
- img->planes[VPX_PLANE_U] = yv12->u_buffer;
- img->planes[VPX_PLANE_V] = yv12->v_buffer;
- img->planes[VPX_PLANE_ALPHA] = NULL;
- img->stride[VPX_PLANE_Y] = yv12->y_stride;
- img->stride[VPX_PLANE_U] = yv12->uv_stride;
- img->stride[VPX_PLANE_V] = yv12->uv_stride;
- img->stride[VPX_PLANE_ALPHA] = yv12->y_stride;
-#if CONFIG_VP9_HIGHBITDEPTH
- if (yv12->flags & YV12_FLAG_HIGHBITDEPTH) {
- // vpx_image_t uses byte strides and a pointer to the first byte
- // of the image.
- img->fmt = (vpx_img_fmt_t)(img->fmt | VPX_IMG_FMT_HIGHBITDEPTH);
- img->bit_depth = yv12->bit_depth;
- img->planes[VPX_PLANE_Y] = (uint8_t*)CONVERT_TO_SHORTPTR(yv12->y_buffer);
- img->planes[VPX_PLANE_U] = (uint8_t*)CONVERT_TO_SHORTPTR(yv12->u_buffer);
- img->planes[VPX_PLANE_V] = (uint8_t*)CONVERT_TO_SHORTPTR(yv12->v_buffer);
- img->planes[VPX_PLANE_ALPHA] = NULL;
- img->stride[VPX_PLANE_Y] = 2 * yv12->y_stride;
- img->stride[VPX_PLANE_U] = 2 * yv12->uv_stride;
- img->stride[VPX_PLANE_V] = 2 * yv12->uv_stride;
- img->stride[VPX_PLANE_ALPHA] = 2 * yv12->y_stride;
- }
-#endif // CONFIG_VP9_HIGHBITDEPTH
- img->bps = bps;
- img->user_priv = user_priv;
- img->img_data = yv12->buffer_alloc;
- img->img_data_owner = 0;
- img->self_allocd = 0;
-}
-
-static vpx_codec_err_t image2yuvconfig(const vpx_image_t *img,
- YV12_BUFFER_CONFIG *yv12) {
- yv12->y_buffer = img->planes[VPX_PLANE_Y];
- yv12->u_buffer = img->planes[VPX_PLANE_U];
- yv12->v_buffer = img->planes[VPX_PLANE_V];
-
- yv12->y_crop_width = img->d_w;
- yv12->y_crop_height = img->d_h;
- yv12->render_width = img->r_w;
- yv12->render_height = img->r_h;
- yv12->y_width = img->d_w;
- yv12->y_height = img->d_h;
-
- yv12->uv_width = img->x_chroma_shift == 1 ? (1 + yv12->y_width) / 2
- : yv12->y_width;
- yv12->uv_height = img->y_chroma_shift == 1 ? (1 + yv12->y_height) / 2
- : yv12->y_height;
- yv12->uv_crop_width = yv12->uv_width;
- yv12->uv_crop_height = yv12->uv_height;
-
- yv12->y_stride = img->stride[VPX_PLANE_Y];
- yv12->uv_stride = img->stride[VPX_PLANE_U];
- yv12->color_space = img->cs;
- yv12->color_range = img->range;
-
-#if CONFIG_VP9_HIGHBITDEPTH
- if (img->fmt & VPX_IMG_FMT_HIGHBITDEPTH) {
- // In vpx_image_t
- // planes point to uint8 address of start of data
- // stride counts uint8s to reach next row
- // In YV12_BUFFER_CONFIG
- // y_buffer, u_buffer, v_buffer point to uint16 address of data
- // stride and border counts in uint16s
- // This means that all the address calculations in the main body of code
- // should work correctly.
- // However, before we do any pixel operations we need to cast the address
- // to a uint16 ponter and double its value.
- yv12->y_buffer = CONVERT_TO_BYTEPTR(yv12->y_buffer);
- yv12->u_buffer = CONVERT_TO_BYTEPTR(yv12->u_buffer);
- yv12->v_buffer = CONVERT_TO_BYTEPTR(yv12->v_buffer);
- yv12->y_stride >>= 1;
- yv12->uv_stride >>= 1;
- yv12->flags = YV12_FLAG_HIGHBITDEPTH;
- } else {
- yv12->flags = 0;
- }
- yv12->border = (yv12->y_stride - img->w) / 2;
-#else
- yv12->border = (img->stride[VPX_PLANE_Y] - img->w) / 2;
-#endif // CONFIG_VP9_HIGHBITDEPTH
- yv12->subsampling_x = img->x_chroma_shift;
- yv12->subsampling_y = img->y_chroma_shift;
- return VPX_CODEC_OK;
-}
-
-#endif // VP10_VP10_IFACE_COMMON_H_
+++ /dev/null
-##
-## Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-##
-## Use of this source code is governed by a BSD-style license
-## that can be found in the LICENSE file in the root of the source
-## tree. An additional intellectual property rights grant can be found
-## in the file PATENTS. All contributing project authors may
-## be found in the AUTHORS file in the root of the source tree.
-##
-
-VP10_CX_EXPORTS += exports_enc
-
-VP10_CX_SRCS-yes += $(VP10_COMMON_SRCS-yes)
-VP10_CX_SRCS-no += $(VP10_COMMON_SRCS-no)
-VP10_CX_SRCS_REMOVE-yes += $(VP10_COMMON_SRCS_REMOVE-yes)
-VP10_CX_SRCS_REMOVE-no += $(VP10_COMMON_SRCS_REMOVE-no)
-
-VP10_CX_SRCS-yes += vp10_cx_iface.c
-
-VP10_CX_SRCS-yes += encoder/avg.c
-VP10_CX_SRCS-yes += encoder/bitstream.c
-VP10_CX_SRCS-yes += encoder/context_tree.c
-VP10_CX_SRCS-yes += encoder/context_tree.h
-VP10_CX_SRCS-yes += encoder/cost.h
-VP10_CX_SRCS-yes += encoder/cost.c
-VP10_CX_SRCS-yes += encoder/dct.c
-VP10_CX_SRCS-$(CONFIG_VP9_TEMPORAL_DENOISING) += encoder/denoiser.c
-VP10_CX_SRCS-$(CONFIG_VP9_TEMPORAL_DENOISING) += encoder/denoiser.h
-VP10_CX_SRCS-yes += encoder/encodeframe.c
-VP10_CX_SRCS-yes += encoder/encodeframe.h
-VP10_CX_SRCS-yes += encoder/encodemb.c
-VP10_CX_SRCS-yes += encoder/encodemv.c
-VP10_CX_SRCS-yes += encoder/ethread.h
-VP10_CX_SRCS-yes += encoder/ethread.c
-VP10_CX_SRCS-yes += encoder/extend.c
-VP10_CX_SRCS-yes += encoder/firstpass.c
-VP10_CX_SRCS-yes += encoder/block.h
-VP10_CX_SRCS-yes += encoder/bitstream.h
-VP10_CX_SRCS-yes += encoder/encodemb.h
-VP10_CX_SRCS-yes += encoder/encodemv.h
-VP10_CX_SRCS-yes += encoder/extend.h
-VP10_CX_SRCS-yes += encoder/firstpass.h
-VP10_CX_SRCS-yes += encoder/lookahead.c
-VP10_CX_SRCS-yes += encoder/lookahead.h
-VP10_CX_SRCS-yes += encoder/mcomp.h
-VP10_CX_SRCS-yes += encoder/encoder.h
-VP10_CX_SRCS-yes += encoder/quantize.h
-VP10_CX_SRCS-yes += encoder/ratectrl.h
-VP10_CX_SRCS-yes += encoder/rd.h
-VP10_CX_SRCS-yes += encoder/rdopt.h
-VP10_CX_SRCS-yes += encoder/tokenize.h
-VP10_CX_SRCS-yes += encoder/treewriter.h
-VP10_CX_SRCS-yes += encoder/mcomp.c
-VP10_CX_SRCS-yes += encoder/encoder.c
-VP10_CX_SRCS-yes += encoder/palette.h
-VP10_CX_SRCS-yes += encoder/palette.c
-VP10_CX_SRCS-yes += encoder/picklpf.c
-VP10_CX_SRCS-yes += encoder/picklpf.h
-VP10_CX_SRCS-yes += encoder/quantize.c
-VP10_CX_SRCS-yes += encoder/ratectrl.c
-VP10_CX_SRCS-yes += encoder/rd.c
-VP10_CX_SRCS-yes += encoder/rdopt.c
-VP10_CX_SRCS-yes += encoder/segmentation.c
-VP10_CX_SRCS-yes += encoder/segmentation.h
-VP10_CX_SRCS-yes += encoder/speed_features.c
-VP10_CX_SRCS-yes += encoder/speed_features.h
-VP10_CX_SRCS-yes += encoder/subexp.c
-VP10_CX_SRCS-yes += encoder/subexp.h
-VP10_CX_SRCS-yes += encoder/resize.c
-VP10_CX_SRCS-yes += encoder/resize.h
-VP10_CX_SRCS-$(CONFIG_INTERNAL_STATS) += encoder/blockiness.c
-
-VP10_CX_SRCS-yes += encoder/tokenize.c
-VP10_CX_SRCS-yes += encoder/treewriter.c
-VP10_CX_SRCS-yes += encoder/aq_variance.c
-VP10_CX_SRCS-yes += encoder/aq_variance.h
-VP10_CX_SRCS-yes += encoder/aq_cyclicrefresh.c
-VP10_CX_SRCS-yes += encoder/aq_cyclicrefresh.h
-VP10_CX_SRCS-yes += encoder/aq_complexity.c
-VP10_CX_SRCS-yes += encoder/aq_complexity.h
-VP10_CX_SRCS-yes += encoder/skin_detection.c
-VP10_CX_SRCS-yes += encoder/skin_detection.h
-ifeq ($(CONFIG_VP9_POSTPROC),yes)
-VP10_CX_SRCS-$(CONFIG_INTERNAL_STATS) += common/postproc.h
-VP10_CX_SRCS-$(CONFIG_INTERNAL_STATS) += common/postproc.c
-endif
-VP10_CX_SRCS-yes += encoder/temporal_filter.c
-VP10_CX_SRCS-yes += encoder/temporal_filter.h
-VP10_CX_SRCS-yes += encoder/mbgraph.c
-VP10_CX_SRCS-yes += encoder/mbgraph.h
-
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/avg_intrin_sse2.c
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/temporal_filter_apply_sse2.asm
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/quantize_sse2.c
-ifeq ($(CONFIG_VP9_HIGHBITDEPTH),yes)
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/highbd_block_error_intrin_sse2.c
-endif
-
-ifeq ($(CONFIG_USE_X86INC),yes)
-VP10_CX_SRCS-$(HAVE_MMX) += encoder/x86/dct_mmx.asm
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/error_sse2.asm
-endif
-
-ifeq ($(ARCH_X86_64),yes)
-ifeq ($(CONFIG_USE_X86INC),yes)
-VP10_CX_SRCS-$(HAVE_SSSE3) += encoder/x86/quantize_ssse3_x86_64.asm
-VP10_CX_SRCS-$(HAVE_SSSE3) += encoder/x86/dct_ssse3_x86_64.asm
-endif
-endif
-
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/dct_sse2.c
-VP10_CX_SRCS-$(HAVE_SSSE3) += encoder/x86/dct_ssse3.c
-
-ifeq ($(CONFIG_VP9_TEMPORAL_DENOISING),yes)
-VP10_CX_SRCS-$(HAVE_SSE2) += encoder/x86/denoiser_sse2.c
-endif
-
-VP10_CX_SRCS-$(HAVE_AVX2) += encoder/x86/error_intrin_avx2.c
-
-ifneq ($(CONFIG_VP9_HIGHBITDEPTH),yes)
-VP10_CX_SRCS-$(HAVE_NEON) += encoder/arm/neon/dct_neon.c
-VP10_CX_SRCS-$(HAVE_NEON) += encoder/arm/neon/error_neon.c
-endif
-VP10_CX_SRCS-$(HAVE_NEON) += encoder/arm/neon/avg_neon.c
-VP10_CX_SRCS-$(HAVE_NEON) += encoder/arm/neon/quantize_neon.c
-
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/avg_msa.c
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/error_msa.c
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/fdct4x4_msa.c
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/fdct8x8_msa.c
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/fdct16x16_msa.c
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/fdct_msa.h
-VP10_CX_SRCS-$(HAVE_MSA) += encoder/mips/msa/temporal_filter_msa.c
-
-VP10_CX_SRCS-yes := $(filter-out $(VP10_CX_SRCS_REMOVE-yes),$(VP10_CX_SRCS-yes))
+++ /dev/null
-##
-## Copyright (c) 2010 The WebM project authors. All Rights Reserved.
-##
-## Use of this source code is governed by a BSD-style license
-## that can be found in the LICENSE file in the root of the source
-## tree. An additional intellectual property rights grant can be found
-## in the file PATENTS. All contributing project authors may
-## be found in the AUTHORS file in the root of the source tree.
-##
-
-VP10_DX_EXPORTS += exports_dec
-
-VP10_DX_SRCS-yes += $(VP10_COMMON_SRCS-yes)
-VP10_DX_SRCS-no += $(VP10_COMMON_SRCS-no)
-VP10_DX_SRCS_REMOVE-yes += $(VP10_COMMON_SRCS_REMOVE-yes)
-VP10_DX_SRCS_REMOVE-no += $(VP10_COMMON_SRCS_REMOVE-no)
-
-VP10_DX_SRCS-yes += vp10_dx_iface.c
-
-VP10_DX_SRCS-yes += decoder/decodemv.c
-VP10_DX_SRCS-yes += decoder/decodeframe.c
-VP10_DX_SRCS-yes += decoder/decodeframe.h
-VP10_DX_SRCS-yes += decoder/detokenize.c
-VP10_DX_SRCS-yes += decoder/decodemv.h
-VP10_DX_SRCS-yes += decoder/detokenize.h
-VP10_DX_SRCS-yes += decoder/dthread.c
-VP10_DX_SRCS-yes += decoder/dthread.h
-VP10_DX_SRCS-yes += decoder/decoder.c
-VP10_DX_SRCS-yes += decoder/decoder.h
-VP10_DX_SRCS-yes += decoder/dsubexp.c
-VP10_DX_SRCS-yes += decoder/dsubexp.h
-
-VP10_DX_SRCS-yes := $(filter-out $(VP10_DX_SRCS_REMOVE-yes),$(VP10_DX_SRCS-yes))
projects / platform / upstream / libvpx.git / commitdiff