Consistently use get_prob(), clip_prob() and newly added clip_pixel().
authorRonald S. Bultje <rbultje@google.com>
Mon, 10 Dec 2012 20:09:07 +0000 (12:09 -0800)
committerRonald S. Bultje <rbultje@google.com>
Wed, 12 Dec 2012 18:01:19 +0000 (10:01 -0800)
commit4d0ec7aacd2227b1b98d1f5100bde64c7797b962
tree5e2e16ade1e5a05e66108f43ab992c8632b69fc1
parentd1244659757ff64212f7beebc6c8e45909d5113c
Consistently use get_prob(), clip_prob() and newly added clip_pixel().

Add a function clip_pixel() to clip a pixel value to the [0,255] range
of allowed values, and use this where-ever appropriate (e.g. prediction,
reconstruction). Likewise, consistently use the recently added function
clip_prob(), which calculates a binary probability in the [1,255] range.
If possible, try to use get_prob() or its sister get_binary_prob() to
calculate binary probabilities, for consistency.

Since in some places, this means that binary probability calculations
are changed (we use {255,256}*count0/(total) in a range of places,
and all of these are now changed to use 256*count0+(total>>1)/total),
this changes the encoding result, so this patch warrants some extensive
testing.

Change-Id: Ibeeff8d886496839b8e0c0ace9ccc552351f7628
20 files changed:
vp9/common/vp9_common.h
vp9/common/vp9_entropy.c
vp9/common/vp9_entropymode.c
vp9/common/vp9_entropymv.c
vp9/common/vp9_filter.c
vp9/common/vp9_idctllm.c
vp9/common/vp9_pred_common.c
vp9/common/vp9_recon.c
vp9/common/vp9_reconinter.c
vp9/common/vp9_reconintra.c
vp9/common/vp9_reconintra4x4.c
vp9/common/vp9_treecoder.c
vp9/common/vp9_treecoder.h
vp9/decoder/vp9_dequantize.c
vp9/encoder/vp9_bitstream.c
vp9/encoder/vp9_mcomp.c
vp9/encoder/vp9_onyx_if.c
vp9/encoder/vp9_rdopt.c
vp9/encoder/vp9_segmentation.c
vp9/encoder/vp9_tokenize.c