lagarith: reallocate rgb_planes when needed

Fixes invalid writes on pixel format changes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 4c3e1956ee35fdcc5ffdb28782050164b4623c0b)
(cherry picked from commit bd57e783437f990c3ac4747eeebe20332e103980)

lagarith: avoid infinite loop in lag_rac_refill()

range == 0 happens with corrupted files

CC:libav-stable@libav.org
(cherry picked from commit de6dfa2bb82df916a67e5036b0ef96a944781ed3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 8bce2c60b8ebc31899d576dde3bbe6205faae97d)

lagarith: pad RGB buffer by 1 byte.

For left HFYU prediction, we predict from the buffer buf+1 using 8- or
16-byte reads. This means that aligning the buffer by 16 bytes is in
itself not sufficient, because if the width itself is 16- or 8-byte
aligned, the buffer will not be padded, and thus a read of size 16 at
buf+1 will overflow boundaries at the right edge. Padding the buffer by
1 byte is sufficient to not overflow its boundaries.

Fixes bug 342.

(cherry picked from commit 98d0d19208959766a58f13dd6a678d1f765a26ac)

truemotion1: check the header size

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 2240e2078d53d3cfce8ff1dda64e58fa72038602)
(cherry picked from commit 76b40a9bf93e387d98aa7dc02ec7a8d13f51722f)

shorten: pad the internal bitstream buffer

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 1713eec29add37b654ec6bf262b843d139c1ffc6)
(cherry picked from commit 5881ec0ea58a95403bd375b63f22d49905cdd8e5)

samplefmt: avoid integer overflow in av_samples_get_buffer_size()

CC:libav-stable@libav.org
(cherry picked from commit 0e830094ad0dc251613a0aa3234d9c5c397e02e6)
(cherry picked from commit e9b3abd49890e958c745ea46a9f4f91b6b4baa58)

Conflicts:
libavutil/samplefmt.c

h264: Fix a typo from the previous commit

f777504f640260337974848c7d5d7a3f064bbb45 changed a - in +

CC: libav-stable@libav.org
(cherry picked from commit d922c5a5fbaf0b6c73bd8c81ae059bc6e406961c)
(cherry picked from commit 3ce77e04c2ca4b9e7fa6b94b51e8d7c5f188da86)
(cherry picked from commit 8cba6f58c8acaa0ca6749110a2746bbe60ff2dab)

h264: Lower bound check for slice offsets

And use the value from the specification.

Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit f777504f640260337974848c7d5d7a3f064bbb45)
(cherry picked from commit 5bd083d0216d9ee649039c84999fb61386536ac1)

Conflicts:
libavcodec/h264.c

(cherry picked from commit 41380e017afcca3119acb560c08a60a97d416c3c)

Conflicts:
libavcodec/h264.c

rpza: limit the number of blocks to the total remaining blocks in the frame

Fixes invalid writes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 77bb0004bbe18f1498cfecdc68db5f10808b6599)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Prepare for 0.8.11 Release

lavf: make av_probe_input_buffer more robust

Always use the actually read size as the offset instead of making
possibly invalid assumptions.

Addresses: CVE-2012-6618

(cherry picked from commit 2115a3597457231a6e5c0527fe0ff8550f64b733)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavformat/utils.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8575f5362f98c937758b20ff8512d6767a56208e)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Updated Changelog for 0.8.10

oggparseogm: check timing variables

Fixes a potential divide by zero.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 75647dea6f7db79b409bad66a119f5c73da730f3)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit bf7c240a50f8ed99a42e08bb7a8a70262cce34ad)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

mathematics: remove asserts from av_rescale_rnd()

It is a public function, it must not assert on its parameters.

(cherry picked from commit 94a417acc05cc5151b473abc0bf51fad26f8c5a0)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 03bfd8419fbaf9c72b293457437bd508dea64736)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

vc1: Always reset numref when parsing a new frame header.

Fixes an issue where the B-frame coding mode switches from interlaced
fields to interlaced frames, causing incorrect decisions in the motion
compensation code and resulting in visual artifacts.

CC: libav-stable@libav.org
Signed-off-by: Tim Walker <tdskywalker@gmail.com>
(cherry picked from commit dd2d0039b6405dc724e4fef0d5b8f49530eea3aa)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 3cc8d9bc1ffc6c0888960fb009f12fa3047bb663)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

h264: reset num_reorder_frames if it is invalid

An invalid VUI is not considered a fatal error, so the SPS containing it
may still be used. Leaving an invalid value of num_reorder_frames there
can result in writing over the bounds of H264Context.delayed_pic.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 9ecabd7892ff073ae60ded3fc0a1290f5914ed5c)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/h264_ps.c

(cherry picked from commit 299c5dcfb0cd3debdf07943edfb46f4aeb02ca91)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

h264: check that an IDR NAL only contains I slices

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 8b2e5e42bb9d6a59ede5af2e6df4aaf7750d1195)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 62ed6da016b789eee00e0fff517df4a254e12e5d)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/h264.c

mov: Free an earlier allocated array if allocating a new one

It could probably also be considered an error if the pointer isn't
null at this point, but then we might risk rejecting some
slightly broken files that we might have handled so far.

Sample-Id: 00000496-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 2620df13104ddaa136158eb6bb1195adbf9d7692)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit a1b4d42d31ba700c97d4388153a2a553d71ca0ba)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

segafilm: fix leaks if reading the header fails

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 6892d145a0c80249bd61ee7dd31ec851c5076bcd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f728782c0d30433efa11f1238a16aed994e9b563)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavformat/segafilm.c

h264_cavlc: check the size of the intra PCM data.

Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

(cherry picked from commit b5275ca1a805436ca12540c34dd5ed1671877434)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

cavs: Check for negative cbp

Sample-Id: 00000647-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c85e5f13f6ac9c4c90125e7671d89009e57f9df9)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/cavsdec.c

avi: DV in AVI must be considered single stream

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3485a07977f17b8d4709fb327be4fc29031032b7)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

avutil: use align == 0 for default alignment in audio sample buffer functions

Fixes: http://pad.lv/1264886, http://pad.lv/1241439
(cherry picked from commit 0109a09dc3850eb5dbff84a7bb50eb252a5a8f22)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavutil/avutil.h

flashsv: Check diff_start diff_height values

Fix out of array accesses.

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Adresses: CVE-2013-7015
(cherry picked from commit 57070b1468edc6ac8cb3696c817f3c943975d4c1)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 10d48fe6d3963842319b1d8d738a318020836e72)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

dsputil/pngdsp: fix signed/unsigned type in end comparison

Fixes out of array accesses and integer overflows.

(cherry picked from commit d1916d13e28b87f4b1b214231149e12e1d536b4b)
Adresses: CVE-2013-7010, CVE-2013-7014

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit af9799790d7a6342027e0261b5dd87657abb7a0b)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/pngdsp.c

vqavideo: check chunk sizes before reading chunks

Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bfa1e20127a16392a0b85a4aa4840889)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 13093f9767b922661132a3c1f4b5ba2c7338b660)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f7d18deb73d1dd1b27b2c7062c9a10d168a6c62a)

Addresses: CVE-2013-0865

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit ab434bf0d051008a329d49d0256faa5d64e2bf4d)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

avi: directly resync on DV in AVI read failure

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ceec6e792e4b5baaa23b220f4fd33417631f5288)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Adresses CVE-2013-0856
(cherry picked from commit 61057f4604eb909ac2b37f08c7d2b0ed758fd4bf)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

get_bits: change the failure condition in init_get_bits

Too much code relies in having init_get_bits fed with a valid
buffer and set its dimension to 0.

Check for NULL buffer instead.

(cherry picked from commit 4603ec85ed620e585fc6e2e072c99858ed421855)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

twinvq: Cope with gcc-4.8.2 miscompilation

Apparently gcc-4.8.2 miscompiles enums resulting in a lucky fpe soon
after it.

Passing the enum value as integer makes the ftype == FT_PPC condition
evaluates correctly.

Changelog for 0.8.10

pthread: Avoid spurious wakeups

pthread_wait_cond can wake up unexpectedly (Wikipedia: Spurious_wakeup).

The FF_THREAD_SLICE thread mechanism could spontaneously execute
jobs or allow the caller of avctx->execute to return before all
jobs were complete.

Test both cases to ensure the wakeup is real.

Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 311583e7798237be5cc531d672a9e37f8c729d83)

pthread: Fix deadlock during thread initialization

Sometimes, if pthread_create() failed, then pthread_cond_wait() could
accidentally be called in the worker threads after the uninit function
had already called pthread_cond_broadcast(), leading to a deadlock.

Don't call pthread_cond_wait() if c->done is set.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
(cherry picked from commit 1a5a6ac01b0ad2cf3d2128372ea41f3c1cfc2d3f)

mpegvideo: Initialize chroma_*_shift and codec_tag even if the size is 0

This fixes breakage in a few fate tests on certain setups
(that for some reason didn't break on OS X) after the previous
commit (8812a8057). Currently, some video streams are initialized
in ff_MPV_common_init with width/height set at 0 and only changed
to a proper video size with ff_MPV_common_frame_size_change later.

The breakage was diagnosed by Anton Khirnov.

Signed-off-by: Martin Storsjö <martin@martin.st>

vc1dec: Don't decode slices when the latest slice header failed to decode

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Conflicts:
libavcodec/vc1dec.c

vc1dec: Make sure last_picture is initialized in vc1_decode_skip_blocks

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5e25fdbfe01635cfc650ac4adc27d434b2df0d64)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/vc1dec.c
(cherry picked from commit 494f2d4f9e834db1eaf1a7d0160d497f9802013d)

r3d: Add more input value validation

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Conflicts:
libavformat/r3d.c

fraps: Make the input buffer size checks more strict

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Conflicts:
libavcodec/fraps.c

svq3: Avoid a division by zero

If the height is zero, the decompression will probably end up
failing due to not fitting into the allocated buffer later
anyway, so this doesn't need any more elaborate check.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 601c2015bc16f0b281160292a6a760cbbbb0eacb)

rmdec: Validate the fps value

Abort if it is invalid if strict error checking has been requested.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0f310a6f333b016d336674d086045e8473fdf918)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavformat/rmdec.c

twinvqdec: Check the ibps parameter separately

This is required, since invalid parameters actually could
pass the switch check below.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit c77d409bf95954aceb762dd800d1ee2868c4b0d4)
(cherry picked from commit 9b9aee27f4e43b4a6b0884f8a6f49eb0289d7c09)

asfdec: Check the return value of asf_read_stream_properties

This makes sure errors in setting stream parameters are passed
on to the caller. This avoids successfully opening files while
some parameters aren't filled in properly.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit cc41167aede4c101ad17eeffa8f39bb6c23d3dad)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit fc4d11ec9b4c9710e2dac012d4ed0e7d08c6df7d)

mxfdec: set audio timebase to 1/samplerate

Fixes sync in some samples (e.g. bugs 7581 and 8374 in VLC).
Based on a commit by Matthieu Bouron <matthieu.bouron@gmail.com>

Reported-by: Jean-Baptiste Kempf <jb@videolan.org>
CC: libav-stable@libav.org
(cherry picked from commit 93370d12164236d59645314871a1d6808b2a8ddb)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

pcx: Check the packet size before assuming it fits a palette

This fixes reads out of bounds.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d1d99e3befea5d411ac3aae72dbdecce94f8b547)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/pcx.c
(cherry picked from commit 7e350b7ddd19af856b55634233d609e29baab646)

rpza: Fix a buffer size check

We read 2 bytes for 15 out of 16 pixels, therefore we need to
have at least 30 bytes, not 16.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 7ba0cedbfeff5671b264d1d7e90777057b5714c6)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit f06e39fe6b272a11782c023c31eec43bfce3138d)

xxan: Disallow odd width

Decoded data is always written in pairs within this decoder.
This fixes writes out of bounds.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit aa0dd52434768da64f1f3d8ae92bcf980c1adffc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

xan: Only read within the data that actually was initialized

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit fc739b3eefa0b58d64e7661621da94a94dbc8a82)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 09ace619d6ccb2c0a45b5fdead29f926409fa129)

xan: Use bytestream2 to limit reading to within the buffer

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 30db94dc399f6e4ef8905049d9b740556f0fce47)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 145de32896b37a508f11bcf11dfcc94487301716)

pcx: Consume the whole packet if giving up due to missing palette

Previously, we returned 0, meaning successful decoding but 0
bytes consumed, leading to an infinite loop.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9fb0de86b49e9fb0709a8ad1e1875e35da841887)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 812955a12b190012c134be33a93f27308953eb2f)

pngdec: Stop trying to decode once inflate returns Z_STREAM_END

If the input buffer contains more data after the deflate stream,
the loop previously left running infinitely, with inflate returning
Z_STREAM_END.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a81cad8f86d1feb7e4bfae29e43f3e994935a5c7)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit a63e83cd4b43c3dcef38f7fefe41c002a263af0f)

mov: Make sure the read sample count is nonnegative

This avoids setting a negative number of frames, ending up with a
negative average frame rate.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit c231987662194d009dd91bfc57c678e0e70ca161)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit c10f3fed259c23e6887f68cdf3e7d4ae87026f65)

bfi: Add some very basic sanity checks for input packet sizes

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 640a2427aafa774b83316b7a8c5c2bdc28bfd269)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 10f384e4f5d0ee692cacaf90d629d8bc2178b092)

bfi: Avoid divisions by zero

If a zero-length video packet is to be returned, just return
AVERROR(EAGAIN) and switch back to the audio stream.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9fc7184d1a9af8d97b3fc5c2ef9d0a647d6617ea)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ad1223d6bcc69e1639951aedcdae40822bf41042)

electronicarts: Add more sanity checking for the number of channels

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a9221e39600a31ee13e736e9e47743cde23f0280)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavformat/electronicarts.c
(cherry picked from commit a89868d714705af1b0b004fa790a889e9ba792cd)

riffdec: Add sanity checks for the sample rate

This avoids a division by zero for G726.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d07aa3f02b73ab1371c13ac7898338380ca0932b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 607863acaec85671f8c2afd81079ae4c605e3468)

mvi: Add sanity checking for the audio frame size

This avoids a division by zero.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 28ff439efd2362fb21e1a78610737f2e26a72d8f)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 04d2f9ace3fb6e880f3488770fc5a39de5b63cbb)

xwma: Avoid division by zero

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit adc09136a4a63b152630abeacb22c56541eacf60)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 2ff935a06008fb1959ff633962fbc728762c33cb)

avidec: Make sure a packet is large enough before reading its data

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8d07258bb6063d0780ce2d39443d6dc6d8eedc5a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavformat/avidec.c
(cherry picked from commit 2e4c649b3e62fdd158b5a9a0f973d3b186a23e94)

vqf: Make sure the bitrate is in the valid range

Even if the sample rate is valid, an invalid bitrate could
pass the mode combination test below.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 68ff9981283a56c731f00c2ee7901103665092fc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 60701469ab9f526841ae81444236425f87916adb)

vqf: Make sure sample_rate is set to a valid value

This avoids divisions by zero later (and possibly assertions in
time base scaling), since an invalid rate_flag combined with an
invalid bitrate below could pass the mode combination test.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9277050e2918e0a0df9689721a188a604d886616)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 4d60ab62e05decc562645cd6f813f7c9e69637ee)

vc1dec: Undo mpegvideo initialization if unable to allocate tables

Previously, s->context_initialized was left set to 1
if ff_vc1_decode_init_alloc_tables failed, skipping the
initialization completely on the next decode call.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b772b0e28eba6abf76d86ee8c6e459a86642db5a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

vc1dec: Fix leaks in ff_vc1_decode_init_alloc_tables on errors

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit ede508443e4bf57dc1e019fac81bf6244b88fbd3)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit b62704891d2353679e012555ac9e9a49ee63d497)

wnv1: Make sure the input packet is large enough

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 91be1103fd1f79d381edf268c32f4166b6c3b6d8)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 0c8c6b4419e00d13197a4aea5456b398dca24df0)

dca: Validate the lfe parameter

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a9d50bb578ec04c085a25f1e023f75e0e4499d5e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

rl2: Avoid a division by zero

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 3ca14aa5964ea5d11f7a15f9fff17924d6096d44)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ce1dacb435460dda1f9d453eaaeac44bd502aca4)

wtv: Add more sanity checks for a length read from the file

Also make sure the existing length check can't overflow.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 83c285f88016b087c2f0f4b9ef356ad8ef12d947)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 78dc022f6f8a8b87773a209e0fcbea2d5b48396f)

segafilm: Validate the number of audio channels

This avoids divisions by zero later.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 82e266c6d3fbf3cc74e515b883e66543381a0f2c)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 5379c5184b9fe9ef06234638f5629d4c80056e04)

qpeg: Add checks for running out of rows in qpeg_decode_inter

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 7a5a55722749a3ab77941914707277b147322cbe)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 4d90550cf95eac0451465116d6e53bac37b96927)

mpegaudiodec: Validate that the number of channels fits at the given offset

This is similar to the fix in 35cbc98b.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e9d61de96c113ee0ef8082833c7e682df0e23eec)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit bacf5db1962a6955ce80eea6bbc86c6970d7d360)

asv1: Verify the amount of extradata

The init function reads one byte of extradata.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit f50803354c6acb4575379d7c54ca48ec5d36dd61)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

idroqdec: Make sure a video stream has been allocated before returning packets

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit bcbe4f3ceb6ee0210d3a401963518906c8b9b230)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit de75bc01cda53acfbd9f901639695ade8e650c43)

rv10: Validate the dimensions set from the container

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5372cda67109848d22146289e401669266217e80)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 0b0f1cd44ece180e12795cfc8d0a0ac5ea3ebe2c)

xmv: Add more sanity checks for parameters read from the bitstream

Since the number of channels is multiplied by 36 and assigned to
to a uint16_t, make sure this calculation didn't overflow. (In
certain cases the calculation could overflow leaving the
truncated block_align at 0, leading to divisions by zero later.)

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d4c2a3740fb95f952a87ba320d2bf31f126bdf68)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 00516b5491fbd99e4057f21eae231fc02cc596e3)

ffv1: Make sure at least one slice context is initialized

This avoids crashes when initializing the range coder for
the first slice context.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b1db33159fdc2da4bdd8c75e4ff9a7dd0ef2f0c2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

truemotion2: Use av_freep properly in an error path

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit c39f7eba01cd656e8f0eed592f93d11814736650)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit eac1c3f384eab770d42468f4f244156c1735701d)

eacmv: Make sure a reference frame exists before referencing it

This is similar to an existing check for the second-last frame
from 062421e3.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit ea78a348d86a3a733f6c1e0a65cfdd8283d924b9)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/eacmv.c
(cherry picked from commit 2e12af4587613dd5b2c3431e5c8194d73b03434f)

mpeg4videodec: Check the width/height in mpeg4_decode_sprite_trajectory

This avoids a potential division by zero.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit f875a732e36786d49f3650e3235272891a820600)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit c8c93795e4afd04c2c5b74e29e8dec29b6a76b81)

ivi_common: Make sure color planes have been initialized

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a92538b7c0defc86c55fb91f55dfa36aad192673)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 38bd229af9c4fa5897fc1a69e73a04c55f78647f)

oggparseogm: Convert to use bytestream2

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 19b9659f3174599e8685d329c4330b1ea8c4c6db)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

rv34: Check the return value from ff_rv34_decode_init

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 711c970168297683860422e95d6b7e37ee3c8367)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 20c8f176293e7520c6205b664e25ecf8a711253e)

matroskadec: Verify realaudio codec parameters

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 569d18aa9dc989c37bb4d4b968026fe5afa6fff9)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 9f7a8b8f8f6ad024410232d926b774261ef2ef36)

mace: Make sure that the channel count is set to a valid value

Also return a proper error code.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e1f3847f860a1094a46be4c5f10db8df616c3135)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/mace.c

svq3: Check for any negative return value from ff_h264_check_intra_pred_mode

Also pass on any returned error code.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1115689d54ea95a084421f5a182b8dc56cbff978)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/svq3.c

vp3: Check the framerate for validity

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 6fc8226e29055858f28973bb3d27b63b3b65e616)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit b4c479a82adbb1301e3e549cd80cdd65208ddd05)

cavsdec: Make sure a sequence header has been decoded before decoding pictures

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e90a6846c2c006fbebd00e1f2789f4a86fafacef)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/cavsdec.c

sierravmd: Do sanity checking of frame sizes

Limit the size to INT_MAX/2 (for simplicity) to be sure that
size + BYTES_PER_FRAME_RECORD won't overflow.

Also factorize other existing error return paths.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0ef1660a6365ce60ead8858936b6f3f8ea862826)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 153deed18bed43d16b272e8681b2a9b988d2682a)

omadec: Properly check lengths before incrementing the position

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 342c43d154e586bc022c86b168fe8d36f69da9d3)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 9eba02d5dd7036294ea350cb772822deec95b867)

mpc8: Make sure the first stream exists before parsing the seek table

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 17d57848fc14e82f76a65ffb25c90f2f011dc4a0)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 557df77eab7d3726c34221aeb999afe9e7818d52)

mpc8: Check the seek table size parsed from the bitstream

Limit the size to INT_MAX/2 (for simplicity) to be sure that
size + FF_INPUT_BUFFER_PADDING_SIZE won't overflow.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 459f2b393a3f89ed08d10fbceb4738d1429f268e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit f8a72f041c049e812dfa1f32156327e9778f5710)

zmbvdec: Check the buffer size for uncompressed data

Also don't pointlessly set the buffer size to 1 after copying
one packet.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0d61f260010707f3028b818e8b24598e1a83d696)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

ape: Don't allow the seektable to be omitted

The seektable is required for filling in ape->frames[i].pos
further down.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 183b9d843a9533774fabd3984a52f3987001acbc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

shorten: Break out of loop looking for fmt chunk if none is found

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b26742cc308552f242ee2bf93b07a3ff509f4edc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

shorten: Use a checked bytestream reader for the wave header

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 49568851bf1700e3d9ea9cda29208d0df3c2c38b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

smacker: Make sure we don't fill in huffman codes out of range

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0679cec6e8802643bbe6d5f68ca1110a7d3171da)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

smacker: Avoid integer overflow when allocating packets

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 710b0e27025948b7511821c2f888ff2d74a59e14)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

smacker: Don't return packets in unallocated streams

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8d928023f953a28692ba27071a448259134b103b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

dsicin: Add some basic sanity checks for fields read from the file

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 48d6556dd46d4f4fac10d0f4a819e314887cd50e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

arm: Don't clobber callee saved registers in scalarproduct

q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4a9ada22df443cc38be77cc5e492694)

Signed-off-by: Martin Storsjö <martin@martin.st>

Prepare for 0.8.10 Release

roqvideodec: check dimensions validity

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae610451170cd5a28b33950006ff0bd23036845)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee26d352a52eb9f7fcd8d9167fb4a5ba015b612)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 488f87be873506abb01d67708a67c10a4dd29283)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 52b18c1fde65efac7f6e6104b76d39bf8d0a34ee)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

qdm2: check array index before use, fix out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a7ee6281f7ef1c29284e3a4cadfe0f227ffde1ed)

CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 39bec05ed42e505d17877b0c23f16322f9b5883b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0b2b8ab979624b0cce673d5e99255482d7c553ad)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>