2 * Copyright (C) 2008 David Schleef <ds@schleef.org>
3 * Copyright (C) 2011 Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk>.
4 * Copyright (C) 2011 Nokia Corporation. All rights reserved.
5 * Contact: Stefan Kost <stefan.kost@nokia.com>
6 * Copyright (C) 2012 Collabora Ltd.
7 * Author : Edward Hervey <edward@collabora.com>
9 * This library is free software; you can redistribute it and/or
10 * modify it under the terms of the GNU Library General Public
11 * License as published by the Free Software Foundation; either
12 * version 2 of the License, or (at your option) any later version.
14 * This library is distributed in the hope that it will be useful,
15 * but WITHOUT ANY WARRANTY; without even the implied warranty of
16 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
17 * Library General Public License for more details.
19 * You should have received a copy of the GNU Library General Public
20 * License along with this library; if not, write to the
21 * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
22 * Boston, MA 02110-1301, USA.
26 * SECTION:gstvideodecoder
27 * @title: GstVideoDecoder
28 * @short_description: Base class for video decoders
30 * This base class is for video decoders turning encoded data into raw video
33 * The GstVideoDecoder base class and derived subclasses should cooperate as
38 * * Initially, GstVideoDecoder calls @start when the decoder element
39 * is activated, which allows the subclass to perform any global setup.
41 * * GstVideoDecoder calls @set_format to inform the subclass of caps
42 * describing input video data that it is about to receive, including
43 * possibly configuration data.
44 * While unlikely, it might be called more than once, if changing input
45 * parameters require reconfiguration.
47 * * Incoming data buffers are processed as needed, described in Data
50 * * GstVideoDecoder calls @stop at end of all processing.
54 * * The base class gathers input data, and optionally allows subclass
55 * to parse this into subsequently manageable chunks, typically
56 * corresponding to and referred to as 'frames'.
58 * * Each input frame is provided in turn to the subclass' @handle_frame
60 * * When the subclass enables the subframe mode with `gst_video_decoder_set_subframe_mode`,
61 * the base class will provide to the subclass the same input frame with
62 * different input buffers to the subclass @handle_frame
63 * callback. During this call, the subclass needs to take
64 * ownership of the input_buffer as @GstVideoCodecFrame.input_buffer
65 * will have been changed before the next subframe buffer is received.
66 * The subclass will call `gst_video_decoder_have_last_subframe`
67 * when a new input frame can be created by the base class.
68 * Every subframe will share the same @GstVideoCodecFrame.output_buffer
69 * to write the decoding result. The subclass is responsible to protect
72 * * If codec processing results in decoded data, the subclass should call
73 * @gst_video_decoder_finish_frame to have decoded data pushed
74 * downstream. In subframe mode
75 * the subclass should call @gst_video_decoder_finish_subframe until the
76 * last subframe where it should call @gst_video_decoder_finish_frame.
77 * The subclass can detect the last subframe using GST_VIDEO_BUFFER_FLAG_MARKER
78 * on buffers or using its own logic to collect the subframes.
79 * In case of decoding failure, the subclass must call
80 * @gst_video_decoder_drop_frame or @gst_video_decoder_drop_subframe,
81 * to allow the base class to do timestamp and offset tracking, and possibly
82 * to requeue the frame for a later attempt in the case of reverse playback.
86 * * The GstVideoDecoder class calls @stop to inform the subclass that data
87 * parsing will be stopped.
93 * * When the pipeline is seeked or otherwise flushed, the subclass is
94 * informed via a call to its @reset callback, with the hard parameter
95 * set to true. This indicates the subclass should drop any internal data
96 * queues and timestamps and prepare for a fresh set of buffers to arrive
97 * for parsing and decoding.
101 * * At end-of-stream, the subclass @parse function may be called some final
102 * times with the at_eos parameter set to true, indicating that the element
103 * should not expect any more data to be arriving, and it should parse and
104 * remaining frames and call gst_video_decoder_have_frame() if possible.
106 * The subclass is responsible for providing pad template caps for
107 * source and sink pads. The pads need to be named "sink" and "src". It also
108 * needs to provide information about the output caps, when they are known.
109 * This may be when the base class calls the subclass' @set_format function,
110 * though it might be during decoding, before calling
111 * @gst_video_decoder_finish_frame. This is done via
112 * @gst_video_decoder_set_output_state
114 * The subclass is also responsible for providing (presentation) timestamps
115 * (likely based on corresponding input ones). If that is not applicable
116 * or possible, the base class provides limited framerate based interpolation.
118 * Similarly, the base class provides some limited (legacy) seeking support
119 * if specifically requested by the subclass, as full-fledged support
120 * should rather be left to upstream demuxer, parser or alike. This simple
121 * approach caters for seeking and duration reporting using estimated input
122 * bitrates. To enable it, a subclass should call
123 * @gst_video_decoder_set_estimate_rate to enable handling of incoming
126 * The base class provides some support for reverse playback, in particular
127 * in case incoming data is not packetized or upstream does not provide
128 * fragments on keyframe boundaries. However, the subclass should then be
129 * prepared for the parsing and frame processing stage to occur separately
130 * (in normal forward processing, the latter immediately follows the former),
131 * The subclass also needs to ensure the parsing stage properly marks
132 * keyframes, unless it knows the upstream elements will do so properly for
135 * The bare minimum that a functional subclass needs to implement is:
137 * * Provide pad templates
138 * * Inform the base class of output caps via
139 * @gst_video_decoder_set_output_state
141 * * Parse input data, if it is not considered packetized from upstream
142 * Data will be provided to @parse which should invoke
143 * @gst_video_decoder_add_to_frame and @gst_video_decoder_have_frame to
144 * separate the data belonging to each video frame.
146 * * Accept data in @handle_frame and provide decoded results to
147 * @gst_video_decoder_finish_frame, or call @gst_video_decoder_drop_frame.
156 * * Add a flag/boolean for I-frame-only/image decoders so we can do extra
157 * features, like applying QoS on input (as opposed to after the frame is
159 * * Add a flag/boolean for decoders that require keyframes, so the base
160 * class can automatically discard non-keyframes before one has arrived
161 * * Detect reordered frame/timestamps and fix the pts/dts
162 * * Support for GstIndex (or shall we not care ?)
163 * * Calculate actual latency based on input/output timestamp/frame_number
164 * and if it exceeds the recorded one, save it and emit a GST_MESSAGE_LATENCY
165 * * Emit latency message when it changes
169 /* Implementation notes:
170 * The Video Decoder base class operates in 2 primary processing modes, depending
171 * on whether forward or reverse playback is requested.
174 * * Incoming buffer -> @parse() -> add_to_frame()/have_frame() ->
175 * handle_frame() -> push downstream
177 * Reverse playback is more complicated, since it involves gathering incoming
178 * data regions as we loop backwards through the upstream data. The processing
179 * concept (using incoming buffers as containing one frame each to simplify
182 * Upstream data we want to play:
183 * Buffer encoded order: 1 2 3 4 5 6 7 8 9 EOS
185 * Groupings: AAAAAAA BBBBBBB CCCCCCC
188 * Buffer reception order: 7 8 9 4 5 6 1 2 3 EOS
190 * Discont flag: D D D
192 * - Each Discont marks a discont in the decoding order.
193 * - The keyframes mark where we can start decoding.
195 * Initially, we prepend incoming buffers to the gather queue. Whenever the
196 * discont flag is set on an incoming buffer, the gather queue is flushed out
197 * before the new buffer is collected.
199 * The above data will be accumulated in the gather queue like this:
201 * gather queue: 9 8 7
204 * When buffer 4 is received (with a DISCONT), we flush the gather queue like
208 * take head of queue and prepend to parse queue (this reverses the
209 * sequence, so parse queue is 7 -> 8 -> 9)
211 * Next, we process the parse queue, which now contains all un-parsed packets
212 * (including any leftover ones from the previous decode section)
214 * for each buffer now in the parse queue:
215 * Call the subclass parse function, prepending each resulting frame to
216 * the parse_gather queue. Buffers which precede the first one that
217 * produces a parsed frame are retained in the parse queue for
218 * re-processing on the next cycle of parsing.
220 * The parse_gather queue now contains frame objects ready for decoding,
222 * parse_gather: 9 -> 8 -> 7
224 * while (parse_gather)
225 * Take the head of the queue and prepend it to the decode queue
226 * If the frame was a keyframe, process the decode queue
227 * decode is now 7-8-9
229 * Processing the decode queue results in frames with attached output buffers
230 * stored in the 'output_queue' ready for outputting in reverse order.
232 * After we flushed the gather queue and parsed it, we add 4 to the (now empty)
233 * gather queue. We get the following situation:
236 * decode queue: 7 8 9
238 * After we received 5 (Keyframe) and 6:
240 * gather queue: 6 5 4
241 * decode queue: 7 8 9
243 * When we receive 1 (DISCONT) which triggers a flush of the gather queue:
245 * Copy head of the gather queue (6) to decode queue:
248 * decode queue: 6 7 8 9
250 * Copy head of the gather queue (5) to decode queue. This is a keyframe so we
251 * can start decoding.
254 * decode queue: 5 6 7 8 9
256 * Decode frames in decode queue, store raw decoded data in output queue, we
257 * can take the head of the decode queue and prepend the decoded result in the
262 * output queue: 9 8 7 6 5
264 * Now output all the frames in the output queue, picking a frame from the
267 * Copy head of the gather queue (4) to decode queue, we flushed the gather
268 * queue and can now store input buffer in the gather queue:
273 * When we receive EOS, the queue looks like:
275 * gather queue: 3 2 1
278 * Fill decode queue, first keyframe we copy is 2:
281 * decode queue: 2 3 4
287 * output queue: 4 3 2
289 * Leftover buffer 1 cannot be decoded and must be discarded.
292 #include "gstvideodecoder.h"
293 #include "gstvideoutils.h"
294 #include "gstvideoutilsprivate.h"
296 #include <gst/video/video.h>
297 #include <gst/video/video-event.h>
298 #include <gst/video/gstvideopool.h>
299 #include <gst/video/gstvideometa.h>
302 GST_DEBUG_CATEGORY (videodecoder_debug);
303 #define GST_CAT_DEFAULT videodecoder_debug
306 #define DEFAULT_QOS TRUE
307 #define DEFAULT_MAX_ERRORS GST_VIDEO_DECODER_MAX_ERRORS
308 #define DEFAULT_MIN_FORCE_KEY_UNIT_INTERVAL 0
309 #define DEFAULT_DISCARD_CORRUPTED_FRAMES FALSE
310 #define DEFAULT_AUTOMATIC_REQUEST_SYNC_POINTS FALSE
311 #define DEFAULT_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS (GST_VIDEO_DECODER_REQUEST_SYNC_POINT_DISCARD_INPUT | GST_VIDEO_DECODER_REQUEST_SYNC_POINT_CORRUPT_OUTPUT)
313 /* Used for request_sync_point_frame_number. These are out of range for the
314 * frame numbers and can be given special meaning */
315 #define REQUEST_SYNC_POINT_PENDING G_MAXUINT + 1
316 #define REQUEST_SYNC_POINT_UNSET G_MAXUINT64
323 PROP_MIN_FORCE_KEY_UNIT_INTERVAL,
324 PROP_DISCARD_CORRUPTED_FRAMES,
325 PROP_AUTOMATIC_REQUEST_SYNC_POINTS,
326 PROP_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS,
329 struct _GstVideoDecoderPrivate
331 /* FIXME introduce a context ? */
334 GstAllocator *allocator;
335 GstAllocationParams params;
339 GstAdapter *input_adapter;
340 /* assembles current frame */
341 GstAdapter *output_adapter;
343 /* Whether we attempt to convert newsegment from bytes to
344 * time using a bitrate estimation */
345 gboolean do_estimate_rate;
347 /* Whether input is considered packetized or not */
350 /* whether input is considered as subframes */
351 gboolean subframe_mode;
356 gboolean had_output_data;
357 gboolean had_input_data;
359 gboolean needs_format;
360 /* input_segment are output_segment identical */
361 gboolean in_out_segment_sync;
363 /* TRUE if we have an active set of instant rate flags */
364 gboolean decode_flags_override;
365 GstSegmentFlags decode_flags;
367 /* ... being tracked here;
368 * only available during parsing or when doing subframe decoding */
369 GstVideoCodecFrame *current_frame;
370 /* events that should apply to the current frame */
371 /* FIXME 2.0: Use a GQueue or similar, see GstVideoCodecFrame::events */
372 GList *current_frame_events;
373 /* events that should be pushed before the next frame */
374 /* FIXME 2.0: Use a GQueue or similar, see GstVideoCodecFrame::events */
375 GList *pending_events;
377 /* relative offset of input data */
378 guint64 input_offset;
379 /* relative offset of frame */
380 guint64 frame_offset;
381 /* tracking ts and offsets */
384 /* last outgoing ts */
385 GstClockTime last_timestamp_out;
386 /* incoming pts - dts */
387 GstClockTime pts_delta;
388 gboolean reordered_output;
390 /* FIXME: Consider using a GQueue or other better fitting data structure */
391 /* reverse playback */
396 /* collected parsed frames */
398 /* frames to be handled == decoded */
400 /* collected output - of buffer objects, not frames */
401 GList *output_queued;
404 /* base_picture_number is the picture number of the reference picture */
405 guint64 base_picture_number;
406 /* combine with base_picture_number, framerate and calcs to yield (presentation) ts */
407 GstClockTime base_timestamp;
410 GstClockTime min_force_key_unit_interval;
411 gboolean discard_corrupted_frames;
413 /* Key unit related state */
414 gboolean needs_sync_point;
415 GstVideoDecoderRequestSyncPointFlags request_sync_point_flags;
416 guint64 request_sync_point_frame_number;
417 GstClockTime last_force_key_unit_time;
418 /* -1 if we saw no sync point yet */
419 guint64 distance_from_sync;
421 gboolean automatic_request_sync_points;
422 GstVideoDecoderRequestSyncPointFlags automatic_request_sync_point_flags;
424 guint32 system_frame_number;
425 guint32 decode_frame_number;
427 GQueue frames; /* Protected with OBJECT_LOCK */
428 GstVideoCodecState *input_state;
429 GstVideoCodecState *output_state; /* OBJECT_LOCK and STREAM_LOCK */
430 gboolean output_state_changed;
434 gdouble proportion; /* OBJECT_LOCK */
435 GstClockTime earliest_time; /* OBJECT_LOCK */
436 GstClockTime qos_frame_duration; /* OBJECT_LOCK */
438 /* qos messages: frames dropped/processed */
442 /* Outgoing byte size ? */
449 /* upstream stream tags (global tags are passed through as-is) */
450 GstTagList *upstream_tags;
454 GstTagMergeMode tags_merge_mode;
456 gboolean tags_changed;
459 gboolean use_default_pad_acceptcaps;
461 #ifndef GST_DISABLE_DEBUG
462 /* Diagnostic time for reporting the time
463 * from flush to first output */
464 GstClockTime last_reset_time;
468 static GstElementClass *parent_class = NULL;
469 static gint private_offset = 0;
471 /* cached quark to avoid contention on the global quark table lock */
472 #define META_TAG_VIDEO meta_tag_video_quark
473 static GQuark meta_tag_video_quark;
475 static void gst_video_decoder_class_init (GstVideoDecoderClass * klass);
476 static void gst_video_decoder_init (GstVideoDecoder * dec,
477 GstVideoDecoderClass * klass);
479 static void gst_video_decoder_finalize (GObject * object);
480 static void gst_video_decoder_get_property (GObject * object, guint property_id,
481 GValue * value, GParamSpec * pspec);
482 static void gst_video_decoder_set_property (GObject * object, guint property_id,
483 const GValue * value, GParamSpec * pspec);
485 static gboolean gst_video_decoder_setcaps (GstVideoDecoder * dec,
487 static gboolean gst_video_decoder_sink_event (GstPad * pad, GstObject * parent,
489 static gboolean gst_video_decoder_src_event (GstPad * pad, GstObject * parent,
491 static GstFlowReturn gst_video_decoder_chain (GstPad * pad, GstObject * parent,
493 static gboolean gst_video_decoder_sink_query (GstPad * pad, GstObject * parent,
495 static GstStateChangeReturn gst_video_decoder_change_state (GstElement *
496 element, GstStateChange transition);
497 static gboolean gst_video_decoder_src_query (GstPad * pad, GstObject * parent,
499 static void gst_video_decoder_reset (GstVideoDecoder * decoder, gboolean full,
500 gboolean flush_hard);
502 static GstFlowReturn gst_video_decoder_decode_frame (GstVideoDecoder * decoder,
503 GstVideoCodecFrame * frame);
505 static void gst_video_decoder_push_event_list (GstVideoDecoder * decoder,
507 static GstClockTime gst_video_decoder_get_frame_duration (GstVideoDecoder *
508 decoder, GstVideoCodecFrame * frame);
509 static GstVideoCodecFrame *gst_video_decoder_new_frame (GstVideoDecoder *
511 static GstFlowReturn gst_video_decoder_clip_and_push_buf (GstVideoDecoder *
512 decoder, GstBuffer * buf);
513 static GstFlowReturn gst_video_decoder_flush_parse (GstVideoDecoder * dec,
516 static void gst_video_decoder_clear_queues (GstVideoDecoder * dec);
518 static gboolean gst_video_decoder_sink_event_default (GstVideoDecoder * decoder,
520 static gboolean gst_video_decoder_src_event_default (GstVideoDecoder * decoder,
522 static gboolean gst_video_decoder_decide_allocation_default (GstVideoDecoder *
523 decoder, GstQuery * query);
524 static gboolean gst_video_decoder_propose_allocation_default (GstVideoDecoder *
525 decoder, GstQuery * query);
526 static gboolean gst_video_decoder_negotiate_default (GstVideoDecoder * decoder);
527 static GstFlowReturn gst_video_decoder_parse_available (GstVideoDecoder * dec,
528 gboolean at_eos, gboolean new_buffer);
529 static gboolean gst_video_decoder_negotiate_unlocked (GstVideoDecoder *
531 static gboolean gst_video_decoder_sink_query_default (GstVideoDecoder * decoder,
533 static gboolean gst_video_decoder_src_query_default (GstVideoDecoder * decoder,
536 static gboolean gst_video_decoder_transform_meta_default (GstVideoDecoder *
537 decoder, GstVideoCodecFrame * frame, GstMeta * meta);
539 static gboolean gst_video_decoder_handle_missing_data_default (GstVideoDecoder *
540 decoder, GstClockTime timestamp, GstClockTime duration);
542 static void gst_video_decoder_copy_metas (GstVideoDecoder * decoder,
543 GstVideoCodecFrame * frame, GstBuffer * src_buffer,
544 GstBuffer * dest_buffer);
546 static void gst_video_decoder_request_sync_point_internal (GstVideoDecoder *
547 dec, GstClockTime deadline, GstVideoDecoderRequestSyncPointFlags flags);
549 /* we can't use G_DEFINE_ABSTRACT_TYPE because we need the klass in the _init
550 * method to get to the padtemplates */
552 gst_video_decoder_get_type (void)
554 static gsize type = 0;
556 if (g_once_init_enter (&type)) {
558 static const GTypeInfo info = {
559 sizeof (GstVideoDecoderClass),
562 (GClassInitFunc) gst_video_decoder_class_init,
565 sizeof (GstVideoDecoder),
567 (GInstanceInitFunc) gst_video_decoder_init,
570 _type = g_type_register_static (GST_TYPE_ELEMENT,
571 "GstVideoDecoder", &info, G_TYPE_FLAG_ABSTRACT);
574 g_type_add_instance_private (_type, sizeof (GstVideoDecoderPrivate));
576 g_once_init_leave (&type, _type);
581 static inline GstVideoDecoderPrivate *
582 gst_video_decoder_get_instance_private (GstVideoDecoder * self)
584 return (G_STRUCT_MEMBER_P (self, private_offset));
588 gst_video_decoder_class_init (GstVideoDecoderClass * klass)
590 GObjectClass *gobject_class;
591 GstElementClass *gstelement_class;
593 gobject_class = G_OBJECT_CLASS (klass);
594 gstelement_class = GST_ELEMENT_CLASS (klass);
596 GST_DEBUG_CATEGORY_INIT (videodecoder_debug, "videodecoder", 0,
597 "Base Video Decoder");
599 parent_class = g_type_class_peek_parent (klass);
601 if (private_offset != 0)
602 g_type_class_adjust_private_offset (klass, &private_offset);
604 gobject_class->finalize = gst_video_decoder_finalize;
605 gobject_class->get_property = gst_video_decoder_get_property;
606 gobject_class->set_property = gst_video_decoder_set_property;
608 gstelement_class->change_state =
609 GST_DEBUG_FUNCPTR (gst_video_decoder_change_state);
611 klass->sink_event = gst_video_decoder_sink_event_default;
612 klass->src_event = gst_video_decoder_src_event_default;
613 klass->decide_allocation = gst_video_decoder_decide_allocation_default;
614 klass->propose_allocation = gst_video_decoder_propose_allocation_default;
615 klass->negotiate = gst_video_decoder_negotiate_default;
616 klass->sink_query = gst_video_decoder_sink_query_default;
617 klass->src_query = gst_video_decoder_src_query_default;
618 klass->transform_meta = gst_video_decoder_transform_meta_default;
619 klass->handle_missing_data = gst_video_decoder_handle_missing_data_default;
622 * GstVideoDecoder:qos:
624 * If set to %TRUE the decoder will handle QoS events received
625 * from downstream elements.
626 * This includes dropping output frames which are detected as late
627 * using the metrics reported by those events.
631 g_object_class_install_property (gobject_class, PROP_QOS,
632 g_param_spec_boolean ("qos", "Quality of Service",
633 "Handle Quality-of-Service events from downstream",
634 DEFAULT_QOS, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
637 * GstVideoDecoder:max-errors:
639 * Maximum number of tolerated consecutive decode errors. See
640 * gst_video_decoder_set_max_errors() for more details.
644 g_object_class_install_property (gobject_class, PROP_MAX_ERRORS,
645 g_param_spec_int ("max-errors", "Max errors",
646 "Max consecutive decoder errors before returning flow error",
647 -1, G_MAXINT, DEFAULT_MAX_ERRORS,
648 G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
651 * GstVideoDecoder:min-force-key-unit-interval:
653 * Minimum interval between force-key-unit events sent upstream by the
654 * decoder. Setting this to 0 will cause every event to be handled, setting
655 * this to %GST_CLOCK_TIME_NONE will cause every event to be ignored.
657 * See gst_video_event_new_upstream_force_key_unit() for more details about
658 * force-key-unit events.
662 g_object_class_install_property (gobject_class,
663 PROP_MIN_FORCE_KEY_UNIT_INTERVAL,
664 g_param_spec_uint64 ("min-force-key-unit-interval",
665 "Minimum Force Keyunit Interval",
666 "Minimum interval between force-keyunit requests in nanoseconds", 0,
667 G_MAXUINT64, DEFAULT_MIN_FORCE_KEY_UNIT_INTERVAL,
668 G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
671 * GstVideoDecoder:discard-corrupted-frames:
673 * If set to %TRUE the decoder will discard frames that are marked as
674 * corrupted instead of outputting them.
678 g_object_class_install_property (gobject_class, PROP_DISCARD_CORRUPTED_FRAMES,
679 g_param_spec_boolean ("discard-corrupted-frames",
680 "Discard Corrupted Frames",
681 "Discard frames marked as corrupted instead of outputting them",
682 DEFAULT_DISCARD_CORRUPTED_FRAMES,
683 G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
686 * GstVideoDecoder:automatic-request-sync-points:
688 * If set to %TRUE the decoder will automatically request sync points when
689 * it seems like a good idea, e.g. if the first frames are not key frames or
690 * if packet loss was reported by upstream.
694 g_object_class_install_property (gobject_class,
695 PROP_AUTOMATIC_REQUEST_SYNC_POINTS,
696 g_param_spec_boolean ("automatic-request-sync-points",
697 "Discard Corrupted Frames",
698 "Discard frames marked as corrupted instead of outputting them",
699 DEFAULT_AUTOMATIC_REQUEST_SYNC_POINTS,
700 G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
703 * GstVideoDecoder:automatic-request-sync-point-flags:
705 * GstVideoDecoderRequestSyncPointFlags to use for the automatically
706 * requested sync points if `automatic-request-sync-points` is enabled.
710 g_object_class_install_property (gobject_class,
711 PROP_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS,
712 g_param_spec_flags ("automatic-request-sync-point-flags",
713 "Discard Corrupted Frames",
714 "Discard frames marked as corrupted instead of outputting them",
715 GST_TYPE_VIDEO_DECODER_REQUEST_SYNC_POINT_FLAGS,
716 DEFAULT_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS,
717 G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
719 meta_tag_video_quark = g_quark_from_static_string (GST_META_TAG_VIDEO_STR);
723 gst_video_decoder_init (GstVideoDecoder * decoder, GstVideoDecoderClass * klass)
725 GstPadTemplate *pad_template;
728 GST_DEBUG_OBJECT (decoder, "gst_video_decoder_init");
730 decoder->priv = gst_video_decoder_get_instance_private (decoder);
733 gst_element_class_get_pad_template (GST_ELEMENT_CLASS (klass), "sink");
734 g_return_if_fail (pad_template != NULL);
736 decoder->sinkpad = pad = gst_pad_new_from_template (pad_template, "sink");
738 gst_pad_set_chain_function (pad, GST_DEBUG_FUNCPTR (gst_video_decoder_chain));
739 gst_pad_set_event_function (pad,
740 GST_DEBUG_FUNCPTR (gst_video_decoder_sink_event));
741 gst_pad_set_query_function (pad,
742 GST_DEBUG_FUNCPTR (gst_video_decoder_sink_query));
743 gst_element_add_pad (GST_ELEMENT (decoder), decoder->sinkpad);
746 gst_element_class_get_pad_template (GST_ELEMENT_CLASS (klass), "src");
747 g_return_if_fail (pad_template != NULL);
749 decoder->srcpad = pad = gst_pad_new_from_template (pad_template, "src");
751 gst_pad_set_event_function (pad,
752 GST_DEBUG_FUNCPTR (gst_video_decoder_src_event));
753 gst_pad_set_query_function (pad,
754 GST_DEBUG_FUNCPTR (gst_video_decoder_src_query));
755 gst_element_add_pad (GST_ELEMENT (decoder), decoder->srcpad);
757 gst_segment_init (&decoder->input_segment, GST_FORMAT_TIME);
758 gst_segment_init (&decoder->output_segment, GST_FORMAT_TIME);
760 g_rec_mutex_init (&decoder->stream_lock);
762 decoder->priv->input_adapter = gst_adapter_new ();
763 decoder->priv->output_adapter = gst_adapter_new ();
764 decoder->priv->packetized = TRUE;
765 decoder->priv->needs_format = FALSE;
767 g_queue_init (&decoder->priv->frames);
768 g_queue_init (&decoder->priv->timestamps);
771 decoder->priv->do_qos = DEFAULT_QOS;
772 decoder->priv->max_errors = GST_VIDEO_DECODER_MAX_ERRORS;
774 decoder->priv->min_latency = 0;
775 decoder->priv->max_latency = 0;
777 decoder->priv->automatic_request_sync_points =
778 DEFAULT_AUTOMATIC_REQUEST_SYNC_POINTS;
779 decoder->priv->automatic_request_sync_point_flags =
780 DEFAULT_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS;
782 gst_video_decoder_reset (decoder, TRUE, TRUE);
785 static GstVideoCodecState *
786 _new_input_state (GstCaps * caps)
788 GstVideoCodecState *state;
789 GstStructure *structure;
790 const GValue *codec_data;
792 state = g_slice_new0 (GstVideoCodecState);
793 state->ref_count = 1;
794 gst_video_info_init (&state->info);
795 if (G_UNLIKELY (!gst_video_info_from_caps (&state->info, caps)))
797 state->caps = gst_caps_ref (caps);
799 structure = gst_caps_get_structure (caps, 0);
801 codec_data = gst_structure_get_value (structure, "codec_data");
802 if (codec_data && G_VALUE_TYPE (codec_data) == GST_TYPE_BUFFER)
803 state->codec_data = GST_BUFFER (g_value_dup_boxed (codec_data));
809 g_slice_free (GstVideoCodecState, state);
814 static GstVideoCodecState *
815 _new_output_state (GstVideoFormat fmt, GstVideoInterlaceMode interlace_mode,
816 guint width, guint height, GstVideoCodecState * reference,
817 gboolean copy_interlace_mode)
819 GstVideoCodecState *state;
821 state = g_slice_new0 (GstVideoCodecState);
822 state->ref_count = 1;
823 gst_video_info_init (&state->info);
824 if (!gst_video_info_set_interlaced_format (&state->info, fmt, interlace_mode,
826 g_slice_free (GstVideoCodecState, state);
831 GstVideoInfo *tgt, *ref;
834 ref = &reference->info;
836 /* Copy over extra fields from reference state */
837 if (copy_interlace_mode)
838 tgt->interlace_mode = ref->interlace_mode;
839 tgt->flags = ref->flags;
840 tgt->chroma_site = ref->chroma_site;
841 tgt->colorimetry = ref->colorimetry;
842 GST_DEBUG ("reference par %d/%d fps %d/%d",
843 ref->par_n, ref->par_d, ref->fps_n, ref->fps_d);
844 tgt->par_n = ref->par_n;
845 tgt->par_d = ref->par_d;
846 tgt->fps_n = ref->fps_n;
847 tgt->fps_d = ref->fps_d;
848 tgt->views = ref->views;
850 GST_VIDEO_INFO_FIELD_ORDER (tgt) = GST_VIDEO_INFO_FIELD_ORDER (ref);
852 if (GST_VIDEO_INFO_MULTIVIEW_MODE (ref) != GST_VIDEO_MULTIVIEW_MODE_NONE) {
853 GST_VIDEO_INFO_MULTIVIEW_MODE (tgt) = GST_VIDEO_INFO_MULTIVIEW_MODE (ref);
854 GST_VIDEO_INFO_MULTIVIEW_FLAGS (tgt) =
855 GST_VIDEO_INFO_MULTIVIEW_FLAGS (ref);
857 /* Default to MONO, overridden as needed by sub-classes */
858 GST_VIDEO_INFO_MULTIVIEW_MODE (tgt) = GST_VIDEO_MULTIVIEW_MODE_MONO;
859 GST_VIDEO_INFO_MULTIVIEW_FLAGS (tgt) = GST_VIDEO_MULTIVIEW_FLAGS_NONE;
863 GST_DEBUG ("reference par %d/%d fps %d/%d",
864 state->info.par_n, state->info.par_d,
865 state->info.fps_n, state->info.fps_d);
871 gst_video_decoder_setcaps (GstVideoDecoder * decoder, GstCaps * caps)
873 GstVideoDecoderClass *decoder_class;
874 GstVideoCodecState *state;
877 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
879 GST_DEBUG_OBJECT (decoder, "setcaps %" GST_PTR_FORMAT, caps);
881 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
883 if (decoder->priv->input_state) {
884 GST_DEBUG_OBJECT (decoder,
885 "Checking if caps changed old %" GST_PTR_FORMAT " new %" GST_PTR_FORMAT,
886 decoder->priv->input_state->caps, caps);
887 if (gst_caps_is_equal (decoder->priv->input_state->caps, caps))
888 goto caps_not_changed;
891 state = _new_input_state (caps);
893 if (G_UNLIKELY (state == NULL))
896 if (decoder_class->set_format)
897 ret = decoder_class->set_format (decoder, state);
902 if (decoder->priv->input_state)
903 gst_video_codec_state_unref (decoder->priv->input_state);
904 decoder->priv->input_state = state;
906 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
912 GST_DEBUG_OBJECT (decoder, "Caps did not change - ignore");
913 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
920 GST_WARNING_OBJECT (decoder, "Failed to parse caps");
921 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
927 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
928 GST_WARNING_OBJECT (decoder, "Subclass refused caps");
929 gst_video_codec_state_unref (state);
935 gst_video_decoder_finalize (GObject * object)
937 GstVideoDecoder *decoder;
939 decoder = GST_VIDEO_DECODER (object);
941 GST_DEBUG_OBJECT (object, "finalize");
943 g_rec_mutex_clear (&decoder->stream_lock);
945 if (decoder->priv->input_adapter) {
946 g_object_unref (decoder->priv->input_adapter);
947 decoder->priv->input_adapter = NULL;
949 if (decoder->priv->output_adapter) {
950 g_object_unref (decoder->priv->output_adapter);
951 decoder->priv->output_adapter = NULL;
954 if (decoder->priv->input_state)
955 gst_video_codec_state_unref (decoder->priv->input_state);
956 if (decoder->priv->output_state)
957 gst_video_codec_state_unref (decoder->priv->output_state);
959 if (decoder->priv->pool) {
960 gst_object_unref (decoder->priv->pool);
961 decoder->priv->pool = NULL;
964 if (decoder->priv->allocator) {
965 gst_object_unref (decoder->priv->allocator);
966 decoder->priv->allocator = NULL;
969 G_OBJECT_CLASS (parent_class)->finalize (object);
973 gst_video_decoder_get_property (GObject * object, guint property_id,
974 GValue * value, GParamSpec * pspec)
976 GstVideoDecoder *dec = GST_VIDEO_DECODER (object);
977 GstVideoDecoderPrivate *priv = dec->priv;
979 switch (property_id) {
981 g_value_set_boolean (value, priv->do_qos);
983 case PROP_MAX_ERRORS:
984 g_value_set_int (value, gst_video_decoder_get_max_errors (dec));
986 case PROP_MIN_FORCE_KEY_UNIT_INTERVAL:
987 g_value_set_uint64 (value, priv->min_force_key_unit_interval);
989 case PROP_DISCARD_CORRUPTED_FRAMES:
990 g_value_set_boolean (value, priv->discard_corrupted_frames);
992 case PROP_AUTOMATIC_REQUEST_SYNC_POINTS:
993 g_value_set_boolean (value, priv->automatic_request_sync_points);
995 case PROP_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS:
996 g_value_set_flags (value, priv->automatic_request_sync_point_flags);
999 G_OBJECT_WARN_INVALID_PROPERTY_ID (object, property_id, pspec);
1005 gst_video_decoder_set_property (GObject * object, guint property_id,
1006 const GValue * value, GParamSpec * pspec)
1008 GstVideoDecoder *dec = GST_VIDEO_DECODER (object);
1009 GstVideoDecoderPrivate *priv = dec->priv;
1011 switch (property_id) {
1013 priv->do_qos = g_value_get_boolean (value);
1015 case PROP_MAX_ERRORS:
1016 gst_video_decoder_set_max_errors (dec, g_value_get_int (value));
1018 case PROP_MIN_FORCE_KEY_UNIT_INTERVAL:
1019 priv->min_force_key_unit_interval = g_value_get_uint64 (value);
1021 case PROP_DISCARD_CORRUPTED_FRAMES:
1022 priv->discard_corrupted_frames = g_value_get_boolean (value);
1024 case PROP_AUTOMATIC_REQUEST_SYNC_POINTS:
1025 priv->automatic_request_sync_points = g_value_get_boolean (value);
1027 case PROP_AUTOMATIC_REQUEST_SYNC_POINT_FLAGS:
1028 priv->automatic_request_sync_point_flags = g_value_get_flags (value);
1031 G_OBJECT_WARN_INVALID_PROPERTY_ID (object, property_id, pspec);
1036 /* hard == FLUSH, otherwise discont */
1037 static GstFlowReturn
1038 gst_video_decoder_flush (GstVideoDecoder * dec, gboolean hard)
1040 GstVideoDecoderClass *klass = GST_VIDEO_DECODER_GET_CLASS (dec);
1041 GstFlowReturn ret = GST_FLOW_OK;
1043 GST_LOG_OBJECT (dec, "flush hard %d", hard);
1045 /* Inform subclass */
1047 GST_FIXME_OBJECT (dec, "GstVideoDecoder::reset() is deprecated");
1048 klass->reset (dec, hard);
1054 /* and get (re)set for the sequel */
1055 gst_video_decoder_reset (dec, FALSE, hard);
1061 gst_video_decoder_create_merged_tags_event (GstVideoDecoder * dec)
1063 GstTagList *merged_tags;
1065 GST_LOG_OBJECT (dec, "upstream : %" GST_PTR_FORMAT, dec->priv->upstream_tags);
1066 GST_LOG_OBJECT (dec, "decoder : %" GST_PTR_FORMAT, dec->priv->tags);
1067 GST_LOG_OBJECT (dec, "mode : %d", dec->priv->tags_merge_mode);
1070 gst_tag_list_merge (dec->priv->upstream_tags, dec->priv->tags,
1071 dec->priv->tags_merge_mode);
1073 GST_DEBUG_OBJECT (dec, "merged : %" GST_PTR_FORMAT, merged_tags);
1075 if (merged_tags == NULL)
1078 if (gst_tag_list_is_empty (merged_tags)) {
1079 gst_tag_list_unref (merged_tags);
1083 return gst_event_new_tag (merged_tags);
1087 gst_video_decoder_push_event (GstVideoDecoder * decoder, GstEvent * event)
1089 switch (GST_EVENT_TYPE (event)) {
1090 case GST_EVENT_SEGMENT:
1094 gst_event_copy_segment (event, &segment);
1096 GST_DEBUG_OBJECT (decoder, "segment %" GST_SEGMENT_FORMAT, &segment);
1098 if (segment.format != GST_FORMAT_TIME) {
1099 GST_DEBUG_OBJECT (decoder, "received non TIME newsegment");
1103 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1104 decoder->output_segment = segment;
1105 decoder->priv->in_out_segment_sync =
1106 gst_segment_is_equal (&decoder->input_segment, &segment);
1107 decoder->priv->last_timestamp_out = GST_CLOCK_TIME_NONE;
1108 decoder->priv->earliest_time = GST_CLOCK_TIME_NONE;
1109 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1116 GST_DEBUG_OBJECT (decoder, "pushing event %s",
1117 gst_event_type_get_name (GST_EVENT_TYPE (event)));
1119 return gst_pad_push_event (decoder->srcpad, event);
1122 static GstFlowReturn
1123 gst_video_decoder_parse_available (GstVideoDecoder * dec, gboolean at_eos,
1124 gboolean new_buffer)
1126 GstVideoDecoderClass *decoder_class = GST_VIDEO_DECODER_GET_CLASS (dec);
1127 GstVideoDecoderPrivate *priv = dec->priv;
1128 GstFlowReturn ret = GST_FLOW_OK;
1129 gsize was_available, available;
1132 available = gst_adapter_available (priv->input_adapter);
1134 while (available || new_buffer) {
1136 /* current frame may have been parsed and handled,
1137 * so we need to set up a new one when asking subclass to parse */
1138 if (priv->current_frame == NULL)
1139 priv->current_frame = gst_video_decoder_new_frame (dec);
1141 was_available = available;
1142 ret = decoder_class->parse (dec, priv->current_frame,
1143 priv->input_adapter, at_eos);
1144 if (ret != GST_FLOW_OK)
1147 /* if the subclass returned success (GST_FLOW_OK), it is expected
1148 * to have collected and submitted a frame, i.e. it should have
1149 * called gst_video_decoder_have_frame(), or at least consumed a
1150 * few bytes through gst_video_decoder_add_to_frame().
1152 * Otherwise, this is an implementation bug, and we error out
1153 * after 2 failed attempts */
1154 available = gst_adapter_available (priv->input_adapter);
1155 if (!priv->current_frame || available != was_available)
1157 else if (++inactive == 2)
1158 goto error_inactive;
1166 GST_ERROR_OBJECT (dec, "Failed to consume data. Error in subclass?");
1167 return GST_FLOW_ERROR;
1171 /* This function has to be called with the stream lock taken. */
1172 static GstFlowReturn
1173 gst_video_decoder_drain_out (GstVideoDecoder * dec, gboolean at_eos)
1175 GstVideoDecoderClass *decoder_class = GST_VIDEO_DECODER_GET_CLASS (dec);
1176 GstVideoDecoderPrivate *priv = dec->priv;
1177 GstFlowReturn ret = GST_FLOW_OK;
1179 if (dec->input_segment.rate > 0.0) {
1180 /* Forward mode, if unpacketized, give the child class
1181 * a final chance to flush out packets */
1182 if (!priv->packetized) {
1183 ret = gst_video_decoder_parse_available (dec, TRUE, FALSE);
1187 if (decoder_class->finish)
1188 ret = decoder_class->finish (dec);
1190 if (decoder_class->drain) {
1191 ret = decoder_class->drain (dec);
1193 GST_FIXME_OBJECT (dec, "Sub-class should implement drain()");
1197 /* Reverse playback mode */
1198 ret = gst_video_decoder_flush_parse (dec, TRUE);
1205 _flush_events (GstPad * pad, GList * events)
1209 for (tmp = events; tmp; tmp = tmp->next) {
1210 if (GST_EVENT_TYPE (tmp->data) != GST_EVENT_EOS &&
1211 GST_EVENT_TYPE (tmp->data) != GST_EVENT_SEGMENT &&
1212 GST_EVENT_IS_STICKY (tmp->data)) {
1213 gst_pad_store_sticky_event (pad, GST_EVENT_CAST (tmp->data));
1215 gst_event_unref (tmp->data);
1217 g_list_free (events);
1222 /* Must be called holding the GST_VIDEO_DECODER_STREAM_LOCK */
1224 gst_video_decoder_negotiate_default_caps (GstVideoDecoder * decoder)
1226 GstCaps *caps, *templcaps;
1227 GstVideoCodecState *state;
1231 GstStructure *structure;
1233 templcaps = gst_pad_get_pad_template_caps (decoder->srcpad);
1234 caps = gst_pad_peer_query_caps (decoder->srcpad, templcaps);
1236 gst_caps_unref (templcaps);
1241 if (!caps || gst_caps_is_empty (caps) || gst_caps_is_any (caps))
1244 GST_LOG_OBJECT (decoder, "peer caps %" GST_PTR_FORMAT, caps);
1246 /* before fixating, try to use whatever upstream provided */
1247 caps = gst_caps_make_writable (caps);
1248 caps_size = gst_caps_get_size (caps);
1249 if (decoder->priv->input_state && decoder->priv->input_state->caps) {
1250 GstCaps *sinkcaps = decoder->priv->input_state->caps;
1251 GstStructure *structure = gst_caps_get_structure (sinkcaps, 0);
1254 if (gst_structure_get_int (structure, "width", &width)) {
1255 for (i = 0; i < caps_size; i++) {
1256 gst_structure_set (gst_caps_get_structure (caps, i), "width",
1257 G_TYPE_INT, width, NULL);
1261 if (gst_structure_get_int (structure, "height", &height)) {
1262 for (i = 0; i < caps_size; i++) {
1263 gst_structure_set (gst_caps_get_structure (caps, i), "height",
1264 G_TYPE_INT, height, NULL);
1269 for (i = 0; i < caps_size; i++) {
1270 structure = gst_caps_get_structure (caps, i);
1271 /* Random I420 1280x720 for fixation */
1272 if (gst_structure_has_field (structure, "format"))
1273 gst_structure_fixate_field_string (structure, "format", "I420");
1275 gst_structure_set (structure, "format", G_TYPE_STRING, "I420", NULL);
1277 if (gst_structure_has_field (structure, "width"))
1278 gst_structure_fixate_field_nearest_int (structure, "width", 1280);
1280 gst_structure_set (structure, "width", G_TYPE_INT, 1280, NULL);
1282 if (gst_structure_has_field (structure, "height"))
1283 gst_structure_fixate_field_nearest_int (structure, "height", 720);
1285 gst_structure_set (structure, "height", G_TYPE_INT, 720, NULL);
1287 caps = gst_caps_fixate (caps);
1289 if (!caps || !gst_video_info_from_caps (&info, caps))
1292 GST_INFO_OBJECT (decoder,
1293 "Chose default caps %" GST_PTR_FORMAT " for initial gap", caps);
1295 gst_video_decoder_set_output_state (decoder, info.finfo->format,
1296 info.width, info.height, decoder->priv->input_state);
1297 gst_video_codec_state_unref (state);
1298 gst_caps_unref (caps);
1305 gst_caps_unref (caps);
1311 gst_video_decoder_handle_missing_data_default (GstVideoDecoder *
1312 decoder, GstClockTime timestamp, GstClockTime duration)
1314 GstVideoDecoderPrivate *priv;
1316 priv = decoder->priv;
1318 if (priv->automatic_request_sync_points) {
1319 GstClockTime deadline =
1320 gst_segment_to_running_time (&decoder->input_segment, GST_FORMAT_TIME,
1323 GST_DEBUG_OBJECT (decoder,
1324 "Requesting sync point for missing data at running time %"
1325 GST_TIME_FORMAT " timestamp %" GST_TIME_FORMAT " with duration %"
1326 GST_TIME_FORMAT, GST_TIME_ARGS (deadline), GST_TIME_ARGS (timestamp),
1327 GST_TIME_ARGS (duration));
1329 gst_video_decoder_request_sync_point_internal (decoder, deadline,
1330 priv->automatic_request_sync_point_flags);
1337 gst_video_decoder_sink_event_default (GstVideoDecoder * decoder,
1340 GstVideoDecoderClass *decoder_class;
1341 GstVideoDecoderPrivate *priv;
1342 gboolean ret = FALSE;
1343 gboolean forward_immediate = FALSE;
1345 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
1347 priv = decoder->priv;
1349 switch (GST_EVENT_TYPE (event)) {
1350 case GST_EVENT_STREAM_START:
1352 GstFlowReturn flow_ret = GST_FLOW_OK;
1354 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1355 flow_ret = gst_video_decoder_drain_out (decoder, FALSE);
1356 ret = (flow_ret == GST_FLOW_OK);
1358 GST_DEBUG_OBJECT (decoder, "received STREAM_START. Clearing taglist");
1359 /* Flush upstream tags after a STREAM_START */
1360 if (priv->upstream_tags) {
1361 gst_tag_list_unref (priv->upstream_tags);
1362 priv->upstream_tags = NULL;
1363 priv->tags_changed = TRUE;
1365 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1367 /* Forward STREAM_START immediately. Everything is drained after
1368 * the STREAM_START event and we can forward this event immediately
1369 * now without having buffers out of order.
1371 forward_immediate = TRUE;
1374 case GST_EVENT_CAPS:
1378 gst_event_parse_caps (event, &caps);
1379 ret = gst_video_decoder_setcaps (decoder, caps);
1380 gst_event_unref (event);
1384 case GST_EVENT_SEGMENT_DONE:
1386 GstFlowReturn flow_ret = GST_FLOW_OK;
1388 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1389 flow_ret = gst_video_decoder_drain_out (decoder, FALSE);
1390 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1391 ret = (flow_ret == GST_FLOW_OK);
1393 /* Forward SEGMENT_DONE immediately. This is required
1394 * because no buffer or serialized event might come
1395 * after SEGMENT_DONE and nothing could trigger another
1396 * _finish_frame() call.
1398 * The subclass can override this behaviour by overriding
1399 * the ::sink_event() vfunc and not chaining up to the
1400 * parent class' ::sink_event() until a later time.
1402 forward_immediate = TRUE;
1407 GstFlowReturn flow_ret = GST_FLOW_OK;
1409 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1410 flow_ret = gst_video_decoder_drain_out (decoder, TRUE);
1411 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1412 ret = (flow_ret == GST_FLOW_OK);
1414 /* Error out even if EOS was ok when we had input, but no output */
1415 if (ret && priv->had_input_data && !priv->had_output_data) {
1416 GST_ELEMENT_ERROR (decoder, STREAM, DECODE,
1417 ("No valid frames decoded before end of stream"),
1418 ("no valid frames found"));
1421 /* Forward EOS immediately. This is required because no
1422 * buffer or serialized event will come after EOS and
1423 * nothing could trigger another _finish_frame() call.
1425 * The subclass can override this behaviour by overriding
1426 * the ::sink_event() vfunc and not chaining up to the
1427 * parent class' ::sink_event() until a later time.
1429 forward_immediate = TRUE;
1434 GstClockTime timestamp, duration;
1435 GstGapFlags gap_flags = 0;
1436 GstFlowReturn flow_ret = GST_FLOW_OK;
1437 gboolean needs_reconfigure = FALSE;
1439 GList *frame_events;
1441 gst_event_parse_gap (event, ×tamp, &duration);
1442 gst_event_parse_gap_flags (event, &gap_flags);
1444 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1445 /* If this is not missing data, or the subclass does not handle it
1446 * specifically, then drain out the decoder and forward the event
1448 if ((gap_flags & GST_GAP_FLAG_MISSING_DATA) == 0
1449 || !decoder_class->handle_missing_data
1450 || decoder_class->handle_missing_data (decoder, timestamp,
1452 if (decoder->input_segment.flags & GST_SEEK_FLAG_TRICKMODE_KEY_UNITS)
1453 flow_ret = gst_video_decoder_drain_out (decoder, FALSE);
1454 ret = (flow_ret == GST_FLOW_OK);
1456 /* Ensure we have caps before forwarding the event */
1457 if (!decoder->priv->output_state) {
1458 if (!gst_video_decoder_negotiate_default_caps (decoder)) {
1459 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1460 GST_ELEMENT_ERROR (decoder, STREAM, FORMAT, (NULL),
1461 ("Decoder output not negotiated before GAP event."));
1462 forward_immediate = TRUE;
1465 needs_reconfigure = TRUE;
1468 needs_reconfigure = gst_pad_check_reconfigure (decoder->srcpad)
1469 || needs_reconfigure;
1470 if (decoder->priv->output_state_changed || needs_reconfigure) {
1471 if (!gst_video_decoder_negotiate_unlocked (decoder)) {
1472 GST_WARNING_OBJECT (decoder, "Failed to negotiate with downstream");
1473 gst_pad_mark_reconfigure (decoder->srcpad);
1477 GST_DEBUG_OBJECT (decoder, "Pushing all pending serialized events"
1479 events = decoder->priv->pending_events;
1480 frame_events = decoder->priv->current_frame_events;
1481 decoder->priv->pending_events = NULL;
1482 decoder->priv->current_frame_events = NULL;
1484 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1486 gst_video_decoder_push_event_list (decoder, events);
1487 gst_video_decoder_push_event_list (decoder, frame_events);
1489 /* Forward GAP immediately. Everything is drained after
1490 * the GAP event and we can forward this event immediately
1491 * now without having buffers out of order.
1493 forward_immediate = TRUE;
1495 gst_clear_event (&event);
1499 case GST_EVENT_CUSTOM_DOWNSTREAM:
1502 GstFlowReturn flow_ret = GST_FLOW_OK;
1504 if (gst_video_event_parse_still_frame (event, &in_still)) {
1506 GST_DEBUG_OBJECT (decoder, "draining current data for still-frame");
1507 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1508 flow_ret = gst_video_decoder_drain_out (decoder, FALSE);
1509 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1510 ret = (flow_ret == GST_FLOW_OK);
1512 /* Forward STILL_FRAME immediately. Everything is drained after
1513 * the STILL_FRAME event and we can forward this event immediately
1514 * now without having buffers out of order.
1516 forward_immediate = TRUE;
1520 case GST_EVENT_SEGMENT:
1524 gst_event_copy_segment (event, &segment);
1526 if (segment.format == GST_FORMAT_TIME) {
1527 GST_DEBUG_OBJECT (decoder,
1528 "received TIME SEGMENT %" GST_SEGMENT_FORMAT, &segment);
1532 GST_DEBUG_OBJECT (decoder,
1533 "received SEGMENT %" GST_SEGMENT_FORMAT, &segment);
1535 /* handle newsegment as a result from our legacy simple seeking */
1536 /* note that initial 0 should convert to 0 in any case */
1537 if (priv->do_estimate_rate &&
1538 gst_pad_query_convert (decoder->sinkpad, GST_FORMAT_BYTES,
1539 segment.start, GST_FORMAT_TIME, &start)) {
1540 /* best attempt convert */
1541 /* as these are only estimates, stop is kept open-ended to avoid
1542 * premature cutting */
1543 GST_DEBUG_OBJECT (decoder,
1544 "converted to TIME start %" GST_TIME_FORMAT,
1545 GST_TIME_ARGS (start));
1546 segment.start = start;
1547 segment.stop = GST_CLOCK_TIME_NONE;
1548 segment.time = start;
1550 gst_event_unref (event);
1551 event = gst_event_new_segment (&segment);
1553 goto newseg_wrong_format;
1557 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1559 /* Update the decode flags in the segment if we have an instant-rate
1560 * override active */
1561 GST_OBJECT_LOCK (decoder);
1562 if (!priv->decode_flags_override)
1563 priv->decode_flags = segment.flags;
1565 segment.flags &= ~GST_SEGMENT_INSTANT_FLAGS;
1566 segment.flags |= priv->decode_flags & GST_SEGMENT_INSTANT_FLAGS;
1569 priv->base_timestamp = GST_CLOCK_TIME_NONE;
1570 priv->base_picture_number = 0;
1572 decoder->input_segment = segment;
1573 decoder->priv->in_out_segment_sync = FALSE;
1575 GST_OBJECT_UNLOCK (decoder);
1576 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1580 case GST_EVENT_INSTANT_RATE_CHANGE:
1582 GstSegmentFlags flags;
1585 gst_event_parse_instant_rate_change (event, NULL, &flags);
1587 GST_OBJECT_LOCK (decoder);
1588 priv->decode_flags_override = TRUE;
1589 priv->decode_flags = flags;
1591 /* Update the input segment flags */
1592 seg = &decoder->input_segment;
1593 seg->flags &= ~GST_SEGMENT_INSTANT_FLAGS;
1594 seg->flags |= priv->decode_flags & GST_SEGMENT_INSTANT_FLAGS;
1595 GST_OBJECT_UNLOCK (decoder);
1598 case GST_EVENT_FLUSH_STOP:
1602 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1603 for (l = priv->frames.head; l; l = l->next) {
1604 GstVideoCodecFrame *frame = l->data;
1606 frame->events = _flush_events (decoder->srcpad, frame->events);
1608 priv->current_frame_events = _flush_events (decoder->srcpad,
1609 decoder->priv->current_frame_events);
1611 /* well, this is kind of worse than a DISCONT */
1612 gst_video_decoder_flush (decoder, TRUE);
1613 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1614 /* Forward FLUSH_STOP immediately. This is required because it is
1615 * expected to be forwarded immediately and no buffers are queued
1618 forward_immediate = TRUE;
1625 gst_event_parse_tag (event, &tags);
1627 if (gst_tag_list_get_scope (tags) == GST_TAG_SCOPE_STREAM) {
1628 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1629 if (priv->upstream_tags != tags) {
1630 if (priv->upstream_tags)
1631 gst_tag_list_unref (priv->upstream_tags);
1632 priv->upstream_tags = gst_tag_list_ref (tags);
1633 GST_INFO_OBJECT (decoder, "upstream tags: %" GST_PTR_FORMAT, tags);
1635 gst_event_unref (event);
1636 event = gst_video_decoder_create_merged_tags_event (decoder);
1637 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1647 /* Forward non-serialized events immediately, and all other
1648 * events which can be forwarded immediately without potentially
1649 * causing the event to go out of order with other events and
1650 * buffers as decided above.
1653 if (!GST_EVENT_IS_SERIALIZED (event) || forward_immediate) {
1654 ret = gst_video_decoder_push_event (decoder, event);
1656 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
1657 decoder->priv->current_frame_events =
1658 g_list_prepend (decoder->priv->current_frame_events, event);
1659 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
1666 newseg_wrong_format:
1668 GST_DEBUG_OBJECT (decoder, "received non TIME newsegment");
1669 gst_event_unref (event);
1676 gst_video_decoder_sink_event (GstPad * pad, GstObject * parent,
1679 GstVideoDecoder *decoder;
1680 GstVideoDecoderClass *decoder_class;
1681 gboolean ret = FALSE;
1683 decoder = GST_VIDEO_DECODER (parent);
1684 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
1686 GST_DEBUG_OBJECT (decoder, "received event %d, %s", GST_EVENT_TYPE (event),
1687 GST_EVENT_TYPE_NAME (event));
1689 if (decoder_class->sink_event)
1690 ret = decoder_class->sink_event (decoder, event);
1695 /* perform upstream byte <-> time conversion (duration, seeking)
1696 * if subclass allows and if enough data for moderately decent conversion */
1697 static inline gboolean
1698 gst_video_decoder_do_byte (GstVideoDecoder * dec)
1702 GST_OBJECT_LOCK (dec);
1703 ret = dec->priv->do_estimate_rate && (dec->priv->bytes_out > 0)
1704 && (dec->priv->time > GST_SECOND);
1705 GST_OBJECT_UNLOCK (dec);
1711 gst_video_decoder_do_seek (GstVideoDecoder * dec, GstEvent * event)
1715 GstSeekType start_type, end_type;
1717 gint64 start, start_time, end_time;
1718 GstSegment seek_segment;
1721 gst_event_parse_seek (event, &rate, &format, &flags, &start_type,
1722 &start_time, &end_type, &end_time);
1724 /* we'll handle plain open-ended flushing seeks with the simple approach */
1726 GST_DEBUG_OBJECT (dec, "unsupported seek: rate");
1730 if (start_type != GST_SEEK_TYPE_SET) {
1731 GST_DEBUG_OBJECT (dec, "unsupported seek: start time");
1735 if ((end_type != GST_SEEK_TYPE_SET && end_type != GST_SEEK_TYPE_NONE) ||
1736 (end_type == GST_SEEK_TYPE_SET && end_time != GST_CLOCK_TIME_NONE)) {
1737 GST_DEBUG_OBJECT (dec, "unsupported seek: end time");
1741 if (!(flags & GST_SEEK_FLAG_FLUSH)) {
1742 GST_DEBUG_OBJECT (dec, "unsupported seek: not flushing");
1746 memcpy (&seek_segment, &dec->output_segment, sizeof (seek_segment));
1747 gst_segment_do_seek (&seek_segment, rate, format, flags, start_type,
1748 start_time, end_type, end_time, NULL);
1749 start_time = seek_segment.position;
1751 if (!gst_pad_query_convert (dec->sinkpad, GST_FORMAT_TIME, start_time,
1752 GST_FORMAT_BYTES, &start)) {
1753 GST_DEBUG_OBJECT (dec, "conversion failed");
1757 seqnum = gst_event_get_seqnum (event);
1758 event = gst_event_new_seek (1.0, GST_FORMAT_BYTES, flags,
1759 GST_SEEK_TYPE_SET, start, GST_SEEK_TYPE_NONE, -1);
1760 gst_event_set_seqnum (event, seqnum);
1762 GST_DEBUG_OBJECT (dec, "seeking to %" GST_TIME_FORMAT " at byte offset %"
1763 G_GINT64_FORMAT, GST_TIME_ARGS (start_time), start);
1765 return gst_pad_push_event (dec->sinkpad, event);
1769 gst_video_decoder_src_event_default (GstVideoDecoder * decoder,
1772 GstVideoDecoderPrivate *priv;
1773 gboolean res = FALSE;
1775 priv = decoder->priv;
1777 GST_DEBUG_OBJECT (decoder,
1778 "received event %d, %s", GST_EVENT_TYPE (event),
1779 GST_EVENT_TYPE_NAME (event));
1781 switch (GST_EVENT_TYPE (event)) {
1782 case GST_EVENT_SEEK:
1787 GstSeekType start_type, stop_type;
1789 gint64 tstart, tstop;
1792 gst_event_parse_seek (event, &rate, &format, &flags, &start_type, &start,
1794 seqnum = gst_event_get_seqnum (event);
1796 /* upstream gets a chance first */
1797 if ((res = gst_pad_push_event (decoder->sinkpad, event)))
1800 /* if upstream fails for a time seek, maybe we can help if allowed */
1801 if (format == GST_FORMAT_TIME) {
1802 if (gst_video_decoder_do_byte (decoder))
1803 res = gst_video_decoder_do_seek (decoder, event);
1807 /* ... though a non-time seek can be aided as well */
1808 /* First bring the requested format to time */
1810 gst_pad_query_convert (decoder->srcpad, format, start,
1811 GST_FORMAT_TIME, &tstart)))
1814 gst_pad_query_convert (decoder->srcpad, format, stop,
1815 GST_FORMAT_TIME, &tstop)))
1818 /* then seek with time on the peer */
1819 event = gst_event_new_seek (rate, GST_FORMAT_TIME,
1820 flags, start_type, tstart, stop_type, tstop);
1821 gst_event_set_seqnum (event, seqnum);
1823 res = gst_pad_push_event (decoder->sinkpad, event);
1830 GstClockTimeDiff diff;
1831 GstClockTime timestamp;
1833 gst_event_parse_qos (event, &type, &proportion, &diff, ×tamp);
1835 GST_OBJECT_LOCK (decoder);
1836 priv->proportion = proportion;
1837 if (G_LIKELY (GST_CLOCK_TIME_IS_VALID (timestamp))) {
1838 if (G_UNLIKELY (diff > 0)) {
1839 priv->earliest_time = timestamp + 2 * diff + priv->qos_frame_duration;
1841 priv->earliest_time = timestamp + diff;
1844 priv->earliest_time = GST_CLOCK_TIME_NONE;
1846 GST_OBJECT_UNLOCK (decoder);
1848 GST_DEBUG_OBJECT (decoder,
1849 "got QoS %" GST_TIME_FORMAT ", %" GST_STIME_FORMAT ", %g",
1850 GST_TIME_ARGS (timestamp), GST_STIME_ARGS (diff), proportion);
1852 res = gst_pad_push_event (decoder->sinkpad, event);
1856 res = gst_pad_push_event (decoder->sinkpad, event);
1863 GST_DEBUG_OBJECT (decoder, "could not convert format");
1868 gst_video_decoder_src_event (GstPad * pad, GstObject * parent, GstEvent * event)
1870 GstVideoDecoder *decoder;
1871 GstVideoDecoderClass *decoder_class;
1872 gboolean ret = FALSE;
1874 decoder = GST_VIDEO_DECODER (parent);
1875 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
1877 GST_DEBUG_OBJECT (decoder, "received event %d, %s", GST_EVENT_TYPE (event),
1878 GST_EVENT_TYPE_NAME (event));
1880 if (decoder_class->src_event)
1881 ret = decoder_class->src_event (decoder, event);
1887 gst_video_decoder_src_query_default (GstVideoDecoder * dec, GstQuery * query)
1889 GstPad *pad = GST_VIDEO_DECODER_SRC_PAD (dec);
1890 gboolean res = TRUE;
1892 GST_LOG_OBJECT (dec, "handling query: %" GST_PTR_FORMAT, query);
1894 switch (GST_QUERY_TYPE (query)) {
1895 case GST_QUERY_POSITION:
1900 /* upstream gets a chance first */
1901 if ((res = gst_pad_peer_query (dec->sinkpad, query))) {
1902 GST_LOG_OBJECT (dec, "returning peer response");
1906 /* Refuse BYTES format queries. If it made sense to
1907 * answer them, upstream would have already */
1908 gst_query_parse_position (query, &format, NULL);
1910 if (format == GST_FORMAT_BYTES) {
1911 GST_LOG_OBJECT (dec, "Ignoring BYTES position query");
1915 /* we start from the last seen time */
1916 time = dec->priv->last_timestamp_out;
1917 /* correct for the segment values */
1918 time = gst_segment_to_stream_time (&dec->output_segment,
1919 GST_FORMAT_TIME, time);
1921 GST_LOG_OBJECT (dec,
1922 "query %p: our time: %" GST_TIME_FORMAT, query, GST_TIME_ARGS (time));
1924 /* and convert to the final format */
1925 if (!(res = gst_pad_query_convert (pad, GST_FORMAT_TIME, time,
1929 gst_query_set_position (query, format, value);
1931 GST_LOG_OBJECT (dec,
1932 "query %p: we return %" G_GINT64_FORMAT " (format %u)", query, value,
1936 case GST_QUERY_DURATION:
1940 /* upstream in any case */
1941 if ((res = gst_pad_query_default (pad, GST_OBJECT (dec), query)))
1944 gst_query_parse_duration (query, &format, NULL);
1945 /* try answering TIME by converting from BYTE if subclass allows */
1946 if (format == GST_FORMAT_TIME && gst_video_decoder_do_byte (dec)) {
1949 if (gst_pad_peer_query_duration (dec->sinkpad, GST_FORMAT_BYTES,
1951 GST_LOG_OBJECT (dec, "upstream size %" G_GINT64_FORMAT, value);
1952 if (gst_pad_query_convert (dec->sinkpad,
1953 GST_FORMAT_BYTES, value, GST_FORMAT_TIME, &value)) {
1954 gst_query_set_duration (query, GST_FORMAT_TIME, value);
1961 case GST_QUERY_CONVERT:
1963 GstFormat src_fmt, dest_fmt;
1964 gint64 src_val, dest_val;
1966 GST_DEBUG_OBJECT (dec, "convert query");
1968 gst_query_parse_convert (query, &src_fmt, &src_val, &dest_fmt, &dest_val);
1969 GST_OBJECT_LOCK (dec);
1970 if (dec->priv->output_state != NULL)
1971 res = __gst_video_rawvideo_convert (dec->priv->output_state,
1972 src_fmt, src_val, &dest_fmt, &dest_val);
1975 GST_OBJECT_UNLOCK (dec);
1978 gst_query_set_convert (query, src_fmt, src_val, dest_fmt, dest_val);
1981 case GST_QUERY_LATENCY:
1984 GstClockTime min_latency, max_latency;
1986 res = gst_pad_peer_query (dec->sinkpad, query);
1988 gst_query_parse_latency (query, &live, &min_latency, &max_latency);
1989 GST_DEBUG_OBJECT (dec, "Peer qlatency: live %d, min %"
1990 GST_TIME_FORMAT " max %" GST_TIME_FORMAT, live,
1991 GST_TIME_ARGS (min_latency), GST_TIME_ARGS (max_latency));
1993 GST_OBJECT_LOCK (dec);
1994 min_latency += dec->priv->min_latency;
1995 if (max_latency == GST_CLOCK_TIME_NONE
1996 || dec->priv->max_latency == GST_CLOCK_TIME_NONE)
1997 max_latency = GST_CLOCK_TIME_NONE;
1999 max_latency += dec->priv->max_latency;
2000 GST_OBJECT_UNLOCK (dec);
2002 gst_query_set_latency (query, live, min_latency, max_latency);
2007 res = gst_pad_query_default (pad, GST_OBJECT (dec), query);
2012 GST_ERROR_OBJECT (dec, "query failed");
2017 gst_video_decoder_src_query (GstPad * pad, GstObject * parent, GstQuery * query)
2019 GstVideoDecoder *decoder;
2020 GstVideoDecoderClass *decoder_class;
2021 gboolean ret = FALSE;
2023 decoder = GST_VIDEO_DECODER (parent);
2024 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
2026 GST_DEBUG_OBJECT (decoder, "received query %d, %s", GST_QUERY_TYPE (query),
2027 GST_QUERY_TYPE_NAME (query));
2029 if (decoder_class->src_query)
2030 ret = decoder_class->src_query (decoder, query);
2036 * gst_video_decoder_proxy_getcaps:
2037 * @decoder: a #GstVideoDecoder
2038 * @caps: (allow-none): initial caps
2039 * @filter: (allow-none): filter caps
2041 * Returns caps that express @caps (or sink template caps if @caps == NULL)
2042 * restricted to resolution/format/... combinations supported by downstream
2045 * Returns: (transfer full): a #GstCaps owned by caller
2050 gst_video_decoder_proxy_getcaps (GstVideoDecoder * decoder, GstCaps * caps,
2053 return __gst_video_element_proxy_getcaps (GST_ELEMENT_CAST (decoder),
2054 GST_VIDEO_DECODER_SINK_PAD (decoder),
2055 GST_VIDEO_DECODER_SRC_PAD (decoder), caps, filter);
2059 gst_video_decoder_sink_getcaps (GstVideoDecoder * decoder, GstCaps * filter)
2061 GstVideoDecoderClass *klass;
2064 klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
2067 caps = klass->getcaps (decoder, filter);
2069 caps = gst_video_decoder_proxy_getcaps (decoder, NULL, filter);
2071 GST_LOG_OBJECT (decoder, "Returning caps %" GST_PTR_FORMAT, caps);
2077 gst_video_decoder_sink_query_default (GstVideoDecoder * decoder,
2080 GstPad *pad = GST_VIDEO_DECODER_SINK_PAD (decoder);
2081 GstVideoDecoderPrivate *priv;
2082 gboolean res = FALSE;
2084 priv = decoder->priv;
2086 GST_LOG_OBJECT (decoder, "handling query: %" GST_PTR_FORMAT, query);
2088 switch (GST_QUERY_TYPE (query)) {
2089 case GST_QUERY_CONVERT:
2091 GstFormat src_fmt, dest_fmt;
2092 gint64 src_val, dest_val;
2094 gst_query_parse_convert (query, &src_fmt, &src_val, &dest_fmt, &dest_val);
2095 GST_OBJECT_LOCK (decoder);
2097 __gst_video_encoded_video_convert (priv->bytes_out, priv->time,
2098 src_fmt, src_val, &dest_fmt, &dest_val);
2099 GST_OBJECT_UNLOCK (decoder);
2102 gst_query_set_convert (query, src_fmt, src_val, dest_fmt, dest_val);
2105 case GST_QUERY_ALLOCATION:{
2106 GstVideoDecoderClass *klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
2108 if (klass->propose_allocation)
2109 res = klass->propose_allocation (decoder, query);
2112 case GST_QUERY_CAPS:{
2113 GstCaps *filter, *caps;
2115 gst_query_parse_caps (query, &filter);
2116 caps = gst_video_decoder_sink_getcaps (decoder, filter);
2117 gst_query_set_caps_result (query, caps);
2118 gst_caps_unref (caps);
2122 case GST_QUERY_ACCEPT_CAPS:{
2123 if (decoder->priv->use_default_pad_acceptcaps) {
2125 gst_pad_query_default (GST_VIDEO_DECODER_SINK_PAD (decoder),
2126 GST_OBJECT_CAST (decoder), query);
2129 GstCaps *allowed_caps;
2130 GstCaps *template_caps;
2133 gst_query_parse_accept_caps (query, &caps);
2135 template_caps = gst_pad_get_pad_template_caps (pad);
2136 accept = gst_caps_is_subset (caps, template_caps);
2137 gst_caps_unref (template_caps);
2141 gst_pad_query_caps (GST_VIDEO_DECODER_SINK_PAD (decoder), caps);
2143 accept = gst_caps_can_intersect (caps, allowed_caps);
2145 gst_caps_unref (allowed_caps);
2148 gst_query_set_accept_caps_result (query, accept);
2154 res = gst_pad_query_default (pad, GST_OBJECT (decoder), query);
2161 GST_DEBUG_OBJECT (decoder, "query failed");
2167 gst_video_decoder_sink_query (GstPad * pad, GstObject * parent,
2170 GstVideoDecoder *decoder;
2171 GstVideoDecoderClass *decoder_class;
2172 gboolean ret = FALSE;
2174 decoder = GST_VIDEO_DECODER (parent);
2175 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
2177 GST_DEBUG_OBJECT (decoder, "received query %d, %s", GST_QUERY_TYPE (query),
2178 GST_QUERY_TYPE_NAME (query));
2180 if (decoder_class->sink_query)
2181 ret = decoder_class->sink_query (decoder, query);
2186 typedef struct _Timestamp Timestamp;
2192 GstClockTime duration;
2197 timestamp_free (Timestamp * ts)
2199 g_slice_free (Timestamp, ts);
2203 gst_video_decoder_add_buffer_info (GstVideoDecoder * decoder,
2206 GstVideoDecoderPrivate *priv = decoder->priv;
2209 if (!GST_BUFFER_PTS_IS_VALID (buffer) &&
2210 !GST_BUFFER_DTS_IS_VALID (buffer) &&
2211 !GST_BUFFER_DURATION_IS_VALID (buffer) &&
2212 GST_BUFFER_FLAGS (buffer) == 0) {
2213 /* Save memory - don't bother storing info
2214 * for buffers with no distinguishing info */
2218 ts = g_slice_new (Timestamp);
2220 GST_LOG_OBJECT (decoder,
2221 "adding PTS %" GST_TIME_FORMAT " DTS %" GST_TIME_FORMAT
2222 " (offset:%" G_GUINT64_FORMAT ")",
2223 GST_TIME_ARGS (GST_BUFFER_PTS (buffer)),
2224 GST_TIME_ARGS (GST_BUFFER_DTS (buffer)), priv->input_offset);
2226 ts->offset = priv->input_offset;
2227 ts->pts = GST_BUFFER_PTS (buffer);
2228 ts->dts = GST_BUFFER_DTS (buffer);
2229 ts->duration = GST_BUFFER_DURATION (buffer);
2230 ts->flags = GST_BUFFER_FLAGS (buffer);
2232 g_queue_push_tail (&priv->timestamps, ts);
2234 if (g_queue_get_length (&priv->timestamps) > 40) {
2235 GST_WARNING_OBJECT (decoder,
2236 "decoder timestamp list getting long: %d timestamps,"
2237 "possible internal leaking?", g_queue_get_length (&priv->timestamps));
2242 gst_video_decoder_get_buffer_info_at_offset (GstVideoDecoder *
2243 decoder, guint64 offset, GstClockTime * pts, GstClockTime * dts,
2244 GstClockTime * duration, guint * flags)
2246 #ifndef GST_DISABLE_GST_DEBUG
2247 guint64 got_offset = 0;
2252 *pts = GST_CLOCK_TIME_NONE;
2253 *dts = GST_CLOCK_TIME_NONE;
2254 *duration = GST_CLOCK_TIME_NONE;
2257 g = decoder->priv->timestamps.head;
2260 if (ts->offset <= offset) {
2261 GList *next = g->next;
2262 #ifndef GST_DISABLE_GST_DEBUG
2263 got_offset = ts->offset;
2267 *duration = ts->duration;
2269 g_queue_delete_link (&decoder->priv->timestamps, g);
2271 timestamp_free (ts);
2277 GST_LOG_OBJECT (decoder,
2278 "got PTS %" GST_TIME_FORMAT " DTS %" GST_TIME_FORMAT " flags %x @ offs %"
2279 G_GUINT64_FORMAT " (wanted offset:%" G_GUINT64_FORMAT ")",
2280 GST_TIME_ARGS (*pts), GST_TIME_ARGS (*dts), *flags, got_offset, offset);
2283 #if !GLIB_CHECK_VERSION(2, 60, 0)
2284 #define g_queue_clear_full queue_clear_full
2286 queue_clear_full (GQueue * queue, GDestroyNotify free_func)
2290 while ((data = g_queue_pop_head (queue)) != NULL)
2296 gst_video_decoder_clear_queues (GstVideoDecoder * dec)
2298 GstVideoDecoderPrivate *priv = dec->priv;
2300 g_list_free_full (priv->output_queued,
2301 (GDestroyNotify) gst_mini_object_unref);
2302 priv->output_queued = NULL;
2304 g_list_free_full (priv->gather, (GDestroyNotify) gst_mini_object_unref);
2305 priv->gather = NULL;
2306 g_list_free_full (priv->decode, (GDestroyNotify) gst_video_codec_frame_unref);
2307 priv->decode = NULL;
2308 g_list_free_full (priv->parse, (GDestroyNotify) gst_mini_object_unref);
2310 g_list_free_full (priv->parse_gather,
2311 (GDestroyNotify) gst_video_codec_frame_unref);
2312 priv->parse_gather = NULL;
2313 g_queue_clear_full (&priv->frames,
2314 (GDestroyNotify) gst_video_codec_frame_unref);
2318 gst_video_decoder_reset (GstVideoDecoder * decoder, gboolean full,
2319 gboolean flush_hard)
2321 GstVideoDecoderPrivate *priv = decoder->priv;
2323 GST_DEBUG_OBJECT (decoder, "reset full %d", full);
2325 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
2327 if (full || flush_hard) {
2328 gst_segment_init (&decoder->input_segment, GST_FORMAT_UNDEFINED);
2329 gst_segment_init (&decoder->output_segment, GST_FORMAT_UNDEFINED);
2330 gst_video_decoder_clear_queues (decoder);
2331 decoder->priv->in_out_segment_sync = TRUE;
2333 if (priv->current_frame) {
2334 gst_video_codec_frame_unref (priv->current_frame);
2335 priv->current_frame = NULL;
2338 g_list_free_full (priv->current_frame_events,
2339 (GDestroyNotify) gst_event_unref);
2340 priv->current_frame_events = NULL;
2341 g_list_free_full (priv->pending_events, (GDestroyNotify) gst_event_unref);
2342 priv->pending_events = NULL;
2344 priv->error_count = 0;
2345 priv->had_output_data = FALSE;
2346 priv->had_input_data = FALSE;
2348 GST_OBJECT_LOCK (decoder);
2349 priv->earliest_time = GST_CLOCK_TIME_NONE;
2350 priv->proportion = 0.5;
2351 priv->decode_flags_override = FALSE;
2353 priv->request_sync_point_flags = 0;
2354 priv->request_sync_point_frame_number = REQUEST_SYNC_POINT_UNSET;
2355 priv->last_force_key_unit_time = GST_CLOCK_TIME_NONE;
2356 GST_OBJECT_UNLOCK (decoder);
2357 priv->distance_from_sync = -1;
2361 if (priv->input_state)
2362 gst_video_codec_state_unref (priv->input_state);
2363 priv->input_state = NULL;
2364 GST_OBJECT_LOCK (decoder);
2365 if (priv->output_state)
2366 gst_video_codec_state_unref (priv->output_state);
2367 priv->output_state = NULL;
2369 priv->qos_frame_duration = 0;
2370 GST_OBJECT_UNLOCK (decoder);
2373 gst_tag_list_unref (priv->tags);
2375 priv->tags_merge_mode = GST_TAG_MERGE_APPEND;
2376 if (priv->upstream_tags) {
2377 gst_tag_list_unref (priv->upstream_tags);
2378 priv->upstream_tags = NULL;
2380 priv->tags_changed = FALSE;
2381 priv->reordered_output = FALSE;
2384 priv->processed = 0;
2386 priv->decode_frame_number = 0;
2387 priv->base_picture_number = 0;
2390 GST_DEBUG_OBJECT (decoder, "deactivate pool %" GST_PTR_FORMAT,
2392 gst_buffer_pool_set_active (priv->pool, FALSE);
2393 gst_object_unref (priv->pool);
2397 if (priv->allocator) {
2398 gst_object_unref (priv->allocator);
2399 priv->allocator = NULL;
2403 priv->discont = TRUE;
2405 priv->base_timestamp = GST_CLOCK_TIME_NONE;
2406 priv->last_timestamp_out = GST_CLOCK_TIME_NONE;
2407 priv->pts_delta = GST_CLOCK_TIME_NONE;
2409 priv->input_offset = 0;
2410 priv->frame_offset = 0;
2411 gst_adapter_clear (priv->input_adapter);
2412 gst_adapter_clear (priv->output_adapter);
2413 g_queue_clear_full (&priv->timestamps, (GDestroyNotify) timestamp_free);
2415 GST_OBJECT_LOCK (decoder);
2416 priv->bytes_out = 0;
2418 GST_OBJECT_UNLOCK (decoder);
2420 #ifndef GST_DISABLE_DEBUG
2421 priv->last_reset_time = gst_util_get_timestamp ();
2424 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
2427 static GstFlowReturn
2428 gst_video_decoder_chain_forward (GstVideoDecoder * decoder,
2429 GstBuffer * buf, gboolean at_eos)
2431 GstVideoDecoderPrivate *priv;
2432 GstVideoDecoderClass *klass;
2433 GstFlowReturn ret = GST_FLOW_OK;
2435 klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
2436 priv = decoder->priv;
2438 g_return_val_if_fail (priv->packetized || klass->parse, GST_FLOW_ERROR);
2440 /* Draining on DISCONT is handled in chain_reverse() for reverse playback,
2441 * and this function would only be called to get everything collected GOP
2442 * by GOP in the parse_gather list */
2443 if (decoder->input_segment.rate > 0.0 && GST_BUFFER_IS_DISCONT (buf)
2444 && (decoder->input_segment.flags & GST_SEEK_FLAG_TRICKMODE_KEY_UNITS))
2445 ret = gst_video_decoder_drain_out (decoder, FALSE);
2447 if (priv->current_frame == NULL)
2448 priv->current_frame = gst_video_decoder_new_frame (decoder);
2450 if (!priv->packetized)
2451 gst_video_decoder_add_buffer_info (decoder, buf);
2453 priv->input_offset += gst_buffer_get_size (buf);
2455 if (priv->packetized) {
2456 GstVideoCodecFrame *frame;
2457 gboolean was_keyframe = FALSE;
2459 frame = priv->current_frame;
2461 frame->abidata.ABI.num_subframes++;
2462 if (gst_video_decoder_get_subframe_mode (decoder)) {
2463 /* End the frame if the marker flag is set */
2464 if (!GST_BUFFER_FLAG_IS_SET (buf, GST_VIDEO_BUFFER_FLAG_MARKER)
2465 && (decoder->input_segment.rate > 0.0))
2466 priv->current_frame = gst_video_codec_frame_ref (frame);
2468 priv->current_frame = NULL;
2470 priv->current_frame = frame;
2473 if (!GST_BUFFER_FLAG_IS_SET (buf, GST_BUFFER_FLAG_DELTA_UNIT)) {
2474 was_keyframe = TRUE;
2475 GST_DEBUG_OBJECT (decoder, "Marking current_frame as sync point");
2476 GST_VIDEO_CODEC_FRAME_SET_SYNC_POINT (frame);
2479 if (frame->input_buffer) {
2480 gst_video_decoder_copy_metas (decoder, frame, frame->input_buffer, buf);
2481 gst_buffer_unref (frame->input_buffer);
2483 frame->input_buffer = buf;
2485 if (decoder->input_segment.rate < 0.0) {
2486 priv->parse_gather = g_list_prepend (priv->parse_gather, frame);
2487 priv->current_frame = NULL;
2489 ret = gst_video_decoder_decode_frame (decoder, frame);
2490 if (!gst_video_decoder_get_subframe_mode (decoder))
2491 priv->current_frame = NULL;
2493 /* If in trick mode and it was a keyframe, drain decoder to avoid extra
2494 * latency. Only do this for forwards playback as reverse playback handles
2495 * draining on keyframes in flush_parse(), and would otherwise call back
2496 * from drain_out() to here causing an infinite loop.
2497 * Also this function is only called for reverse playback to gather frames
2498 * GOP by GOP, and does not do any actual decoding. That would be done by
2500 if (ret == GST_FLOW_OK && was_keyframe && decoder->input_segment.rate > 0.0
2501 && (decoder->input_segment.flags & GST_SEEK_FLAG_TRICKMODE_KEY_UNITS))
2502 ret = gst_video_decoder_drain_out (decoder, FALSE);
2504 gst_adapter_push (priv->input_adapter, buf);
2506 ret = gst_video_decoder_parse_available (decoder, at_eos, TRUE);
2509 if (ret == GST_VIDEO_DECODER_FLOW_NEED_DATA)
2515 static GstFlowReturn
2516 gst_video_decoder_flush_decode (GstVideoDecoder * dec)
2518 GstVideoDecoderPrivate *priv = dec->priv;
2519 GstFlowReturn res = GST_FLOW_OK;
2521 GstVideoCodecFrame *current_frame = NULL;
2522 gboolean last_subframe;
2523 GST_DEBUG_OBJECT (dec, "flushing buffers to decode");
2525 walk = priv->decode;
2528 GstVideoCodecFrame *frame = (GstVideoCodecFrame *) (walk->data);
2529 last_subframe = TRUE;
2530 /* In subframe mode, we need to get rid of intermediary frames
2531 * created during the buffer gather stage. That's why that we keep a current
2532 * frame as the main frame and drop all the frame afterwhile until the end
2533 * of the subframes batch.
2535 if (gst_video_decoder_get_subframe_mode (dec)) {
2536 if (current_frame == NULL) {
2537 current_frame = gst_video_codec_frame_ref (frame);
2539 if (current_frame->input_buffer) {
2540 gst_video_decoder_copy_metas (dec, current_frame,
2541 current_frame->input_buffer, current_frame->output_buffer);
2542 gst_buffer_unref (current_frame->input_buffer);
2544 current_frame->input_buffer = gst_buffer_ref (frame->input_buffer);
2545 gst_video_codec_frame_unref (frame);
2547 last_subframe = GST_BUFFER_FLAG_IS_SET (current_frame->input_buffer,
2548 GST_VIDEO_BUFFER_FLAG_MARKER);
2550 current_frame = frame;
2553 GST_DEBUG_OBJECT (dec, "decoding frame %p buffer %p, PTS %" GST_TIME_FORMAT
2554 ", DTS %" GST_TIME_FORMAT, frame, frame->input_buffer,
2555 GST_TIME_ARGS (GST_BUFFER_PTS (frame->input_buffer)),
2556 GST_TIME_ARGS (GST_BUFFER_DTS (frame->input_buffer)));
2560 priv->decode = g_list_delete_link (priv->decode, walk);
2562 /* decode buffer, resulting data prepended to queue */
2563 res = gst_video_decoder_decode_frame (dec, current_frame);
2564 if (res != GST_FLOW_OK)
2566 if (!gst_video_decoder_get_subframe_mode (dec)
2568 current_frame = NULL;
2575 /* gst_video_decoder_flush_parse is called from the
2576 * chain_reverse() function when a buffer containing
2577 * a DISCONT - indicating that reverse playback
2578 * looped back to the next data block, and therefore
2579 * all available data should be fed through the
2580 * decoder and frames gathered for reversed output
2582 static GstFlowReturn
2583 gst_video_decoder_flush_parse (GstVideoDecoder * dec, gboolean at_eos)
2585 GstVideoDecoderPrivate *priv = dec->priv;
2586 GstFlowReturn res = GST_FLOW_OK;
2588 GstVideoDecoderClass *decoder_class;
2590 decoder_class = GST_VIDEO_DECODER_GET_CLASS (dec);
2592 GST_DEBUG_OBJECT (dec, "flushing buffers to parsing");
2594 /* Reverse the gather list, and prepend it to the parse list,
2595 * then flush to parse whatever we can */
2596 priv->gather = g_list_reverse (priv->gather);
2597 priv->parse = g_list_concat (priv->gather, priv->parse);
2598 priv->gather = NULL;
2600 /* clear buffer and decoder state */
2601 gst_video_decoder_flush (dec, FALSE);
2605 GstBuffer *buf = GST_BUFFER_CAST (walk->data);
2606 GList *next = walk->next;
2608 GST_DEBUG_OBJECT (dec, "parsing buffer %p, PTS %" GST_TIME_FORMAT
2609 ", DTS %" GST_TIME_FORMAT " flags %x", buf,
2610 GST_TIME_ARGS (GST_BUFFER_PTS (buf)),
2611 GST_TIME_ARGS (GST_BUFFER_DTS (buf)), GST_BUFFER_FLAGS (buf));
2613 /* parse buffer, resulting frames prepended to parse_gather queue */
2614 gst_buffer_ref (buf);
2615 res = gst_video_decoder_chain_forward (dec, buf, at_eos);
2617 /* if we generated output, we can discard the buffer, else we
2618 * keep it in the queue */
2619 if (priv->parse_gather) {
2620 GST_DEBUG_OBJECT (dec, "parsed buffer to %p", priv->parse_gather->data);
2621 priv->parse = g_list_delete_link (priv->parse, walk);
2622 gst_buffer_unref (buf);
2624 GST_DEBUG_OBJECT (dec, "buffer did not decode, keeping");
2629 walk = priv->parse_gather;
2631 GstVideoCodecFrame *frame = (GstVideoCodecFrame *) (walk->data);
2634 /* this is reverse playback, check if we need to apply some segment
2635 * to the output before decoding, as during decoding the segment.rate
2636 * must be used to determine if a buffer should be pushed or added to
2637 * the output list for reverse pushing.
2639 * The new segment is not immediately pushed here because we must
2640 * wait for negotiation to happen before it can be pushed to avoid
2641 * pushing a segment before caps event. Negotiation only happens
2642 * when finish_frame is called.
2644 for (walk2 = frame->events; walk2;) {
2646 GstEvent *event = walk2->data;
2648 walk2 = g_list_next (walk2);
2649 if (GST_EVENT_TYPE (event) <= GST_EVENT_SEGMENT) {
2651 if (GST_EVENT_TYPE (event) == GST_EVENT_SEGMENT) {
2654 GST_DEBUG_OBJECT (dec, "Segment at frame %p %" GST_TIME_FORMAT,
2655 frame, GST_TIME_ARGS (GST_BUFFER_PTS (frame->input_buffer)));
2656 gst_event_copy_segment (event, &segment);
2657 if (segment.format == GST_FORMAT_TIME) {
2658 dec->output_segment = segment;
2659 dec->priv->in_out_segment_sync =
2660 gst_segment_is_equal (&dec->input_segment, &segment);
2663 dec->priv->pending_events =
2664 g_list_append (dec->priv->pending_events, event);
2665 frame->events = g_list_delete_link (frame->events, cur);
2672 /* now we can process frames. Start by moving each frame from the parse_gather
2673 * to the decode list, reverse the order as we go, and stopping when/if we
2674 * copy a keyframe. */
2675 GST_DEBUG_OBJECT (dec, "checking parsed frames for a keyframe to decode");
2676 walk = priv->parse_gather;
2678 GstVideoCodecFrame *frame = (GstVideoCodecFrame *) (walk->data);
2680 /* remove from the gather list */
2681 priv->parse_gather = g_list_remove_link (priv->parse_gather, walk);
2683 /* move it to the front of the decode queue */
2684 priv->decode = g_list_concat (walk, priv->decode);
2686 /* if we copied a keyframe, flush and decode the decode queue */
2687 if (GST_VIDEO_CODEC_FRAME_IS_SYNC_POINT (frame)) {
2688 GST_DEBUG_OBJECT (dec, "found keyframe %p with PTS %" GST_TIME_FORMAT
2689 ", DTS %" GST_TIME_FORMAT, frame,
2690 GST_TIME_ARGS (GST_BUFFER_PTS (frame->input_buffer)),
2691 GST_TIME_ARGS (GST_BUFFER_DTS (frame->input_buffer)));
2692 res = gst_video_decoder_flush_decode (dec);
2693 if (res != GST_FLOW_OK)
2696 /* We need to tell the subclass to drain now.
2697 * We prefer the drain vfunc, but for backward-compat
2698 * we use a finish() vfunc if drain isn't implemented */
2699 if (decoder_class->drain) {
2700 GST_DEBUG_OBJECT (dec, "Draining");
2701 res = decoder_class->drain (dec);
2702 } else if (decoder_class->finish) {
2703 GST_FIXME_OBJECT (dec, "Sub-class should implement drain(). "
2704 "Calling finish() for backwards-compat");
2705 res = decoder_class->finish (dec);
2708 if (res != GST_FLOW_OK)
2711 /* now send queued data downstream */
2712 walk = priv->output_queued;
2714 GstBuffer *buf = GST_BUFFER_CAST (walk->data);
2716 priv->output_queued =
2717 g_list_delete_link (priv->output_queued, priv->output_queued);
2719 if (G_LIKELY (res == GST_FLOW_OK)) {
2720 /* avoid stray DISCONT from forward processing,
2721 * which have no meaning in reverse pushing */
2722 GST_BUFFER_FLAG_UNSET (buf, GST_BUFFER_FLAG_DISCONT);
2724 /* Last chance to calculate a timestamp as we loop backwards
2725 * through the list */
2726 if (GST_BUFFER_TIMESTAMP (buf) != GST_CLOCK_TIME_NONE)
2727 priv->last_timestamp_out = GST_BUFFER_TIMESTAMP (buf);
2728 else if (priv->last_timestamp_out != GST_CLOCK_TIME_NONE &&
2729 GST_BUFFER_DURATION (buf) != GST_CLOCK_TIME_NONE) {
2730 GST_BUFFER_TIMESTAMP (buf) =
2731 priv->last_timestamp_out - GST_BUFFER_DURATION (buf);
2732 priv->last_timestamp_out = GST_BUFFER_TIMESTAMP (buf);
2733 GST_LOG_OBJECT (dec,
2734 "Calculated TS %" GST_TIME_FORMAT " working backwards",
2735 GST_TIME_ARGS (priv->last_timestamp_out));
2738 res = gst_video_decoder_clip_and_push_buf (dec, buf);
2740 gst_buffer_unref (buf);
2743 walk = priv->output_queued;
2746 /* clear buffer and decoder state again
2747 * before moving to the previous keyframe */
2748 gst_video_decoder_flush (dec, FALSE);
2751 walk = priv->parse_gather;
2758 static GstFlowReturn
2759 gst_video_decoder_chain_reverse (GstVideoDecoder * dec, GstBuffer * buf)
2761 GstVideoDecoderPrivate *priv = dec->priv;
2762 GstFlowReturn result = GST_FLOW_OK;
2764 /* if we have a discont, move buffers to the decode list */
2765 if (!buf || GST_BUFFER_IS_DISCONT (buf)) {
2766 GST_DEBUG_OBJECT (dec, "received discont");
2768 /* parse and decode stuff in the gather and parse queues */
2769 result = gst_video_decoder_flush_parse (dec, FALSE);
2772 if (G_LIKELY (buf)) {
2773 GST_DEBUG_OBJECT (dec, "gathering buffer %p of size %" G_GSIZE_FORMAT ", "
2774 "PTS %" GST_TIME_FORMAT ", DTS %" GST_TIME_FORMAT ", dur %"
2775 GST_TIME_FORMAT, buf, gst_buffer_get_size (buf),
2776 GST_TIME_ARGS (GST_BUFFER_PTS (buf)),
2777 GST_TIME_ARGS (GST_BUFFER_DTS (buf)),
2778 GST_TIME_ARGS (GST_BUFFER_DURATION (buf)));
2780 /* add buffer to gather queue */
2781 priv->gather = g_list_prepend (priv->gather, buf);
2787 static GstFlowReturn
2788 gst_video_decoder_chain (GstPad * pad, GstObject * parent, GstBuffer * buf)
2790 GstVideoDecoder *decoder;
2791 GstFlowReturn ret = GST_FLOW_OK;
2793 decoder = GST_VIDEO_DECODER (parent);
2795 if (G_UNLIKELY (!decoder->priv->input_state && decoder->priv->needs_format))
2796 goto not_negotiated;
2798 GST_LOG_OBJECT (decoder,
2799 "chain PTS %" GST_TIME_FORMAT ", DTS %" GST_TIME_FORMAT " duration %"
2800 GST_TIME_FORMAT " size %" G_GSIZE_FORMAT " flags %x",
2801 GST_TIME_ARGS (GST_BUFFER_PTS (buf)),
2802 GST_TIME_ARGS (GST_BUFFER_DTS (buf)),
2803 GST_TIME_ARGS (GST_BUFFER_DURATION (buf)),
2804 gst_buffer_get_size (buf), GST_BUFFER_FLAGS (buf));
2806 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
2809 * requiring the pad to be negotiated makes it impossible to use
2810 * oggdemux or filesrc ! decoder */
2812 if (decoder->input_segment.format == GST_FORMAT_UNDEFINED) {
2814 GstSegment *segment = &decoder->input_segment;
2816 GST_WARNING_OBJECT (decoder,
2817 "Received buffer without a new-segment. "
2818 "Assuming timestamps start from 0.");
2820 gst_segment_init (segment, GST_FORMAT_TIME);
2822 event = gst_event_new_segment (segment);
2824 decoder->priv->current_frame_events =
2825 g_list_prepend (decoder->priv->current_frame_events, event);
2828 decoder->priv->had_input_data = TRUE;
2830 if (decoder->input_segment.rate > 0.0)
2831 ret = gst_video_decoder_chain_forward (decoder, buf, FALSE);
2833 ret = gst_video_decoder_chain_reverse (decoder, buf);
2835 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
2841 GST_ELEMENT_ERROR (decoder, CORE, NEGOTIATION, (NULL),
2842 ("decoder not initialized"));
2843 gst_buffer_unref (buf);
2844 return GST_FLOW_NOT_NEGOTIATED;
2848 static GstStateChangeReturn
2849 gst_video_decoder_change_state (GstElement * element, GstStateChange transition)
2851 GstVideoDecoder *decoder;
2852 GstVideoDecoderClass *decoder_class;
2853 GstStateChangeReturn ret;
2855 decoder = GST_VIDEO_DECODER (element);
2856 decoder_class = GST_VIDEO_DECODER_GET_CLASS (element);
2858 switch (transition) {
2859 case GST_STATE_CHANGE_NULL_TO_READY:
2860 /* open device/library if needed */
2861 if (decoder_class->open && !decoder_class->open (decoder))
2864 case GST_STATE_CHANGE_READY_TO_PAUSED:
2865 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
2866 gst_video_decoder_reset (decoder, TRUE, TRUE);
2867 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
2869 /* Initialize device/library if needed */
2870 if (decoder_class->start && !decoder_class->start (decoder))
2877 ret = GST_ELEMENT_CLASS (parent_class)->change_state (element, transition);
2879 switch (transition) {
2880 case GST_STATE_CHANGE_PAUSED_TO_READY:{
2881 gboolean stopped = TRUE;
2883 if (decoder_class->stop)
2884 stopped = decoder_class->stop (decoder);
2886 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
2887 gst_video_decoder_reset (decoder, TRUE, TRUE);
2888 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
2895 case GST_STATE_CHANGE_READY_TO_NULL:
2896 /* close device/library if needed */
2897 if (decoder_class->close && !decoder_class->close (decoder))
2909 GST_ELEMENT_ERROR (decoder, LIBRARY, INIT, (NULL),
2910 ("Failed to open decoder"));
2911 return GST_STATE_CHANGE_FAILURE;
2916 GST_ELEMENT_ERROR (decoder, LIBRARY, INIT, (NULL),
2917 ("Failed to start decoder"));
2918 return GST_STATE_CHANGE_FAILURE;
2923 GST_ELEMENT_ERROR (decoder, LIBRARY, INIT, (NULL),
2924 ("Failed to stop decoder"));
2925 return GST_STATE_CHANGE_FAILURE;
2930 GST_ELEMENT_ERROR (decoder, LIBRARY, INIT, (NULL),
2931 ("Failed to close decoder"));
2932 return GST_STATE_CHANGE_FAILURE;
2936 static GstVideoCodecFrame *
2937 gst_video_decoder_new_frame (GstVideoDecoder * decoder)
2939 GstVideoDecoderPrivate *priv = decoder->priv;
2940 GstVideoCodecFrame *frame;
2942 frame = g_slice_new0 (GstVideoCodecFrame);
2944 frame->ref_count = 1;
2946 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
2947 frame->system_frame_number = priv->system_frame_number;
2948 priv->system_frame_number++;
2949 frame->decode_frame_number = priv->decode_frame_number;
2950 priv->decode_frame_number++;
2952 frame->dts = GST_CLOCK_TIME_NONE;
2953 frame->pts = GST_CLOCK_TIME_NONE;
2954 frame->duration = GST_CLOCK_TIME_NONE;
2955 frame->events = priv->current_frame_events;
2956 priv->current_frame_events = NULL;
2958 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
2960 GST_LOG_OBJECT (decoder, "Created new frame %p (sfn:%d)",
2961 frame, frame->system_frame_number);
2967 gst_video_decoder_push_event_list (GstVideoDecoder * decoder, GList * events)
2971 /* events are stored in reverse order */
2972 for (l = g_list_last (events); l; l = g_list_previous (l)) {
2973 GST_LOG_OBJECT (decoder, "pushing %s event", GST_EVENT_TYPE_NAME (l->data));
2974 gst_video_decoder_push_event (decoder, l->data);
2976 g_list_free (events);
2980 gst_video_decoder_prepare_finish_frame (GstVideoDecoder *
2981 decoder, GstVideoCodecFrame * frame, gboolean dropping)
2983 GstVideoDecoderPrivate *priv = decoder->priv;
2984 GList *l, *events = NULL;
2987 #ifndef GST_DISABLE_GST_DEBUG
2988 GST_LOG_OBJECT (decoder, "n %d in %" G_GSIZE_FORMAT " out %" G_GSIZE_FORMAT,
2989 priv->frames.length,
2990 gst_adapter_available (priv->input_adapter),
2991 gst_adapter_available (priv->output_adapter));
2994 sync = GST_VIDEO_CODEC_FRAME_IS_SYNC_POINT (frame);
2996 GST_LOG_OBJECT (decoder,
2997 "finish frame %p (#%d)(sub=#%d) sync:%d PTS:%" GST_TIME_FORMAT " DTS:%"
2999 frame, frame->system_frame_number, frame->abidata.ABI.num_subframes,
3000 sync, GST_TIME_ARGS (frame->pts), GST_TIME_ARGS (frame->dts));
3002 /* Push all pending events that arrived before this frame */
3003 for (l = priv->frames.head; l; l = l->next) {
3004 GstVideoCodecFrame *tmp = l->data;
3007 events = g_list_concat (tmp->events, events);
3015 if (dropping || !decoder->priv->output_state) {
3016 /* Push before the next frame that is not dropped */
3017 decoder->priv->pending_events =
3018 g_list_concat (events, decoder->priv->pending_events);
3020 gst_video_decoder_push_event_list (decoder, decoder->priv->pending_events);
3021 decoder->priv->pending_events = NULL;
3023 gst_video_decoder_push_event_list (decoder, events);
3026 /* Check if the data should not be displayed. For example altref/invisible
3027 * frame in vp8. In this case we should not update the timestamps. */
3028 if (GST_VIDEO_CODEC_FRAME_IS_DECODE_ONLY (frame))
3031 /* If the frame is meant to be output but we don't have an output_buffer
3032 * we have a problem :) */
3033 if (G_UNLIKELY ((frame->output_buffer == NULL) && !dropping))
3034 goto no_output_buffer;
3036 if (GST_CLOCK_TIME_IS_VALID (frame->pts)) {
3037 if (frame->pts != priv->base_timestamp) {
3038 GST_DEBUG_OBJECT (decoder,
3039 "sync timestamp %" GST_TIME_FORMAT " diff %" GST_STIME_FORMAT,
3040 GST_TIME_ARGS (frame->pts),
3041 GST_STIME_ARGS (GST_CLOCK_DIFF (frame->pts,
3042 decoder->output_segment.start)));
3043 priv->base_timestamp = frame->pts;
3044 priv->base_picture_number = frame->decode_frame_number;
3048 if (frame->duration == GST_CLOCK_TIME_NONE) {
3049 frame->duration = gst_video_decoder_get_frame_duration (decoder, frame);
3050 GST_LOG_OBJECT (decoder,
3051 "Guessing duration %" GST_TIME_FORMAT " for frame...",
3052 GST_TIME_ARGS (frame->duration));
3055 /* PTS is expected montone ascending,
3056 * so a good guess is lowest unsent DTS */
3058 GstClockTime min_ts = GST_CLOCK_TIME_NONE;
3059 GstVideoCodecFrame *oframe = NULL;
3060 gboolean seen_none = FALSE;
3062 /* some maintenance regardless */
3063 for (l = priv->frames.head; l; l = l->next) {
3064 GstVideoCodecFrame *tmp = l->data;
3066 if (!GST_CLOCK_TIME_IS_VALID (tmp->abidata.ABI.ts)) {
3071 if (!GST_CLOCK_TIME_IS_VALID (min_ts) || tmp->abidata.ABI.ts < min_ts) {
3072 min_ts = tmp->abidata.ABI.ts;
3076 /* save a ts if needed */
3077 if (oframe && oframe != frame) {
3078 oframe->abidata.ABI.ts = frame->abidata.ABI.ts;
3081 /* and set if needed;
3082 * valid delta means we have reasonable DTS input */
3083 /* also, if we ended up reordered, means this approach is conflicting
3084 * with some sparse existing PTS, and so it does not work out */
3085 if (!priv->reordered_output &&
3086 !GST_CLOCK_TIME_IS_VALID (frame->pts) && !seen_none &&
3087 GST_CLOCK_TIME_IS_VALID (priv->pts_delta)) {
3088 frame->pts = min_ts + priv->pts_delta;
3089 GST_DEBUG_OBJECT (decoder,
3090 "no valid PTS, using oldest DTS %" GST_TIME_FORMAT,
3091 GST_TIME_ARGS (frame->pts));
3094 /* some more maintenance, ts2 holds PTS */
3095 min_ts = GST_CLOCK_TIME_NONE;
3097 for (l = priv->frames.head; l; l = l->next) {
3098 GstVideoCodecFrame *tmp = l->data;
3100 if (!GST_CLOCK_TIME_IS_VALID (tmp->abidata.ABI.ts2)) {
3105 if (!GST_CLOCK_TIME_IS_VALID (min_ts) || tmp->abidata.ABI.ts2 < min_ts) {
3106 min_ts = tmp->abidata.ABI.ts2;
3110 /* save a ts if needed */
3111 if (oframe && oframe != frame) {
3112 oframe->abidata.ABI.ts2 = frame->abidata.ABI.ts2;
3115 /* if we detected reordered output, then PTS are void,
3116 * however those were obtained; bogus input, subclass etc */
3117 if (priv->reordered_output && !seen_none) {
3118 GST_DEBUG_OBJECT (decoder, "invalidating PTS");
3119 frame->pts = GST_CLOCK_TIME_NONE;
3122 if (!GST_CLOCK_TIME_IS_VALID (frame->pts) && !seen_none) {
3123 frame->pts = min_ts;
3124 GST_DEBUG_OBJECT (decoder,
3125 "no valid PTS, using oldest PTS %" GST_TIME_FORMAT,
3126 GST_TIME_ARGS (frame->pts));
3131 if (frame->pts == GST_CLOCK_TIME_NONE) {
3132 /* Last ditch timestamp guess: Just add the duration to the previous
3133 * frame. If it's the first frame, just use the segment start. */
3134 if (frame->duration != GST_CLOCK_TIME_NONE) {
3135 if (GST_CLOCK_TIME_IS_VALID (priv->last_timestamp_out))
3136 frame->pts = priv->last_timestamp_out + frame->duration;
3137 else if (frame->dts != GST_CLOCK_TIME_NONE) {
3138 frame->pts = frame->dts;
3139 GST_LOG_OBJECT (decoder,
3140 "Setting DTS as PTS %" GST_TIME_FORMAT " for frame...",
3141 GST_TIME_ARGS (frame->pts));
3142 } else if (decoder->output_segment.rate > 0.0)
3143 frame->pts = decoder->output_segment.start;
3144 GST_INFO_OBJECT (decoder,
3145 "Guessing PTS=%" GST_TIME_FORMAT " for frame... DTS=%"
3146 GST_TIME_FORMAT, GST_TIME_ARGS (frame->pts),
3147 GST_TIME_ARGS (frame->dts));
3148 } else if (sync && frame->dts != GST_CLOCK_TIME_NONE) {
3149 frame->pts = frame->dts;
3150 GST_LOG_OBJECT (decoder,
3151 "Setting DTS as PTS %" GST_TIME_FORMAT " for frame...",
3152 GST_TIME_ARGS (frame->pts));
3156 if (GST_CLOCK_TIME_IS_VALID (priv->last_timestamp_out)) {
3157 if (frame->pts < priv->last_timestamp_out) {
3158 GST_WARNING_OBJECT (decoder,
3159 "decreasing timestamp (%" GST_TIME_FORMAT " < %"
3160 GST_TIME_FORMAT ")",
3161 GST_TIME_ARGS (frame->pts), GST_TIME_ARGS (priv->last_timestamp_out));
3162 priv->reordered_output = TRUE;
3163 /* make it a bit less weird downstream */
3164 frame->pts = priv->last_timestamp_out;
3168 if (GST_CLOCK_TIME_IS_VALID (frame->pts))
3169 priv->last_timestamp_out = frame->pts;
3176 GST_ERROR_OBJECT (decoder, "No buffer to output !");
3181 * gst_video_decoder_release_frame:
3182 * @dec: a #GstVideoDecoder
3183 * @frame: (transfer full): the #GstVideoCodecFrame to release
3185 * Similar to gst_video_decoder_drop_frame(), but simply releases @frame
3186 * without any processing other than removing it from list of pending frames,
3187 * after which it is considered finished and released.
3192 gst_video_decoder_release_frame (GstVideoDecoder * dec,
3193 GstVideoCodecFrame * frame)
3197 /* unref once from the list */
3198 GST_VIDEO_DECODER_STREAM_LOCK (dec);
3199 link = g_queue_find (&dec->priv->frames, frame);
3201 gst_video_codec_frame_unref (frame);
3202 g_queue_delete_link (&dec->priv->frames, link);
3204 if (frame->events) {
3205 dec->priv->pending_events =
3206 g_list_concat (frame->events, dec->priv->pending_events);
3207 frame->events = NULL;
3209 GST_VIDEO_DECODER_STREAM_UNLOCK (dec);
3211 /* unref because this function takes ownership */
3212 gst_video_codec_frame_unref (frame);
3215 /* called with STREAM_LOCK */
3217 gst_video_decoder_post_qos_drop (GstVideoDecoder * dec, GstClockTime timestamp)
3219 GstClockTime stream_time, jitter, earliest_time, qostime;
3220 GstSegment *segment;
3221 GstMessage *qos_msg;
3223 dec->priv->dropped++;
3225 /* post QoS message */
3226 GST_OBJECT_LOCK (dec);
3227 proportion = dec->priv->proportion;
3228 earliest_time = dec->priv->earliest_time;
3229 GST_OBJECT_UNLOCK (dec);
3231 segment = &dec->output_segment;
3232 if (G_UNLIKELY (segment->format == GST_FORMAT_UNDEFINED))
3233 segment = &dec->input_segment;
3235 gst_segment_to_stream_time (segment, GST_FORMAT_TIME, timestamp);
3236 qostime = gst_segment_to_running_time (segment, GST_FORMAT_TIME, timestamp);
3237 jitter = GST_CLOCK_DIFF (qostime, earliest_time);
3239 gst_message_new_qos (GST_OBJECT_CAST (dec), FALSE, qostime, stream_time,
3240 timestamp, GST_CLOCK_TIME_NONE);
3241 gst_message_set_qos_values (qos_msg, jitter, proportion, 1000000);
3242 gst_message_set_qos_stats (qos_msg, GST_FORMAT_BUFFERS,
3243 dec->priv->processed, dec->priv->dropped);
3244 gst_element_post_message (GST_ELEMENT_CAST (dec), qos_msg);
3248 * gst_video_decoder_drop_frame:
3249 * @dec: a #GstVideoDecoder
3250 * @frame: (transfer full): the #GstVideoCodecFrame to drop
3252 * Similar to gst_video_decoder_finish_frame(), but drops @frame in any
3253 * case and posts a QoS message with the frame's details on the bus.
3254 * In any case, the frame is considered finished and released.
3256 * Returns: a #GstFlowReturn, usually GST_FLOW_OK.
3259 gst_video_decoder_drop_frame (GstVideoDecoder * dec, GstVideoCodecFrame * frame)
3261 GST_LOG_OBJECT (dec, "drop frame %p", frame);
3263 if (gst_video_decoder_get_subframe_mode (dec))
3264 GST_DEBUG_OBJECT (dec, "Drop subframe %d. Must be the last one.",
3265 frame->abidata.ABI.num_subframes);
3267 GST_VIDEO_DECODER_STREAM_LOCK (dec);
3269 gst_video_decoder_prepare_finish_frame (dec, frame, TRUE);
3271 GST_DEBUG_OBJECT (dec, "dropping frame %" GST_TIME_FORMAT,
3272 GST_TIME_ARGS (frame->pts));
3274 gst_video_decoder_post_qos_drop (dec, frame->pts);
3276 /* now free the frame */
3277 gst_video_decoder_release_frame (dec, frame);
3279 GST_VIDEO_DECODER_STREAM_UNLOCK (dec);
3285 * gst_video_decoder_drop_subframe:
3286 * @dec: a #GstVideoDecoder
3287 * @frame: (transfer full): the #GstVideoCodecFrame
3290 * The frame is not considered finished until the whole frame
3291 * is finished or dropped by the subclass.
3293 * Returns: a #GstFlowReturn, usually GST_FLOW_OK.
3298 gst_video_decoder_drop_subframe (GstVideoDecoder * dec,
3299 GstVideoCodecFrame * frame)
3301 g_return_val_if_fail (gst_video_decoder_get_subframe_mode (dec),
3302 GST_FLOW_NOT_SUPPORTED);
3304 GST_LOG_OBJECT (dec, "drop subframe %p num=%d", frame->input_buffer,
3305 gst_video_decoder_get_input_subframe_index (dec, frame));
3307 GST_VIDEO_DECODER_STREAM_LOCK (dec);
3309 gst_video_codec_frame_unref (frame);
3311 GST_VIDEO_DECODER_STREAM_UNLOCK (dec);
3317 gst_video_decoder_transform_meta_default (GstVideoDecoder *
3318 decoder, GstVideoCodecFrame * frame, GstMeta * meta)
3320 const GstMetaInfo *info = meta->info;
3321 const gchar *const *tags;
3322 const gchar *const supported_tags[] = {
3323 GST_META_TAG_VIDEO_STR,
3324 GST_META_TAG_VIDEO_ORIENTATION_STR,
3325 GST_META_TAG_VIDEO_SIZE_STR,
3329 tags = gst_meta_api_type_get_tags (info->api);
3335 if (!g_strv_contains (supported_tags, *tags))
3345 GstVideoDecoder *decoder;
3346 GstVideoCodecFrame *frame;
3351 foreach_metadata (GstBuffer * inbuf, GstMeta ** meta, gpointer user_data)
3353 CopyMetaData *data = user_data;
3354 GstVideoDecoder *decoder = data->decoder;
3355 GstVideoDecoderClass *klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
3356 GstVideoCodecFrame *frame = data->frame;
3357 GstBuffer *buffer = data->buffer;
3358 const GstMetaInfo *info = (*meta)->info;
3359 gboolean do_copy = FALSE;
3361 if (gst_meta_api_type_has_tag (info->api, _gst_meta_tag_memory)) {
3362 /* never call the transform_meta with memory specific metadata */
3363 GST_DEBUG_OBJECT (decoder, "not copying memory specific metadata %s",
3364 g_type_name (info->api));
3366 } else if (klass->transform_meta) {
3367 do_copy = klass->transform_meta (decoder, frame, *meta);
3368 GST_DEBUG_OBJECT (decoder, "transformed metadata %s: copy: %d",
3369 g_type_name (info->api), do_copy);
3372 /* we only copy metadata when the subclass implemented a transform_meta
3373 * function and when it returns %TRUE */
3374 if (do_copy && info->transform_func) {
3375 GstMetaTransformCopy copy_data = { FALSE, 0, -1 };
3376 GST_DEBUG_OBJECT (decoder, "copy metadata %s", g_type_name (info->api));
3377 /* simply copy then */
3379 info->transform_func (buffer, *meta, inbuf, _gst_meta_transform_copy,
3386 gst_video_decoder_copy_metas (GstVideoDecoder * decoder,
3387 GstVideoCodecFrame * frame, GstBuffer * src_buffer, GstBuffer * dest_buffer)
3389 GstVideoDecoderClass *decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
3391 if (decoder_class->transform_meta) {
3392 if (G_LIKELY (frame)) {
3395 data.decoder = decoder;
3397 data.buffer = dest_buffer;
3398 gst_buffer_foreach_meta (src_buffer, foreach_metadata, &data);
3400 GST_WARNING_OBJECT (decoder,
3401 "Can't copy metadata because input frame disappeared");
3407 * gst_video_decoder_finish_frame:
3408 * @decoder: a #GstVideoDecoder
3409 * @frame: (transfer full): a decoded #GstVideoCodecFrame
3411 * @frame should have a valid decoded data buffer, whose metadata fields
3412 * are then appropriately set according to frame data and pushed downstream.
3413 * If no output data is provided, @frame is considered skipped.
3414 * In any case, the frame is considered finished and released.
3416 * After calling this function the output buffer of the frame is to be
3417 * considered read-only. This function will also change the metadata
3420 * Returns: a #GstFlowReturn resulting from sending data downstream
3423 gst_video_decoder_finish_frame (GstVideoDecoder * decoder,
3424 GstVideoCodecFrame * frame)
3426 GstFlowReturn ret = GST_FLOW_OK;
3427 GstVideoDecoderPrivate *priv = decoder->priv;
3428 GstBuffer *output_buffer;
3429 gboolean needs_reconfigure = FALSE;
3431 GST_LOG_OBJECT (decoder, "finish frame %p", frame);
3433 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
3435 needs_reconfigure = gst_pad_check_reconfigure (decoder->srcpad);
3436 if (G_UNLIKELY (priv->output_state_changed || (priv->output_state
3437 && needs_reconfigure))) {
3438 if (!gst_video_decoder_negotiate_unlocked (decoder)) {
3439 gst_pad_mark_reconfigure (decoder->srcpad);
3440 if (GST_PAD_IS_FLUSHING (decoder->srcpad))
3441 ret = GST_FLOW_FLUSHING;
3443 ret = GST_FLOW_NOT_NEGOTIATED;
3448 gst_video_decoder_prepare_finish_frame (decoder, frame, FALSE);
3451 if (priv->tags_changed) {
3452 GstEvent *tags_event;
3454 tags_event = gst_video_decoder_create_merged_tags_event (decoder);
3456 if (tags_event != NULL)
3457 gst_video_decoder_push_event (decoder, tags_event);
3459 priv->tags_changed = FALSE;
3462 /* no buffer data means this frame is skipped */
3463 if (!frame->output_buffer || GST_VIDEO_CODEC_FRAME_IS_DECODE_ONLY (frame)) {
3464 GST_DEBUG_OBJECT (decoder,
3465 "skipping frame %" GST_TIME_FORMAT " because not output was produced",
3466 GST_TIME_ARGS (frame->pts));
3470 /* Mark output as corrupted if the subclass requested so and we're either
3471 * still before the sync point after the request, or we don't even know the
3472 * frame number of the sync point yet (it is 0) */
3473 GST_OBJECT_LOCK (decoder);
3474 if (frame->system_frame_number <= priv->request_sync_point_frame_number
3475 && priv->request_sync_point_frame_number != REQUEST_SYNC_POINT_UNSET) {
3476 if (priv->request_sync_point_flags &
3477 GST_VIDEO_DECODER_REQUEST_SYNC_POINT_CORRUPT_OUTPUT) {
3478 GST_DEBUG_OBJECT (decoder,
3479 "marking frame %" GST_TIME_FORMAT
3480 " as corrupted because it is still before the sync point",
3481 GST_TIME_ARGS (frame->pts));
3482 GST_VIDEO_CODEC_FRAME_FLAG_SET (frame,
3483 GST_VIDEO_CODEC_FRAME_FLAG_CORRUPTED);
3486 /* Reset to -1 to mark it as unset now that we've reached the frame */
3487 priv->request_sync_point_frame_number = REQUEST_SYNC_POINT_UNSET;
3489 GST_OBJECT_UNLOCK (decoder);
3491 if (priv->discard_corrupted_frames
3492 && (GST_VIDEO_CODEC_FRAME_FLAG_IS_SET (frame,
3493 GST_VIDEO_CODEC_FRAME_FLAG_CORRUPTED)
3494 || GST_BUFFER_FLAG_IS_SET (frame->output_buffer,
3495 GST_BUFFER_FLAG_CORRUPTED))) {
3496 GST_DEBUG_OBJECT (decoder,
3497 "skipping frame %" GST_TIME_FORMAT " because it is corrupted",
3498 GST_TIME_ARGS (frame->pts));
3502 /* We need a writable buffer for the metadata changes below */
3503 output_buffer = frame->output_buffer =
3504 gst_buffer_make_writable (frame->output_buffer);
3506 GST_BUFFER_FLAG_UNSET (output_buffer, GST_BUFFER_FLAG_DELTA_UNIT);
3508 GST_BUFFER_PTS (output_buffer) = frame->pts;
3509 GST_BUFFER_DTS (output_buffer) = GST_CLOCK_TIME_NONE;
3510 GST_BUFFER_DURATION (output_buffer) = frame->duration;
3512 GST_BUFFER_OFFSET (output_buffer) = GST_BUFFER_OFFSET_NONE;
3513 GST_BUFFER_OFFSET_END (output_buffer) = GST_BUFFER_OFFSET_NONE;
3515 if (priv->discont) {
3516 GST_BUFFER_FLAG_SET (output_buffer, GST_BUFFER_FLAG_DISCONT);
3519 if (GST_VIDEO_CODEC_FRAME_FLAG_IS_SET (frame,
3520 GST_VIDEO_CODEC_FRAME_FLAG_CORRUPTED)) {
3521 GST_DEBUG_OBJECT (decoder,
3522 "marking frame %" GST_TIME_FORMAT " as corrupted",
3523 GST_TIME_ARGS (frame->pts));
3524 GST_BUFFER_FLAG_SET (output_buffer, GST_BUFFER_FLAG_CORRUPTED);
3527 gst_video_decoder_copy_metas (decoder, frame, frame->input_buffer,
3528 frame->output_buffer);
3530 /* Get an additional ref to the buffer, which is going to be pushed
3531 * downstream, the original ref is owned by the frame
3533 output_buffer = gst_buffer_ref (output_buffer);
3535 /* Release frame so the buffer is writable when we push it downstream
3536 * if possible, i.e. if the subclass does not hold additional references
3539 gst_video_decoder_release_frame (decoder, frame);
3542 if (decoder->output_segment.rate < 0.0
3543 && !(decoder->output_segment.flags & GST_SEEK_FLAG_TRICKMODE_KEY_UNITS)) {
3544 GST_LOG_OBJECT (decoder, "queued frame");
3545 priv->output_queued = g_list_prepend (priv->output_queued, output_buffer);
3547 ret = gst_video_decoder_clip_and_push_buf (decoder, output_buffer);
3552 gst_video_decoder_release_frame (decoder, frame);
3553 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
3558 * gst_video_decoder_finish_subframe:
3559 * @decoder: a #GstVideoDecoder
3560 * @frame: (transfer full): the #GstVideoCodecFrame
3562 * Indicate that a subframe has been finished to be decoded
3563 * by the subclass. This method should be called for all subframes
3564 * except the last subframe where @gst_video_decoder_finish_frame
3565 * should be called instead.
3567 * Returns: a #GstFlowReturn, usually GST_FLOW_OK.
3572 gst_video_decoder_finish_subframe (GstVideoDecoder * decoder,
3573 GstVideoCodecFrame * frame)
3575 g_return_val_if_fail (gst_video_decoder_get_subframe_mode (decoder),
3576 GST_FLOW_NOT_SUPPORTED);
3578 GST_LOG_OBJECT (decoder, "finish subframe %p num=%d", frame->input_buffer,
3579 gst_video_decoder_get_input_subframe_index (decoder, frame));
3581 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
3582 frame->abidata.ABI.subframes_processed++;
3583 gst_video_codec_frame_unref (frame);
3585 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
3590 /* With stream lock, takes the frame reference */
3591 static GstFlowReturn
3592 gst_video_decoder_clip_and_push_buf (GstVideoDecoder * decoder, GstBuffer * buf)
3594 GstFlowReturn ret = GST_FLOW_OK;
3595 GstVideoDecoderPrivate *priv = decoder->priv;
3596 guint64 start, stop;
3597 guint64 cstart, cstop;
3598 GstSegment *segment;
3599 GstClockTime duration;
3601 /* Check for clipping */
3602 start = GST_BUFFER_PTS (buf);
3603 duration = GST_BUFFER_DURATION (buf);
3605 /* store that we have valid decoded data */
3606 priv->had_output_data = TRUE;
3608 stop = GST_CLOCK_TIME_NONE;
3610 if (GST_CLOCK_TIME_IS_VALID (start) && GST_CLOCK_TIME_IS_VALID (duration)) {
3611 stop = start + duration;
3612 } else if (GST_CLOCK_TIME_IS_VALID (start)
3613 && !GST_CLOCK_TIME_IS_VALID (duration)) {
3614 /* If we don't clip away buffers that far before the segment we
3615 * can cause the pipeline to lockup. This can happen if audio is
3616 * properly clipped, and thus the audio sink does not preroll yet
3617 * but the video sink prerolls because we already outputted a
3618 * buffer here... and then queues run full.
3620 * In the worst case we will clip one buffer too many here now if no
3621 * framerate is given, no buffer duration is given and the actual
3622 * framerate is lower than 25fps */
3623 stop = start + 40 * GST_MSECOND;
3626 segment = &decoder->output_segment;
3627 if (gst_segment_clip (segment, GST_FORMAT_TIME, start, stop, &cstart, &cstop)) {
3628 GST_BUFFER_PTS (buf) = cstart;
3630 if (stop != GST_CLOCK_TIME_NONE && GST_CLOCK_TIME_IS_VALID (duration))
3631 GST_BUFFER_DURATION (buf) = cstop - cstart;
3633 GST_LOG_OBJECT (decoder,
3634 "accepting buffer inside segment: %" GST_TIME_FORMAT " %"
3635 GST_TIME_FORMAT " seg %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT
3636 " time %" GST_TIME_FORMAT,
3637 GST_TIME_ARGS (cstart),
3638 GST_TIME_ARGS (cstop),
3639 GST_TIME_ARGS (segment->start), GST_TIME_ARGS (segment->stop),
3640 GST_TIME_ARGS (segment->time));
3642 GST_LOG_OBJECT (decoder,
3643 "dropping buffer outside segment: %" GST_TIME_FORMAT
3644 " %" GST_TIME_FORMAT
3645 " seg %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT
3646 " time %" GST_TIME_FORMAT,
3647 GST_TIME_ARGS (start), GST_TIME_ARGS (stop),
3648 GST_TIME_ARGS (segment->start),
3649 GST_TIME_ARGS (segment->stop), GST_TIME_ARGS (segment->time));
3650 /* only check and return EOS if upstream still
3651 * in the same segment and interested as such */
3652 if (decoder->priv->in_out_segment_sync) {
3653 if (segment->rate >= 0) {
3654 if (GST_BUFFER_PTS (buf) >= segment->stop)
3656 } else if (GST_BUFFER_PTS (buf) < segment->start) {
3660 gst_buffer_unref (buf);
3664 /* Is buffer too late (QoS) ? */
3665 if (priv->do_qos && GST_CLOCK_TIME_IS_VALID (priv->earliest_time)
3666 && GST_CLOCK_TIME_IS_VALID (cstart)) {
3667 GstClockTime deadline =
3668 gst_segment_to_running_time (segment, GST_FORMAT_TIME, cstart);
3669 if (GST_CLOCK_TIME_IS_VALID (deadline) && deadline < priv->earliest_time) {
3670 GST_WARNING_OBJECT (decoder,
3671 "Dropping frame due to QoS. start:%" GST_TIME_FORMAT " deadline:%"
3672 GST_TIME_FORMAT " earliest_time:%" GST_TIME_FORMAT,
3673 GST_TIME_ARGS (start), GST_TIME_ARGS (deadline),
3674 GST_TIME_ARGS (priv->earliest_time));
3675 gst_video_decoder_post_qos_drop (decoder, cstart);
3676 gst_buffer_unref (buf);
3677 priv->discont = TRUE;
3682 /* Set DISCONT flag here ! */
3684 if (priv->discont) {
3685 GST_DEBUG_OBJECT (decoder, "Setting discont on output buffer");
3686 GST_BUFFER_FLAG_SET (buf, GST_BUFFER_FLAG_DISCONT);
3687 priv->discont = FALSE;
3690 /* update rate estimate */
3691 GST_OBJECT_LOCK (decoder);
3692 priv->bytes_out += gst_buffer_get_size (buf);
3693 if (GST_CLOCK_TIME_IS_VALID (duration)) {
3694 priv->time += duration;
3696 /* FIXME : Use difference between current and previous outgoing
3697 * timestamp, and relate to difference between current and previous
3699 /* better none than nothing valid */
3700 priv->time = GST_CLOCK_TIME_NONE;
3702 GST_OBJECT_UNLOCK (decoder);
3704 GST_DEBUG_OBJECT (decoder, "pushing buffer %p of size %" G_GSIZE_FORMAT ", "
3705 "PTS %" GST_TIME_FORMAT ", dur %" GST_TIME_FORMAT, buf,
3706 gst_buffer_get_size (buf),
3707 GST_TIME_ARGS (GST_BUFFER_PTS (buf)),
3708 GST_TIME_ARGS (GST_BUFFER_DURATION (buf)));
3710 /* we got data, so note things are looking up again, reduce
3711 * the error count, if there is one */
3712 if (G_UNLIKELY (priv->error_count))
3713 priv->error_count = 0;
3715 #ifndef GST_DISABLE_DEBUG
3716 if (G_UNLIKELY (priv->last_reset_time != GST_CLOCK_TIME_NONE)) {
3717 GstClockTime elapsed = gst_util_get_timestamp () - priv->last_reset_time;
3719 /* First buffer since reset, report how long we took */
3720 GST_INFO_OBJECT (decoder, "First buffer since flush took %" GST_TIME_FORMAT
3721 " to produce", GST_TIME_ARGS (elapsed));
3722 priv->last_reset_time = GST_CLOCK_TIME_NONE;
3726 /* release STREAM_LOCK not to block upstream
3727 * while pushing buffer downstream */
3728 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
3729 ret = gst_pad_push (decoder->srcpad, buf);
3730 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
3737 * gst_video_decoder_add_to_frame:
3738 * @decoder: a #GstVideoDecoder
3739 * @n_bytes: the number of bytes to add
3741 * Removes next @n_bytes of input data and adds it to currently parsed frame.
3744 gst_video_decoder_add_to_frame (GstVideoDecoder * decoder, int n_bytes)
3746 GstVideoDecoderPrivate *priv = decoder->priv;
3749 GST_LOG_OBJECT (decoder, "add %d bytes to frame", n_bytes);
3754 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
3755 if (gst_adapter_available (priv->output_adapter) == 0) {
3756 priv->frame_offset =
3757 priv->input_offset - gst_adapter_available (priv->input_adapter);
3759 buf = gst_adapter_take_buffer (priv->input_adapter, n_bytes);
3761 gst_adapter_push (priv->output_adapter, buf);
3762 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
3766 * gst_video_decoder_get_pending_frame_size:
3767 * @decoder: a #GstVideoDecoder
3769 * Returns the number of bytes previously added to the current frame
3770 * by calling gst_video_decoder_add_to_frame().
3772 * Returns: The number of bytes pending for the current frame
3777 gst_video_decoder_get_pending_frame_size (GstVideoDecoder * decoder)
3779 GstVideoDecoderPrivate *priv = decoder->priv;
3782 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
3783 ret = gst_adapter_available (priv->output_adapter);
3784 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
3786 GST_LOG_OBJECT (decoder, "Current pending frame has %" G_GSIZE_FORMAT "bytes",
3793 gst_video_decoder_get_frame_duration (GstVideoDecoder * decoder,
3794 GstVideoCodecFrame * frame)
3796 GstVideoCodecState *state = decoder->priv->output_state;
3798 /* it's possible that we don't have a state yet when we are dropping the
3799 * initial buffers */
3801 return GST_CLOCK_TIME_NONE;
3803 if (state->info.fps_d == 0 || state->info.fps_n == 0) {
3804 return GST_CLOCK_TIME_NONE;
3807 /* FIXME: For interlaced frames this needs to take into account
3808 * the number of valid fields in the frame
3811 return gst_util_uint64_scale (GST_SECOND, state->info.fps_d,
3816 * gst_video_decoder_have_frame:
3817 * @decoder: a #GstVideoDecoder
3819 * Gathers all data collected for currently parsed frame, gathers corresponding
3820 * metadata and passes it along for further processing, i.e. @handle_frame.
3822 * Returns: a #GstFlowReturn
3825 gst_video_decoder_have_frame (GstVideoDecoder * decoder)
3827 GstVideoDecoderPrivate *priv = decoder->priv;
3830 GstClockTime pts, dts, duration;
3832 GstFlowReturn ret = GST_FLOW_OK;
3834 GST_LOG_OBJECT (decoder, "have_frame at offset %" G_GUINT64_FORMAT,
3835 priv->frame_offset);
3837 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
3839 n_available = gst_adapter_available (priv->output_adapter);
3841 buffer = gst_adapter_take_buffer (priv->output_adapter, n_available);
3843 buffer = gst_buffer_new_and_alloc (0);
3846 if (priv->current_frame->input_buffer) {
3847 gst_video_decoder_copy_metas (decoder, priv->current_frame,
3848 priv->current_frame->input_buffer, buffer);
3849 gst_buffer_unref (priv->current_frame->input_buffer);
3851 priv->current_frame->input_buffer = buffer;
3853 gst_video_decoder_get_buffer_info_at_offset (decoder,
3854 priv->frame_offset, &pts, &dts, &duration, &flags);
3856 GST_BUFFER_PTS (buffer) = pts;
3857 GST_BUFFER_DTS (buffer) = dts;
3858 GST_BUFFER_DURATION (buffer) = duration;
3859 GST_BUFFER_FLAGS (buffer) = flags;
3861 GST_LOG_OBJECT (decoder, "collected frame size %d, "
3862 "PTS %" GST_TIME_FORMAT ", DTS %" GST_TIME_FORMAT ", dur %"
3863 GST_TIME_FORMAT, n_available, GST_TIME_ARGS (pts), GST_TIME_ARGS (dts),
3864 GST_TIME_ARGS (duration));
3866 if (!GST_BUFFER_FLAG_IS_SET (buffer, GST_BUFFER_FLAG_DELTA_UNIT)) {
3867 GST_DEBUG_OBJECT (decoder, "Marking as sync point");
3868 GST_VIDEO_CODEC_FRAME_SET_SYNC_POINT (priv->current_frame);
3871 if (GST_BUFFER_FLAG_IS_SET (buffer, GST_BUFFER_FLAG_CORRUPTED)) {
3872 GST_DEBUG_OBJECT (decoder, "Marking as corrupted");
3873 GST_VIDEO_CODEC_FRAME_FLAG_SET (priv->current_frame,
3874 GST_VIDEO_CODEC_FRAME_FLAG_CORRUPTED);
3877 /* In reverse playback, just capture and queue frames for later processing */
3878 if (decoder->input_segment.rate < 0.0) {
3879 priv->parse_gather =
3880 g_list_prepend (priv->parse_gather, priv->current_frame);
3881 priv->current_frame = NULL;
3883 GstVideoCodecFrame *frame = priv->current_frame;
3884 frame->abidata.ABI.num_subframes++;
3885 /* In subframe mode, we keep a ref for ourselves
3886 * as this frame will be kept during the data collection
3887 * in parsed mode. The frame reference will be released by
3888 * finish_(sub)frame or drop_(sub)frame.*/
3889 if (gst_video_decoder_get_subframe_mode (decoder))
3890 gst_video_codec_frame_ref (priv->current_frame);
3892 priv->current_frame = NULL;
3894 /* Decode the frame, which gives away our ref */
3895 ret = gst_video_decoder_decode_frame (decoder, frame);
3898 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
3903 /* Pass the frame in priv->current_frame through the
3904 * handle_frame() callback for decoding and passing to gvd_finish_frame(),
3905 * or dropping by passing to gvd_drop_frame() */
3906 static GstFlowReturn
3907 gst_video_decoder_decode_frame (GstVideoDecoder * decoder,
3908 GstVideoCodecFrame * frame)
3910 GstVideoDecoderPrivate *priv = decoder->priv;
3911 GstVideoDecoderClass *decoder_class;
3912 GstFlowReturn ret = GST_FLOW_OK;
3914 decoder_class = GST_VIDEO_DECODER_GET_CLASS (decoder);
3916 /* FIXME : This should only have to be checked once (either the subclass has an
3917 * implementation, or it doesn't) */
3918 g_return_val_if_fail (decoder_class->handle_frame != NULL, GST_FLOW_ERROR);
3919 g_return_val_if_fail (frame != NULL, GST_FLOW_ERROR);
3921 frame->pts = GST_BUFFER_PTS (frame->input_buffer);
3922 frame->dts = GST_BUFFER_DTS (frame->input_buffer);
3923 frame->duration = GST_BUFFER_DURATION (frame->input_buffer);
3925 gst_segment_to_running_time (&decoder->input_segment, GST_FORMAT_TIME,
3928 /* For keyframes, PTS = DTS + constant_offset, usually 0 to 3 frame
3930 /* FIXME upstream can be quite wrong about the keyframe aspect,
3931 * so we could be going off here as well,
3932 * maybe let subclass decide if it really is/was a keyframe */
3933 if (GST_VIDEO_CODEC_FRAME_IS_SYNC_POINT (frame)) {
3934 priv->distance_from_sync = 0;
3936 GST_OBJECT_LOCK (decoder);
3937 priv->request_sync_point_flags &=
3938 ~GST_VIDEO_DECODER_REQUEST_SYNC_POINT_DISCARD_INPUT;
3939 if (priv->request_sync_point_frame_number == REQUEST_SYNC_POINT_PENDING)
3940 priv->request_sync_point_frame_number = frame->system_frame_number;
3941 GST_OBJECT_UNLOCK (decoder);
3943 if (GST_CLOCK_TIME_IS_VALID (frame->pts)
3944 && GST_CLOCK_TIME_IS_VALID (frame->dts)) {
3945 /* just in case they are not equal as might ideally be,
3946 * e.g. quicktime has a (positive) delta approach */
3947 priv->pts_delta = frame->pts - frame->dts;
3948 GST_DEBUG_OBJECT (decoder, "PTS delta %d ms",
3949 (gint) (priv->pts_delta / GST_MSECOND));
3952 if (priv->distance_from_sync == -1 && priv->automatic_request_sync_points) {
3953 GST_DEBUG_OBJECT (decoder,
3954 "Didn't receive a keyframe yet, requesting sync point");
3955 gst_video_decoder_request_sync_point (decoder, frame,
3956 priv->automatic_request_sync_point_flags);
3959 GST_OBJECT_LOCK (decoder);
3960 if ((priv->needs_sync_point && priv->distance_from_sync == -1)
3961 || (priv->request_sync_point_flags &
3962 GST_VIDEO_DECODER_REQUEST_SYNC_POINT_DISCARD_INPUT)) {
3963 GST_WARNING_OBJECT (decoder,
3964 "Subclass requires a sync point but we didn't receive one yet, discarding input");
3965 GST_OBJECT_UNLOCK (decoder);
3966 if (priv->automatic_request_sync_points) {
3967 gst_video_decoder_request_sync_point (decoder, frame,
3968 priv->automatic_request_sync_point_flags);
3970 gst_video_decoder_release_frame (decoder, frame);
3973 GST_OBJECT_UNLOCK (decoder);
3975 priv->distance_from_sync++;
3978 frame->distance_from_sync = priv->distance_from_sync;
3980 if (frame->abidata.ABI.num_subframes == 1) {
3981 frame->abidata.ABI.ts = frame->dts;
3982 frame->abidata.ABI.ts2 = frame->pts;
3985 GST_LOG_OBJECT (decoder,
3986 "frame %p PTS %" GST_TIME_FORMAT ", DTS %" GST_TIME_FORMAT ", dist %d",
3987 frame, GST_TIME_ARGS (frame->pts), GST_TIME_ARGS (frame->dts),
3988 frame->distance_from_sync);
3989 /* FIXME: suboptimal way to add a unique frame to the list, in case of subframe mode. */
3990 if (!g_queue_find (&priv->frames, frame)) {
3991 g_queue_push_tail (&priv->frames, gst_video_codec_frame_ref (frame));
3993 GST_LOG_OBJECT (decoder,
3994 "Do not add an existing frame used to decode subframes");
3997 if (priv->frames.length > 10) {
3998 GST_DEBUG_OBJECT (decoder, "decoder frame list getting long: %d frames,"
3999 "possible internal leaking?", priv->frames.length);
4002 /* do something with frame */
4003 ret = decoder_class->handle_frame (decoder, frame);
4004 if (ret != GST_FLOW_OK)
4005 GST_DEBUG_OBJECT (decoder, "flow error %s", gst_flow_get_name (ret));
4007 /* the frame has either been added to parse_gather or sent to
4008 handle frame so there is no need to unref it */
4014 * gst_video_decoder_get_output_state:
4015 * @decoder: a #GstVideoDecoder
4017 * Get the #GstVideoCodecState currently describing the output stream.
4019 * Returns: (transfer full): #GstVideoCodecState describing format of video data.
4021 GstVideoCodecState *
4022 gst_video_decoder_get_output_state (GstVideoDecoder * decoder)
4024 GstVideoCodecState *state = NULL;
4026 GST_OBJECT_LOCK (decoder);
4027 if (decoder->priv->output_state)
4028 state = gst_video_codec_state_ref (decoder->priv->output_state);
4029 GST_OBJECT_UNLOCK (decoder);
4034 static GstVideoCodecState *
4035 _set_interlaced_output_state (GstVideoDecoder * decoder,
4036 GstVideoFormat fmt, GstVideoInterlaceMode interlace_mode, guint width,
4037 guint height, GstVideoCodecState * reference, gboolean copy_interlace_mode)
4039 GstVideoDecoderPrivate *priv = decoder->priv;
4040 GstVideoCodecState *state;
4042 g_assert ((copy_interlace_mode
4043 && interlace_mode == GST_VIDEO_INTERLACE_MODE_PROGRESSIVE)
4044 || !copy_interlace_mode);
4046 GST_DEBUG_OBJECT (decoder,
4047 "fmt:%d, width:%d, height:%d, interlace-mode: %s, reference:%p", fmt,
4048 width, height, gst_video_interlace_mode_to_string (interlace_mode),
4051 /* Create the new output state */
4053 _new_output_state (fmt, interlace_mode, width, height, reference,
4054 copy_interlace_mode);
4058 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4060 GST_OBJECT_LOCK (decoder);
4061 /* Replace existing output state by new one */
4062 if (priv->output_state)
4063 gst_video_codec_state_unref (priv->output_state);
4064 priv->output_state = gst_video_codec_state_ref (state);
4066 if (priv->output_state != NULL && priv->output_state->info.fps_n > 0) {
4067 priv->qos_frame_duration =
4068 gst_util_uint64_scale (GST_SECOND, priv->output_state->info.fps_d,
4069 priv->output_state->info.fps_n);
4071 priv->qos_frame_duration = 0;
4073 priv->output_state_changed = TRUE;
4074 GST_OBJECT_UNLOCK (decoder);
4076 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4082 * gst_video_decoder_set_output_state:
4083 * @decoder: a #GstVideoDecoder
4084 * @fmt: a #GstVideoFormat
4085 * @width: The width in pixels
4086 * @height: The height in pixels
4087 * @reference: (allow-none) (transfer none): An optional reference #GstVideoCodecState
4089 * Creates a new #GstVideoCodecState with the specified @fmt, @width and @height
4090 * as the output state for the decoder.
4091 * Any previously set output state on @decoder will be replaced by the newly
4094 * If the subclass wishes to copy over existing fields (like pixel aspec ratio,
4095 * or framerate) from an existing #GstVideoCodecState, it can be provided as a
4098 * If the subclass wishes to override some fields from the output state (like
4099 * pixel-aspect-ratio or framerate) it can do so on the returned #GstVideoCodecState.
4101 * The new output state will only take effect (set on pads and buffers) starting
4102 * from the next call to #gst_video_decoder_finish_frame().
4104 * Returns: (transfer full): the newly configured output state.
4106 GstVideoCodecState *
4107 gst_video_decoder_set_output_state (GstVideoDecoder * decoder,
4108 GstVideoFormat fmt, guint width, guint height,
4109 GstVideoCodecState * reference)
4111 return _set_interlaced_output_state (decoder, fmt,
4112 GST_VIDEO_INTERLACE_MODE_PROGRESSIVE, width, height, reference, TRUE);
4116 * gst_video_decoder_set_interlaced_output_state:
4117 * @decoder: a #GstVideoDecoder
4118 * @fmt: a #GstVideoFormat
4119 * @width: The width in pixels
4120 * @height: The height in pixels
4121 * @interlace_mode: A #GstVideoInterlaceMode
4122 * @reference: (allow-none) (transfer none): An optional reference #GstVideoCodecState
4124 * Same as #gst_video_decoder_set_output_state() but also allows you to also set
4125 * the interlacing mode.
4127 * Returns: (transfer full): the newly configured output state.
4131 GstVideoCodecState *
4132 gst_video_decoder_set_interlaced_output_state (GstVideoDecoder * decoder,
4133 GstVideoFormat fmt, GstVideoInterlaceMode interlace_mode, guint width,
4134 guint height, GstVideoCodecState * reference)
4136 return _set_interlaced_output_state (decoder, fmt, interlace_mode, width,
4137 height, reference, FALSE);
4142 * gst_video_decoder_get_oldest_frame:
4143 * @decoder: a #GstVideoDecoder
4145 * Get the oldest pending unfinished #GstVideoCodecFrame
4147 * Returns: (transfer full): oldest pending unfinished #GstVideoCodecFrame.
4149 GstVideoCodecFrame *
4150 gst_video_decoder_get_oldest_frame (GstVideoDecoder * decoder)
4152 GstVideoCodecFrame *frame = NULL;
4154 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4155 if (decoder->priv->frames.head)
4156 frame = gst_video_codec_frame_ref (decoder->priv->frames.head->data);
4157 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4159 return (GstVideoCodecFrame *) frame;
4163 * gst_video_decoder_get_frame:
4164 * @decoder: a #GstVideoDecoder
4165 * @frame_number: system_frame_number of a frame
4167 * Get a pending unfinished #GstVideoCodecFrame
4169 * Returns: (transfer full): pending unfinished #GstVideoCodecFrame identified by @frame_number.
4171 GstVideoCodecFrame *
4172 gst_video_decoder_get_frame (GstVideoDecoder * decoder, int frame_number)
4175 GstVideoCodecFrame *frame = NULL;
4177 GST_DEBUG_OBJECT (decoder, "frame_number : %d", frame_number);
4179 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4180 for (g = decoder->priv->frames.head; g; g = g->next) {
4181 GstVideoCodecFrame *tmp = g->data;
4183 if (tmp->system_frame_number == frame_number) {
4184 frame = gst_video_codec_frame_ref (tmp);
4188 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4194 * gst_video_decoder_get_frames:
4195 * @decoder: a #GstVideoDecoder
4197 * Get all pending unfinished #GstVideoCodecFrame
4199 * Returns: (transfer full) (element-type GstVideoCodecFrame): pending unfinished #GstVideoCodecFrame.
4202 gst_video_decoder_get_frames (GstVideoDecoder * decoder)
4206 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4208 g_list_copy_deep (decoder->priv->frames.head,
4209 (GCopyFunc) gst_video_codec_frame_ref, NULL);
4210 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4216 gst_video_decoder_decide_allocation_default (GstVideoDecoder * decoder,
4219 GstCaps *outcaps = NULL;
4220 GstBufferPool *pool = NULL;
4221 guint size, min, max;
4222 GstAllocator *allocator = NULL;
4223 GstAllocationParams params;
4224 GstStructure *config;
4225 gboolean update_pool, update_allocator;
4228 gst_query_parse_allocation (query, &outcaps, NULL);
4229 gst_video_info_init (&vinfo);
4231 gst_video_info_from_caps (&vinfo, outcaps);
4233 /* we got configuration from our peer or the decide_allocation method,
4235 if (gst_query_get_n_allocation_params (query) > 0) {
4236 /* try the allocator */
4237 gst_query_parse_nth_allocation_param (query, 0, &allocator, ¶ms);
4238 update_allocator = TRUE;
4241 gst_allocation_params_init (¶ms);
4242 update_allocator = FALSE;
4245 if (gst_query_get_n_allocation_pools (query) > 0) {
4246 gst_query_parse_nth_allocation_pool (query, 0, &pool, &size, &min, &max);
4247 size = MAX (size, vinfo.size);
4254 update_pool = FALSE;
4258 /* no pool, we can make our own */
4259 GST_DEBUG_OBJECT (decoder, "no pool, making new pool");
4260 pool = gst_video_buffer_pool_new ();
4264 config = gst_buffer_pool_get_config (pool);
4265 gst_buffer_pool_config_set_params (config, outcaps, size, min, max);
4266 gst_buffer_pool_config_set_allocator (config, allocator, ¶ms);
4268 GST_DEBUG_OBJECT (decoder,
4269 "setting config %" GST_PTR_FORMAT " in pool %" GST_PTR_FORMAT, config,
4271 if (!gst_buffer_pool_set_config (pool, config)) {
4272 config = gst_buffer_pool_get_config (pool);
4274 /* If change are not acceptable, fallback to generic pool */
4275 if (!gst_buffer_pool_config_validate_params (config, outcaps, size, min,
4277 GST_DEBUG_OBJECT (decoder, "unsupported pool, making new pool");
4279 gst_object_unref (pool);
4280 pool = gst_video_buffer_pool_new ();
4281 gst_buffer_pool_config_set_params (config, outcaps, size, min, max);
4282 gst_buffer_pool_config_set_allocator (config, allocator, ¶ms);
4285 if (!gst_buffer_pool_set_config (pool, config))
4289 if (update_allocator)
4290 gst_query_set_nth_allocation_param (query, 0, allocator, ¶ms);
4292 gst_query_add_allocation_param (query, allocator, ¶ms);
4294 gst_object_unref (allocator);
4297 gst_query_set_nth_allocation_pool (query, 0, pool, size, min, max);
4299 gst_query_add_allocation_pool (query, pool, size, min, max);
4302 gst_object_unref (pool);
4308 gst_object_unref (allocator);
4310 gst_object_unref (pool);
4311 GST_ELEMENT_ERROR (decoder, RESOURCE, SETTINGS,
4312 ("Failed to configure the buffer pool"),
4313 ("Configuration is most likely invalid, please report this issue."));
4318 gst_video_decoder_propose_allocation_default (GstVideoDecoder * decoder,
4325 gst_video_decoder_negotiate_pool (GstVideoDecoder * decoder, GstCaps * caps)
4327 GstVideoDecoderClass *klass;
4328 GstQuery *query = NULL;
4329 GstBufferPool *pool = NULL;
4330 GstAllocator *allocator;
4331 GstAllocationParams params;
4332 gboolean ret = TRUE;
4334 klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
4336 query = gst_query_new_allocation (caps, TRUE);
4338 GST_DEBUG_OBJECT (decoder, "do query ALLOCATION");
4340 if (!gst_pad_peer_query (decoder->srcpad, query)) {
4341 GST_DEBUG_OBJECT (decoder, "didn't get downstream ALLOCATION hints");
4344 g_assert (klass->decide_allocation != NULL);
4345 ret = klass->decide_allocation (decoder, query);
4347 GST_DEBUG_OBJECT (decoder, "ALLOCATION (%d) params: %" GST_PTR_FORMAT, ret,
4351 goto no_decide_allocation;
4353 /* we got configuration from our peer or the decide_allocation method,
4355 if (gst_query_get_n_allocation_params (query) > 0) {
4356 gst_query_parse_nth_allocation_param (query, 0, &allocator, ¶ms);
4359 gst_allocation_params_init (¶ms);
4362 if (gst_query_get_n_allocation_pools (query) > 0)
4363 gst_query_parse_nth_allocation_pool (query, 0, &pool, NULL, NULL, NULL);
4366 gst_object_unref (allocator);
4368 goto no_decide_allocation;
4371 if (decoder->priv->allocator)
4372 gst_object_unref (decoder->priv->allocator);
4373 decoder->priv->allocator = allocator;
4374 decoder->priv->params = params;
4376 if (decoder->priv->pool) {
4377 /* do not set the bufferpool to inactive here, it will be done
4378 * on its finalize function. As videodecoder do late renegotiation
4379 * it might happen that some element downstream is already using this
4380 * same bufferpool and deactivating it will make it fail.
4381 * Happens when a downstream element changes from passthrough to
4382 * non-passthrough and gets this same bufferpool to use */
4383 GST_DEBUG_OBJECT (decoder, "unref pool %" GST_PTR_FORMAT,
4384 decoder->priv->pool);
4385 gst_object_unref (decoder->priv->pool);
4387 decoder->priv->pool = pool;
4390 GST_DEBUG_OBJECT (decoder, "activate pool %" GST_PTR_FORMAT, pool);
4391 gst_buffer_pool_set_active (pool, TRUE);
4395 gst_query_unref (query);
4400 no_decide_allocation:
4402 GST_WARNING_OBJECT (decoder, "Subclass failed to decide allocation");
4408 gst_video_decoder_negotiate_default (GstVideoDecoder * decoder)
4410 GstVideoCodecState *state = decoder->priv->output_state;
4411 gboolean ret = TRUE;
4412 GstVideoCodecFrame *frame;
4417 GST_DEBUG_OBJECT (decoder,
4418 "Trying to negotiate the pool with out setting the o/p format");
4419 ret = gst_video_decoder_negotiate_pool (decoder, NULL);
4423 g_return_val_if_fail (GST_VIDEO_INFO_WIDTH (&state->info) != 0, FALSE);
4424 g_return_val_if_fail (GST_VIDEO_INFO_HEIGHT (&state->info) != 0, FALSE);
4426 /* If the base class didn't set any multiview params, assume mono
4428 if (GST_VIDEO_INFO_MULTIVIEW_MODE (&state->info) ==
4429 GST_VIDEO_MULTIVIEW_MODE_NONE) {
4430 GST_VIDEO_INFO_MULTIVIEW_MODE (&state->info) =
4431 GST_VIDEO_MULTIVIEW_MODE_MONO;
4432 GST_VIDEO_INFO_MULTIVIEW_FLAGS (&state->info) =
4433 GST_VIDEO_MULTIVIEW_FLAGS_NONE;
4436 GST_DEBUG_OBJECT (decoder, "output_state par %d/%d fps %d/%d",
4437 state->info.par_n, state->info.par_d,
4438 state->info.fps_n, state->info.fps_d);
4440 if (state->caps == NULL)
4441 state->caps = gst_video_info_to_caps (&state->info);
4443 incaps = gst_pad_get_current_caps (GST_VIDEO_DECODER_SINK_PAD (decoder));
4445 GstStructure *in_struct;
4447 in_struct = gst_caps_get_structure (incaps, 0);
4448 if (gst_structure_has_field (in_struct, "mastering-display-info") ||
4449 gst_structure_has_field (in_struct, "content-light-level")) {
4452 /* prefer upstream information */
4453 state->caps = gst_caps_make_writable (state->caps);
4454 if ((s = gst_structure_get_string (in_struct, "mastering-display-info"))) {
4455 gst_caps_set_simple (state->caps,
4456 "mastering-display-info", G_TYPE_STRING, s, NULL);
4459 if ((s = gst_structure_get_string (in_struct, "content-light-level"))) {
4460 gst_caps_set_simple (state->caps,
4461 "content-light-level", G_TYPE_STRING, s, NULL);
4464 if (gst_structure_has_field (in_struct, "hdr-format")) {
4466 state->caps = gst_caps_make_writable (state->caps);
4468 if ((s = gst_structure_get_string (in_struct, "hdr-format"))) {
4469 gst_caps_set_simple (state->caps, "hdr-format", G_TYPE_STRING, s, NULL);
4473 gst_caps_unref (incaps);
4476 if (state->allocation_caps == NULL)
4477 state->allocation_caps = gst_caps_ref (state->caps);
4479 GST_DEBUG_OBJECT (decoder, "setting caps %" GST_PTR_FORMAT, state->caps);
4481 /* Push all pending pre-caps events of the oldest frame before
4483 frame = decoder->priv->frames.head ? decoder->priv->frames.head->data : NULL;
4484 if (frame || decoder->priv->current_frame_events) {
4488 events = &frame->events;
4490 events = &decoder->priv->current_frame_events;
4493 for (l = g_list_last (*events); l;) {
4494 GstEvent *event = GST_EVENT (l->data);
4497 if (GST_EVENT_TYPE (event) < GST_EVENT_CAPS) {
4498 gst_video_decoder_push_event (decoder, event);
4501 *events = g_list_delete_link (*events, tmp);
4508 prevcaps = gst_pad_get_current_caps (decoder->srcpad);
4509 if (!prevcaps || !gst_caps_is_equal (prevcaps, state->caps)) {
4511 GST_DEBUG_OBJECT (decoder, "decoder src pad has currently NULL caps");
4513 ret = gst_pad_set_caps (decoder->srcpad, state->caps);
4516 GST_DEBUG_OBJECT (decoder,
4517 "current src pad and output state caps are the same");
4520 gst_caps_unref (prevcaps);
4524 decoder->priv->output_state_changed = FALSE;
4525 /* Negotiate pool */
4526 ret = gst_video_decoder_negotiate_pool (decoder, state->allocation_caps);
4533 gst_video_decoder_negotiate_unlocked (GstVideoDecoder * decoder)
4535 GstVideoDecoderClass *klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
4536 gboolean ret = TRUE;
4538 if (G_LIKELY (klass->negotiate))
4539 ret = klass->negotiate (decoder);
4545 * gst_video_decoder_negotiate:
4546 * @decoder: a #GstVideoDecoder
4548 * Negotiate with downstream elements to currently configured #GstVideoCodecState.
4549 * Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if
4552 * Returns: %TRUE if the negotiation succeeded, else %FALSE.
4555 gst_video_decoder_negotiate (GstVideoDecoder * decoder)
4557 GstVideoDecoderClass *klass;
4558 gboolean ret = TRUE;
4560 g_return_val_if_fail (GST_IS_VIDEO_DECODER (decoder), FALSE);
4562 klass = GST_VIDEO_DECODER_GET_CLASS (decoder);
4564 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4565 gst_pad_check_reconfigure (decoder->srcpad);
4566 if (klass->negotiate) {
4567 ret = klass->negotiate (decoder);
4569 gst_pad_mark_reconfigure (decoder->srcpad);
4571 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4577 * gst_video_decoder_allocate_output_buffer:
4578 * @decoder: a #GstVideoDecoder
4580 * Helper function that allocates a buffer to hold a video frame for @decoder's
4581 * current #GstVideoCodecState.
4583 * You should use gst_video_decoder_allocate_output_frame() instead of this
4584 * function, if possible at all.
4586 * Returns: (transfer full): allocated buffer, or NULL if no buffer could be
4587 * allocated (e.g. when downstream is flushing or shutting down)
4590 gst_video_decoder_allocate_output_buffer (GstVideoDecoder * decoder)
4593 GstBuffer *buffer = NULL;
4594 gboolean needs_reconfigure = FALSE;
4596 GST_DEBUG ("alloc src buffer");
4598 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4599 needs_reconfigure = gst_pad_check_reconfigure (decoder->srcpad);
4600 if (G_UNLIKELY (!decoder->priv->output_state
4601 || decoder->priv->output_state_changed || needs_reconfigure)) {
4602 if (!gst_video_decoder_negotiate_unlocked (decoder)) {
4603 if (decoder->priv->output_state) {
4604 GST_DEBUG_OBJECT (decoder, "Failed to negotiate, fallback allocation");
4605 gst_pad_mark_reconfigure (decoder->srcpad);
4608 GST_DEBUG_OBJECT (decoder, "Failed to negotiate, output_buffer=NULL");
4609 goto failed_allocation;
4614 flow = gst_buffer_pool_acquire_buffer (decoder->priv->pool, &buffer, NULL);
4616 if (flow != GST_FLOW_OK) {
4617 GST_INFO_OBJECT (decoder, "couldn't allocate output buffer, flow %s",
4618 gst_flow_get_name (flow));
4619 if (decoder->priv->output_state && decoder->priv->output_state->info.size)
4622 goto failed_allocation;
4624 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4629 GST_INFO_OBJECT (decoder,
4630 "Fallback allocation, creating new buffer which doesn't belongs to any buffer pool");
4632 gst_buffer_new_allocate (NULL, decoder->priv->output_state->info.size,
4636 GST_ERROR_OBJECT (decoder, "Failed to allocate the buffer..");
4637 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4643 * gst_video_decoder_allocate_output_frame:
4644 * @decoder: a #GstVideoDecoder
4645 * @frame: a #GstVideoCodecFrame
4647 * Helper function that allocates a buffer to hold a video frame for @decoder's
4648 * current #GstVideoCodecState. Subclass should already have configured video
4649 * state and set src pad caps.
4651 * The buffer allocated here is owned by the frame and you should only
4652 * keep references to the frame, not the buffer.
4654 * Returns: %GST_FLOW_OK if an output buffer could be allocated
4657 gst_video_decoder_allocate_output_frame (GstVideoDecoder *
4658 decoder, GstVideoCodecFrame * frame)
4660 return gst_video_decoder_allocate_output_frame_with_params (decoder, frame,
4665 * gst_video_decoder_allocate_output_frame_with_params:
4666 * @decoder: a #GstVideoDecoder
4667 * @frame: a #GstVideoCodecFrame
4668 * @params: a #GstBufferPoolAcquireParams
4670 * Same as #gst_video_decoder_allocate_output_frame except it allows passing
4671 * #GstBufferPoolAcquireParams to the sub call gst_buffer_pool_acquire_buffer.
4673 * Returns: %GST_FLOW_OK if an output buffer could be allocated
4678 gst_video_decoder_allocate_output_frame_with_params (GstVideoDecoder *
4679 decoder, GstVideoCodecFrame * frame, GstBufferPoolAcquireParams * params)
4681 GstFlowReturn flow_ret;
4682 GstVideoCodecState *state;
4684 gboolean needs_reconfigure = FALSE;
4686 g_return_val_if_fail (decoder->priv->output_state, GST_FLOW_NOT_NEGOTIATED);
4687 g_return_val_if_fail (frame->output_buffer == NULL, GST_FLOW_ERROR);
4689 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4691 state = decoder->priv->output_state;
4692 if (state == NULL) {
4693 g_warning ("Output state should be set before allocating frame");
4696 num_bytes = GST_VIDEO_INFO_SIZE (&state->info);
4697 if (num_bytes == 0) {
4698 g_warning ("Frame size should not be 0");
4702 needs_reconfigure = gst_pad_check_reconfigure (decoder->srcpad);
4703 if (G_UNLIKELY (decoder->priv->output_state_changed || needs_reconfigure)) {
4704 if (!gst_video_decoder_negotiate_unlocked (decoder)) {
4705 gst_pad_mark_reconfigure (decoder->srcpad);
4706 if (GST_PAD_IS_FLUSHING (decoder->srcpad)) {
4707 GST_DEBUG_OBJECT (decoder,
4708 "Failed to negotiate a pool: pad is flushing");
4710 } else if (!decoder->priv->pool || decoder->priv->output_state_changed) {
4711 GST_DEBUG_OBJECT (decoder,
4712 "Failed to negotiate a pool and no previous pool to reuse");
4715 GST_DEBUG_OBJECT (decoder,
4716 "Failed to negotiate a pool, falling back to the previous pool");
4721 GST_LOG_OBJECT (decoder, "alloc buffer size %d", num_bytes);
4723 flow_ret = gst_buffer_pool_acquire_buffer (decoder->priv->pool,
4724 &frame->output_buffer, params);
4726 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4731 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4732 return GST_FLOW_FLUSHING;
4735 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4736 return GST_FLOW_ERROR;
4740 * gst_video_decoder_get_max_decode_time:
4741 * @decoder: a #GstVideoDecoder
4742 * @frame: a #GstVideoCodecFrame
4744 * Determines maximum possible decoding time for @frame that will
4745 * allow it to decode and arrive in time (as determined by QoS events).
4746 * In particular, a negative result means decoding in time is no longer possible
4747 * and should therefore occur as soon/skippy as possible.
4749 * Returns: max decoding time.
4752 gst_video_decoder_get_max_decode_time (GstVideoDecoder *
4753 decoder, GstVideoCodecFrame * frame)
4755 GstClockTimeDiff deadline;
4756 GstClockTime earliest_time;
4758 GST_OBJECT_LOCK (decoder);
4759 earliest_time = decoder->priv->earliest_time;
4760 if (GST_CLOCK_TIME_IS_VALID (earliest_time)
4761 && GST_CLOCK_TIME_IS_VALID (frame->deadline))
4762 deadline = GST_CLOCK_DIFF (earliest_time, frame->deadline);
4764 deadline = G_MAXINT64;
4766 GST_LOG_OBJECT (decoder, "earliest %" GST_TIME_FORMAT
4767 ", frame deadline %" GST_TIME_FORMAT ", deadline %" GST_STIME_FORMAT,
4768 GST_TIME_ARGS (earliest_time), GST_TIME_ARGS (frame->deadline),
4769 GST_STIME_ARGS (deadline));
4771 GST_OBJECT_UNLOCK (decoder);
4777 * gst_video_decoder_get_qos_proportion:
4778 * @decoder: a #GstVideoDecoder
4779 * current QoS proportion, or %NULL
4781 * Returns: The current QoS proportion.
4786 gst_video_decoder_get_qos_proportion (GstVideoDecoder * decoder)
4790 g_return_val_if_fail (GST_IS_VIDEO_DECODER (decoder), 1.0);
4792 GST_OBJECT_LOCK (decoder);
4793 proportion = decoder->priv->proportion;
4794 GST_OBJECT_UNLOCK (decoder);
4800 _gst_video_decoder_error (GstVideoDecoder * dec, gint weight,
4801 GQuark domain, gint code, gchar * txt, gchar * dbg, const gchar * file,
4802 const gchar * function, gint line)
4805 GST_WARNING_OBJECT (dec, "error: %s", txt);
4807 GST_WARNING_OBJECT (dec, "error: %s", dbg);
4808 dec->priv->error_count += weight;
4809 dec->priv->discont = TRUE;
4810 if (dec->priv->max_errors >= 0 &&
4811 dec->priv->error_count > dec->priv->max_errors) {
4812 gst_element_message_full (GST_ELEMENT (dec), GST_MESSAGE_ERROR,
4813 domain, code, txt, dbg, file, function, line);
4814 return GST_FLOW_ERROR;
4823 * gst_video_decoder_set_max_errors:
4824 * @dec: a #GstVideoDecoder
4825 * @num: max tolerated errors
4827 * Sets numbers of tolerated decoder errors, where a tolerated one is then only
4828 * warned about, but more than tolerated will lead to fatal error. You can set
4829 * -1 for never returning fatal errors. Default is set to
4830 * GST_VIDEO_DECODER_MAX_ERRORS.
4832 * The '-1' option was added in 1.4
4835 gst_video_decoder_set_max_errors (GstVideoDecoder * dec, gint num)
4837 g_return_if_fail (GST_IS_VIDEO_DECODER (dec));
4839 dec->priv->max_errors = num;
4843 * gst_video_decoder_get_max_errors:
4844 * @dec: a #GstVideoDecoder
4846 * Returns: currently configured decoder tolerated error count.
4849 gst_video_decoder_get_max_errors (GstVideoDecoder * dec)
4851 g_return_val_if_fail (GST_IS_VIDEO_DECODER (dec), 0);
4853 return dec->priv->max_errors;
4857 * gst_video_decoder_set_needs_format:
4858 * @dec: a #GstVideoDecoder
4859 * @enabled: new state
4861 * Configures decoder format needs. If enabled, subclass needs to be
4862 * negotiated with format caps before it can process any data. It will then
4863 * never be handed any data before it has been configured.
4864 * Otherwise, it might be handed data without having been configured and
4865 * is then expected being able to do so either by default
4866 * or based on the input data.
4871 gst_video_decoder_set_needs_format (GstVideoDecoder * dec, gboolean enabled)
4873 g_return_if_fail (GST_IS_VIDEO_DECODER (dec));
4875 dec->priv->needs_format = enabled;
4879 * gst_video_decoder_get_needs_format:
4880 * @dec: a #GstVideoDecoder
4882 * Queries decoder required format handling.
4884 * Returns: %TRUE if required format handling is enabled.
4889 gst_video_decoder_get_needs_format (GstVideoDecoder * dec)
4893 g_return_val_if_fail (GST_IS_VIDEO_DECODER (dec), FALSE);
4895 result = dec->priv->needs_format;
4901 * gst_video_decoder_set_packetized:
4902 * @decoder: a #GstVideoDecoder
4903 * @packetized: whether the input data should be considered as packetized.
4905 * Allows baseclass to consider input data as packetized or not. If the
4906 * input is packetized, then the @parse method will not be called.
4909 gst_video_decoder_set_packetized (GstVideoDecoder * decoder,
4910 gboolean packetized)
4912 decoder->priv->packetized = packetized;
4916 * gst_video_decoder_get_packetized:
4917 * @decoder: a #GstVideoDecoder
4919 * Queries whether input data is considered packetized or not by the
4922 * Returns: TRUE if input data is considered packetized.
4925 gst_video_decoder_get_packetized (GstVideoDecoder * decoder)
4927 return decoder->priv->packetized;
4931 * gst_video_decoder_have_last_subframe:
4932 * @decoder: a #GstVideoDecoder
4933 * @frame: (transfer none): the #GstVideoCodecFrame to update
4935 * Indicates that the last subframe has been processed by the decoder
4936 * in @frame. This will release the current frame in video decoder
4937 * allowing to receive new frames from upstream elements. This method
4938 * must be called in the subclass @handle_frame callback.
4940 * Returns: a #GstFlowReturn, usually GST_FLOW_OK.
4945 gst_video_decoder_have_last_subframe (GstVideoDecoder * decoder,
4946 GstVideoCodecFrame * frame)
4948 g_return_val_if_fail (gst_video_decoder_get_subframe_mode (decoder),
4950 /* unref once from the list */
4951 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
4952 if (decoder->priv->current_frame == frame) {
4953 gst_video_codec_frame_unref (decoder->priv->current_frame);
4954 decoder->priv->current_frame = NULL;
4956 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
4962 * gst_video_decoder_set_subframe_mode:
4963 * @decoder: a #GstVideoDecoder
4964 * @subframe_mode: whether the input data should be considered as subframes.
4966 * If this is set to TRUE, it informs the base class that the subclass
4967 * can receive the data at a granularity lower than one frame.
4969 * Note that in this mode, the subclass has two options. It can either
4970 * require the presence of a GST_VIDEO_BUFFER_FLAG_MARKER to mark the
4971 * end of a frame. Or it can operate in such a way that it will decode
4972 * a single frame at a time. In this second case, every buffer that
4973 * arrives to the element is considered part of the same frame until
4974 * gst_video_decoder_finish_frame() is called.
4976 * In either case, the same #GstVideoCodecFrame will be passed to the
4977 * GstVideoDecoderClass:handle_frame vmethod repeatedly with a
4978 * different GstVideoCodecFrame:input_buffer every time until the end of the
4979 * frame has been signaled using either method.
4980 * This method must be called during the decoder subclass @set_format call.
4985 gst_video_decoder_set_subframe_mode (GstVideoDecoder * decoder,
4986 gboolean subframe_mode)
4988 decoder->priv->subframe_mode = subframe_mode;
4992 * gst_video_decoder_get_subframe_mode:
4993 * @decoder: a #GstVideoDecoder
4995 * Queries whether input data is considered as subframes or not by the
4996 * base class. If FALSE, each input buffer will be considered as a full
4999 * Returns: TRUE if input data is considered as sub frames.
5004 gst_video_decoder_get_subframe_mode (GstVideoDecoder * decoder)
5006 return decoder->priv->subframe_mode;
5010 * gst_video_decoder_get_input_subframe_index:
5011 * @decoder: a #GstVideoDecoder
5012 * @frame: (transfer none): the #GstVideoCodecFrame to update
5014 * Queries the number of the last subframe received by
5015 * the decoder baseclass in the @frame.
5017 * Returns: the current subframe index received in subframe mode, 1 otherwise.
5022 gst_video_decoder_get_input_subframe_index (GstVideoDecoder * decoder,
5023 GstVideoCodecFrame * frame)
5025 return frame->abidata.ABI.num_subframes;
5029 * gst_video_decoder_get_processed_subframe_index:
5030 * @decoder: a #GstVideoDecoder
5031 * @frame: (transfer none): the #GstVideoCodecFrame to update
5033 * Queries the number of subframes in the frame processed by
5034 * the decoder baseclass.
5036 * Returns: the current subframe processed received in subframe mode.
5041 gst_video_decoder_get_processed_subframe_index (GstVideoDecoder * decoder,
5042 GstVideoCodecFrame * frame)
5044 return frame->abidata.ABI.subframes_processed;
5048 * gst_video_decoder_set_estimate_rate:
5049 * @dec: a #GstVideoDecoder
5050 * @enabled: whether to enable byte to time conversion
5052 * Allows baseclass to perform byte to time estimated conversion.
5055 gst_video_decoder_set_estimate_rate (GstVideoDecoder * dec, gboolean enabled)
5057 g_return_if_fail (GST_IS_VIDEO_DECODER (dec));
5059 dec->priv->do_estimate_rate = enabled;
5063 * gst_video_decoder_get_estimate_rate:
5064 * @dec: a #GstVideoDecoder
5066 * Returns: currently configured byte to time conversion setting
5069 gst_video_decoder_get_estimate_rate (GstVideoDecoder * dec)
5071 g_return_val_if_fail (GST_IS_VIDEO_DECODER (dec), 0);
5073 return dec->priv->do_estimate_rate;
5077 * gst_video_decoder_set_latency:
5078 * @decoder: a #GstVideoDecoder
5079 * @min_latency: minimum latency
5080 * @max_latency: maximum latency
5082 * Lets #GstVideoDecoder sub-classes tell the baseclass what the decoder
5083 * latency is. Will also post a LATENCY message on the bus so the pipeline
5084 * can reconfigure its global latency.
5087 gst_video_decoder_set_latency (GstVideoDecoder * decoder,
5088 GstClockTime min_latency, GstClockTime max_latency)
5090 g_return_if_fail (GST_CLOCK_TIME_IS_VALID (min_latency));
5091 g_return_if_fail (max_latency >= min_latency);
5093 GST_OBJECT_LOCK (decoder);
5094 decoder->priv->min_latency = min_latency;
5095 decoder->priv->max_latency = max_latency;
5096 GST_OBJECT_UNLOCK (decoder);
5098 gst_element_post_message (GST_ELEMENT_CAST (decoder),
5099 gst_message_new_latency (GST_OBJECT_CAST (decoder)));
5103 * gst_video_decoder_get_latency:
5104 * @decoder: a #GstVideoDecoder
5105 * @min_latency: (out) (allow-none): address of variable in which to store the
5106 * configured minimum latency, or %NULL
5107 * @max_latency: (out) (allow-none): address of variable in which to store the
5108 * configured mximum latency, or %NULL
5110 * Query the configured decoder latency. Results will be returned via
5111 * @min_latency and @max_latency.
5114 gst_video_decoder_get_latency (GstVideoDecoder * decoder,
5115 GstClockTime * min_latency, GstClockTime * max_latency)
5117 GST_OBJECT_LOCK (decoder);
5119 *min_latency = decoder->priv->min_latency;
5121 *max_latency = decoder->priv->max_latency;
5122 GST_OBJECT_UNLOCK (decoder);
5126 * gst_video_decoder_merge_tags:
5127 * @decoder: a #GstVideoDecoder
5128 * @tags: (allow-none): a #GstTagList to merge, or NULL to unset
5129 * previously-set tags
5130 * @mode: the #GstTagMergeMode to use, usually #GST_TAG_MERGE_REPLACE
5132 * Sets the audio decoder tags and how they should be merged with any
5133 * upstream stream tags. This will override any tags previously-set
5134 * with gst_audio_decoder_merge_tags().
5136 * Note that this is provided for convenience, and the subclass is
5137 * not required to use this and can still do tag handling on its own.
5142 gst_video_decoder_merge_tags (GstVideoDecoder * decoder,
5143 const GstTagList * tags, GstTagMergeMode mode)
5145 g_return_if_fail (GST_IS_VIDEO_DECODER (decoder));
5146 g_return_if_fail (tags == NULL || GST_IS_TAG_LIST (tags));
5147 g_return_if_fail (tags == NULL || mode != GST_TAG_MERGE_UNDEFINED);
5149 GST_VIDEO_DECODER_STREAM_LOCK (decoder);
5150 if (decoder->priv->tags != tags) {
5151 if (decoder->priv->tags) {
5152 gst_tag_list_unref (decoder->priv->tags);
5153 decoder->priv->tags = NULL;
5154 decoder->priv->tags_merge_mode = GST_TAG_MERGE_APPEND;
5157 decoder->priv->tags = gst_tag_list_ref ((GstTagList *) tags);
5158 decoder->priv->tags_merge_mode = mode;
5161 GST_DEBUG_OBJECT (decoder, "set decoder tags to %" GST_PTR_FORMAT, tags);
5162 decoder->priv->tags_changed = TRUE;
5164 GST_VIDEO_DECODER_STREAM_UNLOCK (decoder);
5168 * gst_video_decoder_get_buffer_pool:
5169 * @decoder: a #GstVideoDecoder
5171 * Returns: (transfer full): the instance of the #GstBufferPool used
5172 * by the decoder; free it after use it
5175 gst_video_decoder_get_buffer_pool (GstVideoDecoder * decoder)
5177 g_return_val_if_fail (GST_IS_VIDEO_DECODER (decoder), NULL);
5179 if (decoder->priv->pool)
5180 return gst_object_ref (decoder->priv->pool);
5186 * gst_video_decoder_get_allocator:
5187 * @decoder: a #GstVideoDecoder
5188 * @allocator: (out) (allow-none) (transfer full): the #GstAllocator
5190 * @params: (out) (allow-none) (transfer full): the
5191 * #GstAllocationParams of @allocator
5193 * Lets #GstVideoDecoder sub-classes to know the memory @allocator
5194 * used by the base class and its @params.
5196 * Unref the @allocator after use it.
5199 gst_video_decoder_get_allocator (GstVideoDecoder * decoder,
5200 GstAllocator ** allocator, GstAllocationParams * params)
5202 g_return_if_fail (GST_IS_VIDEO_DECODER (decoder));
5205 *allocator = decoder->priv->allocator ?
5206 gst_object_ref (decoder->priv->allocator) : NULL;
5209 *params = decoder->priv->params;
5213 * gst_video_decoder_set_use_default_pad_acceptcaps:
5214 * @decoder: a #GstVideoDecoder
5215 * @use: if the default pad accept-caps query handling should be used
5217 * Lets #GstVideoDecoder sub-classes decide if they want the sink pad
5218 * to use the default pad query handler to reply to accept-caps queries.
5220 * By setting this to true it is possible to further customize the default
5221 * handler with %GST_PAD_SET_ACCEPT_INTERSECT and
5222 * %GST_PAD_SET_ACCEPT_TEMPLATE
5227 gst_video_decoder_set_use_default_pad_acceptcaps (GstVideoDecoder * decoder,
5230 decoder->priv->use_default_pad_acceptcaps = use;
5234 gst_video_decoder_request_sync_point_internal (GstVideoDecoder * dec,
5235 GstClockTime deadline, GstVideoDecoderRequestSyncPointFlags flags)
5237 GstEvent *fku = NULL;
5238 GstVideoDecoderPrivate *priv;
5240 g_return_if_fail (GST_IS_VIDEO_DECODER (dec));
5244 GST_OBJECT_LOCK (dec);
5246 /* Check if we're allowed to send a new force-keyunit event.
5247 * frame->deadline is set to the running time of the PTS. */
5248 if (priv->min_force_key_unit_interval == 0 ||
5249 deadline == GST_CLOCK_TIME_NONE ||
5250 (priv->min_force_key_unit_interval != GST_CLOCK_TIME_NONE &&
5251 (priv->last_force_key_unit_time == GST_CLOCK_TIME_NONE
5252 || (priv->last_force_key_unit_time +
5253 priv->min_force_key_unit_interval <= deadline)))) {
5254 GST_DEBUG_OBJECT (dec,
5255 "Requesting a new key-unit for frame with deadline %" GST_TIME_FORMAT,
5256 GST_TIME_ARGS (deadline));
5258 gst_video_event_new_upstream_force_key_unit (GST_CLOCK_TIME_NONE, FALSE,
5260 priv->last_force_key_unit_time = deadline;
5262 GST_DEBUG_OBJECT (dec,
5263 "Can't request a new key-unit for frame with deadline %"
5264 GST_TIME_FORMAT, GST_TIME_ARGS (deadline));
5266 priv->request_sync_point_flags |= flags;
5267 /* We don't know yet the frame number of the sync point so set it to a
5268 * frame number higher than any allowed frame number */
5269 priv->request_sync_point_frame_number = REQUEST_SYNC_POINT_PENDING;
5270 GST_OBJECT_UNLOCK (dec);
5273 gst_pad_push_event (dec->sinkpad, fku);
5277 * gst_video_decoder_request_sync_point:
5278 * @dec: a #GstVideoDecoder
5279 * @frame: a #GstVideoCodecFrame
5280 * @flags: #GstVideoDecoderRequestSyncPointFlags
5282 * Allows the #GstVideoDecoder subclass to request from the base class that
5283 * a new sync should be requested from upstream, and that @frame was the frame
5284 * when the subclass noticed that a new sync point is required. A reason for
5285 * the subclass to do this could be missing reference frames, for example.
5287 * The base class will then request a new sync point from upstream as long as
5288 * the time that passed since the last one is exceeding
5289 * #GstVideoDecoder:min-force-key-unit-interval.
5291 * The subclass can signal via @flags how the frames until the next sync point
5292 * should be handled:
5294 * * If %GST_VIDEO_DECODER_REQUEST_SYNC_POINT_DISCARD_INPUT is selected then
5295 * all following input frames until the next sync point are discarded.
5296 * This can be useful if the lack of a sync point will prevent all further
5297 * decoding and the decoder implementation is not very robust in handling
5298 * missing references frames.
5299 * * If %GST_VIDEO_DECODER_REQUEST_SYNC_POINT_CORRUPT_OUTPUT is selected
5300 * then all output frames following @frame are marked as corrupted via
5301 * %GST_BUFFER_FLAG_CORRUPTED. Corrupted frames can be automatically
5302 * dropped by the base class, see #GstVideoDecoder:discard-corrupted-frames.
5303 * Subclasses can manually mark frames as corrupted via %GST_VIDEO_CODEC_FRAME_FLAG_CORRUPTED
5304 * before calling gst_video_decoder_finish_frame().
5309 gst_video_decoder_request_sync_point (GstVideoDecoder * dec,
5310 GstVideoCodecFrame * frame, GstVideoDecoderRequestSyncPointFlags flags)
5312 g_return_if_fail (GST_IS_VIDEO_DECODER (dec));
5313 g_return_if_fail (frame != NULL);
5315 gst_video_decoder_request_sync_point_internal (dec, frame->deadline, flags);
5319 * gst_video_decoder_set_needs_sync_point:
5320 * @dec: a #GstVideoDecoder
5321 * @enabled: new state
5323 * Configures whether the decoder requires a sync point before it starts
5324 * outputting data in the beginning. If enabled, the base class will discard
5325 * all non-sync point frames in the beginning and after a flush and does not
5326 * pass it to the subclass.
5328 * If the first frame is not a sync point, the base class will request a sync
5329 * point via the force-key-unit event.
5334 gst_video_decoder_set_needs_sync_point (GstVideoDecoder * dec, gboolean enabled)
5336 g_return_if_fail (GST_IS_VIDEO_DECODER (dec));
5338 dec->priv->needs_sync_point = enabled;
5342 * gst_video_decoder_get_needs_sync_point:
5343 * @dec: a #GstVideoDecoder
5345 * Queries if the decoder requires a sync point before it starts outputting
5346 * data in the beginning.
5348 * Returns: %TRUE if a sync point is required in the beginning.
5353 gst_video_decoder_get_needs_sync_point (GstVideoDecoder * dec)
5357 g_return_val_if_fail (GST_IS_VIDEO_DECODER (dec), FALSE);
5359 result = dec->priv->needs_sync_point;