From cbc46176e0567f2856dd81bc24184856edd33651 Mon Sep 17 00:00:00 2001 From: Wim Taymans Date: Tue, 2 Oct 2012 16:15:19 +0200 Subject: [PATCH] pwg: rewite data-access chapter Rewrite the data-access chapter so that we talk about appsrc instead of the fakesrc hacks. --- docs/manual/advanced-dataaccess.xml | 367 +++++++++++++++++++++++++----------- tests/examples/manual/Makefile.am | 6 +- 2 files changed, 259 insertions(+), 114 deletions(-) diff --git a/docs/manual/advanced-dataaccess.xml b/docs/manual/advanced-dataaccess.xml index f066d11..b8f6544 100644 --- a/docs/manual/advanced-dataaccess.xml +++ b/docs/manual/advanced-dataaccess.xml @@ -54,6 +54,8 @@ cb_have_data (GstPad *pad, GstBuffer *buffer; buffer = GST_PAD_PROBE_INFO_BUFFER (info); + + buffer = gst_buffer_make_writable (buffer); gst_buffer_map (buffer, &map, GST_MAP_WRITE); @@ -69,6 +71,8 @@ cb_have_data (GstPad *pad, } gst_buffer_unmap (buffer, &map); + GST_PAD_PROBE_INFO_DATA (info) = buffer; + return TRUE; } @@ -145,21 +149,21 @@ main (gint argc, looking for. - The above example is not really correct though. Strictly speaking, a - pad probe callback is only allowed to modify the buffer content if the - buffer is writable. Whether this is the case or not depends a lot on - the pipeline and the elements involved. Often enough, this is the case, - but sometimes it is not, and if it is not then unexpected modification - of the data or metadata can introduce bugs that are very hard to debug - and track down. You can check if a buffer is writable with - gst_buffer_is_writable (). Since you - can pass back a different buffer than the one passed in, it is a good - idea to make the buffer writable in the callback function. + Strictly speaking, a pad probe callback is only allowed to modify the + buffer content if the buffer is writable. Whether this is the case or + not depends a lot on the pipeline and the elements involved. Often + enough, this is the case, but sometimes it is not, and if it is not + then unexpected modification of the data or metadata can introduce + bugs that are very hard to debug and track down. You can check if a + buffer is writable with gst_buffer_is_writable (). + Since you can pass back a different buffer than the one passed in, + it is a good idea to make the buffer writable in the callback function + with gst_buffer_make_writable (). Pad probes are suited best for looking at data as it passes through - the pipeline. If you need to modify data, you should write your own - GStreamer element. Base classes like GstAudioFilter, GstVideoFilter or + the pipeline. If you need to modify data, you should better write your + own GStreamer element. Base classes like GstAudioFilter, GstVideoFilter or GstBaseTransform make this fairly easy. @@ -168,7 +172,8 @@ main (gint argc, an identity element into the pipeline and connect to its "handoff" signal. The identity element also provides a few useful debugging tools like the "dump" property or the "last-message" property (the latter is - enabled by passing the '-v' switch to gst-launch). + enabled by passing the '-v' switch to gst-launch and by setting the + silent property on the identity to FALSE). @@ -178,124 +183,225 @@ main (gint argc, Many people have expressed the wish to use their own sources to inject data into a pipeline. Some people have also expressed the wish to grab the output in a pipeline and take care of the actual output inside - their application. While either of these methods are stongly - discouraged, &GStreamer; offers hacks to do this. However, - there is no support for those methods. If it doesn't work, - you're on your own. Also, synchronization, thread-safety and other - things that you've been able to take for granted so far are no longer - guaranteed if you use any of those methods. It's always better to - simply write a plugin and have the pipeline schedule and manage it. + their application. While either of these methods are strongly + discouraged, &GStreamer; offers support for this. + Beware! You need to know what you are doing. Since + you don't have any support from a base class you need to thoroughly + understand state changes and synchronization. If it doesn't work, + there are a million ways to shoot yourself in the foot. It's always + better to simply write a plugin and have the base class manage it. See the Plugin Writer's Guide for more information on this topic. Also see the next section, which will explain how to embed plugins statically in your application. - - New - API was developed to make data insertion and extraction easy - for applications. It can be found as GstAppSrc and GstAppSink in the - - gst-plugins-base module. - - After all those disclaimers, let's start. There's three possible - elements that you can use for the above-mentioned purposes. Those are - called fakesrc (an imaginary source), - fakesink (an imaginary sink) and identity - (an imaginary filter). The same method applies to each of those - elements. Here, we will discuss how to use those elements to insert - (using fakesrc) or grab (using fakesink or identity) data from a + There's two possible elements that you can use for the above-mentioned + purposes. Those are called appsrc (an imaginary source) + and appsink (an imaginary sink). The same method applies + to each of those elements. Here, we will discuss how to use those + elements to insert (using appsrc) or grab (using appsink) data from a pipeline, and how to set negotiation. - Those who're paying close attention will notice that the purpose - of identity is almost identical to that of probes. Indeed, this is - true. Probes allow for the same purpose, and a bunch more, and - with less overhead plus dynamic removing/adding of handlers, but - apart from those, probes and identity have the same purpose, just - in a completely different implementation type. + Both appsrc and appsink provide 2 sets of API. One API uses standard + GObject (action) signals and properties. The same API is also + available as a regular C api. The C api is more performant but + requires you to link to the app library in order to use the elements. - - Inserting or grabbing data + + Inserting data with appsrc - The three before-mentioned elements (fakesrc, fakesink and identity) - each have a handoff signal that will be called in - the _get ()- (fakesrc) or _chain - ()-function (identity, fakesink). In the signal handler, - you can set (fakesrc) or get (identity, fakesink) data to/from the - provided buffer. Note that in the case of fakesrc, you have to set - the size of the provided buffer using the sizemax - property. For both fakesrc and fakesink, you also have to set the - signal-handoffs property for this method to work. + First we look at some examples for appsrc, which lets you insert data + into the pipeline from the application. Appsrc has some configuration + options that define how it will operate. You should decide about the + following configurations: + + + + Will the appsrc operate in push or pull mode. The stream-type + property can be used to control this. stream-type of + random-access will activate pull mode scheduling + while the other stream-types activate push mode. + + + + + The caps of the buffers that appsrc will push out. This needs to + be configured with the caps property. The caps must be set to a + fixed caps and will be used to negotiate a format downstream. + + + + + It the appsrc operates in live mode or not. This can be configured + with the is-live property. When operating in live-mode it is + important to configure the min-latency and max-latency in appsrc. + The min-latency should be set to the amount of time it takes between + capturing a buffer and when it is pushed inside appsrc. + In live mode, you should timestamp the buffers with the pipeline + running-time when the first byte of the buffer was captured before + feeding them to appsrc. You can let appsrc do the timestaping with + the do-timestamp property (but then the min-latency must be set + to 0 because it timestamps based on the running-time when the buffer + entered appsrc). + + + + + The format of the SEGMENT event that appsrc will push. The format + has implications for how the running-time of the buffers will + be calculated so you must be sure you understand this. For + live sources you probably want to set the format property to + GST_FORMAT_TIME. For non-live source it depends on the media type + that you are handling. If you plan to timestamp the buffers, you + should probably put a GST_FORMAT_TIME format, otherwise + GST_FORMAT_BYTES might be appropriate. + + + + + If appsrc operates in random-access mode, it is important to configure + the size property of appsrc with the number of bytes in the stream. + This will allow downstream elements to know the size of the media and + alows them to seek to the end of the stream when needed. + + + - Note that your handoff function should not - block, since this will block pipeline iteration. Also, do not try - to use all sort of weird hacks in such functions to accomplish - something that looks like synchronization or so; it's not the right - way and will lead to issues elsewhere. If you're doing any of this, - you're basically misunderstanding the &GStreamer; design. + The main way of handling data to appsrc is by using the function + gst_app_src_push_buffer () or by emiting the + push-buffer action signal. This will put the buffer onto a queue from + which appsrc will read from in its streaming thread. It is important + to note that data transport will not happen from the thread that + performed the push-buffer call. - - - - Forcing a format - Sometimes, when using fakesrc as a source in your pipeline, you'll - want to set a specific format, for example a video size and format - or an audio bitsize and number of channels. You can do this by - forcing a specific GstCaps on the pipeline, - which is possible by using filtered caps. You - can set a filtered caps on a link by using the - capsfilter element in between the two elements, and - specifying a GstCaps as - caps property on this element. It will then - only allow types matching that specified capability set for - negotiation. See also . + The max-bytes property controls how much data can be + queued in appsrc before appsrc considers the queue full. A filled + internal queue will always signal the enough-data + signal, which signals the application that it should stop pushing + data into appsrc. The block property will cause appsrc to + block the push-buffer method until free data becomes available again. + + + When the internal queue is running out of data, the + need-data signal is emitted, which signals the application + that it should start pushing more data into appsrc. - - - - Example application - This example application will generate black/white (it switches - every second) video to an X-window output by using fakesrc as a - source and using filtered caps to force a format. Since the depth - of the image depends on your X-server settings, we use a colorspace - conversion element to make sure that the output to your X server - will have the correct bitdepth. You can also set timestamps on the - provided buffers to override the fixed framerate. + In addition to the need-data and enough-data + signals, appsrc can emit the seek-data signal when the + stream-mode property is set to seekable + or random-access. The signal argument will contain the + new desired position in the stream expressed in the unit set with the + format property. After receiving the seek-data signal, + the application should push-buffers from the new position. - -#include <string.h> /* for memset () */ + + When the last byte is pushed into appsrc, you must call + gst_app_src_end_of_stream () to make it send + an EOS downstream. + + + These signals allow the application to operate appsrc in push and + pull mode as will be explained next. + + + + Using appsrc in push mode + + When appsrc is configured in push mode (stream-type is stream or + seekable), the application repeatedly calls the push-buffer method + with a new buffer. Optionally, the queue size in the appsrc can be + controlled with the enough-data and need-data signals by respectively + stopping/starting the push-buffer calls. The value of the + min-percent property defines how empty the internal appsrc queue + needs to be before the need-data signal will be fired. You can set + this to some value >0 to avoid completely draining the queue. + + + When the stream-type is set to seekable, don't forget to implement + a seek-data callback. + + + Use this model when implementing various network protocols or + hardware devices. + + + + + Using appsrc in pull mode + + In the pull model, data is fed to appsrc from the need-data signal + handler. You should push exactly the amount of bytes requested in the + need-data signal. You are only allowed to push less bytes when you are + at the end of the stream. + + + Use this model for file access or other randomly accessable sources. + + + + + Appsrc example + + This example application will generate black/white (it switches + every second) video to an Xv-window output by using appsrc as a + source with caps to force a format. We use a colorspace + conversion element to make sure that we feed the right format to + your X server. We configure a video stream with a variable framerate + (0/1) and we set the timestamps on the outgoing buffers in such + a way that we play 2 frames per second. + + + Note how we use the pull mode method of pushing new buffers into + appsrc although appsrc is running in push mode. + + #include <gst/gst.h> +static GMainLoop *loop; + static void -cb_handoff (GstElement *fakesrc, - GstBuffer *buffer, - GstPad *pad, - gpointer user_data) +cb_need_data (GstElement *appsrc, + guint unused_size, + gpointer user_data) { static gboolean white = FALSE; - GstMapInfo info; - - gst_buffer_map (buffer, &info, GST_MAP_WRITE); + static GstClockTime timestamp = 0; + GstBuffer *buffer; + guint size; + GstFlowReturn ret; + + size = 385 * 288 * 2; + + buffer = gst_buffer_new_allocate (NULL, size, NULL); /* this makes the image black/white */ - memset (info.data, white ? 0xff : 0x0, info.size); + gst_buffer_memset (buffer, 0, white ? 0xff : 0x0, size); + white = !white; - gst_buffer_unmap (buffer, &info); + GST_BUFFER_PTS (buffer) = timestamp; + GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2); + + timestamp += GST_BUFFER_DURATION (buffer); + + g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret); + + if (ret != GST_FLOW_OK) { + /* something wrong, stop pushing */ + g_main_loop_quit (loop); + } } gint main (gint argc, gchar *argv[]) { - GstElement *pipeline, *fakesrc, *flt, *conv, *videosink; - GMainLoop *loop; + GstElement *pipeline, *appsrc, *conv, *videosink; /* init GStreamer */ gst_init (&argc, &argv); @@ -303,28 +409,26 @@ main (gint argc, /* setup pipeline */ pipeline = gst_pipeline_new ("pipeline"); - fakesrc = gst_element_factory_make ("fakesrc", "source"); - flt = gst_element_factory_make ("capsfilter", "flt"); + appsrc = gst_element_factory_make ("appsrc", "source"); conv = gst_element_factory_make ("videoconvert", "conv"); videosink = gst_element_factory_make ("xvimagesink", "videosink"); /* setup */ - g_object_set (G_OBJECT (flt), "caps", + g_object_set (G_OBJECT (appsrc), "caps", gst_caps_new_simple ("video/x-raw", "format", G_TYPE_STRING, "RGB16", "width", G_TYPE_INT, 384, "height", G_TYPE_INT, 288, - "framerate", GST_TYPE_FRACTION, 1, 1, + "framerate", GST_TYPE_FRACTION, 0, 1, NULL), NULL); - gst_bin_add_many (GST_BIN (pipeline), fakesrc, flt, conv, videosink, NULL); - gst_element_link_many (fakesrc, flt, conv, videosink, NULL); + gst_bin_add_many (GST_BIN (pipeline), appsrc, conv, videosink, NULL); + gst_element_link_many (appsrc, conv, videosink, NULL); - /* setup fake source */ - g_object_set (G_OBJECT (fakesrc), - "signal-handoffs", TRUE, - "sizemax", 384 * 288 * 2, - "sizetype", 2, NULL); - g_signal_connect (fakesrc, "handoff", G_CALLBACK (cb_handoff), NULL); + /* setup appsrc */ + g_object_set (G_OBJECT (appsrc), + "stream-type", 0, + "format", GST_FORMAT_TIME, NULL); + g_signal_connect (appsrc, "need-data", G_CALLBACK (cb_need_data), NULL); /* play */ gst_element_set_state (pipeline, GST_STATE_PLAYING); @@ -333,10 +437,51 @@ main (gint argc, /* clean up */ gst_element_set_state (pipeline, GST_STATE_NULL); gst_object_unref (GST_OBJECT (pipeline)); + g_main_loop_unref (loop); return 0; } - + + + + + + Grabbing data with appsink + + The two before-mentioned elements (fakesrc, fakesink and identity) + each have a handoff signal that will be called in + the _get ()- (fakesrc) or _chain + ()-function (identity, fakesink). In the signal handler, + you can set (fakesrc) or get (identity, fakesink) data to/from the + provided buffer. Note that in the case of fakesrc, you have to set + the size of the provided buffer using the sizemax + property. For both fakesrc and fakesink, you also have to set the + signal-handoffs property for this method to work. + + + Note that your handoff function should not + block, since this will block pipeline iteration. Also, do not try + to use all sort of weird hacks in such functions to accomplish + something that looks like synchronization or so; it's not the right + way and will lead to issues elsewhere. If you're doing any of this, + you're basically misunderstanding the &GStreamer; design. + + + + + Forcing a format + + Sometimes you'll want to set a specific format, for example a video + size and format or an audio bitsize and number of channels. You can + do this by forcing a specific GstCaps on + the pipeline, which is possible by using + filtered caps. You can set a filtered caps on + a link by using the capsfilter element in between the + two elements, and specifying a GstCaps as + caps property on this element. It will then + only allow types matching that specified capability set for + negotiation. See also . + diff --git a/tests/examples/manual/Makefile.am b/tests/examples/manual/Makefile.am index 65d997c..454c1fc 100644 --- a/tests/examples/manual/Makefile.am +++ b/tests/examples/manual/Makefile.am @@ -36,7 +36,7 @@ EXAMPLES = \ query \ typefind \ probe \ - fakesrc \ + appsrc \ playbin \ decodebin @@ -50,7 +50,7 @@ BUILT_SOURCES = \ query.c \ typefind.c dynamic.c \ probe.c \ - fakesrc.c \ + appsrc.c \ playbin.c decodebin.c CLEANFILES = core core.* test-registry.* *.gcno *.gcda $(BUILT_SOURCES) @@ -86,7 +86,7 @@ typefind.c dynamic.c: $(top_srcdir)/docs/manual/advanced-autoplugging.xml probe.c: $(top_srcdir)/docs/manual/advanced-dataaccess.xml $(PERL_PATH) $(srcdir)/extract.pl $@ $< -fakesrc.c: $(top_srcdir)/docs/manual/advanced-dataaccess.xml +appsrc.c: $(top_srcdir)/docs/manual/advanced-dataaccess.xml $(PERL_PATH) $(srcdir)/extract.pl $@ $< playbin.c decodebin.c: $(top_srcdir)/docs/manual/highlevel-components.xml -- 2.7.4