using-appsink-appsrc-in-qt.md

   1 # Using appsink/appsrc in Qt
   2
   3 ## Goal
   4
   5 For those times when you need to stream data into or out of GStreamer
   6 through your application, GStreamer includes two helpful elements:
   7
   8   - `appsink` - Allows applications to easily extract data from a
   9     GStreamer pipeline
  10   - `appsrc` - Allows applications to easily stream data into a
  11     GStreamer pipeline
  12
  13 This tutorial will demonstrate how to use both of them by constructing a
  14 pipeline to decode an audio file, stream it into an application's code,
  15 then stream it back into your audio output device. All this, using
  16 QtGStreamer.
  17
  18 ## Steps
  19
  20 First, the files. These are also available in the
  21 `examples/appsink-src` directory of the QGstreamer SDK.
  22
  23 **CMakeLists.txt**
  24
  25 ```
  26 project(qtgst-example-appsink-src)
  27 find_package(QtGStreamer REQUIRED)
  28 find_package(Qt4 REQUIRED)
  29 include_directories(${QTGSTREAMER_INCLUDES} ${QT_QTCORE_INCLUDE_DIRS})
  30 add_definitions(${QTGSTREAMER_DEFINITIONS})
  31 set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${QTGSTREAMER_FLAGS}")
  32 add_executable(appsink-src main.cpp)
  33 target_link_libraries(appsink-src ${QTGSTREAMER_UTILS_LIBRARIES} ${QT_QTCORE_LIBRARIES})
  34 ```
  35
  36 **main.cpp**
  37
  38 ``` c
  39 #include <iostream>
  40 #include <QtCore/QCoreApplication>
  41 #include <QGlib/Error>
  42 #include <QGlib/Connect>
  43 #include <QGst/Init>
  44 #include <QGst/Bus>
  45 #include <QGst/Pipeline>
  46 #include <QGst/Parse>
  47 #include <QGst/Message>
  48 #include <QGst/Utils/ApplicationSink>
  49 #include <QGst/Utils/ApplicationSource>
  50
  51 class MySink : public QGst::Utils::ApplicationSink
  52 {
  53 public:
  54     MySink(QGst::Utils::ApplicationSource *src)
  55         : QGst::Utils::ApplicationSink(), m_src(src) {}
  56 protected:
  57     virtual void eos()
  58     {
  59         m_src->endOfStream();
  60     }
  61     virtual QGst::FlowReturn newBuffer()
  62     {
  63         m_src->pushBuffer(pullBuffer());
  64         return QGst::FlowOk;
  65     }
  66 private:
  67     QGst::Utils::ApplicationSource *m_src;
  68 };
  69
  70 class Player : public QCoreApplication
  71 {
  72 public:
  73     Player(int argc, char **argv);
  74     ~Player();
  75 private:
  76     void onBusMessage(const QGst::MessagePtr & message);
  77 private:
  78     QGst::Utils::ApplicationSource m_src;
  79     MySink m_sink;
  80     QGst::PipelinePtr pipeline1;
  81     QGst::PipelinePtr pipeline2;
  82 };
  83 Player::Player(int argc, char **argv)
  84     : QCoreApplication(argc, argv), m_sink(&m_src)
  85 {
  86     QGst::init(&argc, &argv);
  87     if (argc <= 1) {
  88         std::cerr << "Usage: " << argv[0] << " <audio_file>" << std::endl;
  89         std::exit(1);
  90     }
  91     const char *caps = "audio/x-raw-int,channels=1,rate=8000,"
  92                        "signed=(boolean)true,width=16,depth=16,endianness=1234";
  93     /* source pipeline */
  94     QString pipe1Descr = QString("filesrc location=\"%1\" ! "
  95                                  "decodebin2 ! "
  96                                  "audioconvert ! "
  97                                  "audioresample ! "
  98                                  "appsink name=\"mysink\" caps=\"%2\"").arg(argv[1], caps);
  99     pipeline1 = QGst::Parse::launch(pipe1Descr).dynamicCast<QGst::Pipeline>();
 100     m_sink.setElement(pipeline1->getElementByName("mysink"));
 101     QGlib::connect(pipeline1->bus(), "message::error", this, &Player::onBusMessage);
 102     pipeline1->bus()->addSignalWatch();
 103     /* sink pipeline */
 104     QString pipe2Descr = QString("appsrc name=\"mysrc\" caps=\"%1\" ! autoaudiosink").arg(caps);
 105     pipeline2 = QGst::Parse::launch(pipe2Descr).dynamicCast<QGst::Pipeline>();
 106     m_src.setElement(pipeline2->getElementByName("mysrc"));
 107     QGlib::connect(pipeline2->bus(), "message", this, &Player::onBusMessage);
 108     pipeline2->bus()->addSignalWatch();
 109     /* start playing */
 110     pipeline1->setState(QGst::StatePlaying);
 111     pipeline2->setState(QGst::StatePlaying);
 112 }
 113 Player::~Player()
 114 {
 115     pipeline1->setState(QGst::StateNull);
 116     pipeline2->setState(QGst::StateNull);
 117 }
 118 void Player::onBusMessage(const QGst::MessagePtr & message)
 119 {
 120     switch (message->type()) {
 121     case QGst::MessageEos:
 122         quit();
 123         break;
 124     case QGst::MessageError:
 125         qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
 126         break;
 127     default:
 128         break;
 129     }
 130 }
 131
 132 int main(int argc, char **argv)
 133 {
 134     Player p(argc, argv);
 135     return p.exec();
 136 }
 137 ```
 138
 139 ### Walkthrough
 140
 141 As this is a very simple example, most of the action happens in the
 142 `Player`'s constructor. First, GStreamer is initialized through
 143 `QGst::init()`:
 144
 145 **GStreamer Initialization**
 146
 147 ``` c
 148     QGst::init(&argc, &argv);
 149 ```
 150
 151 Now we can construct the first half of the pipeline:
 152
 153 **Pipeline Setup**
 154
 155 ``` c
 156     const char *caps = "audio/x-raw-int,channels=1,rate=8000,"
 157                        "signed=(boolean)true,width=16,depth=16,endianness=1234";
 158
 159     /* source pipeline */
 160     QString pipe1Descr = QString("filesrc location=\"%1\" ! "
 161                                  "decodebin2 ! "
 162                                  "audioconvert ! "
 163                                  "audioresample ! "
 164                                  "appsink name=\"mysink\" caps=\"%2\"").arg(argv[1], caps);
 165     pipeline1 = QGst::Parse::launch(pipe1Descr).dynamicCast<QGst::Pipeline>();
 166     m_sink.setElement(pipeline1->getElementByName("mysink"));
 167     QGlib::connect(pipeline1->bus(), "message::error", this, &Player::onBusMessage);
 168     pipeline1->bus()->addSignalWatch();
 169 ```
 170
 171 `QGst::Parse::launch()` parses the text description of a pipeline and
 172 returns a `QGst::PipelinePtr`. In this case, the pipeline is composed
 173 of:
 174
 175   - A `filesrc` element to read the file
 176   - `decodebin2` to automatically examine the stream and pick the right
 177     decoder(s)
 178   - `audioconvert` and `audioresample` to convert the output of the
 179     `decodebin2` into the caps specified for the `appsink`
 180   - An `appsink` element with specific caps
 181
 182 Next, we tell our `MySink` class (which is a subclass
 183 of `QGst::Utils::ApplicationSink`) what `appsink` element to use.
 184
 185 The second half of the pipeline is created similarly:
 186
 187 **Second Pipeline**
 188
 189 ``` c
 190     /* sink pipeline */
 191     QString pipe2Descr = QString("appsrc name=\"mysrc\" caps=\"%1\" ! autoaudiosink").arg(caps);
 192     pipeline2 = QGst::Parse::launch(pipe2Descr).dynamicCast<QGst::Pipeline>();
 193     m_src.setElement(pipeline2->getElementByName("mysrc"));
 194     QGlib::connect(pipeline2->bus(), "message", this, &Player::onBusMessage);
 195     pipeline2->bus()->addSignalWatch();
 196 ```
 197
 198 Finally, the pipeline is started:
 199
 200 **Starting the pipeline**
 201
 202 ``` c
 203  /* start playing */
 204     pipeline1->setState(QGst::StatePlaying);
 205     pipeline2->setState(QGst::StatePlaying);
 206 ```
 207
 208 Once the pipelines are started, the first one begins pushing buffers
 209 into the `appsink` element. Our `MySink` class implements the
 210 `newBuffer()` method, which is called by QGStreamer when a new buffer is
 211 ready for processing:
 212
 213 **MySink::newBuffer()**
 214
 215 ``` c
 216     virtual QGst::FlowReturn newBuffer()
 217     {
 218         m_src->pushBuffer(pullBuffer());
 219         return QGst::FlowOk;
 220     }
 221 ```
 222
 223 Our implementation takes the new buffer and pushes it into the
 224 `appsrc` element, which got assigned in the `Player` constructor:
 225
 226 **Player::Player()**
 227
 228 ``` c
 229 Player::Player(int argc, char **argv)
 230     : QCoreApplication(argc, argv), m_sink(&m_src)
 231 ```
 232
 233 From there, buffers flow into the `autoaudiosink` element, which
 234 automatically figures out a way to send it to your speakers.
 235
 236 ## Conclusion
 237
 238 You should now have an understanding of how to push and pull arbitrary
 239 data into and out of a GStreamer pipeline.
 240
 241 It has been a pleasure having you here, and see you soon\!