+++ /dev/null
-RTP auxiliary stream design
-
-
-auxiliary elements
-------------------
-
-There are two kind of auxiliary elements, sender and receiver.
-Let's call them rtpauxsend and rtpauxreceive.
-
-rtpauxsend has always one sink pad and can have unlimited requested src pads.
-If only src pad then it works in SSRC-multiplexed mode, if several src pads
-then it works in session multiplexed mode.
-
-rtpauxreceive has always one ssrc pad and can have unlimited requested sink pads.
-If only one sink pad then it works in SSRC-multiplexed mode, if several sink pads
-then it works in session multiplexed mode.
-
-
-rtpbin and auxiliary elements
-----------------------------
-
--- basic mecanism
-
-rtpbin knows for which session ids the given auxiliary element belong to.
-It's done through "set-aux-send", for rtpauxsend kind, and through
-"set-aux-receive" for rtpauxreceive kind.
-You can call those signals as much as needed for each auxiliary element.
-So for aux elements that work in SSRC-multiplexed mode this signal action is
-called only one time.
-
-The user has to call those action signals before to request the differents
-rtpbin pads.
-rtpbin is in charge to link those auxiliary elements with the sessions,
-and on receiver side, rtpbin has also to handle the link with ssrcdemux.
-
-rtpbin never knows if the given rtpauxsend is actually a rtprtxsend element
-or another aux element.
-rtpbin never knows if the given rtpauxreceive is actually a rtprtxreceive
-element or another aux element.
-rtpbin has to be kept generic so that more aux elements can be added later
-without changing rtpbin.
-
-It's currently not possible to use rtpbin with auxiliary stream from gst-launch.
-We can discuss about having the ability for rtpbin to instanciate itself
-the special aux elements rtprtxsend and rtprtxreceive but they need to be
-configured ("payload-type" and "payload-types" properties) to make retransmission
-work. So having several rtprtxsend and rtprtxreceive in a rtpbin would require
-a lot of properties to manage them form rtpbin.
-And for each auxiliary elements.
-
-If you want to use rtprtxreceive and rtprtpsend from gst-launch you have to use
-rtpsession, ssrcdemux and rtpjitterbuffer elements yourself.
-See gtk-doc of rtprtxreceive for an example.
-
--- requesting the rtpbin's pads on the pipeline receiver side
-
-If rtpauxreceive is set for session, i, j, k then it has to call
-rtpbin::"set-aux-receive" 3 times giving those ids and this aux element.
-It has to be done before requesting the recv_rtp_sink_i, recv_rtp_sink_j, recv_rtp_sink_k.
-For a concrete case rtprtxreceive, if the user wants it for session i, then it has
-to call rtpbin::"set-aux-receive" one time giving i and this aux element.
-Then the user can request recv_rtp_sink_i pad.
-
-Calling rtpbin::"set-aux-receive" does not create the session. It add the given
-session id and aux element to a hashtable(key:session id, value: aux element).
-Then when the user ask for rtpbin.recv_rtp_sink_i, rtpbin lookup if there is an
-aux element for this i session id. If yes it requests a sink pad to this aux
-element and links it with the recv_rtp_src pad of the new gstrtpsession.
-rtpbin also checks that this aux element is connected only one time to ssrcdemux.
-Because rtpauxreceive has only one source pad.
-Each call to request rtpbin.recv_rtp_sink_k will also creates rtpbin.recv_rtp_src_k_ssrc_pt
-as usual. So that the user have it when then it requests rtpbin. (from gst-launch) or
-using on_rtpbinreceive_pad_added callback from an application.
-
--- requesting the rtpbin's pads on the pipeline sender side
-
-For the sender this is similar but a bit more complicated to implement.
-When the user asks for rtpbin.send_rtp_sink_i, rtpbin will lookup in its
-second map (key:session id, value: aux send element). If there is one aux
-element, then it will set the sink pad of this aux sender element to be the ghost
-pad rtpbin.send_rtp_sink_i that the user asked. rtpbin will also request a src
-pad of this aux element to connect it to gstrtpsession_i. It will automatically
-create rtpbin.send_rtp_src_i the usuall way. Then if the user asks
-rtpbin.send_rtp_src_k, then rtpbin will also lookup in that map and request
-another source pad of the aux element and connect it to the new gstrtpsession_k.
+++ /dev/null
-RTP retransmission design
-
-
-GstRTPRetransmissionRequest
----------------------------
-
-Custom upstream event which mainly contains the ssrc and the seqnum of the
-packet which is asked to be retransmisted.
-
-On the pipeline receiver side this event is generated by the
-gstrtpjitterbuffer element. Then it is translated to a NACK to be sent over
-the network.
-
-On the pipeline sender side, this event is generated by the gstrtpsession
-element when it receives a NACK from the network.
-
-
-rtprtxsend element
-------------------
-
--- basic mechanism
-
-rtprtxsend keeps a history of rtp packets that it has already sent.
-When it receives the event GstRTPRetransmissionRequest from the downstream
-gstrtpsession element, it loopkup the requested seqnum in its stored packets.
-If the packet is present in its history, it will create a RTX packet according
-to RFC 4588. Then this rtx packet is pushed to its src pad as other packets.
-
-rtprtxsend works in SSRC-multiplexed mode, so it has one always sink and
-src pad.
-
--- building retransmission packet fron original packet
-
-A rtx packet is mostly the same as an orignal packet, except it has its own
-ssrc and its own seqnum. That's why rtprtxsend works in SSRC-multiplexed mode.
-It also means that the same session is used.
-Another difference between rtx packet and its original is that it inserts the
-original seqnum (OSN: 2 bytes) at the beginning of the payload.
-Also rtprtxsend builds rtx packet without padding, to let other elements do that.
-The last difference is the payload type. For now the user has to set it through
-the rtx-payload-type property. Later it will be automatically retreive this
-information from SDP. See fmtp field as specifies in the RPC4588
-(a=fmtp:99 apt=98) fmtp is the payload type of the retransmission stream
-and apt the payload type of its associated master stream.
-
--- restransmission ssrc and seqnum
-
-To choose rtx_ssrc it randomly selects a number between 0 and 2^32-1 until
-it is different than master_ssrc.
-rtx_seqnum is randomly selected between 0 and 2^16-1
-
--- deeper in the stored buffer history
-
-For the history it uses a GSequence with 2^15-1 as its maximum size.
-Which is resonable as the default value is 100.
-It contains the packets in reverse order they have been sent
-(head:newest, tail:oldest)
-GSequence allows to add and remove an element in constant time (like a queue).
-Also GSequence allows to do a binary search when rtprtxsend lookup in its
-history.
-It's important if it receives a lot of requests or if the history is large.
-
--- pending rtx packets
-
-When looking up in its history, if seqnum is found then it pushes the buffer
-into a GQueue to its tail.
-Before to send the current master stream packet, rtprtxsend sends all the
-buffers which are in this GQueue. Taking care of converting them to rtx
-packets.
-This way, rtx packets are sent in the same order they have been requested.
-(g_list_foreach traverse the queue from head to tail)
-The GQueue is cleared between sending 2 master stream packets.
-So for this GQueue to contain more than one element, it means that rtprtxsend
-receives more than one rtx request between sending 2 master packets.
-
--- collision
-
-When handling a GstRTPCollision event, if the ssrc is its rtx ssrc then
-rtprtxsend clear its history and its pending retransmission queue.
-Then it chooses a rtx_ssrc until it's different than master ssrc.
-If the GstRTPCollision event does not contain its rtx ssrc, for example
-its master ssrc or other, then it just forwards the event to upstream.
-So that it can be handled by the rtppayloader.
-
-
-rtprtxreceive element
-------------------
-
--- basic mechanism
-
-The same rtprtxreceive instance can receive several master streams and several
-retransmission streams.
-So it will try to dynamically associate a rtx ssrc with its master ssrc.
-So that it can reconstruct the original from the proper rtx packet.
-
-The algorithm is based on the fact that seqnums of different streams
-(considering all master and all rtx streams) evolve at a different rate.
-It means that the initial seqnum is random for each one and the offset could
-also be different. So that they are statistically all different at a given
-time. If bad luck then the association is delayed to the next rtx request.
-
-The algorithm also needs to know if a given packet is a rtx packet or not.
-To know this information there is the rtx-payload-types property. For now the
-user as to configure it but later it will be automatically retreive this
-information from SDP.
-It needs to know if the current packet is rtx or not in order to know if
-it can extract the OSN from the payload. Otherwise it would extract the OSN
-even on master streams which means nothing and so it could do bad things.
-In theory maybe it could work but we have this information in SDP so why not
-using it to avoid bad associations.
-
-Note that it also means that several master streams can have the same payload
-type. And also several rtx streams can have the same payload type.
-So the information from SDP which gives us which rtx payload type belong to
-a give master payload type is not enough to do the association between rtx ssrc
-and master ssrc.
-
-rtprtxreceive works in SSRC-multiplexed mode, so it has one always sink and
-src pad.
-
--- deeper in the association algorithm
-
-When it receives a GstRTPRetransmissionRequest event it will remember the ssrc
-and the seqnum from this request.
-
-On incoming packets, if the packet has its ssrc already associated then it
-knows if the ssrc is an rtx ssrc or a master stream ssrc.
-If this is a rtx packet then it recontructs the original and pushs the result to
-src pad as if it was a master packet.
-
-If the ssrc is not yet associated rtprtxreceive checks the payload type.
-if the packet has its payload type marked as rtx then it will extract the OSN
-(original seqnum number) and lookup in its stored requests if a seqnum matchs.
-If found, then it associates the current ssrc to the master ssrc marked in the
-request. If not found it just drops the packet.
-Then it removes the request from the stored requests.
-
-If there are 2 requests with the same seqnum and different ssrc, then the
-couple seqnum,ssrc is removed from the stored requests.
-A stored request actually means that actually the couple seqnum,ssrc is stored.
-If it's happens the request is droped but it avoids to do bad associations.
-In this case the association is just delayed to the next request.
-
--- building original packet from rtx packet
-
-Header, extensions, payload and padding are mostly the same. Except that the
-OSN is removed from the payload. Then ssrc, seqnum, and original payload type
-are correctly set. Original payload type is actually also stored when the
-rtx request is handled.
projects / platform / upstream / gst-plugins-good.git / commitdiff