From f9f59df2caff87ae6c5e138ca81bc1819f8518bf Mon Sep 17 00:00:00 2001 From: David Schleef Date: Fri, 10 Dec 2004 00:56:12 +0000 Subject: [PATCH] more notes Original commit message from CVS: more notes --- docs/random/ds/0.9-planning2 | 129 +++++++++++++++++ docs/random/ds/0.9-suggested-changes | 2 +- docs/random/ds/negotiation | 268 +++++++++++++++++++++++++++++++++++ 3 files changed, 398 insertions(+), 1 deletion(-) create mode 100644 docs/random/ds/0.9-planning2 create mode 100644 docs/random/ds/negotiation diff --git a/docs/random/ds/0.9-planning2 b/docs/random/ds/0.9-planning2 new file mode 100644 index 0000000..c38ce12 --- /dev/null +++ b/docs/random/ds/0.9-planning2 @@ -0,0 +1,129 @@ + +plans: + + - The primary object that applications deal with is a GstPipeline. A + given pipeline is connected to a particular main loop (GMainLoop for + glib, etc.). Calls to gst_ functions for objects owned by that + pipeline must be done from the context of the pipeline's main loop. + Signals fired by elements are marshalled in the pipeline's main + loop. + + Notably, this means the gst_ API is not necessarily thread-safe. + However, it is safe to operate on different GstPipelines from + different threads. This makes it possible, for example, for + rhythmbox to play music and gather metadata from different threads + using different pipelines. Likewise, it's also possible to do + both in the same thread. + + - The primary method of scheduling an element is through a generic + 'iterate()' method. The iterate method explicitly tells the core + what it is waiting for (a specific time, pads to have available + data, etc.), and the core calls the iterate method when these + "triggers" happen. GstElement subclasses will be created to + emulate 0.8-style get/chain/loop methods. Existing elements will + be converted to the new subclasses rather than implement the + iterate method directly, unless there is a compelling reason to + do so. Iterate implementations are expected not to block, ever. + + Rationale: This makes it possible to create completely non-blocking + elements. + + - Scheduling elements will be done in either a threaded or + non-threaded way. The idle handler that is called by a pipeline's + main loop determines which elements are ready to be iterated + (based on their triggers), and puts them into a ready queue. In + the non-threaded case, the idle handler then calls the iterate() + method on each element in the ready queue. In the threaded case, + additional helper threads (which are completely owned by the + pipeline) are used to call the iterate methods. + + Note that in the threaded case, elements may not always be run + in the same thread. + + Some elements are much easier to write if they run in the same + thread as the main loop (i.e., elements that are also GUI widgets). + An element flag can be set to make the manager always call the + iterate method in the manager context (i.e., in the main loop + thread). Also, elements like spider need to make core calls + which may not be allowed from other threads. + + Rationale: Doing all bookkeeping in a single thread/context makes + the core code _much_ simpler. This bookkeeping takes only a + minimal amount of CPU time, less than 5% of the CPU time in a + rhythmbox pipeline. There is very little benefit to spreading + this over multiple CPUs until the number of CPUs is greater than + ~16, and you have _huge_ pipelines. Also, a single-threaded + manager significantly decreases the number of locks necessary + in the core, decreasing lock contention (if any) and making it + easier to understand deadlocks (if any). + + - There are essentially two types of objects/structures. One type + includes objects that are derived from GObject, and are passed in + function calls similarly to gtk. The other type includes objects + (structures, really) that are not reference counted and passed + around similar to how GstCaps works in 0.8. That is, functions + that take 'const GstCaps *' do not take ownership of the passed + object, whereas functions that take 'GstCaps *' do. Similar is + true for return values. + + - The concept of GstBuffer from 0.8 will be split into two types. + One type will focus solely on holding information pertaining to + ownership of a memory area (call this GstMemBuffer), and the + other type will focus solely in transfering information between + elements (call this GstPipeBuffer). In case you get confused, + GstMemBuffers _are not_ transferred between elements, and + GstPipeBuffers _do not_ own the memory they point to. + + In general, GstPipeBuffers point to (and reference) a GstMemBuffer. + GstMemBuffers are GObjects. GstPipeBuffers are structs, like + GstCaps. GstPipeBuffers have timestamps, durations, and flags. + GstMemBuffers contain read/write flags. There are no subbuffers + for either type, because they are not necessary. Essentially, + GstPipeBuffers completely replace the concept of subbuffers. + + (I'd like to continue to use the name GstBuffer for GstPipeBuffers, + since its usage is much more common in elements.) + + Rationale: Memory regions need an ultimate owner and reference + counting. However, chunks passed around between elements need + to be small and efficient. These goals are non-overlapping and + conflicting, and thus are inappropriate to be combined in the + same class. + + - Core objects should have very few (if any) public fields. This + means that accessor macros will all be eliminated and replaced + with accessor functions. + + Rationale: This makes it possible to change the core more during + an ABI-stable series. + + - Remove pluggable scheduling. + + Rationale: We need one good scheduler. Having multiple schedulers + is directly opposed to this goal. + + - 0.8-style element states are split up. One state (AppState) + indicates what the application wants the element to be doing, + and is completely under the control of the application. The + other state (ElementState) indicates what the element is actually + doing, and is under control of the element. If the application + wants an element to pause, it sets the AppState to PAUSED, and + the element eventually changes its ElementState to PAUSED (and + fires a signal). If the element has an error or EOS, it sets + its ElementState to SOME_STATE and fires a signal, while the + AppState remains at PLAYING. The actual number and descriptions + of states has not been discussed. + + Rationale: It's pretty obvious that we're mixing concepts for + elements states in 0.8. + + - getcaps() methods will be replaced by an element_allowed_caps() + field in the pad. The primary reason for this is because + renegotiation only needs to happen when circumstances change. + This is more easily done by a field in GstPad and notification + of peers when this changes. + + Somewhere, there's a document I wrote about completely redoing + caps. + + diff --git a/docs/random/ds/0.9-suggested-changes b/docs/random/ds/0.9-suggested-changes index 6008ab8..ca131e4 100644 --- a/docs/random/ds/0.9-suggested-changes +++ b/docs/random/ds/0.9-suggested-changes @@ -145,7 +145,6 @@ object hierarchy: - all bitrates should be bits per second, since they're specified as integers - - categorization of plugins: - categories were intended from the start as a list of keywords, instead of an implicit tree. @@ -158,3 +157,4 @@ object hierarchy: - optimization: - delay parsing caps from the registry until after startup (until they're first requested or maybe an idle handler for the main loop) + diff --git a/docs/random/ds/negotiation b/docs/random/ds/negotiation new file mode 100644 index 0000000..7356a93 --- /dev/null +++ b/docs/random/ds/negotiation @@ -0,0 +1,268 @@ + + +Properties: + + "template_caps" + The union of all possible caps that a pad might have in any + condition. Is a class property, and doesn't change over the + life of an object. Set by the element during class init. + + "internal_caps" + The caps that the element will allow on a pad, given its current + state and the external_caps of all the other pads of the element. + Set by the element. + + "external_caps" + The caps representing what the core will allow on a pad. This is + a combination of the peer's allowed caps, and filtering caps, and + any in-process negotiation restrictions. Set by the core. + + "negotiated_caps" + The format that the element will use. Set by the core. + + "filter_caps" + The caps intersected with everything else during negotiation. Set + by the application. + + "negotiable" + A boolean that indicates that the pad is ready to negotiate. Set + by the element. + + + +Negotiation steps, from the element perspective: + + - During object initialization, an element must set template_caps + and allowed_caps for each pad. + + - In response to changes in the internal state of the element or in + the peer_allowed_caps of any other pad owned by the element, the + element should update allowed_caps of a pad. + + - Pads are initially created with negotiable as FALSE. When an + element wants to indicate that negotiation is allowable, it should + set negotiable to TRUE. + + - In response to a negotiate signal, an element should copy the caps + in "allowed_caps" to "negotiated_caps", and extract any information + needed to handle the format. At this point, "allowed_caps" is + guaranteed to be fixed. + + +Negotiation steps, from the core perspective: + + + + + + + + I'd like to split it up into manageable "projects" + hopefully some of which we can recruit others to do + i'd like a decision about if we drop threads + in what sense? + threads being not required? + threads being not part of the API + or threads being not used + so no thread safety considerations + I would like threads to not be part of the API + requiring apps/plugins to be written thread-safely is insane + the question is this: If someone calls gst_element_set_state (element, state), does he have to marshal it into the correct thread himself or not? + no + reason being: we already have to deal with subthreads + we == the core + then you'll have a lot of problems + i'd like to just drop subthreads and be done with it +--> StyXman (~mdione@200.59.114.71) has joined #gstreamer + and run everything in one thread? + yes + schedulers are free to use threads though + this is obviously a retreat + right + but perhaps a strategic one + otherwise you have to guarantee that you get no races + I've seen this as a viable option for some time + there's stuff that just does not work as is currently + it always seemed difficult to convince other people that it's the right thing to do + for example, wtay + like setting xwindow ids, changing states or app-events + like seeks +<-- iain has quit ("leaving") + all of this has the typical thread issues currently: it sometimes works and sometimes crashes the app + and that's not acceptable to me + let's do it + if you look at my 0.10 code, I've tried to have GstObjects belong to a "context" + this all fits into the idea that I think we should be a lot more like gtk architecturally + for example, gtk doesn't have anything like caps negotiation, where elements call each other recursively + and have functions ensure they're in the right context or otherwise marshal + yes it does + ? + size requests and allocations + but that's more like how I want it -- setting a property and firing a signal telling others that it's updated + as i understand it + though gtk has the bonus of being a tree which we don't have, we have a directed, maybe cyclic graph + and for caps, the graph isn't even directed + gtk is far too simple for gst + true + i can't even come up with an analogy to a video transcoder + but we do some relatively simple stuff in a complicated way +<-- StyXman (~mdione@200.59.114.71) has left #gstreamer + like? + caps negotiation + that's not simple + passing around the data should be + was it you or thomasvs that wanted to split core headers into app headers and plugin headers? + I definitely wanted that + app/plugin/registry to be exact + registry? + for functions that access "internal" stuff? + stuff an app normally doesn't touch + so we can do something like declare the "app" stuff supported for a longer time + oooh, good call + I'm sold + so, now that threads are gone, what are the biggies I want in 0.10? + 1) non-blocking main-loop integrated processing + that's rather important if you don't have threads :) + right + but it works in my gst-01.0 stuff already + minus the elements that need to be ported of cause + which aren't that many fortunately + though some are hard (like gnomevfs), because they have no non-blocking API + a subclass of GstElement that fires up a helper thread? + (dirty, but it works) + probably + - autopluggers would be easier to write if there was a core method + to do what plugidentities do. + no clue how to implement that + there's also a note about bins being more involved with negotiation + that's the next biggie: 2) fix the negotiation process to allow good autoplugging + that was something I thought about last time around + bins? + that sounds like something I'd work on + GstBin + the only reason bhins still exist for me is to group elements + so that a bin subclass could override how its children are negotiated + much like GtkBin, actually + and allocation + hum + how do you solve that issue with links that are only partly inside the bin and partly inside another one? + that would be up to the parent (grandparent?) + so all links must be in the same toplevel bin? + yes + isn't that the case currently? + probably not checked + pfff + currently you can do anything + heh + what about state changes? + dunno + it's not really important for me + apart from the EOS stuff + I'd like states to be split into "what state does the app want me to be in" and "what state am I in" + with the latter being under control of the element + yeah, that'd be nice + that way, you don't have multiple people trying to control the state + we definitely need to fix EOS + i'm not sure how we should handle errors + element errors? + yeah, if we want to explicitly unset errors or not + currently on error the element goes to paused + it's easier with actual_state + in theory i just set it to PLAYING again and continue? + the actual_state goes to PAUSED and fires a signal + dunno, maybe actual_state should go to ERROR + yeah + that'd require gst_element_unset_error though + with actual_state, we can add states without thinking too hard + but if you can just unset it, why not just do it and continue + the app may want to be involved + especially to get the error message + do we want a separate api section for autopluggers? + why would we want that? + or make it part of registry + maybe s/registry/internal/ or some other name + because autopluggers may want to access internal information that we wouldn't expose to apps or plugins + could you add gst/gstcompat.c to your arch repo? + hm + what internal info would an autoplugger need + depends + whatever replaces plugidentities + a simple one (decoder) just needs caps and ranks + I meant for negotiation information + hm + no, i don't want a seperate section for it + just put it in "internal", "extended" or whatever + ok + do we want to plan for the app api to be ABI compatible past 0.10? + er, that's silly + er, kinda silly + API compat + you could achieve ABI compat if you expose all objects just as typedefs + lemme reboot into linux so i can add gstcompat.c + brb +<-- Company has quit (Read error: 104 (Connection reset by peer)) +--> Company (~Company@pD95D53D9.dip.t-dialin.net) has joined #gstreamer +--> ploum (~ploum@21-4.CampusNet.ucl.ac.be) has joined #gstreamer +<-- marv has quit (Read error: 104 (Connection reset by peer)) + so do you still support caps registration? +<-- ploum has quit (Client Quit) + registering media types, fields, descriptions, and default fixate functions? + and validation + yes + that's an important thing we need + otherwise people mess up by either carelessly doing stuff wrong or not understanding what they're supposed to do + whose responsibility is it to register these? + libgstplugins? + a plugin? + do we want this info in the registry so that a plugin can automatically be loaded to provide registration for video/x-raw-rgb? + haven't really thought about that part yet + it's somewhat similar to typefind functions + yeah +--> ultrafunk (~pd@eth779.vic.adsl.internode.on.net) has joined #gstreamer + i'd probably have a seperate Factory for caps info + and have the system load them when you use the caps info stuff + should caps info be manditory? + have you synced your repo? + i'm trying to, but some commit was interrupted last time... + how does one sync a local repository to fd.o? + rsync? + tla archive-mirror + ah + see rhythmbox.org => developers for how it works + and _never_ press ctrl-c in arch +<-- Misirlou has quit (Remote closed the connection) +--> Misirlou (~asdf@c-24-125-118-27.va.client2.attbi.com) has joined #gstreamer + is there a reason why a pad might know (and set) allowed_caps, but not be ready to negotiate? + i don't know a good one + know a bad one? + "arch-mirror: unable to acquire revision lock (internal error in archive-pfs.c(pfs_lock_revision))" + stupidly ordered code + no, don't know one + here's a good question: + should the core negotiate links in a pipeline that are negotiatable, even though some other links are not negotiatable? + this ends up making us negotiate some things multiple times + filesrc ! mad ! audioconvert ! audioscale ! osssink + the core doesn't know if links are negotiable + it will in the future :) + sinesrc ! identity ! osssink + how does it know if sinesrc ! identity is negotiable? + because the allowed_caps on the two pads intersect +--> marv (~ilovekimm@host-216-76-232-134.hsv.bellsouth.net) has joined #gstreamer + I'm pretty sure that I don't want gst_pad_link() to return negotiation errors + ever + you can't avoid that + I want to separate "linkable" from "negotiable" + ah, gst_pad_link + so you can link videotestsrc ! identity ! osssink + i confused it with GstPadLink + yeah, i was thinking about that, too + vts ! osssink won't link, since the pad templates don't intersect + http://web.ics.purdue.edu/~kuliniew/wp/archives/2004/08/07/error-handling/ + :) + right now the vts ! id ! osssink pipeline fails to link + yeah + and goom ! xvimagesink only fails when xvimagesink is at least READY + (assuming your xv can't do RGB) + + + -- 2.7.4