--- /dev/null
+
+plans:
+
+ - The primary object that applications deal with is a GstPipeline. A
+ given pipeline is connected to a particular main loop (GMainLoop for
+ glib, etc.). Calls to gst_ functions for objects owned by that
+ pipeline must be done from the context of the pipeline's main loop.
+ Signals fired by elements are marshalled in the pipeline's main
+ loop.
+
+ Notably, this means the gst_ API is not necessarily thread-safe.
+ However, it is safe to operate on different GstPipelines from
+ different threads. This makes it possible, for example, for
+ rhythmbox to play music and gather metadata from different threads
+ using different pipelines. Likewise, it's also possible to do
+ both in the same thread.
+
+ - The primary method of scheduling an element is through a generic
+ 'iterate()' method. The iterate method explicitly tells the core
+ what it is waiting for (a specific time, pads to have available
+ data, etc.), and the core calls the iterate method when these
+ "triggers" happen. GstElement subclasses will be created to
+ emulate 0.8-style get/chain/loop methods. Existing elements will
+ be converted to the new subclasses rather than implement the
+ iterate method directly, unless there is a compelling reason to
+ do so. Iterate implementations are expected not to block, ever.
+
+ Rationale: This makes it possible to create completely non-blocking
+ elements.
+
+ - Scheduling elements will be done in either a threaded or
+ non-threaded way. The idle handler that is called by a pipeline's
+ main loop determines which elements are ready to be iterated
+ (based on their triggers), and puts them into a ready queue. In
+ the non-threaded case, the idle handler then calls the iterate()
+ method on each element in the ready queue. In the threaded case,
+ additional helper threads (which are completely owned by the
+ pipeline) are used to call the iterate methods.
+
+ Note that in the threaded case, elements may not always be run
+ in the same thread.
+
+ Some elements are much easier to write if they run in the same
+ thread as the main loop (i.e., elements that are also GUI widgets).
+ An element flag can be set to make the manager always call the
+ iterate method in the manager context (i.e., in the main loop
+ thread). Also, elements like spider need to make core calls
+ which may not be allowed from other threads.
+
+ Rationale: Doing all bookkeeping in a single thread/context makes
+ the core code _much_ simpler. This bookkeeping takes only a
+ minimal amount of CPU time, less than 5% of the CPU time in a
+ rhythmbox pipeline. There is very little benefit to spreading
+ this over multiple CPUs until the number of CPUs is greater than
+ ~16, and you have _huge_ pipelines. Also, a single-threaded
+ manager significantly decreases the number of locks necessary
+ in the core, decreasing lock contention (if any) and making it
+ easier to understand deadlocks (if any).
+
+ - There are essentially two types of objects/structures. One type
+ includes objects that are derived from GObject, and are passed in
+ function calls similarly to gtk. The other type includes objects
+ (structures, really) that are not reference counted and passed
+ around similar to how GstCaps works in 0.8. That is, functions
+ that take 'const GstCaps *' do not take ownership of the passed
+ object, whereas functions that take 'GstCaps *' do. Similar is
+ true for return values.
+
+ - The concept of GstBuffer from 0.8 will be split into two types.
+ One type will focus solely on holding information pertaining to
+ ownership of a memory area (call this GstMemBuffer), and the
+ other type will focus solely in transfering information between
+ elements (call this GstPipeBuffer). In case you get confused,
+ GstMemBuffers _are not_ transferred between elements, and
+ GstPipeBuffers _do not_ own the memory they point to.
+
+ In general, GstPipeBuffers point to (and reference) a GstMemBuffer.
+ GstMemBuffers are GObjects. GstPipeBuffers are structs, like
+ GstCaps. GstPipeBuffers have timestamps, durations, and flags.
+ GstMemBuffers contain read/write flags. There are no subbuffers
+ for either type, because they are not necessary. Essentially,
+ GstPipeBuffers completely replace the concept of subbuffers.
+
+ (I'd like to continue to use the name GstBuffer for GstPipeBuffers,
+ since its usage is much more common in elements.)
+
+ Rationale: Memory regions need an ultimate owner and reference
+ counting. However, chunks passed around between elements need
+ to be small and efficient. These goals are non-overlapping and
+ conflicting, and thus are inappropriate to be combined in the
+ same class.
+
+ - Core objects should have very few (if any) public fields. This
+ means that accessor macros will all be eliminated and replaced
+ with accessor functions.
+
+ Rationale: This makes it possible to change the core more during
+ an ABI-stable series.
+
+ - Remove pluggable scheduling.
+
+ Rationale: We need one good scheduler. Having multiple schedulers
+ is directly opposed to this goal.
+
+ - 0.8-style element states are split up. One state (AppState)
+ indicates what the application wants the element to be doing,
+ and is completely under the control of the application. The
+ other state (ElementState) indicates what the element is actually
+ doing, and is under control of the element. If the application
+ wants an element to pause, it sets the AppState to PAUSED, and
+ the element eventually changes its ElementState to PAUSED (and
+ fires a signal). If the element has an error or EOS, it sets
+ its ElementState to SOME_STATE and fires a signal, while the
+ AppState remains at PLAYING. The actual number and descriptions
+ of states has not been discussed.
+
+ Rationale: It's pretty obvious that we're mixing concepts for
+ elements states in 0.8.
+
+ - getcaps() methods will be replaced by an element_allowed_caps()
+ field in the pad. The primary reason for this is because
+ renegotiation only needs to happen when circumstances change.
+ This is more easily done by a field in GstPad and notification
+ of peers when this changes.
+
+ Somewhere, there's a document I wrote about completely redoing
+ caps.
+
+
--- /dev/null
+
+
+Properties:
+
+ "template_caps"
+ The union of all possible caps that a pad might have in any
+ condition. Is a class property, and doesn't change over the
+ life of an object. Set by the element during class init.
+
+ "internal_caps"
+ The caps that the element will allow on a pad, given its current
+ state and the external_caps of all the other pads of the element.
+ Set by the element.
+
+ "external_caps"
+ The caps representing what the core will allow on a pad. This is
+ a combination of the peer's allowed caps, and filtering caps, and
+ any in-process negotiation restrictions. Set by the core.
+
+ "negotiated_caps"
+ The format that the element will use. Set by the core.
+
+ "filter_caps"
+ The caps intersected with everything else during negotiation. Set
+ by the application.
+
+ "negotiable"
+ A boolean that indicates that the pad is ready to negotiate. Set
+ by the element.
+
+
+
+Negotiation steps, from the element perspective:
+
+ - During object initialization, an element must set template_caps
+ and allowed_caps for each pad.
+
+ - In response to changes in the internal state of the element or in
+ the peer_allowed_caps of any other pad owned by the element, the
+ element should update allowed_caps of a pad.
+
+ - Pads are initially created with negotiable as FALSE. When an
+ element wants to indicate that negotiation is allowable, it should
+ set negotiable to TRUE.
+
+ - In response to a negotiate signal, an element should copy the caps
+ in "allowed_caps" to "negotiated_caps", and extract any information
+ needed to handle the format. At this point, "allowed_caps" is
+ guaranteed to be fixed.
+
+
+Negotiation steps, from the core perspective:
+
+
+
+
+
+
+
+<ds> I'd like to split it up into manageable "projects"
+<ds> hopefully some of which we can recruit others to do
+<Company> i'd like a decision about if we drop threads
+<ds> in what sense?
+<ds> threads being not required?
+<Company> threads being not part of the API
+<ds> or threads being not used
+<Company> so no thread safety considerations
+<ds> I would like threads to not be part of the API
+<ds> requiring apps/plugins to be written thread-safely is insane
+<Company> the question is this: If someone calls gst_element_set_state (element, state), does he have to marshal it into the correct thread himself or not?
+<ds> no
+<ds> reason being: we already have to deal with subthreads
+<ds> we == the core
+<Company> then you'll have a lot of problems
+<Company> i'd like to just drop subthreads and be done with it
+--> StyXman (~mdione@200.59.114.71) has joined #gstreamer
+<ds> and run everything in one thread?
+<Company> yes
+<Company> schedulers are free to use threads though
+<ds> this is obviously a retreat
+<Company> right
+<ds> but perhaps a strategic one
+<Company> otherwise you have to guarantee that you get no races
+<ds> I've seen this as a viable option for some time
+<Company> there's stuff that just does not work as is currently
+<ds> it always seemed difficult to convince other people that it's the right thing to do
+<ds> for example, wtay
+<Company> like setting xwindow ids, changing states or app-events
+<Company> like seeks
+<-- iain has quit ("leaving")
+<Company> all of this has the typical thread issues currently: it sometimes works and sometimes crashes the app
+<Company> and that's not acceptable to me
+<ds> let's do it
+<Company> if you look at my 0.10 code, I've tried to have GstObjects belong to a "context"
+<ds> this all fits into the idea that I think we should be a lot more like gtk architecturally
+<ds> for example, gtk doesn't have anything like caps negotiation, where elements call each other recursively
+<Company> and have functions ensure they're in the right context or otherwise marshal
+<Company> yes it does
+<ds> ?
+<Company> size requests and allocations
+<ds> but that's more like how I want it -- setting a property and firing a signal telling others that it's updated
+<ds> as i understand it
+<Company> though gtk has the bonus of being a tree which we don't have, we have a directed, maybe cyclic graph
+<Company> and for caps, the graph isn't even directed
+<Company> gtk is far too simple for gst
+<ds> true
+<Company> i can't even come up with an analogy to a video transcoder
+<ds> but we do some relatively simple stuff in a complicated way
+<-- StyXman (~mdione@200.59.114.71) has left #gstreamer
+<Company> like?
+<ds> caps negotiation
+<Company> that's not simple
+<ds> passing around the data should be
+<ds> was it you or thomasvs that wanted to split core headers into app headers and plugin headers?
+<Company> I definitely wanted that
+<Company> app/plugin/registry to be exact
+<ds> registry?
+<ds> for functions that access "internal" stuff?
+<Company> stuff an app normally doesn't touch
+<Company> so we can do something like declare the "app" stuff supported for a longer time
+<ds> oooh, good call
+<ds> I'm sold
+<Company> so, now that threads are gone, what are the biggies I want in 0.10?
+<Company> 1) non-blocking main-loop integrated processing
+<ds> that's rather important if you don't have threads :)
+<Company> right
+<Company> but it works in my gst-01.0 stuff already
+<Company> minus the elements that need to be ported of cause
+<Company> which aren't that many fortunately
+<Company> though some are hard (like gnomevfs), because they have no non-blocking API
+<ds> a subclass of GstElement that fires up a helper thread?
+<ds> (dirty, but it works)
+<Company> probably
+<ds> - autopluggers would be easier to write if there was a core method
+<ds> to do what plugidentities do.
+<Company> no clue how to implement that
+<ds> there's also a note about bins being more involved with negotiation
+<Company> that's the next biggie: 2) fix the negotiation process to allow good autoplugging
+<ds> that was something I thought about last time around
+<Company> bins?
+<ds> that sounds like something I'd work on
+<ds> GstBin
+<Company> the only reason bhins still exist for me is to group elements
+<ds> so that a bin subclass could override how its children are negotiated
+<ds> much like GtkBin, actually
+<ds> and allocation
+<Company> hum
+<Company> how do you solve that issue with links that are only partly inside the bin and partly inside another one?
+<ds> that would be up to the parent (grandparent?)
+<Company> so all links must be in the same toplevel bin?
+<ds> yes
+<ds> isn't that the case currently?
+<ds> probably not checked
+<Company> pfff
+<Company> currently you can do anything
+<ds> heh
+<ds> what about state changes?
+<Company> dunno
+<Company> it's not really important for me
+<Company> apart from the EOS stuff
+<ds> I'd like states to be split into "what state does the app want me to be in" and "what state am I in"
+<ds> with the latter being under control of the element
+<Company> yeah, that'd be nice
+<ds> that way, you don't have multiple people trying to control the state
+<Company> we definitely need to fix EOS
+<Company> i'm not sure how we should handle errors
+<ds> element errors?
+<Company> yeah, if we want to explicitly unset errors or not
+<Company> currently on error the element goes to paused
+<ds> it's easier with actual_state
+<Company> in theory i just set it to PLAYING again and continue?
+<ds> the actual_state goes to PAUSED and fires a signal
+<Company> dunno, maybe actual_state should go to ERROR
+<ds> yeah
+<Company> that'd require gst_element_unset_error though
+<ds> with actual_state, we can add states without thinking too hard
+<Company> but if you can just unset it, why not just do it and continue
+<ds> the app may want to be involved
+<ds> especially to get the error message
+<ds> do we want a separate api section for autopluggers?
+<Company> why would we want that?
+<ds> or make it part of registry
+<Company> maybe s/registry/internal/ or some other name
+<ds> because autopluggers may want to access internal information that we wouldn't expose to apps or plugins
+<ds> could you add gst/gstcompat.c to your arch repo?
+<Company> hm
+<Company> what internal info would an autoplugger need
+<ds> depends
+<ds> whatever replaces plugidentities
+<Company> a simple one (decoder) just needs caps and ranks
+<ds> I meant for negotiation information
+<Company> hm
+<Company> no, i don't want a seperate section for it
+<Company> just put it in "internal", "extended" or whatever
+<ds> ok
+<ds> do we want to plan for the app api to be ABI compatible past 0.10?
+<ds> er, that's silly
+<ds> er, kinda silly
+<Company> API compat
+<Company> you could achieve ABI compat if you expose all objects just as typedefs
+<Company> lemme reboot into linux so i can add gstcompat.c
+<Company> brb
+<-- Company has quit (Read error: 104 (Connection reset by peer))
+--> Company (~Company@pD95D53D9.dip.t-dialin.net) has joined #gstreamer
+--> ploum (~ploum@21-4.CampusNet.ucl.ac.be) has joined #gstreamer
+<-- marv has quit (Read error: 104 (Connection reset by peer))
+<ds> so do you still support caps registration?
+<-- ploum has quit (Client Quit)
+<ds> registering media types, fields, descriptions, and default fixate functions?
+<ds> and validation
+<Company> yes
+<Company> that's an important thing we need
+<Company> otherwise people mess up by either carelessly doing stuff wrong or not understanding what they're supposed to do
+<ds> whose responsibility is it to register these?
+<ds> libgstplugins?
+<ds> a plugin?
+<ds> do we want this info in the registry so that a plugin can automatically be loaded to provide registration for video/x-raw-rgb?
+<Company> haven't really thought about that part yet
+<ds> it's somewhat similar to typefind functions
+<Company> yeah
+--> ultrafunk (~pd@eth779.vic.adsl.internode.on.net) has joined #gstreamer
+<Company> i'd probably have a seperate Factory for caps info
+<Company> and have the system load them when you use the caps info stuff
+<ds> should caps info be manditory?
+<ds> have you synced your repo?
+<Company> i'm trying to, but some commit was interrupted last time...
+<ds> how does one sync a local repository to fd.o?
+<ds> rsync?
+<Company> tla archive-mirror
+<ds> ah
+<Company> see rhythmbox.org => developers for how it works
+<Company> and _never_ press ctrl-c in arch
+<-- Misirlou has quit (Remote closed the connection)
+--> Misirlou (~asdf@c-24-125-118-27.va.client2.attbi.com) has joined #gstreamer
+<ds> is there a reason why a pad might know (and set) allowed_caps, but not be ready to negotiate?
+<Company> i don't know a good one
+<ds> know a bad one?
+<Company> "arch-mirror: unable to acquire revision lock (internal error in archive-pfs.c(pfs_lock_revision))"
+<Company> stupidly ordered code
+<Company> no, don't know one
+<ds> here's a good question:
+<ds> should the core negotiate links in a pipeline that are negotiatable, even though some other links are not negotiatable?
+<ds> this ends up making us negotiate some things multiple times
+<ds> filesrc ! mad ! audioconvert ! audioscale ! osssink
+<Company> the core doesn't know if links are negotiable
+<ds> it will in the future :)
+<Company> sinesrc ! identity ! osssink
+<Company> how does it know if sinesrc ! identity is negotiable?
+<ds> because the allowed_caps on the two pads intersect
+--> marv (~ilovekimm@host-216-76-232-134.hsv.bellsouth.net) has joined #gstreamer
+<ds> I'm pretty sure that I don't want gst_pad_link() to return negotiation errors
+<ds> ever
+<Company> you can't avoid that
+<ds> I want to separate "linkable" from "negotiable"
+<Company> ah, gst_pad_link
+<ds> so you can link videotestsrc ! identity ! osssink
+<Company> i confused it with GstPadLink
+<Company> yeah, i was thinking about that, too
+<ds> vts ! osssink won't link, since the pad templates don't intersect
+<Company> http://web.ics.purdue.edu/~kuliniew/wp/archives/2004/08/07/error-handling/
+<Company> :)
+<ds> right now the vts ! id ! osssink pipeline fails to link
+<Company> yeah
+<Company> and goom ! xvimagesink only fails when xvimagesink is at least READY
+<Company> (assuming your xv can't do RGB)
+
+
+
projects / platform / upstream / gstreamer.git / commitdiff