+THE GOAL
+--------
What we are trying to achieve:
satisfy:
make distcheck
non-srcdir build (ie, mkdir build; cd build; ../configure; make)
-How it works:
+THE SETUP
+---------
+There is a "mirror" root CVS module that contains "ffmpeg".
+This directory contains a vendor-branch checkout of upstream FFmpeg CVS
+of a given day.
-* configure checks whether or not it should update ffmpeg from CVS by looking
- at the nano version number
- - if it's 1, we're in cvs mode, and it should check it out
- - if it's not 1, we're in prerel or rel mode, and the code should already
- be on disk
- FIXME: we could change this to really check out the source code if some
- required files aren't there just in case someone checks out from CVS
- but CVS is not at nano 1
+On head, the following things have been commited on top of this:
+* patches/, which is a directory with a set of patches, and a series file
+ listing the order, as generated by quilt
+* the result of having all these patches commited (ie, quilt push -a) to the
+ ffmpeg tree.
-* patching of the checked-out copy happens at
+The patched CVS ffmpeg code needs to be commited to CVS so that a checkout
+gives the patched code
-Axioms under which we work:
-- the dist tarball needs to include either
- - the pristine ffmpeg checkout + our patches + a patch mechanism on make
- or
- - the ffmpeg checkout with patches already applied
+The Quilt state .pc hidden directory must NOT be committed to CVS, because
+having CVS subdirs inside it confuses the hell out of quilt and causes it to
+start storing diffs against CVS Entries files, and all hell breaks loose
-- configure/make is not allowed to touch files that already live in the source
- tree; if they need to then they need to be copied first and cleaned
- afterward
+THE WARNING
+-----------
-- it would be very nice if, on update of either the Tag file or the patch set,
- make would know exactly what to do with it.
+***
+NEVER EVER commit stuff in gst-libs/ext/ffmpeg UNLESS your quilt stack is
+completely applied !
+This means, ALWAYS make sure quilt push -a has been run without problems.
+
+What's more, if you want to be on the safe side, make sure that you can
+unapply and reapply without problems, by running quilt pop -a then
+quilt push -a.
+
+The ONLY exception to this is when you're working on a branch to update
+the upstream source you're working with.
+
+***
+
+THE WAY
+-------
+- If you want to hack on our copy of the FFmpeg code, there are some basic
+ rules you need to respect:
+ - you need to use quilt. If you don't use quilt, you can't hack on it.
+ - we separate patches based on the functionality they patch, and whether
+ or not we want to send stuff upstream. Make sure you work in the right
+ patch. use "quilt applied" to check which patches are applied.
+ - before starting to hack, run cvs diff. There should be NO diffs, and
+ NO files listed with question mark. If there are, somebody before you
+ probably made a mistake. To manage the state correctly, it is vital that
+ none of the files are unknown to CVS.
+
+FIRST TIME:
+ - The quilt state is kept in a hidden dir in the gst-libs/ext/ffmpeg dir,
+ but this hidden dir can't be kept in CVS because it confuses patch. Hence
+ when you get a clean gst-ffmpeg checkout you have an ffmpeg tree with
+ patches applied, but no quilt metadata to modify it with.
+
+ - You need to create the quilt metadata in your checkout:
+ 1) Unroll the quilt patches. In gst-libs/ext/ffmpeg, run:
+
+ tac patches/series | while read p; do patch -p1 -R < "patches/$p"; done
+
+ 2) Now, push all the patches to quilt and it will apply them, but now with
+ the appropriate stored metadata:
+
+ quilt push -a
+
+- if you want to add a file to a patchset, you need to:
+ - be in the right patchset
+ - quilt add (file)
+ - cvs add .pc/(patchsetname)/(file)
+ - cvs commit .pc/(patchsetname) (to update the state of quilt in cvs)
+ - edit the file
+ - cvs add the file if it doesn't exist yet
+ - quilt refresh
+ - quilt push -a (This one is IMPORTANT, otherwise you'll have a huge diff)
+ - cvs commit
+
+- if you want to add a patchset, you need to:
+ - go over the procedure with thomas to check it's correct
+ - decide where in the stack to put it. ask for help if you don't know.
+ - go there in the patch stack (use quilt pop/push)
+ - quilt new (patchsetname).patch (don't forget .patch !)
+ - quilt add (files)
+ - cvs add .pc/(patchsetname) the whole tree
+ - cvs commit .pc/(patchsetname)
+ - quilt refresh
+ - quilt push -a
+ - cvs commit
+ - cvs diff (to check if any of the files are unknown to CVS; if they are,
+ you need to add them to CVS)
+
+THE UPSTREAM
+------------
+At some points you want to update the upstream snapshot code to a newer date.
+This is easy if you follow the steps outlined here, but make sure to follow
+them correctly !
+
+- find a good CVS snapshot date for upstream, one that is known to work.
+ You're going to save yourself quite a bit of trouble if you verify this
+ first !
+- check it out to a local directory:
+ cvs -z9 -d:pserver:anonymous@mplayerhq.hu:/cvsroot/ffmpeg export -D '2004-04-11 23:00 GMT' ffmpeg
+- compile it and test it, make sure it works
+
+- in gst-ffmpeg/gst-libs/ext/ffmpeg:
+ - Pre-flight checks:
+ - first make sure you don't have local changes, all files are either in
+ CVS or in .cvsignore patch, the whole quilt stack is applied, and stuff
+ works.
+ - do a quilt pop -a and quilt push -a to verify everything is ok.
+
+ - Branch and rollback:
+ - tag HEAD with the branch root point:
+ cvs tag BRANCH-UPDATE-CVS-2004-04-11-23-00-ROOT
+ - branch:
+ cvs tag -b BRANCH-UPDATE-CVS-2004-04-11-23-00
+ - FIXME: lock cvs HEAD
+ - update local copy to branch:
+ cvs update -r BRANCH-UPDATE-CVS-2004-04-11-23-00
+ - peel off all patches:
+ quilt pop -a
+ - commit this
+ cvs commit
+ - check
+ cvs diff
+ you should only have ? for files that are generated somehow (binaries,
+ build files, ...)
+ you get warnings about cvs not finding files to diff that are in .pc
+ or generated by your patches
+
+
+ - if you want, you can now compare this state of CVS (which should
+ be last upstream CVS combined with your local unapplied quilt state)
+ - remember to NOT do cvs update from here on, since you popped your quilt
+ state all your added files that are also in CVS are not locally present.
+
+ - sync with upstream:
+ - in a temp dir, redo the export:
+ cd ..
+ mkdir tmp
+ cd tmp
+ cvs -z9 -d:pserver:anonymous@mplayerhq.hu:/cvsroot/ffmpeg export -D '2004-04-11 23:00 GMT' ffmpeg
+ - rsync it over the old ffmpeg tree
+ rsync -arv ffmpeg ..
+ - go back and commit this new snapshot
+ cd ../ffmpeg
+ cvs commit
+ - check if any new files got added that you should add to cvs
+ cvs diff
+ This will list a lot of local files missing, from your quilt state,
+ which you shouldn't worry about. Just inspect all the ?'s and add
+ files to cvs that belong to upstream and should be in cvs.
+ - if everything's ok and commited, tag the state:
+ cvs tag UPSTREAM-CVS-2004-04-11-23-00
+
+ - reapply and fix quilt patches one by one
+ - try applying one
+ quilt push
+ - if that didn't work, inspect the patch and figure out how to fix it:
+ - if the patch got applied upstream completely, quilt push will tell
+ you the patch looks like a reverse patch. In that case you can
+ remove the patch from your patches file (patches/series), and
+ remove the .pc/$(patchname) and patches/$(patchname).patch files from
+ cvs.
+ - if the patch conflicts somehow, you can force application with
+ quilt push -f
+ and then resolve all the rejects, and fix the patch completely.
+ Then refresh quilt state with
+ quilt refresh
+ - when the patch is succesfully removed or reworked, commit current state
+ to CVS, then check again if nothing is missing using cvs diff, and
+ resolve problems/apply missing files from your quilt state/...
+
+ - after reapplying your complete quilt state, test locally if the complete
+ gst-ffmpeg module now works. Compile and test. Resolve all problems
+ (upstream errors, missing symbols, missing files, ...) until you have
+ a working module. commit again.
+
+ - merge to head:
+ - update locally back to head
+ cvs update -A
+ - FIXME: unlock cvs HEAD
+ - merge from your branch
+ cvs update -j BRANCH-UPDATE-CVS-2004-04-11-23-00
+ - commit
+ cvs commit
+ - check for diffs
+ cvs diff
+ - tag merge point
+ cvs tag BRANCH-UPDATE-CVS-2004-04-11-23-00-MERGE
+ - add upstream date to "THE RECORDS" below
+
+ - get a drink
+
+THE PLUGIN
+----------
+Some notes on how ffmpeg wrapping inside GStreamer currently works:
+* gstffmpeg{dec,enc,demux,mux}.c are wrappers for specific element types from
+ their ffmpeg counterpart. If you want to wrap a new type of element in
+ wrapper file.
+ The ffmpeg element types, define a whole *list* of elements (in
+ GStreamer, each decoder etc. needs to be its own element).
+ We use a set of tricks for that to keep coding simple: codec
+ mapping and dynamic type creation.
+
+* ffmpeg uses CODEC_ID_* enumerations for their codecs. GStreamer uses caps,
+ which consists of a mimetype and a defined set of properties. In ffmpeg,
+ these properties live in a AVCodecContext struct, which contains anything
+ that could configure any codec (which makes it rather messy, but ohwell).
+ To convert from one to the other, we use codec mapping, which is done in
+ gstffmpegcodecmap.[ch]. This is the most important file in the whole
+ ffmpeg wrapping process! It contains functions to go from a codec type
+ (video or audio - used as the output format for decoding or the input
+ format for encoding), a codec id (to identify each format) or a format id
+ (a string identifying a file format - usually the file format extension)
+ to a GstCaps, and the other way around.
+
+* to define multiple elements in one source file (which all behave similarly),
+ we dynamically create types for each plugin and let all of them operate on
+ the same struct (GstFFMpegDec, GstFFMpegEnc, ...). The functions in
+ gstffmpeg{dec,enc,demux,mux}.c called gst_ffmpeg*_register() do this.
+ The magic is as follows: for each codec or format, ffmpeg has a single
+ AVCodec or AV{Input,Output}Format, which are packed together in a list of
+ supported codecs/formats. We simply walk through the list, for each of
+ those, we check whether gstffmpegcodecmap.c knows about this single one.
+ If it does, we get the GstCaps for each pad template that belongs to it,
+ and register a type for all of those together. We also leave this inside
+ a caching struct, that will later be used by the base_init() function to
+ fill in information about this specific codec in the class struct of this
+ element (pad templates and codec/format information). Since the actual
+ codec information is the only thing that really makes each codec/format
+ different (they all behave the same through the ffmpeg API), we don't
+ really need to do anything else that is codec-specific, so all other
+ functions are rather simple.
+
+* one particular thing that needs mention is how gstffmpeg{mux,demux}.c and
+ gstffmpegprotocol.c interoperate. ffmpeg uses URLProtocols for data input
+ and output. Now, of course, we want to use the *GStreamer* way of doing
+ input and output (filesrc, ...) rather than the ffmpeg way. Therefore, we
+ wrap up a GstPad as a URLProtocol and register this with ffmpeg. This is
+ what gstffmpegprotocol.c does. The URL is called gstreamer://%p, where %p
+ is the address of a GstPad. gstffmpeg{mux,demux}.c then open a file called
+ gstreamer://%p, with %p being their source/sink pad, respectively. This
+ way, we use GStreamer for data input/output through the ffmpeg API. It's
+ rather ugly, but it has worked quite well so far.
+
+* there's lots of things that still need doing. See the TODO file for more
+ information.
+
+THE RECORDS
+-----------
+- list of snapshots used:
+
+ CVS-2004-04-11-23-00
+ * other updates people didn't enter :)
+ CVS-2006-02-17-04-00
+
+THE REMINDERS
+-------------
+* the initial ffmpeg checkout was imported using:
+ - get CVS ffmpeg
+ cvs -z3 -d:pserver:anonymous@mplayerhq.hu:/cvsroot/ffmpeg co -D '2004-03-09 06:00 GMT' ffmpeg
projects / platform / upstream / gst-libav.git / blobdiff