From dc8008a9b943ed5a01c08900c9463d6d8ee49e66 Mon Sep 17 00:00:00 2001 From: Jan Schmidt Date: Mon, 28 Apr 2003 22:09:38 +0000 Subject: [PATCH] A few ideas looking for feedback Original commit message from CVS: A few ideas looking for feedback --- docs/random/thaytan/opengl | 20 ++++++++++++++++++++ docs/random/thaytan/video-overlays | 28 ++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) create mode 100644 docs/random/thaytan/opengl create mode 100644 docs/random/thaytan/video-overlays diff --git a/docs/random/thaytan/opengl b/docs/random/thaytan/opengl new file mode 100644 index 0000000..a796629 --- /dev/null +++ b/docs/random/thaytan/opengl @@ -0,0 +1,20 @@ +# OpenGL usage +How to set up openGL using plugins in such a way that they can +render to an on-screen window, or offscreen equally? + +eg: +superfoo3d ! glwindow +or +superfoo3d ! gloffscreen ! xvideowindow + +This would imply that there is some mime type which connects a glwindow/gloffscreen and a GL using component sensibly. I don't see that there is any actual +data to send, however - the only purpose of the glwindow/gloffscreen is to +set up an openGL context and then 'output' it somehow one the superfoo3d has +finished drawing each frame. + +In the case of glwindow, 'output' it means translate into an on-screen window, +but for gloffscreen it means produce a video packet and go. + +These components really need some other 2 way communication, rather than the +pads metaphor, I think? + diff --git a/docs/random/thaytan/video-overlays b/docs/random/thaytan/video-overlays new file mode 100644 index 0000000..af0697b --- /dev/null +++ b/docs/random/thaytan/video-overlays @@ -0,0 +1,28 @@ +# Compositing and video overlays. +We want a way to composite a video frame and an overlay frame. The video frame is as we expect, the overlay is a normal video frame + an alpha mask. In the following monologue, please consider that I mean RGB/I420/YV12/YUY2 wherever I say RGB. + +So we have a plugin with 2 sinks and one src: + +----------------+ + |video | + | video| + |overlay | + +----------------+ + +What should the format for the overlay data look like? +Ideas: + * RGBA - 4 byte per pixel video frame + * RGB+A - 3 Bytes per pixel video frame, then 1 byte per pixel overlay frame. + A new mime type, or a variation on video/raw? I'm not sure + * Overlay is actually 2 sinks, one takes a RGB/YUV data the other the alpha channel. + +I'm not sure which approach is better, but I think it is probably neatest to +use RGB+A, and then have a separate plugin which has 2 sinks and converts an +RGB/YUV stream plus an alpha stream into an RGB+A stream. The benefit of RGB+A over RGBA in this scenario, is that it is easier (correct me if I'm wrong) to optimise 2 block copies which appends an alpha frame to a RGB frame than it is to +do the many copies required to interleave them into an RGBA stream. + +So, I see this producing a few new plugins: +videooverlay - takes an RGB and an RGB+A frame from 2 sinks, does the overlay (according to some properties) and outputs a result frame in RGB or RGB+A (as negotiated) format on 1 src. +rgb2rgba - takes 1 RGB frame and one A frame from 2 sinks and outputs RGB+A on 1 src. If the A sink is not connected, we just add a fixed alpha channel based on an object-property. +rgb2rgba - takes an RGB+A frame and discards the RGB component. +textoverlay - This plugin, instead of taking a video frame and overlaying text, can just output an RGB+A stream with appropriate timestamping. This prevents duplicating code to composite with an alpha mask and allows us to optimise it in one spot only. + -- 2.7.4