From aa491d7b9b00981a3dedde4b2dd391a0b8df085c Mon Sep 17 00:00:00 2001 From: Seunghun Lee Date: Thu, 22 May 2025 18:40:15 +0900 Subject: [PATCH] wtz-video-shell: Add documentation Change-Id: Ie23883a332b7e88e980f0d3596513e24a770c2af --- protocol/tizen/wtz-video-shell.xml | 499 ++++++++++++++++++++++++++++- 1 file changed, 485 insertions(+), 14 deletions(-) diff --git a/protocol/tizen/wtz-video-shell.xml b/protocol/tizen/wtz-video-shell.xml index 07ba872..ccccd28 100644 --- a/protocol/tizen/wtz-video-shell.xml +++ b/protocol/tizen/wtz-video-shell.xml @@ -25,36 +25,213 @@ THIS SOFTWARE. - - TODO + + The wtz_video_shell protocol is designed to optimize video output in the + Tizen environment by enabling atomic synchronization between surfaces + handled by different processes. It addresses a common use case where UI + rendering and video streaming are managed by separate clients - typically, a + UI application and a media daemon process. + Traditionally, Tizen applications render video playback using multiple + surfaces: a toplevel wl_surface for UI elements and a wl_subsurface for + video content. While the wl_subsurface interface provides set_sync and + set_desync methods to control synchronization between parent and child + surfaces, these are only valid within a single client context. In + multi-process environments, no protocol-level mechanism exists to + synchronize commits across different clients. The wtz_video_shell protocol + introduces a Wayland-native solution to this problem without requiring any + additional inter-process communication. + + Synchronization is achieved by decoupling responsibilities: the media + process handles buffer updates for the video surface, while the UI + application controls destination rectangle, transform, mapping state and + other visual attributes of video content that need to be synchronized. These + properties are applied atomically with the UI application's surface commit, + ensuring visual consistency between the UI and the video frame. + + To support this model, the protocol introduces two key objects: + wtz_video_exported_viewport and wtz_video_viewport_source. The former is + created from an existing wl_subsurface, allowing reuse of its established + semantics such as position, z-order, and commit synchronization behavior. + Once created, the compositor returns a handle (an opaque string) that can be + safely shared with other clients. The video client can then bind this handle + to a wtz_video_viewport_source, enabling it to stream content that is + rendered under the control of the UI application. + + Although this approach diverges from the original intent of wl_subsurface - + which was scoped to single-client use cases - it avoids the complexity and + overhead of adopting an embedding compositor model, offering a lightweight + alternative that is particularly well-suited to resource-constrained + platforms. + + The wtz_video_shell interface is the global entry point for the + wtz_video_shell protocol. + + A client can primarily use this interface to: + - export a wl_subsurface as a wtz_video_exported_viewport, which exposes + destination size, transformation, mapping state controls. + - import that handle in another client and bind it to a wl_surface to + create a wtz_video_viewport_source, which is responsible for attaching + and committing video buffers. + + This interface separates display-related controls from the source buffer + management, which is handled by wtz_video_viewport_source. This + separation is conceptually derived from the wp_viewport interface in the + Wayland protocol, where both source and destination rectangles are + managed in one place. In contrast, wtz_video_shell assigns these roles + across two interfaces: + + - wtz_video_viewport_source: manages the buffer submission and source + rectangle (via set_source) + - wtz_video_exported_viewport: manages the destination size (via + set_destination) and transform (via set_transform) + + + + This request exports a given wl_subsurface as a + wtz_video_exported_viewport. + + The exported viewport can be used to control the destination size, + transformation, and mapping state of the associated surface. Upon + creation, the compositor assigns a unique handle to the viewport, which + can be shared with another client. + + The other client can then import the handle via get_viewport_source to + attach and display video buffers in sync with the exported viewport's + state. + + The wl_subsurface used for wtz_video_exported_viewport cannot have + child subsurfaces. If the specified wl_subsurface has a child surface, a + child_exists protocol error is raised. + + + This request creates a wtz_video_surface from a given wl_surface and + assigns the wtz_video_surface role to it. + + The assigned role marks the surface as eligible to be used as the target + of a wtz_video_viewport_source. Clients must invoke this request before + attemping to import a viewport handle. + + This interface also serves compatibility with existing implementations + that rely on resource ID-based subsurface creation mechanisms. By + introducing wtz_video_surface as a separate role-bearing object, the + protocol preserves the ability to associate a surface with an external + exported viewport handle while maintaining compatibility with previous + workflows. + + In the long term, this usage is expected to be deprecated in favor of a + more direct flow centered around wtz_video_viewport_source creation. + Clients are encouraged to adopt the new interface where possible. + + If the wl_surface already has a role assigned by another protocol, + the compositor will raise the role protocol error. + + + This request allows a client to retrieve the global unique resource ID + of a wl_surface that was exported as a wtz_video_exported_viewport. + + It supports legacy workflows where wl_subsurface objects were created + based on a known resource ID, typically shared between UI and video + processes via out-of-band communication. This mechanism is preserved to + ease the migration from existing systems to the new wtz_video_shell + protocol. + + When the compositor receives this request with a vaild handle, it emits + the global_resource_id event containing the associated wl_surface's + globally unique ID. + + This request is intended for compatibility and will likely be deprecated + in the future revisions as clients migrate to using + wtz_video_viewport_source-based workflows. + + + This event is sent in response to a get_global_resource_id_from_handle + request. It delivers the globally unique resource ID of the wl_surface + that was exported as a wtz_video_exported_viewport. + + If the given handle is invalid or no matching exported viewport exists, + the compositor will send 0 as the ID, indicating an invalid result. + + + The wtz_video_exported_viewport interface allows a client to control the + mapping state, transform, and destination size of a video surface. These + properties affect how the content submitted via a + wtz_video_viewport_source is composed and displayed by the compositor. + + A wtz_video_exported_viewport is considered mapped when both of the + following conditions are met: + + - The associated wl_subsurface is mapped, + - The client has issued the wtz_video_exported_viewport.map request + + Nomally in Wayland, a wl_subsurface is considered mapped only after a + wl_buffer is attached to its wl_surface. However, in this protocol, + buffer submission to the wl_surface used for wtz_video_exported_viewport + is undefined behavior. Submitting buffer to the wl_surface may result in + unexpected behavior. + Instead, mapping is explicitly controlled through the map request of this + interface. In other words, the wtz_video_exported_viewport.map request + effectively replaces the usual "buffer attach" condition for determining + whether the wl_subsurface is mapped. + + Submitting a wl_buffer directly to a surface used for + wtz_video_exported_viewport is currently undefined. This may be defined + in the future to support filling letterboxed areas when using + wtz_video_viewport_source.set_aspect_ratio. + + When a destination size is specified, the size of the associated + wtz_video_viewport_source's wl_surface becomes exactly dst_width and + dst_height and the buffer source is scaled accordingly. If no + destination is set, the surface size becomes the size of the source + rectangle. If that too is unset, the buffer size determines the surface + size. + + Transform is applied before crop-and-scale operations in this order: + + 1. transform + 2. crop and scale + + Position and z-order among sibling wl_subsurfaces are inherited from the + wl_subsurface used during export. Committing behavior follows the sync + mode of the wl_subsurface. See the wl_subsurface specification for more + details. + + The wtz_video_exported_viewport must be destroyed before the associated + wl_subsurface. Calling requests on this object after its wl_subsurface + is destroyed will raise a no_subsurface protocol error. + + Adding child wl_subsurfaces on the wl_subsurface used for this object is + undefined. Attempting to create a child wl_subsurface results in a + child_added protocol error. + + @@ -70,56 +247,224 @@ - + + This event provides a unique, opaque handle string for the exported + viewport. It is sent once, immediately after the object is created. + + The handle can be passed to another client and used with + wtz_video_surface.get_viewport_source to bind a + wtz_video_viewport_source to a wl_surface. This enables video content to + be streamed by another process under the control of this exported + viewport. + + The handle remains valid for the lifetime of this object. Once the + wtz_video_exported_viewport is destroyed, the handle must be considered + invalid. + + - + + This request destroys the exported viewport object and invalidates the + associated handle. + + If a wtz_video_viewport_source is currently bound to this exported + viewport, the connection is broken and the corresponding wl_surface will + be immediately unmapped. The compositor will also emit a + wtz_video_viewport_source.viewport_destroyed event to notify the video + client that the exported viewport has been destroyed. + + After this request, the handle previously emitted via the + wtz_video_exported_viewport.handle event becomes invalid. If a client + attempts to create a wtz_video_viewport_source using this handle, the + compositor will send wtz_video_viewport_source.viewport_destroyed event, + indicating that the exported viewport no longer exists. + + The wl_subsurface associated with this object is not destroyed by this + request and remains valid. - + + This request sets the destination size of the video content in surface + coordinates. The sepcified width and height define the final display + size of the video buffer submitted by the associated + wtz_video_viewport_source. + + If both width and height are set to -1, the previously specified + destination size is cleared. In this case, the size of the video content + will match either the source size set via + wtz_video_viewport_source.set_source, or the size of the submitted + buffer if no source rectangle is specified. + + If either width or height is set to 0 or a negative value other than -1, + the compositor will raise a bad_value protocol error. + + The destination size is double-buffered, and changes take effect on the + next commit of the associated wl_surface. For more information on + double-buffered state, refer to the wl_surface.commit and wl_subsurface + specifications. - - + + - + + This request sets the rotation transform to be applied to the video + content submitted via the associated wtz_video_viewport_source. + + The transform determines how the video buffer is rotated when rendered + on screen. The rotation is applied in the coordinate space of the + compositor's global coordinate system, not the physical orientation of + the output device. + + Using this request in combination with wl_surface.set_buffer_transform + is undefined and may result in unexpected behavior. + + This transform is double-buffered and takes effect on the next commit of + the associated wl_surface - + - + + This request marks the exported viewport as visible, allowing the + associated video content to be shown on screen, provided other + conditions are also met. + + A wtz_video_exported_viewport is considered visible (i.e. mapped) when: + + - The associated wl_subsurface is mapped. + - The client has explicitly called this map request. + + However, the actual visibility on screen also depends on the condition + of associated wtz_video_viewport_source. For example, if no buffer has + been attached and committed to the source, nothing will be shown even if + this exported viewport is mapped. + + Note that unlike regular wl_subsurface objects, this exported viewport + does not require a buffer to be attached directly. The map request + replaces that requirement in determining the viewport's visibility + state. + + Calling this request multiple times has no additional effect. + + The visibility state is double-buffered and applies on the next + wl_surface.commit of the associated wl_subsurface. - + + This request hides the exported viewport, preventing the associated + video content from being displayed on screen. + + Like the map request, this state is double-buffered and takes effect on + the next commit of the associated wl_surface. + + The wtz_video_surface interface is used to assign the video surface role + to a wl_surface. This role is required before the surface can be used to + create a wtz_video_viewport_source. + + This object serves as a management point for video-related behaviors on + the surface. It allows clients to: + + - retrieve a viewport source using an exported handle, + - create a wl_subsurface for backward compatibility, + - set metadata such as a human-readable name, + - Set or unset the surface as a stand-alone mode. + + Once this role is assigned, the wl_surface must not be given any other + role. Attempting to do so will result in wtz_video_shell.role protocol + error. + + The wtz_video_surface is intended to support existing use cases where + clients manage video rendering via resource ID-based wl_subsurface + creation. However, as the protocol evolves, use of + wtz_video_viewport_source is encouraged, and reliance on get_subsurface + reuqest may be deprecated over time. + + This interface may be used exclusively for wtz_video_viewport_source or + wl_subsurface. + + This interface must be destroyed before the associated wl_surface. Calling + requests on this object after its wl_surface is destroyed will raise a + no_surface protocol error. + + - + + + Destroy the wtz_video_surface object and release the role assigned to + the associated wl_surface. After this request, the wl_surface no longer + has the wtz_video_surface role. + + + This request creates a wtz_video_viewport_source object on the + wtz_video_surface by importing an exported viewport via its handle. + + The handle must refer to a valid and currently active + wtz_video_exported_viewport. If the handle is invalid or refers to an + exported viewport that has already been destroyed, the newly created + wtz_video_viewport_source will receive a viewport_destroyed event. + + This request fails with the following protocol errors: + + - role: if the surface is already in use as a + wl_subsurface through get_subsurface. + - no_surface: if the wl_surface associated with this + wtz_video_surface has already been destroyed. + - handle_already_used: if the provided handle is already in use by + another video source. + - viewport_exists: if the wl_surface is currently using another kind of + viewport, such as wl_viewport or tizen_viewport. + + Upon successful creation, the wtz_video_viewport_source becomes + responsible for attaching and committing video buffers that will be + transformed and scaled according to the associated + wtz_video_exported_viewport. + + + This request creates a wl_subsurface for the wl_surface associated with + this wtz_video_surface with given parent wl_surface. + + This request is provided for compatibility with legacy clients that rely + on global resource ID-based wl_subsurface creation. It is not intended + for use in new clients, and its usage is expected to be deprecated in + favor of wtz_video_viewport_source. + + The wl_subsurface created through this request inherits the traditional + behavior defined in the wl_subcompositor protocol. + + If the surface is already in use as a wtz_video_viewport_source through + get_viewport_source, the role protocol error is raised. Once this + request is issued, the wtz_video_surface object holds the role of a + wl_subsurface, and any further attempt to create a viewport source on it + will result in a role protocol error. + + + This request sets a human-readable name for the wtz_video_surface + object. + + The name is intended solely for debugging purposes. Compositors may use + the provided name in logs. + + The name does not affect behavior or rendering in any way, and there is + no guarantee that the compositor will use or display it. + + If called multiple times, the most recent name replaces the previous + one. + + + This request enables legacy standalone mode for the wtz_video_surface. + + When enabled, the associated video surface will no longer be implicitly + unmapped when its wtz_video_exported_viewport or wl_subsurface parent + becomes unmapped. This allows the video surface to remain visible on + screen even if its parent surface is hidden. + + Note that this mode only prevents unmapping due to the parent being + unmapped. It does not cause the video surface to become mapped + automatically if it was previously unmapped. Clients must still + explicitly map the surface through the appropriate protocol mechanisms. + + This feature exists solely for compatibility with legacy behavior and is + not recommended for use in new designs. + + + This request disables standalone mode previously enabled by + set_stand_alone. Once disabled, the video surface will return to default + behavior and may be implicitly unmapped when its parent surface is + unmapped. + + This feature exists solely for compatibility with legacy behavior and is + not recommended for use in new designs. + + + The wtz_video_viewport_source interface represents the source content + that will be displayed through a wtz-video_exported_viewport. + + This interface is associated with a wl_surface and provides control over + the region of the submitted buffer that should be displayed (i.e., the + source rectangle), as well as the ability to maintain a specific aspect + ratio when scaling to the destination size. + + The surface will be considered mapped and therefore visible when the + following conditions are met: + + - The associated wtz_video_exported_viewport is in the mapped state. + - A non-null buffer is attached to the wl_surface. + + The final output on screen is determined by the source rectangle set by + this interface, transformed by the transform setting of the + wtz_video_exported_viewport, and scaled into the destination size also + specified by the exported viewport. + + + + Destroy the wtz_video_viewport_source object. + + Once destroyed, the wl_surface will be immediately unmapped, and its + content will no longer be displayed via the associated + wtz_video_exported_viewport. + @@ -148,6 +559,23 @@ + Define the source rectangle of the buffer attached to the wl_surface + associated with this wtz_video_viewport_source. + + The source rectangle is specified by (x, y, width, height) in + surface-local buffer coordinates. The content within this rectangle will + be used as the input for rendering. The rectangle will be rotated and + scaled based on the associated wtz_video_exported_viewport + configuration. + + If all values are set to -1, the source rectangle is unset, and the + entire buffer will be used as the source area. + + If x or y are negative, or if width or height are zero or negative, the + bad_value protocol error is raised. + + If this request is sent after the associated wl_surface has been + destroyed, the no_surface protocol error is raised. @@ -156,12 +584,55 @@ + + This request specifies the desired aspect ratio for rendering the video + content. The provided integer values (width and height) represent the + intended aspect ratio. + + When a valid aspect ratio is set, the compositor will scale and render + the content within the destination area defined by the associated + wtz_video_exported_viewport, preserving the given aspect ratio. + + The remaining area, if any, is left empty (letterboxed). The visual + result of the empty area is currently undefined and left to the + discretion of the compositor implementation. + + Although attaching a buffer to the wl_subsurface used as the + wtz_video_exported_viewport is currently undefined behavior, it may be + considered in the future as a way to fill the letterboxed area with + custom content or background. + + It both width and height are set to -1, the previously set aspect ratio + is cleared, and the content is scaled to exactly match the destination + size, even if that results in a stretched or squashed image. + + If either width or height is zero or negative, the bad_value protocol + error is raised. + + If this request is sent after the associated wl_surface has been + destroyed, the no_surface protocol error is raised. + + + This event is sent when the wtz_video_viewport_source is no longer + associated with a valid wtz_video_exported_viewport. + + This can happen if: + + - The exported viewport has been destroyed. + - An invalid or expired handle was used during the get_viewport_source + request. + + Upon receiving this event, the client should treat this + wtz_video_viewport_source as disconnected. The wl_surface associated + with this object will be immediately unmapped. + + The client may choose to destroy this object after receiving this event. + - -- 2.34.1