From 39f5db73e253ffa6288be50b2ea5778b0464eed3 Mon Sep 17 00:00:00 2001 From: =?utf8?q?Kristian=20H=C3=B8gsberg?= Date: Thu, 10 Jun 2010 13:05:48 -0400 Subject: [PATCH] Add drag and drop section to spec --- spec/main.tex | 278 ++++++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 222 insertions(+), 56 deletions(-) diff --git a/spec/main.tex b/spec/main.tex index 2adf296..8210a88 100644 --- a/spec/main.tex +++ b/spec/main.tex @@ -13,9 +13,10 @@ \section{Wayland Overview} - - wayland is a protocol for a new display server. - - - wayland is an implementation +\begin{itemize} +\item wayland is a protocol for a new display server. +\item wayland is an implementation +\end{itemize} \subsection{Replacing X11} @@ -68,28 +69,33 @@ the type of event. Events are generated both in repsonse to a request (in which case the request and the event constitutes a round trip) or spontanously when the server state changes. - - state is broadcast on connect, events sent out when state - change. client must listen for these changes and cache the state. - no need (or mechanism) to query server state. +\begin{itemize} +\item state is broadcast on connect, events sent out when state + change. client must listen for these changes and cache the state. + no need (or mechanism) to query server state. - - server will broadcast presence of a number of global objects, - which in turn will broadcast their current state +\item server will broadcast presence of a number of global objects, + which in turn will broadcast their current state +\end{itemize} \subsection{Connect Time} - - no fixed format connect block, the server emits a bunch of events - at connect time - - - presence events for global objects: output, compositor, input devices - +\begin{itemize} +\item no fixed format connect block, the server emits a bunch of + events at connect time +\item presence events for global objects: output, compositor, input + devices +\end{itemize} \subsection{Security and Authentication} - - mostly about access to underlying buffers, need new drm auth - mechanism (the grant-to ioctl idea), need to check the cmd stream? +\begin{itemize} +\item mostly about access to underlying buffers, need new drm auth + mechanism (the grant-to ioctl idea), need to check the cmd stream? - - getting the server socket depends on the compositor type, could be - a system wide name, through fd passing on the session dbus. or the - client is forked by the compositor and the fd is already opened. +\item getting the server socket depends on the compositor type, could + be a system wide name, through fd passing on the session dbus. or + the client is forked by the compositor and the fd is already opened. +\end{itemize} \subsection{Creating Objects} @@ -124,7 +130,7 @@ created by the client global object \begin{itemize} -\item - input group, keyboard, mouse +\item input group, keyboard, mouse \item keyboard map, change events \item pointer motion \item enter, leave, focus @@ -135,39 +141,192 @@ global object \subsection{Output} - - global objects - - a connected screen - - laid out in a big coordinate system - - basically xrandr over wayland +\begin{itemize} +\item global objects +\item a connected screen +\item laid out in a big coordinate system +\item basically xrandr over wayland +\end{itemize} + +\subsection{Drag and Drop} + +Multi-device aware. Orthogonal to rest of wayland, as it is its own +toplevel object. Since the compositor determines the drag target, it +works with transformed surfaces (dragging to a scaled down window in +expose mode, for example). + +Issues: + +\begin{itemize} +\item we can set the cursor image to the current cursor + dragged + object, which will last as long as the drag, but maybe an request to + attach an image to the cursor will be more convenient? + +\item Should drag.send() destroy the object? There's nothing to do + after the data has been transferred. + +\item How do we marshall several mime-types? We could make the drag + setup a multi-step operation: dnd.create, drag.offer(mime-type1, + drag.offer(mime-type2), drag.activate(). The drag object could send + multiple offer events on each motion event. Or we could just + implement an array type, but that's a pain to work with. + +\item Middle-click drag to pop up menu? Ctrl/Shift/Alt drag? + +\item Send a file descriptor over the protocol to let initiator and + source exchange data out of band? + +\item Action? Specify action when creating the drag object? Ask + action? +\end{itemize} + +New objects, requests and events: + + - New toplevel dnd global. One method, creates a drag object: + + dnd.start(new object id, surface, input device, mime types), + + Starts drag for the device, if it's grabbed by the surface. drag + ends when button is released. Caller is responsible for + destroying the drag object. + + - Drag object methods: + + drag.destroy(id), destroy drag object. + + drag.send(id, data), send drag data. + + drag.accept(id, mime type), accept drag offer, called by + target surface. + + - drag object events: + + drag.offer(id, mime-types), sent to potential destination + surfaces to offer drag data. If the device leaves the window + or the originator cancels the drag, this event is sent with + mime-types = NULL. + + drag.target(id, mime-type), sent to drag originator when a + target surface has accepted the offer. if a previous target + goes away, this event is sent with mime-type = NULL. + + drag.data(id, data), sent to target, contains dragged data. + ends transaction on the target side. + +Sequence of events: + +\begin{itemize} +\item The initiator surface receives a click (which grabs the input + device to that surface) and then enough motion to decide that a drag + is starting. Wayland has no subwindows, so it's entirely up to the + application to decide whether or not a draggable object within the + surface was clicked. + +\item The initiator creates a drag object by calling the create\_drag + method on the dnd global object. As for any client created object, + the client allocates the id. The create\_drag method also takes the + originating surface, the device that's dragging and the mime-types + supported. If the surface has indeed grabbed the device passed in, + the server will create an active drag object for the device. If the + grab was released in the meantime, the drag object will be + in-active, that is, the same state as when the grab is released. In + that case, the client will receive a button up event, which will let + it know that the drag finished. To the client it will look like the + drag was immediately cancelled by the grab ending. + + The special mime-type application/x-root-target indicates that the + initiator is looking for drag events to the root window as well. + +\item To indicate the object being dragged, the initiator can replace + the pointer image with an larger image representing the data being + dragged with the cursor image overlaid. The pointer image will + remain in place as long as the grab is in effect, since no other + surfaces receive enter/leave events. + +\item As long as the grab is active (or until the initiator cancels + the drag by destroying the drag object), the drag object will send + "offer" events to surfaces it moves across. As for motion events, + these events contain the surface local coordinates of the device as + well as the list of mime-types offered. When a device leaves a + surface, it will send an offer event with an empty list of + mime-types to indicate that the device left the surface. + +\item If a surface receives an offer event and decides that it's in an + area that can accept a drag event, it should call the accept method + on the drag object in the event. The surface passes a mime-type in + the request, picked from the list in the offer event, to indicate + which of the types it wants. At this point, the surface can update + the appearance of the drop target to give feedback to the user that + the drag has a valid target. If the offer event moves to a + different drop target (the surface decides the offer coordinates is + outside the drop target) or leaves the surface (the offer event has + an empty list of mime-types) it should revert the appearance of the + drop target to the inactive state. A surface can also decide to + retract its drop target (if the drop target disappears or moves, for + example), by calling the accept method with a NULL mime-type. + +\item When a target surface sends an accept request, the drag object + will send a target event to the initiator surface. This tells the + initiator that the drag currently has a potential target and which + of the offered mime-types the target wants. The initiator can + change the pointer image or drag source appearance to reflect this + new state. If the target surface retracts its drop target of if the + surface disappears, a target event with a NULL mime-type will be + sent. + + If the initiator listed application/x-root-target as a valid + mime-type, dragging into the root window will make the drag object + send a target event with the application/x-root-target mime-type. + +\item When the grab is released (indicated by the button release + event), if the drag has an active target, the initiator calls the + send method on the drag object to send the data to be transferred by + the drag operation, in the format requested by the target. The + initiator can then destroy the drag object by calling the destroy + method. + +\item The drop target receives a data event from the drag object with + the requested data. +\end{itemize} + +MIME is defined in RFC's 2045-2049. A registry of MIME types is +maintained by the Internet Assigned Numbers Authority (IANA). + +ftp://ftp.isi.edu/in-notes/iana/assignments/media-types/ + \section{Types of compositors} \subsection{System Compositor} - - ties in with graphical boot - - hosts different types of session compositors - - lets us switch between multiple sessions (fast user switching, +\begin{itemize} +\item ties in with graphical boot +\item hosts different types of session compositors +\item lets us switch between multiple sessions (fast user switching, secure/personal desktop switching) - - multiseat - - linux implementation using libudev, egl, kms, evdev, cairo - - for fullscreen clients, the system compositor can reprogram the +\item multiseat +\item linux implementation using libudev, egl, kms, evdev, cairo +\item for fullscreen clients, the system compositor can reprogram the video scanout address to source fromt the client provided buffer. +\end{itemize} \subsection{Session Compositor} - - nested under the system compositor. nesting is feasible because +\begin{itemize} +\item nested under the system compositor. nesting is feasible because protocol is async, roundtrip would break nesting - - gnome-shell - - moblin - - compiz? - - kde compositor? - - text mode using vte - - rdp session - - fullscreen X session under wayland - - can run without system compositor, on the hw where it makes +\item gnome-shell +\item moblin +\item compiz? +\item kde compositor? +\item text mode using vte +\item rdp session +\item fullscreen X session under wayland +\item can run without system compositor, on the hw where it makes sense - - root window less X server, bridging X windows into a wayland +\item root window less X server, bridging X windows into a wayland session compositor +\end{itemize} \subsection{Embbedding Compositor} @@ -188,8 +347,10 @@ the requests the embedded client needs to inform the host about buffer updates and a mechanism for forwarding input events from the host application. - - firefox embedding flash by being a special purpose compositor to +\begin{itemize} +\item firefox embedding flash by being a special purpose compositor to the plugin +\end{itemize} \section{Implementation} @@ -199,45 +360,50 @@ what's currently implemented \texttt{libwayland-server.so} - - implements protocol side of a compositor - - - minimal, doesn't include any rendering or input device handling - - - helpers for running on egl and evdev, and for nested wayland +\begin{itemize} +\item implements protocol side of a compositor +\item minimal, doesn't include any rendering or input device handling +\item helpers for running on egl and evdev, and for nested wayland +\end{itemize} \subsection{Wayland Client Library} \texttt{libwayland.so} - - minimal, designed to support integration with real toolkits such as +\begin{itemize} +\item minimal, designed to support integration with real toolkits such as Qt, GTK+ or Clutter. - - doesn't cache state, but lets the toolkits cache server state in +\item doesn't cache state, but lets the toolkits cache server state in native objects (GObject or QObject or whatever). +\end{itemize} \subsection{Wayland System Compositor} - - implementation of the system compositor +\item implementation of the system compositor - - uses libudev, eagle (egl), evdev and drm +\item uses libudev, eagle (egl), evdev and drm - - integrates with ConsoleKit, can create new sessions +\item integrates with ConsoleKit, can create new sessions - - allows multi seat setups +\item allows multi seat setups - - configurable through udev rules and maybe /etc/wayland.d type thing +\item configurable through udev rules and maybe /etc/wayland.d type thing +\end{itemize} \subsection{X Server Session} - - xserver module and driver support +\begin{itemize} +\item xserver module and driver support - - uses wayland client library +\item uses wayland client library - - same X.org server as we normally run, the front buffer is a wayland +\item same X.org server as we normally run, the front buffer is a wayland surface but all accel code, 3d and extensions are there - - when full screen the session compositor will scan out from the X +\item when full screen the session compositor will scan out from the X server wayland surface, at which point X is running pretty much as it does natively. +\end{itemize} \end{document} -- 2.7.4