Weston (Wayland) Windowing System
Wayland is a protocol for communication between a display server and its clients. A Wayland server uses the Wayland protocol to communicate with a GUI program, which is a Wayland client. A Wayland server is also called a Wayland compositor, as it also acts as a compositing window manager.
The Weston server, usually just called Weston, is the reference implementation of a Wayland compositor. It manages the displays, including composition of their contents, support for their input device events (touch screen, mouse, keyboard, etc.), and their settings (wallpapers, resolution, multi-monitor display, etc.).
Weston is lightweight compared to X11, and is fast as a compositor. It is suitable for many embedded and mobile use cases.
The current release of Jetson Linux supports Weston 6.0.
Weston/Wayland Architecture
A Wayland server and a set of Weston libraries are collectively known as Weston/Wayland. This diagram describes the Weston/Wayland architecture.
The Wayland server libraries implement the Wayland protocol for Weston. The Weston libraries implement the Wayland compositor, which uses the Linux Kernel Mode Setting (KMS) for setting up the display, performs the compositing using OpenGLES and Direct Rendering Manager (DRM), and manages the Linux input devices.
The Wayland client application uses the Wayland protocol to communicate with the Wayland compositor. It may be an EGL application, an X server (rootless), or another type of display server.
Shells
Weston supports different graphical user interfaces through shell plugins. NVIDIA distributes Weston with two plugins: the Desktop shell and the IVI shell.
The Desktop and IVI shell each provides its own Wayland protocol interface for Wayland clients. Thus a Wayland client written for a specific shell protocol does not work with others.
The desktop shell provides a modern desktop environment similar to X11, plus interfaces to the keyboard and mouse. It is implemented by the library desktop-shell.so. A special desktop shell client, weston-desktop-shell, renders the wallpaper, panel, and other desktop elements.
The IVI shell is designed for In-Vehicle Infotainment (IVI) applications. It provides a GENIVI Layer Manager-compatible API through its controller modules, and a simple protocol for use by IVI applications. It is implemented by the library ivi-shell.so.
Configuration
You can configure the the Weston shell's look and behavior preferences through the configuration file /etc/xdg/weston/weston.ini.
See Ubuntu weston.ini manpage for more information about configuring the Weston shell.
Environment Variables
The environment variables Weston uses are:
• WAYLAND_DEBUG: If set to any value, causes libwayland to print the live protocol to stderr.
• XDG_RUNTIME_DIR: Specifies the directory for Weston's socket and lock files. If set, Weston creates its socket under this directory, and Wayland clients use the socket from this directory to communicate with Weston.
Running Weston 6.0
Use the following procedure to launch Weston.
To launch Weston
1. If the X server is running, stop it:
$ sudo service gdm stop; sudo pkill -9 Xorg
2. Start the Tegra dummy DRM driver:
$ sudo modprobe tegra-udrm modeset=1
3. Create a libgbm.so.1 symlink:
$ sudo ln -sf /usr/lib/aarch64-linux-gnu/tegra/libnvgbm.so /usr/lib/aarch64-linux-gnu/libgbm.so.1
4. Set up the environment and directory permissions:
$ unset DISPLAY
$ mkdir /tmp/xdg
$ chmod 700 /tmp/xdg
5. Choose a valid TTY device:
$ export WESTON_TTY=1
6. Launch Weston.
1. To launch Weston as a root user, enter this command:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston --tty="$WESTON_TTY" --idle-time=0 &
• Optional: Set the environment variable WAYLAND_DEBUG to server to dump protocol messages from the server to stderr:
$ export WAYLAND_DEBUG=server
• Optional: Use the --use-egldevice switch to disable GBM swapchains and use EGLStream to write Weston output.
2. To launch Weston as a non-root user with weston-launch, enter these commands:
$ sudo groupadd weston-launch
$ sudo usermod -a -G weston-launch $USER
$ sudo chown root /usr/bin/weston-launch
$ sudo chmod +s /usr/bin/weston-launch
$ weston-launch &
7. Run a Wayland client:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston-simple-egl
• Optional: Set the environment variable WAYLAND_DEBUG to 1 to dump all protocol messages to stderr:
$ WAYLAND_DEBUG=1
8. To run Weston tests, you must load the module weston-test.so with a command line option or a setting in weston.ini. For example, you may specify the option: --modules=/usr/lib/aarch64-linux-gnu/tegra/weston/weston test.so.
9. To launch Weston with ivi-shell and ivi-controller on the device, pass the arguments to the weston binary as: --shell=ivi-shell.so --modules=ivi-controller.so.
Multiple Display Heads
Weston supports multiple display heads with multiple overlays on all Jetson devices. Use weston.ini to set the expected resolution and orientation of the monitors.
If multiple displays are connected, applications are launched on the one that contains the mouse pointer at the time of launch. Once an application is launched, you can drag it across a boundary between displays.
If multiple displays are connected and they have different resolutions, Weston reports the frame rate of the display on which more than 50% of its window is displayed. If a monitor is removed while Weston is running (hot removal), running applications dynamically shift to the remaining active monitor.
Example of weston.ini Display Options
Use this example as a model for setting your own display options in weston.ini.
name=HDMI-A-1
mode=1920 x 1080
transform=90 (0,90,180,270)
Hot-Plugging
Weston monitors events sent by the Direct Rendering Manager (DRM) subsystem using udev. When the display configuration changes, udev receives a HOTPLUG event. It calls drmModeGetResources() to identify connectors that have been added or removed, and takes actions accordingly.
On T210 there is no real kernel mode DRM driver; instead there is a user mode DRM driver, drm-nvdc, and a dummy kernel mode DRM driver, tegra-udrm. These drivers work together to provide a real DRM driver-like interface to Weston. The tegra-udrm driver signals a HOTPLUG event when the display connection state changes.
Verified Use Cases
• Disconnect and connect the HDMI™ display after launching Weston.
• Disconnect and connect the HDMI display while the application nvgldemo is running, and verify that application drawing is visible after connection.
• Launch Weston without an HDMI display connected and then connect an HDMI display.
• Launch Weston without an HDMI display connected, launch the nvgldemo application, and then connect an HDMI display.
All of the use cases above are also verified with DisplayPort displays.
Issues Fixed in Weston
• Application window disappears when display is re-connected.
A fix for this issue has been backported to the downstream Weston 6 codeline from the upstream Weston main codeline. See the freedesktop.org GitLab for more information.
• Weston crashes using ivi-shell when no display is connected.
• Support added for multiple hotplug displays in ivi-shell.
ivi-shell was not originally designed to support hotplug. This is an NVIDIA enhancement.
The fix adds a protocol to ivi-vm which lets the client receive a hotplug event sent by ivi-controller. The client must handle the event during hotplug using the ivi-layermanagement API to relocate surfaces, destroy surfaces, and so on. The client can create listeners to listen for the connector_name, screen_id, and screen_destroy events in the ivi-wm-screen interface to the ivi-wm protocol to get notifications of hot plugging.
Following is some example code:
static void
screen_id(void *data, struct ivi_wm_screen *ivi_wm_screen, uint32_t id);
static void
screen_destroy(void *data,struct ivi_wm_screen *ivi_wm_screen, uint32_t id);
static void
connector_name(void *data, struct ivi_wm_screen *ivi_wm_screen, const char *process_name);
struct ivi_wm_screen_listener wm_screen_listener = {
screen_id,
screen_destroy,
layer_added,
connector_name,
error,
};
ivi_wm_screen_add_listener(ivi_wm_screen, &wm_screen_listener, args);
Known Issues
• There is currently no way for the DRM module to notify Weston that a page flip has failed. Consequently, when Weston is configured to support multiple displays, hot-plugging a display at certain times may cause drmAtomicCommit() to fail with a “page flip pending” assertion.
Compositing Mode in Weston
Jetson devices have multiple planes per head. The major types of planes are:
• Primary plane
• Overlay plane
• Cursor plane
On every frame, Weston evaluates all surfaces and views and chooses one of these strategies to composite them:
• Overlay-only mode: Weston tries to assign a plane for each surface or view. If that is not possible, it shifts to "mixed mode."
• Mixed mode: Weston tries to assign overlay plane(s) to some of the surfaces and views, composite the remaining surfaces and views using GL, and update the result to the primary plane. If that is also not possible, it proceeds with “GL-only mode.”
• GL-only mode: Weston composites all of the surfaces and views using GL, and updates the result to the primary plane.
With all of these strategies, Weston makes sure to preserve the rendering order of the surfaces and views. You can force GL-only mode compositing by setting the environment variable WESTON_FORCE_RENDERER to 1.
The demo application weston-simple-dmabuf-egldevice demonstrates mixed-mode compositing in Weston, where some surfaces are assigned overlay planes and others are composited with GL to the primary plane.
You can use the application weston-debug to verify that overlay planes are being assigned. Use the following procedure:
1. Stop X if it is running and prepare the device for launching Weston:
$ sudo service gdm stop; sudo pkill -9 Xorg
$ unset DISPLAY
$ mkdir /tmp/xdg
$ chmod 700 /tmp/xdg
$ export WESTON_TTY=1
$ sudo modprobe tegra-udrm modeset=1
$ sudo ln -sf /usr/lib/aarch64-linux-gnu/tegra/libnvgbm.so /usr/lib/aarch64-linux-gnu/libgbm.so.1
2. Launch Weston with the --debug switch:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston --tty="$WESTON_TTY" --idle-time=0 --debug &
3. Launch a few Weston client applications:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston-simple-egl &
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston-simple-dmabuf-egldevice &
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston-simple-dmabuf-egldevice &
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston-simple-dmabuf-egldevice &
4. Run the weston-debug application:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg ./weston-debug -a
5. Look for information like that shown below in the weston-debug output. Blue colored surfaces are overlay-composited and green-colored surfaces are GL-composited.
Layer 5 (pos 0x50000000):
View 0 (role xdg_toplevel, PID 8080, surface ID 3, top-level window 'simple-dmabuf-egldevice', 0x33c798b0):
position: (714, 860) -> (970, 1116)
[not opaque]
outputs: 0 (HDMI-A-1) (primary)
dmabuf buffer
format: 0x34325258 XRGB8888
modifier: 0x3800000000fe014
View 1 (role xdg_toplevel, PID 8073, surface ID 3, top-level window 'simple-dmabuf-egldevice', 0x33bb9500):
position: (209, 591) -> (465, 847)
[not opaque]
outputs: 0 (HDMI-A-1) (primary)
dmabuf buffer
format: 0x34325258 XRGB8888
modifier: 0x3800000000fe014
View 2 (role xdg_toplevel, PID 8066, surface ID 3, top-level window 'simple-dmabuf-egldevice', 0x31769110):
position: (1129, 339) -> (1385, 595)
[not opaque]
outputs: 0 (HDMI-A-1) (primary)
dmabuf buffer
format: 0x34325258 XRGB8888
modifier: 0x3800000000fe014
View 3 (role xdg_toplevel, PID 8051, surface ID 9, top-level window 'simple-egl', 0x314c6430):
position: (1343, 642) -> (1593, 892)
[not opaque]
outputs: 0 (HDMI-A-1) (primary)
EGL buffer
Layer 6 (pos 0x2):
...
[view] evaluating view 0x33c798b0 for output HDMI-A-1 (0)
[view] view 0x33c798b0 format: XRGB8888
[overlay] provisionally placing view 0x33c798b0 on overlay 4002 in mixed mode
[view] evaluating view 0x33bb9500 for output HDMI-A-1 (0)
[view] view 0x33bb9500 format: XRGB8888
[overlay] provisionally placing view 0x33bb9500 on overlay 4001 in mixed mode
[view] evaluating view 0x31769110 for output HDMI-A-1 (0)
[view] view 0x31769110 format: XRGB8888
[overlay] not placing view 0x31769110 on overlay: no free overlay planes
[view] evaluating view 0x314c6430 for output HDMI-A-1 (0)
[overlay] not placing view 0x314c6430 on overlay: couldn't get fb
[repaint] Using mixed state composition
[repaint] view 0x33c798b0 on Overlay plane 4002
[repaint] view 0x33bb9500 on Overlay plane 4001
[repaint] view 0x31769110 using renderer composition
[repaint] view 0x314c6430 using renderer composition
NV16/NV24 in Weston
The Weston GL compositor adds support for NV16 and NV24 content via shared memory buffers.
Applications that use the WL_SHM_FORMAT_NV24 enum can substitute DRM_FORMAT_NV24, as by convention both enums use symbols defined in drm_fourcc.h.
Shared-Memory Rendering
The Weston test color-format.weston exercises shared-memory rendering for all formats that Weston currently supports. Though implemented as a test case, it is intended for visual verification, and it must be manually terminated.
To run the test case
1. Start Weston with the weston-test.so module, for example:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston --tty="$WESTON_TTY" --idle-time=0 --modules=weston-test.so &
2. Enter this command to execute the test application:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg color-format.weston
The application displays eight color test patterns, each rendered using a different color format. All should appear substantially identical, with minor variations caused by the limitations of the formats.
3. To terminate the test, enter Ctrl-C.
DMA Buffer Rendering
DRM now supports NV16 and NV24 color formats. To use these formats, construct an EGLImage from the DMA buffer via the exposed EXT_image_dma_buf_import and EXT_image_dma_buf_import_modifiers extensions.
Currently only block-linear DMA buffer layouts are supported. DRM_FORMAT_NV21, DRM_FORMAT_NV16, and DRM_FORMAT_NV24 have been added to supported_drm_formats.
To run a verified use case (video playback test)
• Enter this command:
$ gst-launch-1.0 filesrc location=<mp4_src_file> ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)<format>' ! nvdrmvideosink
Where:
• <mp4_src_file> is the pathname of the apt source file
• <format> is NV16 or NV24
The gst-launch-1.0 command is a GStreamer pipeline to display video. It uses the nvvidconv plugin to convert other formats in the stream to NV16/NV24. The video is displayed with the nvdrmvideosink plugin, which internally uses DRM.
VPR Memory
• With nvbuf_utils: Add the NvBufferTag_PROTECTED tag to the NvBufferTag member of struct NvBufferCreateParams and pass it to NvBufferCreate().
• With GBM: Specify the GBM_BO_USE_PROTECTED usage flag for gbm_bo_create().
• With EGLStream: Add the EGL_PROTECTED_CONTENT_EXT attribute when creating eglSurface objects. Specify the texture parameter GL_TEXTURE_PROTECTED_EXT for protected textures.
To write to protected textures with GL you must also use a protected context, which you may create by specifying the EGL_PROTECTED_CONTENT_EXT attribute. Note that you must also specify this attribute when you import dmabuf into EGL, in the case where the dmabuf is created in VPR memory.
To run a sample case
• Run the weston-simple-dmabuf-egldevice app with the -p switch.
To verify the correctness of the result, examine it on the screen. It should be the same as when you run the app without using VPR memory.
Current Limitations
The VPR content can only be rendered on the screen with overlay GL composition. GL does not support protected content yet. Such support will be added in later releases.
Thus, if there are several windows on the screen exceeding the limit for overlay composition, protected content is not rendered for the excess windows.
HDCP support is not available yet. It will be supported in a future release. For Jetson Nano devices, though, you can display VPR content onscreen even without establishing an HDCP link.
Weston dma-buf Support
Clients can post dma-buf buffers to Weston using the Linux DMA-BUF Unstable V1 Protocol of Wayland. Dma-buf support with Weston has following components.
Buffer Allocation
GBM is commonly used to allocate buffers, which are backed by the dma-buf file descriptor.
This code snippet shows how to allocate a buffer using GBM:
drm_fd = open("/dev/dri/card0", O_RDWR);
device = gbm_create_device(drm_fd);
bo = gbm_bo_create_with_modifiers(device, width, height,
format, modifiers, modifiers_count);
Weston supports the formats DRM_FORMAT_XRGB8888 and DRM_FORMAT_ARGB8888.
A modifier is an encoding of vendor-specific buffer layout parameters. Weston supports modifiers are supported by DRM (read the IN_FORMATS plane property blob) and by EGL (call eglQueryDmaBufModifiersEXT() ).
Buffer Write/read from CPU
You can use this API function to write data directly into a GBM buffer:
gbm_bo_write(bo, user_buffer, sizeof user_buffer);
You also can memory map a GBM buffer and get a CPU-accessible address to the data written to or read from it. The following code snippet shows how:
uint32_t dst_stride = 0;
void *gbo_mapping = NULL;
// Map bo to CPU accessible address.
char *dst_ptr = gbm_bo_map(bo,
0, 0,
width,
height,
GBM_BO_TRANSFER_READ_WRITE,
&dst_stride,
&gbo_mapping);
// Write data to or read data from dst_ptr.
. . .
// Unmap bo.
gbm_bo_unmap(bo, gbo_mapping);
Wayland Protocol to Post dma-buf Buffers to Weston
Weston uses the Generic Buffer Management (GBM) library to allocate buffers, which are backed by dma-buf file descriptors. Client applications must call the following GBM functions to get the dma-buf file descriptor and associated parameters from the GBM buffer object. To post the dma-buf to a Wayland surface, the file descriptor and parameters must be provided via Linux DMA-BUF Unstable V1 Protocol.
The following code snippet shows the main calls involved.
stride = gbm_bo_get_stride_for_plane(bo, 0);
offset = gbm_bo_get_offset(bo, 0);
handle = gbm_bo_get_handle_for_plane(bo, 0);
modifier = gbm_bo_get_modifier(bo);
drmPrimeHandleToFD(drm_fd, handle, 0, &dmabuf_fd);
zwp_linux_buffer_params_v1_add(params, dmabuf_fds, 0, offset,
stride, modifier >> 32,
modifier & 0xffffffff);
zwp_linux_buffer_params_v1_create(params, width, height,
format, flags);
GL Renderer in Weston
The GL renderer in Weston creates an EGLImage from a dma-buf object that it receives with Linux DMA-BUF Unstable V1 Protocol using the EGL_EXT_image_dma_buf_import and EGL_EXT_image_dma_buf_import_modifiers extensions. Weston obtains the GL texture target to be used with this EGLImage by calling eglQueryDmaBufModifiersEXT(), and binds the EGLImage to the texture.
Display Hardware Compositing in Weston
The DRM compositor in Weston calls drmModeAddFB2WithModifiers() to associate a DRM framebuffer ID with the dma-buf it receives via Linux DMA-BUF Unstable V1 Protocol. It then uses the framebuffer ID to present the dma-buf to one of the available display hardware planes using drmModeSetPlane(), in the legacy path, or drmModeAtomicCommit(), in the atomic modeset path. This takes the form of a flip, where the reference to the new buffer is swapped in during vblank.
Weston dma-buf Sample
weston-dmabuf-formats is a simple application to test Weston dmabuf support for different formats, such as XRGB8888, NV12, NV16, NV24.
Enter this command to launch the application:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston-dmabuf-formats --import-format=<format> --loop=<count> --use-gbm
Where:
• <format> is NV16 or NV24.
• <count> is the number of iterations to execute.
• --use-gbm indicates that the application is to use GBM instead of nvbuf_utils for buffer allocation.
weston-debug
The client application weston-debug receives and prints debug messages using the weston-debug protocol. For example, when Weston is recompositing it prints all surface views and descriptions of the decision process regarding their assignment to overlays and GL compositing.
For weston-debug to work, you must run Weston with the ‑‑debug switch. Do not use this switch in production.
You can pass weston-debug additional parameters to choose categories of debug output to display:
$ sudo XDG_RUNTIME_DIR=/tmp/xdg weston –debug
$ sudo XDG_RUNTIME_DIR=/tmp/xdg ./weston-debug -a
Alternatively, you can enable Weston debug output by setting the environment variable WAYLAND_DEBUG=server, and Weston client debug by setting WAYLAND_DEBUG=1.
Gnome-Wayland Desktop Shell Support
With L4T release 32.4.2 NVIDIA supports the Gnome-Wayland windowing system. L4T support for Gnome-Wayland is currently experimental.
This section describes some procedures for using Gnome-Wayland.
To start the windowing system
1. In /etc/systemd/nvuser.sh, change:
GNOME_WAYLAND="0"
To:
GNOME_WAYLAND="1"
2. In /etc/modprobe.d/tegra-udrm.conf, uncomment the line:
#options tegra-udrm modeset=1
(Remove ‘#’ from the start of the line.)
3. In /etc/systemd/nv.sh, change:
DISABLE_MESA_EGL="1"
To:
DISABLE_MESA_EGL="0"
4. In /etc/systemd/system/nvargus-daemon.service, stop the nvargus daemon from being launched on boot by inserting ‘;’ at the beginning of the line:
ExecStart=/usr/sbin/nvargus-daemon
5. Reboot the device. If auto-login is enabled, L4T logs you in to a Wayland session; otherwise it displays a list of available interfaces (e.g. Wayland, command line) and logs you into a session of the type you select.