UI, Menu, and Video Considerations

User Interface/Menu/Pre-Rendered Video Placement

In NVIDIA Surround mode there can be components of games which are undesirable to have on outside displays (landscape orientation), or split between multiple displays. Examples of these components include game menus, pre-rendered videos, and user interface elements such as heads-up displays (HUDs).

Some games are not designed with systems' display configuration in mind. As a result, they can end up stretching menus, subtitles, or pre-rendered in-game videos — like cut scenes — across the multiple displays. This causes difficulty reading and using the menus, and distortion of the videos, which are typically designed for a particular aspect ratio.

Figure 3. In-game video, scaled up with undesirable stretching

Figure 4. In-game video, scaled up until it fits properly into one display without stretching

Sometimes when running in landscape orientation Surround modes, games place parts of the user interface on displays that are in the peripheral regions of a user's eyesight, negatively impacting the experience. This is especially undesirable if a user typically wants to use or read these interface components often, as is the case with a game's map, health/damage indicators, or controls (Figures 5 and 6).

Figure 5. HUD components inconveniently placed on outside displays

Figure 6. HUD components conveniently placed on center display

In contrast, when a user is running in portrait orientation mode, it may actually be preferable to allow interface elements onto the outside displays since they are closer together than in landscape mode, and portrait mode is very close to the 16:9 or 16:10 aspect ratios that many single displays use. As a result, the outside edges of the outside displays are often within a comfortably viewable area (Figures 7 and 8).

Figure 7. HUD inconveniently restricted to center portrait-oriented display

Figure 8. HUD conveniently spans three portrait-oriented displays

In either landscape or portrait orientation, it is usually not desirable to have individual parts of the interface spanning multiple displays, not only because they will be split between displays in normal Surround modes, but also because they may end up partially or wholly occluded in bezel-corrected display modes (Figure 9).

Figure 9. UI components at top and bottom of screen are occluded by bezel in bezel-corrected display mode

NVIDIA Surround technology presents multiple displays as one large display to the game and it can be difficult for a developer to know where the bezels of the actual physical displays reside with respect to the final rendered image. It can also be difficult to know if the current display mode is *bezel-corrected*, which results in some pixels being virtually hidden behind the bezel for the sake of having rendered objects appear to line up across the multiple displays. Bezel-correction is designed to create the perception of looking through a window with panes, rather than just seeing the normal display separated by display bezels. In this way, better immersion is afforded at the expense of visibility of some of the rendered pixels (Figures 10 and 11).

Figure 10. Lack of bezel-corrected mode shows split/elongated view between left and center Surround displays

Figure 11. Bezel-corrected mode shows continuous view between left and center Surround displays, but occludes some pixels

The recommended way to avoid issues with menus and video rendering in Surround modes is to restrict them to the center display, so that they are not stretched in any way. Alternatively, you could scale these elements until they maximize either the width or height of the active display mode, thus enlarging the display area covered while retaining correct proportions. Similarly, the recommended way to avoid the user interface placement issues when in landscape mode is to restrict user interfaces to the center screen, by making sure that none of the rendered interface elements are placed beyond certain boundaries where a display bezel may be. This way interface elements are not rendered on the outside displays, or split between displays.

In Portrait orientation modes, it is generally preferred to allow interface elements to be placed on the outside displays, since the aspect ratio is close to that of 16:9 or 16:10 modes. However, it is still ideal to avoid having them split (or occluded in the case of bezel-corrected modes) between displays (Figure 12).

NVIDIA does not recommend developers look for a display number since the NVIDIA Control Panel allows users to arrange the displays in any order they desire. Instead, it is recommended that you use NVAPI for the most robust programmatic placement of menu, video, and interface elements for Surround configurations. Here is an example solution where we are querying for "visible rects" and then restricting the UI to the visible part of the center display:

NvAPI_Status retVal;
NvU32 displayID = 0;

//We assume that displayArgs, a struct of device properties such as
//resolution and fullscreen state, has already been initialized with
//valid data. Also, we assume that Rect is some previously defined
//struct type which contains member variables to describe location and
//size of an arbitrary display rect.
Rect safeUIRegion;

safeUIRegion.x = 0;
safeUIRegion.y = 0;
safeUIRegion.width = displayArgs.width;
safeUIRegion.height = displayArgs.height;

retVal = NvAPI_DISP_GetGDIPrimaryDisplayId(&displayID);

if (retVal == NVAPI_OK) {
NV_RECT viewports[NV_MOSAIC_MAX_DISPLAYS];
NvU8 isBezelCorrected;

retVal = NvAPI_Mosaic_GetDisplayViewportsByResolution(displayID,
displayArgs.width, displayArgs.height, viewports,
&isBezelCorrected);

if (retVal == NVAPI_OK && displayArgs.isFullscreen) {
//NVIDIA Surround is enabled.

print("NVIDIA Surround topology found.\n");

NvU32 displayCount = 0;

while (viewports[displayCount].top !=
viewports[displayCount].bottom) displayCount++;

//Ensure there is more than one viewport rect and that there is an odd
//number of displays in the Surround configuration if we want to
//restrict the UI to the center display.
if ((displayCount > 1) && (displayCount & 1)) {
bool isSingleDisplayRow = true;

for (i = 1; i < displayCount; i++) {
if ((viewports[0].top != viewports[i].top) ||
(viewports[0].bottom != viewports[i].bottom)) {
isSingleDisplayRow = false;
break;
}
}
//Check to make sure that the displays are in a single row together.
if (isSingleDisplayRow) {
NvU32 centerDisplayIdx = displayCount / 2;

//Check which rect dimension of the center display is larger. If the
//height is larger than the width, we assume the displays are arranged
//in portrait mode and we leave the UI spanning multiple displays (and
//possibly occlude parts of the UI in a bezel-corrected configuration).
//If the width is larger than the height, we store that center
//display's viewport coordinates and size so we can restrict the UI to
//that region for the user's convenience and because we know the UI was
//probably designed with this display orientation/aspect ratio in mind.
//Alternatively, we could choose to just use the rect regions returned
//by NvAPI_Mosaic_GetDisplayViewportsByResolution to place UI elements
//on more than one display, while still restricting them to the visible
//regions.
NvU32 centerWidth = viewports[centerDisplayIdx].right
- viewports[centerDisplayIdx].left + 1; NvU32 centerHeight = viewports[centerDisplayIdx].bottom
- viewports[centerDisplayIdx].top + 1);

if (centerWidth > centerHeight) {
print("Restricting UI to center display.\n");
safeUIRegion.x = viewports[centerDisplayIdx].left;
safeUIRegion.y = viewports[centerDisplayIdx].top;
safeUIRegion.width = centerWidth;
safeUIRegion.height = centerHeight;
}
}
}
}
else if (retVal == NVAPI_MOSAIC_NOT_ACTIVE) {
print("NVIDIA Surround is not enabled.\n");
}
else {
print("NvAPI_Mosaic_GetDisplayViewportsByResolution call
failed.\n");
}
}
else {
print("NvAPI_DISP_GetGDIPrimaryDisplayId call failed.\n");
}
DrawUI(safeUIRegion);

Another way of doing this which does not require the use of NVAPI, but is less robust, is to assume the user has a configuration that is three displays wide based on the aspect ratio of the selected resolution, or let the user select an option in-game to enable this mode. This example defines rough bezel-correction-safe horizontal boundaries for programmatic placement of menu, video, and interface elements:

SingleDisplayWidth = TotalWidth / 3;

//Covers cases for Surround Landscape mode comprised of 5:4 displays
//and up.
if AspectRatio >= 3.75 {
//Define minimum interface x value as the left boundary of the center
//display + 10% of a single display's width (for bezel-corrected
//safety).
HUD.minX = SingleDisplayWidth + (SingleDisplayWidth / 10);

//Define maximum interface x value as the right boundary of the center
//display + 10% of a single display's width (for bezel-corrected
//safety).
HUD.maxX = (2 * SingleDisplayWidth) – (SingleDisplayWidth / 10);
}

else {
//Assume portrait mode, allowing interface elements on all 3 displays,
//but make sure there is no interface element within 10% of 1/3 and 2/3
//of the total display width.
HUD.maxDisp1 = (SingleDisplayWidth) – (SingleDisplayWidth / 10);
HUD.minDisp2 = (SingleDisplayWidth) + (SingleDisplayWidth / 10);
HUD.maxDisp2 = (2 * SingleDisplayWidth)
– (SingleDisplayWidth / 10);
HUD.minDisp3 = (2 * SingleDisplayWidth)
+ (SingleDisplayWidth / 10);
//Bezels are located between HUD.maxDisp1 and HUD.minDisp2 as well as
//between HUD.maxDisp2 and HUD.minDisp3
}

Figure 12. Bezel-correction-safe areas to place UI elements so they are not occluded by bezels for common resolutions and bezel offsets

Yet another solution for these placement issues is to allow the user to have some element of control over the interface so they can offset interface elements by some amount or control their exact location. This could function by means of in-game controls or by exposing interface object positions in a text configuration file (which, for example, could be edited by third party tools or even manually by users).

Mouse Interaction with User Interface

In certain Surround modes, some games have problems with mouse cursor interaction with the user interface. Typically, these problems are caused by the visual representation of the mouse cursor rendering in a different location than the game believes it to be, or hit-targets for the UI elements failing to scale/move with their corresponding visual components.

Because the mouse cursor being rendered in the incorrect location will likely break the ability to play a game, it is important to verify that this works correctly in Surround resolutions, for both landscape and portrait Surround configurations.

In order to make sure that user interface hit-targets work correctly as resolution scales upwards, it is recommended that you either restrict your UI and menu elements to the center display using the methods mentioned in the User Interface/Menu/Pre-rendered Video Placement section, or scale up the hit-targets in the same way that their corresponding visual components are scaled, for all resolutions.

Displaying Enumerated Resolution Mode List in Menus

It is important to display a full list of available modes in a game's graphics options menu because NVIDIA Surround technology adds many new resolution modes (such as bezel-corrected Surround resolutions) to the modes traditionally supported by the NVIDIA driver. In some games, the resolution mode list is designed as a drop down box that does not scroll, and as a result, many resolutions in the longer mode list end up above the top edge or below the bottom edge of the display. This prevents a user from being able to select those resolutions. NVIDIA recommends offering users the option to select all enumerated resolutions from scrollable lists or other compatible control types.

Additionally, to ensure all modes are able to be offered to users, it is important to ensure that a game does not have some pre-defined maximum number of resolution mode list entries in the menu interface, which truncates the list of enumerated modes that is offered to users.

Lastly, it's important to make sure that enumerated resolutions are not filtered in some way, like by only displaying resolutions that are a multiple of some number. This is because Surround users are able to specify arbitrary amounts of bezel correction, and as a result, they can create odd value resolutions (like 6011x1080) that a developer cannot predict.

Aspect Ratio Grouping in Menus

Some games try to make resolution choice easier for users by enumerating the available resolutions, and then sorting them into pre-defined aspect ratio groups in an in-game menu's user interface. For games that do this, it is recommended that one of the following solutions is used, in light of all the new aspect ratio configurations that are possible:

 

 


NVIDIA® GameWorks™ Documentation Rev. 1.0.200608 ©2014-2020. NVIDIA Corporation. All Rights Reserved.