gl_FragCoord should not be used in stereo mode.)
As described earlier, 3D Vision Automatic uses driver heuristics to decide which draw calls need to be stereoized and which ones should not be. In this section, some of the common problem heuristics are described, and their specific behaviors outlined.
For PC based applications, the most common stereoscopic heuristic that applications run into is the state of the Z-buffer. When the Z-buffer is set to NULL, the 3D Vision Automatic driver uses this as a cue that the rendering being actively performed is a blit-with-shader, and disables the stereoscopic portion accordingly.
If you wish to avoid rendering to the Z-buffer while performing an operation, but still need that operation to be stereoized, set the Z-test function to ALWAYS and Z-write-enable to FALSE, while leaving a Z-target bound.
Surface Aspect Ratios
In Direct3D9, all Render Targets (surfaces created with IDirect3DDevice9::CreateRenderTarget), regardless of aspect ratio, are stereoized.
By default, non-square surfaces that are equal to or larger than the back buffer are stereoized. Non-square surfaces smaller than the backbuffer are not stereoized by default.
Square surfaces are (by default) not stereoized. The expected usage for these surfaces is typically projected lights or shadow buffers, and therefore they are not eye-dependent.
In order to apply these heuristics to a Direct3D9 title, applications should create the desired renderable surface with IDirect3DDevice9::CreateTexture, taking care to set the D3DUSAGE_RENDERTARGET bit of the usage parameter.
2D Rendering is typically the area of the rendering engine that requires the most care to get right for a successful stereoscopic title.
To render an object without separation, at the same screen-space position in the left and right eye, the best approach is to render these objects at convergence depth. This depth can be retrieved from NVAPI by calling NvAPI_Stereo_GetConvergenceDepth.
If the W coordinate of the output position from the vertex shader is at this depth, no separation will occur between each eye -- the triangles will be at the same screen space position in each eye. See Specifying a Depth for Stereoization below for suggestions of how to modify the vertex shader output position.
While there are other methods available if your title has a custom 3D Vision Automatic profile, this method is the most robust.
As explained in The Existing Conceptual Pipeline, controlling the W coordinate output from the vertex shader is the key to controlling the apparent depth of rendered objects. There are several methods to modify this value; the appropriate method for your application depends on your current pipeline and requirements. None of the methods described here should affect rendering when running in non-stereoscopic modes, and can be left enabled at all times for free.
The preferred method for modifying this depth is to pipe a constant into the vertex shader, then scale the entire output position of the vertex shader to this value.
For example, you might have a vertex shader that looks like this:
float4x4gMatHudWVP;
structVsOutput
{
float4 Position : SV_POSITION;
float4 TexCoord0 : TEXCOORD0;
};
VsOutputRenderHudVS(float4 pos : POSITION,
float2 texCoord0 : TEXCOORD)
{
VsOutput Out;
Out.Position = mul(pos, gMatHudWVP);
Out.TexCoord0 = texCoord0;
return Out;
}In this method, you would first pipe down an additional constant, then modify the final position by this coordinate. This is shown in the following code snippet:
float4x4gMatHudWVP;
floatgApparentDepth; // Depth at which to render the object
structVsOutput
{
float4 Position : SV_POSITION;
float4 TexCoord0 : TEXCOORD0;
};
VsOutputRenderHudVS(float4 pos : POSITION,
float2 texCoord0 : TEXCOORD)
{
VsOutput Out;
Out.Position = mul(pos, gMatHudWVP);
Out.TexCoord0 = texCoord0;
if (Out.Position.w != 0.0f &&gApparentDepth> 0.0f) {
Out.Position *= (gApparentDepth / Out.Position.w);
}
return Out;
}Another approach to this problem, if your application is passing down a 4-component coordinate, is to scale the entire input coordinate by the desired depth. For example, if you were going to render a vertex at (1,2,3,1), you but you wanted it to render at an apparent eye-depth of 20, you could instead render at the position (20,40,60,20). This would produce the exact same screen space position, but would yield correct apparent depth in stereo.
The final approach to specify an apparent depth to the vertex shader is to modify the transform appropriately. As with modifying the input coordinate (as described in Modify the Input Coordinate on page 18), a scaling is all that is necessary. Apply a final scaling transform to your matrix that scales X, Y, Z and W by the desired apparent depth.
After the perspective divide, you will get the same rasterized position and the same value in the Z-buffer, but with a different amount of stereoscopic offsetting.
A common effect in games is to render HUD elements in the world. Some examples of this are:
In all of these cases, stereoscopic can provide additional information to your user over normal HUD layouts. Specifically, the depth cues given by stereoscopic can indicate whether the object being rendered is in front of or behind potential obstructions.
While these objects are technically part of the HUD, they will feel more connected to the object or location they represent in the world if they have a matching amount of separation. The solution to this is to draw the element at an apparent depth value using one of the methods described in Specifying a Depth for Stereoization.
Accurate screen depth can be computed using the following equation
ScreenDepth=normalize(CameraFwd)∙(ObjectWorldPos- CameraWorldPos)
However, NVIDIA has found that this approximation works without causing eyestrain:
ScreenDepth=length(ObjectWorldPos - CameraWorldPos)
Also note that for floating nameplates (for example), it's usually acceptable to draw the nameplate at the depth of the object to which the nameplate is logically attached.
Crosshairs fall into the category of objects described in Wrong is Right. They are actually a part of the HUD, but they just look wrong when drawn at screen depth.
Instead, NVIDIA recommends that you determine the depth of the object drawn under the hotspot of the crosshair, and draw at that apparent depth. NVIDIA refers to this type of crosshair as a Laser Sight.
Laser Sight Crosshairs appear to be on top of the object they're over, which is typically the desired effect. In general, bounding box depth is sufficient for specifying apparent depth. An application can perform a ray cast from the camera location through the crosshair, and compute the length of the returned vector. This length can be used as the apparent depth to render the crosshair at using any of the techniques covered in Specifying a Depth for Stereoization.
The mouse cursor can be thought of exactly like a crosshair that can move around the screen. The guidance for mouse cursors is unsurprisingly identical to that for crosshairs. See Crosshairs for more information.
A common case in post processing is to unproject from window space to world space, perform some calculations and write a value out.
This will pose a problem for stereoscopic rendering because of the hidden mono-to-stereoscopic clip space transformation. To fix this, the mono-to-stereoscopic transformation will also need to be inverted. A potential example of existing unprojection code written for a DirectX renderer might look like this:
float4x4WVPInv; // World-View-Projection Inverse // Viewport Transform Inverse. Stored as // x = 1 / (ResolutionX / 2) // y = 1 / (ResolutionY / 2) // z = -offsetX (usually -1) // w = -offsetY (usually -1) float4VportXformInv; float4ScreenToClip(float2ScreenPos, floatEyeDepth) { float4ClipPos = float4(ScreenPos.xy * VportXformInv.xy + VportXformInv.zw, 0, EyeDepth); // Move the coordinates to the appropriate distance // for the depth specified. ClipPos.xy *= EyeDepth; // Screen and clip space are inverted in the Y direction // from each other. ClipPos.y = -ClipPos.y; returnClipPos; } float4ScreenToWorld(float2ScreenPos, floatEyeDepth) { float4ClipPos = ScreenToClip(ScreenPos, EyeDepth); returnfloat4(mul(WVPInv,ClipPos).xyz, 1.0f); }
The usual method of fixing this is to use a specially crafted stereoscopic texture, which has eye-specific parameters, and use these results to invert the stereoscopic transform. For example:
float4x4WVPInv; // World-View-Projection Inverse // Viewport Transform Inverse. Stored as // x = 1 / (ResolutionX / 2) // y = 1 / (ResolutionY / 2) // z = -offsetX (usually -1) // w = -offsetY (usually -1) float4VportXformInv; // pixel(0.0).x = finalSeparation // pixel(0,0).y = convergence Texture2d StereoParmsTexture; float4StereoToMonoClipSpace(float4 StereoClipPos) { float4MonoClipPos = StereoClipPos; float2StereoParms = tex2D(StereoParmsTexture, 0.0625).xy; MonoClipPos.x -= StereoParms.x * (MonoClipPos.w – StereoParms.y); returnMonoClipPos; } float4ScreenToClip(float2ScreenPos, floatEyeDepth) { float4ClipPos = float4(ScreenPos.xy * VportXformInv.xy + VportXformInv.zw, 0, EyeDepth); // Move the coordinates to the appropriate distance // for the depth specified. ClipPos.xy *= EyeDepth; // Screen and clip space are inverted in the Y direction // from each other. ClipPos.y = -ClipPos.y; returnClipPos; } float4ScreenToWorld(float2ScreenPos, floatEyeDepth) { float4StereoClipPos = ScreenToClip(ScreenPos, EyeDepth); float4MonoClipPos = StereoToMonoClipSpace(StereoClipPos); returnfloat4(mul(WVPInv,MonoClipPos).xyz, 1.0f); }
The specifics of how to build the StereoParmsTexture are available in the nvstereo.h header file, described in Using nvstereo.h.
Note that this solution can utilize the same code path for both stereoscopic and non-stereoscopic cases—by specifying a 0 value for convergence and separation, the result of the computation will match the inputs. This is useful in reducing the code complexity of dealing with stereoscopic being disabled or enabled, and for whether the underlying hardware does or does not support stereo.
One additional difficulty in post processing comes from correctly determining the screen position in the first place. Often, games will perform the complete transform into screen space directly into the vertex shader, storing the results in the output vertex (typically in a texture coordinate). Unfortunately, this also misses the stereo transform, resulting in an incorrect position. This can be easily rectified by using the VPOS (SV_Position in Direct3D10 and higher) built-in attribute in the pixel shader to lookup the current pixel’s screen position. The other alternative would be to perform the stereo transform manually in the vertex shader, before storing the output screen position.
Alternatively, you can make use of the freely available nvstereo.h header to handle building and updating this texture for you in a performance friendly manner. See Using nvstereo.h for more details.
| Note: On Tegra powered devices, post-process fragment shaders should not have any position dependencies when running in stereo mode (e.g., gl_FragCoord should not be used in stereo mode.)
|
Deferred Renderers under DirectX suffer the unprojection problem described in Post Processing, but to a more extreme degree because unprojection is even more common in deferred renderers.
There are two main approaches to fixing this problem. The first solution is exactly the same as described in Post Processing.
The second solution is to simply skip unprojection altogether by storing the world-space position directly in the G-buffer. This solution, while trivial, is not usually practical due to space and bandwidth constraints.
Special care should be taken when using the scissor rectangle for anything that may be stereoized. Currently there is no way to indicate a scissor rectangle for each eye. As a result, objects drawn with a scissor rectangle will likely be clipped incorrectly for both eyes. There is currently no workaround for this issue.
NVIDIA® GameWorks™ Documentation Rev. 1.0.220830 ©2014-2022. NVIDIA Corporation and affiliates. All Rights Reserved.