Algorithm for Stereoscopic 3D Perspective Generation

The following material is extrapolated from The StereoGraphics Developer's Handbook by Lenny Lipton, for describing how to create effective stereoscopic perspective views. The complete handbook is available for downloading from StereoGraphics web site from the Developer's page.

(Illustrations are reproduced from the handbook; text notes are by Dave Milici; Airplane example originally by Bob Akka, Chasm Graphics. Updated OpenGL and DirectX info.)

Two Separate Left and Right Perspective Views

The perception of 3-dimensional depth is accomplished by generating separate left and right perspective views. It is the gradual differences between the left and right perspectives, called parallax, which provide the visual "cues" for perceiving depth in a 3D environment.

A computer graphics programmer should recognize the perspective projection technique illustrated here. For a stereoscopic application program, the perspective projection step is executed twice, once for each perspective view.

 

 

 

 

 

Viewpoint Separation with Parallel Axes

For an accurate reproduction of the pair of stereoscopic views, it is important to keep the axes of the left and right camera lenses parallel. This insures against distorting the projection images in a manner which cinematographers call "keystoning".

(Note that this may not seem to be intuitively appropriate. One might expect that since we physically rotate our eyeballs inward to converge upon objects in our environment, that we should likewise rotate the camera lens axes inward for reproducing such a scene. However research studies, particularly in the academic field, have shown that such a technique distracts from the overall depth perception effect, due to the afore-mentioned keystone distortion.)

When using the parallel-axes camera lens model, there is an additional step described below which helps observers converge their eyes on a stereoscopically reproduced image. This adjustment step makes the rotated-axes method unncessary.

In a stereoscopic computer graphics application, the left and right viewpoints are simply offset from a central observer viewpoint (which would be the default observer viewpoint in a monoscopic application).

 

 

 

Image Shift Compensation

By default, the "centers" of objects in each of the parallel-projection images will never coincide, unless they are at the far background. So a compensation step is used to the shift the images horizontally back towards each other.

By shifting the projection images, the observer has the ability to control the relative "out-of-screen" or "into-screen" parallax of objects in the scene. Typically the overall parallax effect is determined by a combination of the pre-rendering viewpoint separation step and post-rendering image shift step.

In a stereoscopic computer graphics application, the image shift may be accomplished by using an asymetric viewing frustum (or pyramid), such as supported in OpenGL. The projected images are automatically shifted off-center when rendered through a viewing frustum volume that is asymetrically skewed. In applications where the programmer has no such control over the viewing frutsum, (ie, symetric viewing frustum by default), the image shifts can be obtained by specifiying clipped viewport windows which are horizontally offset from each other. A minor side-effect in this case is that pixels at the far left and right edges are lost in the clipping.

Overall Parallax Control

This technique exactly mimics the traditional stereoscopic camera setup. The cameraperson would photograph a scene with lenses aimed parallel at the scene, and an observer would be able shift the left and right photographs held in the stereoscope viewer for comfortable parallax settings.

Overall parallax control is determined by a combination of both viewpoint separation and image shift adjustment. Increasing viewpoint separation widens the parallax between left and right views for greater overall depth effect. Compensating image shift values determine where the zero-parallax setting is located in the scene, and therefore serve to balance the out-of-screen and into-screen depth effects.

In a computer graphics application, the programmer has the ability to control both viewpoint separation and image shift adjustment throughout the image synthesis process. The user would also be allowed to fine-tune both of these parallax parameters for optimal stereoscopic depth effects.

Program Example Code

The basic steps for stereoscopic rendering presented above may be represented by the following pseudo-code:

SelectRenderingBuffer(LEFT);
SetBufferWindow(0+dxShift, 0, width, height);
RenderPerspective(x-dxViewpt, y, z);
 
SelectRenderingBuffer(RIGHT);
SetBufferWindow(0-dxShift, 0, width, height);
RenderPerspective(x+dxViewpt, y, z);
 
UpdateDisplayBuffers();

In this piece of psuedo-code, x, y, and z refer to the 3D coordinates of the default monoscopic viewpoint, +/- dxViewpt refers to the stereo viewpoint separation, width and height refer to the rendering viewport window dimensions, and +/- dxShift refers to the stereo image shift adjustment.

In an actual application program, the viewpoint separation parameter, dxViewpt, and the image shift parameter, dxShift, should be adjustable by the user for optimum parallax setting. In complex 3D scenes, you might want to have the user adjust these parameters twice at extreme zoom-in and zoom-out positions. With knowledge of parallax settings at each extreme, it is also possible to calculate intermediate parallax settings automatically via interpolation.

OpenGL

C code fragments for stereoscopic rendering with OpenGL using asymetric frustum:

 
// build left-eye perspective view
glDrawBuffer(GL_BACK_LEFT);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustum(-1-dxShift, 1-dxShift, -1, 1, 10);
// additional transforms here...(optional)
glTranslatef(0.0+dxViewpt, 0.0, 0.0);
glMatrixMode(GL_MODELVIEW);
// render objects in scene here...
glFlush();
 
// build right-eye perspective view
glDrawBuffer(GL_BACK_RIGHT);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustum(-1+dxShift, 1+dxShift, -1, 1, 10);
// additional transforms here...(optional)
glTranslatef(0.0-dxViewpt, 0.0, 0.0);
glMatrixMode(GL_MODELVIEW);
// render objects in scene here...
glFlush();
 
// update buffers to display
glutSwapBuffers();
 

This technique can be adapted to existing OpenGL applications by modifying the left/right rendering code fragments into a 2-pass loop surrounding existing rendering code. Appropriate usage of local variables and flags can select left, right, or central perspective rendering for stereoscopic or monoscopic viewing modes.

In some instances, re-constructing the projection matrix each time through the rendering loop may be undesirable or unavailable. In these cases, you could temporaily save and restore the pre-built projection matrix using glPushMatrix() and glPopMatrix(), and solely affect the viewpoint separation with the final matrix operation glTranslatef(). Image shift could be performed by specifying clipped viewport windows with glViewport().

for (i=0; i<2; i++) {
// left or right rendering buffer
glDrawBuffer((i==0) ? GL_BACK_LEFT : GL_BACK_RIGHT);
glViewPort((i==0) ? -dxShift : dxShift, 0, width, height);
// left or right eye perspective view
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glTranslatef((i==0) ? dxViewpt : -dxViewpt, 0.0, 0.0);
glMatrixMode(GL_MODELVIEW);
// render objects in scene here...
glFlush();
glMatrixMode(GL_PROJECTION);
glPopMatrix();
}
 
// update buffers to display
glutSwapBuffers();

Note that setting up a stereoscopic display mode is not addressed here since setting up display modes in general are deliberately not included in the OpenGL specification. Instead the programmer should use the additional OpenGL toolkit libraries or system-specfic support APIs to enable a GL_STEREO compatible display mode.

Complete program examples and support notes can be downloaded from these sites:

Note that the GLUT stereo examples are also available as part of the SciTech GLDirect SDK, and the binary executables are included as the stereo demos in the SciTech GLDirect 2.0 trialware distribution.

Direct3D

C++ code fragments for stereoscopic rendering with Direct3D Retained Mode and stereo extension driver:

// setup subordinate camera views and viewport windows in advance
lpCameraLeft->SetPosition(lpD3DCamera, -dxViewpt, 0.0, 0.0);
lpCameraRight->SetPosition(lpD3DCamera, dxViewpt, 0.0, 0.0);
lpViewportLeft->Configure(0, 0, width-dxShift, height);
lpViewportRight->Configure(dxShift, 0, width-dxShift, height);
 
// in main rendering loop:
 
// render scene from left-eye view in one buffer
lpD3DDevice->SetRenderTarget(lpDDSQuad[iPair+0]);
lpViewportLeft->SetCamera(lpCameraLeft);
lpViewportLeft->Render(lpD3DScene);
 
// render scene from right-eye view in another buffer
lpD3DDevice->SetRenderTarget(lpDDSQuad[iPair+1]);
lpViewportRight->SetCamera(lpCameraRight);
lpViewportRight->Render(lpD3DScene);
 
// use stereo extension API to update stereo quad buffers
FlipStereo(lpDDSPrimary, lpDDSQuad[iPair+0], lpDDSQuad[iPair+1], 0);
iPair = (iPair) ? 0 : 2;
 

This example takes advantage of the 3D object heirarchy of a Direct3D Retained Mode application. Subordinate 3D reference frames from the default camera viewpoint are created for lpCameraLeft and lpCameraRight with relative geometry, so no matrix operations are necessary. Separate viewport windows lpViewportLeft and lpViewportRight are used for affecting image shifts in the renderings. DirectDraw surfaces are explicitly referenced as stereo quad buffers, lpDDSQuad[4], for Direct3D rendering targets.

This example also uses the FlipStereo() API extension proposed by StereoGraphics to Microsoft's DirectX team. The "placeholder" support for stereo in DirectX versions prior to DX7 will not work as specified because the separately specified left and right DirectDraw surfaces can not be updated synchronously. The DirectX programmers have acknowledged this oversight, and have incorporated linked stereo surface support for later releases of DirectX (DX7).

To date, only DirectX 7 provides the more correct API for stereoscopic display support. However this can only be implemented by few OEMs which can actually provide hardware-based stereo page-flipping capability, and none have actually done so. The FlipStereo() functionality was folded in to the existing Flip() function using the added DDFLIP_STEREO flag. As such, this means that only fullscreen display modes could be used for stereo page flipping. Refer to the DX7 D3DIM SDK for code examples using the D3D Framework library. DirectX 8 changed the D3D API all over again, and does not support stereo surfaces (most likely due to lack of OEM support for stereo in the DirectX 7 API).

// use DirectDraw7 API to update stereo quad buffers
lpDDSPrimary->Flip(lpDDSQuad[iPair+0], DDFLIP_STEREO);
iPair = (iPair) ? 0 : 2;

SciTech Software has a DirectDraw Stereo SDK specifically to provide comprehensive stereo device support. The DDStereo library supports many more chipsets which do not have hardware stereo capability by using software page-flip emulation, and is able to support them in windowed display modes too. Instead of using DirectDraw Flip() or Blt() functions, the program uses the DDStereo library's DDS_scheduleFlip() function to handle all stereo format updating. Refer to the SciTech DDStereo SDK for complete code examples.

// use SciTech DDStereo API to update stereo quad buffers
DDS_scheduleFlip(lpDDSQuad[iPair+0], lpDDSQuad[iPair+1]);
iPair = (iPair) ? 0 : 2;

The following are source code file excerpts from DirectX SDK examples modified by DaveM. The latest versions use the DDStereo API and cannot be compiled without the complete DDStereo SDK present.

If you have permission to access SciTech's public file server using Perforce, you can find the complete DDStereo SDK examples located in the ..\public\examples\ddstereo directory path. The binary executables are included as part of the stereo demos in SciTech Display Doctor 7.0 for Windows and must use the accompanying SciTech Nucleus drivers for operation in stereo display modes.

Contact Dave Milici at Ironic Research Labs via email: davem@ironicresearch.com

Back