Session 4: Laboratory component:

Examples: Modeling with dynamic texture


Modulated texture vs. traditional texture while rendering a rotating quadrilateral with a wreath


Reanimation of a flower


Changing illumination on a face

Software system

The following software system will be described and used in the hands-on lab. Printed documentation is in the tutorial notes handed out at the lectures. Research project overview.

1. Image capture and tracking

The video capture streams video from a digital camera over the ieee1394 network into a real-time visual tracking and capture process "mexv2", implemented based on XVision2. A separate process implements the GUI. An object capture involves moving the camera to capture the scene from various poses approximately covering the desired range of viewpoints that the rendering has to support. Any ieee1394 compliant camera from $100 web cams (e.g. Pyro, iBot) to professional high frame rate cameras (e.g. Basler) can be used. With better cameras the raw digitized (Bayer pattern) signal is used. Cheap cameras are used in YUV422 mode. Better cameras obvioulsy give a better image quality for the texture generation, but also more accurate and stable tracking.

2. Geometric structure computation and editor

From the tracked point a structure-from-motion (SFM) method is used to compute the non-Euclidean viewing geometry. Using the stratified approach an dself calibration this is upgraded to a Euclidean structure and set of camera matrices. The editor allows stepping through the captured sequence and viewing tracked points (black) and reprojected structure (blue). Any inconsistent points (incorrect correspondences due to tracking errors) can be removed or adjusted. (select button, left click on point(s) to select click middle button to exit select mode), then delete button or move button.)

After verifying the geometric structure, the image sequence is processed into a dynamic texture basis. (Process button)

3. Real-time renderer

The real time renderer takes a geometric structure, the dynamic texture basis, and implements a glut-based interface for rendering the scene while using the mouse for viewpoint control (hold left button and move to rotate about the viewing sphere, middle button to translate in x,y, right button for z).

SW package

Update: We now also have a faster streaming renderer for windows, and a number of captured models. See this WWW page

The complete linux system is available below.

System requirements

capture_system.tar.gz (22MB to download)