Session 4: Laboratory component:
Examples: Modeling with dynamic texture
Software system
The following software system will be described and used in the hands-on
lab. Printed documentation is in the tutorial notes handed out at the lectures.
Research project overview.
1. Image capture and tracking
The video capture streams video from a digital camera over the ieee1394
network into a real-time visual tracking and capture process "mexv2",
implemented based on
XVision2.
A separate process implements the GUI. An object capture involves moving
the camera to capture the scene from various poses approximately covering
the desired range of viewpoints that the rendering has to support.
Any ieee1394 compliant camera from $100 web cams (e.g. Pyro, iBot) to
professional high frame rate cameras (e.g. Basler) can be used. With better
cameras the raw digitized (Bayer pattern) signal is used. Cheap
cameras are used in YUV422 mode. Better cameras obvioulsy give a
better image quality for the texture generation, but also more
accurate and stable tracking.
2. Geometric structure computation and editor
From the tracked point a structure-from-motion (SFM) method is used
to compute the non-Euclidean viewing geometry. Using the stratified
approach an dself calibration this is upgraded to a Euclidean structure
and set of camera matrices. The editor allows stepping through the captured
sequence and viewing tracked points (black) and reprojected structure (blue).
Any inconsistent points (incorrect correspondences due to tracking errors)
can be removed or adjusted. (select button, left click on point(s) to select
click middle button to exit select mode), then delete button or move button.)
After verifying the geometric structure, the image sequence is processed
into a dynamic texture basis. (Process button)
3. Real-time renderer
The real time renderer takes a geometric structure, the dynamic texture
basis, and implements a glut-based interface for rendering the scene
while using the mouse for viewpoint control (hold left button and move to
rotate about the viewing sphere, middle button to translate in x,y, right button for z).
SW package
Update: We now also have a faster streaming renderer for windows, and
a number of captured models. See this WWW page
The complete linux system is available below.
System requirements
capture_system.tar.gz (22MB to download)