1 Introduction (Slides pdf, html)

The confluence of theory and methods in computer graphics and computer vision. Survey of recent methods and applicaitons in image-based modeling and rendering. Survey of related areas and applications. [3,14,5]

A survey with videos and intuitive explanations of both what we will cover in detail as well as other related IBR.

2 Camera models, non-Euclidean viewing geometries and their use in computer graphics (Slides pdf , html)

Basic concepts about viewing geometry. Single view geometry. Camera models and calibration. Application to raysets: light field, lumigraph, concentric mosaics, space carving. [8,10,19,20,28]

3 Scene model building and representation in non-Euclidean framework (Slides pdf , html)

Acquiring depth. Two view geometry. Depth from stereo. Epipolar constraint. Application in warping with depth: Laveau and Faugeras, plenoptic modeling, layer depth images

Acquiring geometric models from uncalibrated images. Multiview geometry. Stratification of projective, affine, eucledian geometry. Projective reconstruction. Affine reconstruction. Self calibration. Bundle adjustment. Application to hybrid visual modeling and rendering approaches:

[13,27,30,29,26,7,4]

4 Texturing of models using recent image-based approaches (Slides pdf , html)

Standard texture warps. Texturing from images: Combined reflectance and light. Issues in texture alignments and coordinate mappings. View dependent textures. Bump mapping. Relief textures. Texturing with depth from stereo images. Combining and representing a texture variability as a spatial basis. Modulating time-varying textures. Estimating texture variation from sample scene images. [15,1,22,24,18,25,21,16]

5 Principles and practice of real time tracking on consumer hardware (Slides html)

Tracking as a dynamic systems problem. Tracking versus correspondance. Tracking of individual features: points, lines, blobs. Parameterized SSD tracking of regions. Robust methods. Combining trackers with geometry to track whole objects and scene. Mixing Real-time video pipeline and software structure fro tracking under Linux. Some direct applications of tracking in VR. [11,12,2]

6 Real-time rendering of image-based models and implementation tricks on consumer-grade graphics hardware (Slides pdf , html)

Capabilities of current graphics acceleration and how it helps image-based modeling and rendering. Intro to writing code for graphics HW. NVidia register combiners, DirectX, CG. Fast numerical linear algebra for texture modulation and tracking. [6]

7 Hands-on demo and PC lab (Slides html)

Video capture and real-time tracking on a pc with analog and digital consumer cameras. Building and editing image based models. Incorporating and rendering image-based object in virtual scenes. [23,9]

Bibliography

1
M. Black, D. Fleet, and Y. Yacoob.
Robustly estimating changes in image appearance.
Computer Vision and Image Understanding, 78(1):8-31, 2000.

2
Michael J. Black and Allan D. Jepson.
Eigentracking: Robust matching and tracking of articulated objects using a view-based representation.
In ECCV (1), pages 329-342, 1996.

3
D. Cobzas and M. Jagersand.
A comparison of non-euclidean image-based rendering.
In Proceedings of Graphics Interface, 2001.

4
D. Cobzas and M. Jagersand.
A comparison of viewing geometries for augmented reality.
In Scandinavian Conference on Image Analysis (SCIA) - submission, 2003.

5
D. Cobzas, M. Jagersand, and K. Yerex.
Editing real world scenes: Augmented reality with image-based rendering (poster).
In IEEE Virtual Reality, 2003.

6
D. Cobzas, K. Yerex, and M. Jagersand.
Dynamic textures for image-based rendering of fine-scale 3d structure and animation of non-rigid motion.
In Eurographics, 2002.

7
P. E. Debevec, C. J. Taylor, and J. Malik.
Modeling and rendering architecture from phtographs.
In Computer Graphics (SIGGRAPH'96), 1996.

8
O. D. Faugeras.
Three Dimensional Computer Vision: A Geometric Viewpoint.
MIT Press, Boston, 1993.

9
G. A. Geist and V. S. Sunderam.
Network-based concurrent computing on the PVM system.
j-CPE, 4(4):293-311, June 1992.

10
S. J. Gortler, R. Grzeszczuk, and R. Szeliski.
The lumigraph.
In Computer Graphics (SIGGRAPH'96), pages 43-54, 1996.

11
Gregory D. Hager and Peter N. Belhumeur.
Efficient region tracking with parametric models of geometry and illumination.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025-1039, 1998.

12
Gregory D. Hager and Kentaro Toyama.
X vision: A portable substrate for real-time vision applications.
Computer Vision and Image Understanding, 69(1):23-37, 1998.

13
R. I. Hartley and A. Zisserman.
Multiple View Geometry in Computer Vision.
Cambridge University Press, 2000.

14
M. Jagersand.
Saliency maps and attention selection in scale and spatial coordinates: An information theoretic approach.
In Fifth International Conference on Computer Vision, 1995.

15
M. Jagersand.
Image based view synthesis of articulated agents.
In Computer Vision and Pattern Recognition, 1997.

16
M. Jagersand and D. Cobzas.
Modulating view-dependent textures.
In SIGGRAPH - submission, 2003.

17
K. Kutulakos and S. Seitz.
A theory of shape by shape carving.
International Journal of Computer Vision, 38:197-216, 2000.

18
Kuang-Chih Lee, Jeffrey Ho, and David Kriegman.
Nine points of light: Acquiring subspaces for face recognition under variable lighting.
In Computer Vision and Pattern Recognition, 2001.

19
M. Levoy and P. Hanrahan.
Light field rendering.
In Computer Graphics (SIGGRAPH'96), pages 31-42, 1996.

20
L. McMillan and G. Bishop.
Plenoptic modeling: Am image-based rendering system.
In Computer Graphics (SIGGRAPH'95), pages 39-46, 1995.

21
H. Murase and S. Nayar.
Visual learning and recognition of 3d objects from appearance.
International Journal of Computer Vision, 14:5-24, 1995.

22
Manuel M. Oliviera, Gary Bishop, and David McAllister.
Relief texture mapping.
In Computer Graphics (SIGGRAPH'00), 2000.

23
W. H. Press, B.P. Flannery, S. A. Teukolsky, and W. T. Ve tterling.
Numerical Recipies in C: The Art of Scientific Computing.
Cambridge University Press, Cambridge, England, 1992.

24
S. M. Seitz and C. R. Dyer.
View morphing.
In Computer Graphics (SIGGRAPH'96), pages 21-30, 1996.

25
Amnon Shashua.
Geometry and Photometry in 3D Visual Recognition.
PhD thesis, MIT, 1993.

26
I. Stamos and P. K. Allen.
Integration of range and image sensing for photorealistic 3d modeling.
In ICRA, 2000.

27
Peter Sturm and Bill Triggs.
A factorization based algorithm for multi-image projective structure and motion.
In ECCV (2), pages 709-720, 1996.

28
R. Szeliski.
Video mosaics for virtual environments.
IEEE Computer Graphics and Applications, pages 22-30, March 1996.

29
C. Tomasi and T. Kanade.
Shape and motion from image streams under orthography: A factorization method.
International Journal of Computer Vision, 9:137-154, 1992.

30
D. Weinshall and C. Tomasi.
Linear and incremental aquisition of invariant shape models from image sequences.
In Proc. of 4th Int. Conf. on Compute Vision, pages 675-682, 1993.

Visual Computing 2003-02-20