Possible research project topics

Below are several sample topics for research projects. Other topics are possible as well as long as they are consistant with our research (se my research pages), and possible to support in our lab.

Digital 3D model creation from uncalibrated 2D video

Creating scene and object models is central to content creation for 3-dimensional digital media. As technology is shifting from traditional analog video and photography to digital visual media, it becomes possible to not only handle 2D images, but to store, transmit and view 3D models. However, current techniques for creating 3D models involve cumbersome and labor intensive manual entry of the 3D geometry in modeling programs (essentially 3D versions of 2D computer drawing programs). Modeling currently is a bottleneck to the widespread adoption of 3D visual media. While the high cost of 3D modeling can be motivated in some cases, e.g. CAD modeling in mass production and special effects for major feature films, it currently prevents the transition from 2D to 3D visualizations in mainstream applications.

In this project we develop an approach to automatic 3D model construction from multiple 2D views. This enables users to create 3D models based on standard 2D images and video input. An object can be scanned into a model by simply rotating it in front of a video camera and a scene can be scanned using a handheld video camera to capture different viewpoints. The technical basis of the approach involve recently developed methods in non-Euclidean geometry which show that reconstruction is possible without cumbersome calibration.

For this project we already has an lab infrastructure and a set of programs which perform the capture. (See www.cs.ualberta.ca/~vis/ibmr ) However, to make its use appealing in a wide set of applications we need to integrate it with existing modeling and rendering systems, as well as show its usefulness in a test production.

Specifically for the summer we seek to one computer science person to work on programming a plug-in for our system which integrates it with "Maya" an industry standard modeling and rendering system.

We seek a second person with talent in both arts as well as some computer science knowledge to plan and produce a demonstration of the system. This will involve capturing a number real world objects and characters, modeling a scene, and then integrating the components into a virtual 3D world and use this to produce a computer animation. The proposed topic for the animation would involve recreating a historical scene from museum objects for educational purposes.

Local, triangle-based dynamic texturing

To render realistic images and movies from 3D digital models texturing is used to endow the surface of the model with fine scale properties. In conventional modeling both the 3D geometry and the texture is designed by hand. This is tedious and we instead work with methods rooted in computer vision and non-Euclidean geometry where both the 3D geometry and appearance is captured from real world objects and scenes. A well known challenge with this approach is that the digital model is never quite as rich and accurate as the real world object. If not compensates this causes visual artifacts in the rendered images. We have developed an approach called "dynamic texturing" which instead of using a static texture image renders a time varying, view dependent texture, which compensates for small misalignments in the 3D model and removes these visual artifacts. Currently the dynamic texture is coded as a global texture basis depending on the object-to-camera view. However, and object is represented by numerous facets, each having a different orientation with respect to the camera. A better approach would be to code the view dependent texture w.r.t. each facet. This particularly would improve the visual quality of rendering when perspective effects are large, e.g. in close up views.

This project involves a student with key experience in graphics programming to write a shader-based implementation of our dynamic texturing algorithm, and integrate the shader into our real-time image-based rendering system.

Real-time image-based tele-presence

Telepresence aims to create a high-fidelity simulation of natural human communication and collaboration, on a network of physically distant computers. This involves capturing and modeling the geometry and appearance of remote humans and objects, and unifying these models into one virtual animated scene bringing the physically remote locations into virtual proximity. Particularly, this research is on doing the visual part of telepresence using regular (un-calibrated) cameras, e.g. digital web-cams, and from the 2D real-time video information only build the immersive 3D virtual scene. This is in contrast to traditional computer graphics, which requires a-priori 3D models that are both difficult and expensive to obtain (e.g. by hand-editing or laser scanning)

Specifically the following will be researched and implemented:

The combination of all three of these aspects will enable modeling of complex articulated motion from images. With a method for easily acquiring models and tracking the pose of humans and other complex objects, new applications in HCI and visual computing become tractable. As a pinacle application, we will implement a telepresence system where a human is modeled from video and their three dimensional representation is transmitted over a network, along with streaming audio as a complete communication system.

Investigation of vision-based control for human-assistive robotics

Service robotics and prothesis for the elderly and handicapped are emerging areas in sensory-based intelligent robotics. While robotics has been successfully applied in engineered (e.g. manufacturing) applications, traditional methods for robot control, calibrated in a global Euclidean frame, have proven difficult to apply in everyday (unstructured) human environments. In addition, traditional robots are inflexible and unnatural for the human to program.

In contrast, humans effortlessly interact physically and visually in the world -- a human can easily pick up an visible object, and can also watch a whole task, learn, and transform the visual information into the necessary motor(muscle) movements. In vision-based robotics, instead of conventional programming, the human can show the robot what to do using gestures carrying symbolic (what) and deictic (where) information. In this implementation, the robot and human share the same visual frame, and the robot interprets the visual directions to carry out the task.

A main challenge for hand-eye coordination in uncalibrated environments is how to transform visual information into the motor frame and how to use it for motion control to ensure stable and convergent behavior. In visual feedback motion control this is a continuous process and the instantaneous coordinate transforms can be estimated on-line using Broyden methods from optimization theory. Researchers have solved some example tasks using these ideas, but complete and coherent principles for the application of uncalibrated or partially calibrated methods to arbitrary task are lacking.

In a summer project one or more of following can be studied:

This project bridges computer vision, robotics, rehabilitation engineering and psychophysics on the study of human movement and human interaction with robots. At the U of Alberta precedents for such links already exists in focused projects, proposed or under way, between professors Jagersand (CS), Weber (Kinesiology), Jones (Biomed Engineering) and Pearson (Physiology). Returning to the previous point, even severely handicapped persons (quadroplegics) retain accurate eye movement control. An interesting cross-disciplinary application would be to build an eye-tracker based user interface, where the patient uses his or her gaze direction to control a service robot.