Possible research project topics

Below are several sample topics for research projects. Other topics are possible as well as long as they are consistant with our research (se my research pages), and possible to support in our lab.

Digital 3D model creation from uncalibrated 2D video

Creating scene and object models is central to content creation for 3-dimensional digital media. As technology is shifting from traditional analog video and photography to digital visual media, it becomes possible to not only handle 2D images, but to store, transmit and view 3D models. However, current techniques for creating 3D models involve cumbersome and labor intensive manual entry of the 3D geometry in modeling programs (essentially 3D versions of 2D computer drawing programs). Modeling currently is a bottleneck to the widespread adoption of 3D visual media. While the high cost of 3D modeling can be motivated in some cases, e.g. CAD modeling in mass production and special effects for major feature films, it currently prevents the transition from 2D to 3D visualizations in mainstream applications.

In this project we develop an approach to automatic 3D model construction from multiple 2D views. This enables users to create 3D models based on standard 2D images and video input. An object can be scanned into a model by simply rotating it in front of a video camera and a scene can be scanned using a handheld video camera to capture different viewpoints. The technical basis of the approach involve recently developed methods in non-Euclidean geometry which show that reconstruction is possible without cumbersome calibration.

For this project we already has an lab infrastructure and a set of programs which perform the capture. (See www.cs.ualberta.ca/~vis/ibmr ) However, to make its use appealing in a wide set of applications we need to integrate it with existing modeling and rendering systems, as well as show its usefulness in a test production.

Specifically for the summer we seek to one computer science person to work on programming a plug-in for our system which integrates it with "Maya" an industry standard modeling and rendering system.

We seek a second person with talent in both arts as well as some computer science knowledge to plan and produce a demonstration of the system. This will involve capturing a number real world objects and characters, modeling a scene, and then integrating the components into a virtual 3D world and use this to produce a computer animation. The proposed topic for the animation would involve recreating a historical scene from museum objects for educational purposes.

Local, triangle-based dynamic texturing

To render realistic images and movies from 3D digital models texturing is used to endow the surface of the model with fine scale properties. In conventional modeling both the 3D geometry and the texture is designed by hand. This is tedious and we instead work with methods rooted in computer vision and non-Euclidean geometry where both the 3D geometry and appearance is captured from real world objects and scenes. A well known challenge with this approach is that the digital model is never quite as rich and accurate as the real world object. If not compensates this causes visual artifacts in the rendered images. We have developed an approach called "dynamic texturing" which instead of using a static texture image renders a time varying, view dependent texture, which compensates for small misalignments in the 3D model and removes these visual artifacts. Currently the dynamic texture is coded as a global texture basis depending on the object-to-camera view. However, and object is represented by numerous facets, each having a different orientation with respect to the camera. A better approach would be to code the view dependent texture w.r.t. each facet. This particularly would improve the visual quality of rendering when perspective effects are large, e.g. in close up views.

This project involves a student with key experience in graphics programming to write a shader-based implementation of our dynamic texturing algorithm, and integrate the shader into our real-time image-based rendering system.

Real-time image-based tele-presence

Telepresence aims to create a high-fidelity simulation of natural human communication and collaboration, on a network of physically distant computers. This involves capturing and modeling the geometry and appearance of remote humans and objects, and unifying these models into one virtual animated scene bringing the physically remote locations into virtual proximity. Particularly, this research is on doing the visual part of telepresence using regular (un-calibrated) cameras, e.g. digital web-cams, and from the 2D real-time video information only build the immersive 3D virtual scene. This is in contrast to traditional computer graphics, which requires a-priori 3D models that are both difficult and expensive to obtain (e.g. by hand-editing or laser scanning)

Specifically the following will be researched and implemented:

360 degree Tracking In order to capture an object from all sides, fiducial points on the object must be tracked whenever they are visible. Our current system is limited to about 30-40 degrees of rotation because it requires that all tracked points be visible at all times. To overcome this, trackable points must be detected, added and removed automatically as they come in and out of view. This involves finding an apropriate methods for detecting areas which will track well, and detecting if and when a particular tracker has lost its target.
Modeling articulated and non-rigid objects To model complex objects (such as humans), more than simple rigid structure is required. Articulated motion should be detected and extracted from the tracked image projections. Methods for geometry extraction from tracked data must be reformulated to accommodate articulated object. Non-rigid motion can be represented by the dynamic textures (as discussed in the Thesis section).
Dealing with occluded textures With support for 360 degree views, and more complicated, self-occluding objects, the entire texture will not be visible in every training image. A method must be found to detect the visibility of all parts of the texture in the training data. Then, only visible parts of the texture will be extracted during the capture stage when the sample textures are analyzed.

The combination of all three of these aspects will enable modeling of complex articulated motion from images. With a method for easily acquiring models and tracking the pose of humans and other complex objects, new applications in HCI and visual computing become tractable. As a pinacle application, we will implement a telepresence system where a human is modeled from video and their three dimensional representation is transmitted over a network, along with streaming audio as a complete communication system.

Investigation of vision-based control for human-assistive robotics

Service robotics and prothesis for the elderly and handicapped are emerging areas in sensory-based intelligent robotics. While robotics has been successfully applied in engineered (e.g. manufacturing) applications, traditional methods for robot control, calibrated in a global Euclidean frame, have proven difficult to apply in everyday (unstructured) human environments. In addition, traditional robots are inflexible and unnatural for the human to program.

In contrast, humans effortlessly interact physically and visually in the world -- a human can easily pick up an visible object, and can also watch a whole task, learn, and transform the visual information into the necessary motor(muscle) movements. In vision-based robotics, instead of conventional programming, the human can show the robot what to do using gestures carrying symbolic (what) and deictic (where) information. In this implementation, the robot and human share the same visual frame, and the robot interprets the visual directions to carry out the task.

A main challenge for hand-eye coordination in uncalibrated environments is how to transform visual information into the motor frame and how to use it for motion control to ensure stable and convergent behavior. In visual feedback motion control this is a continuous process and the instantaneous coordinate transforms can be estimated on-line using Broyden methods from optimization theory. Researchers have solved some example tasks using these ideas, but complete and coherent principles for the application of uncalibrated or partially calibrated methods to arbitrary task are lacking.

In a summer project one or more of following can be studied:

How to formulate typical human manipulation tasks using information from uncalibrated or partially calibrated camera-robot systems. Hespana, Dodds and Hager have recently theoretically proved that the level of camera calibration affects what manipulation tasks are solvable, e.g. for a weakly calibrated camera exactly those tasks expressable by a projectively invariant task function are solvable. We intend to study to what the least amount of information needed for classes of normal human tasks by looking for the least restrictive formulation and invariance needed. This tells us if a task is at all solvable from visual information given particular vision system assumptions, but not how to generate the motions to solve the task.
How to go from visual specifications to actual robot motion. Published vision based control have used simple linear controllers. The stability and convergence of these depend on the task (objective) function properties. We intend to study how to compose geometric task specifications into convex (solvable) task functions, and how to formulate stable, convergent controllers for these.
To implement and study several experimental tasks, such as a robot that can pick up and manipulate objects visually pointed to by a human supervisor. Human-assistive robotics is an area where experimental evaluation of real tasks often suggests new principles for the robotic solutions. The resulting systems can serve as prototypes for how to implement real robotic assistants in home care and rehabilitation.
Current robotic hardware, ie industrial arms, need to be modified to support this kind of control. In particular, low level access directly to the joint servos need to be provided in corrent, velocity and position modes. In this project a student talented in real-time programming and electrical engineering will work togehter with other researchers in the group on providing the hardware and software access. In particular the responsibilities of the new person will be to modify the hardware in an exisiting Unimation PUMA 760 controller to connect the analog controllers and amplifiers with a PC-card based AD/DA card (Servo-2-go card), and hence bypassing the built in inflexible digital controller. Nex the new person will develop the SW drivers for the new HW in collaboration with the otjher researchers to support the use of the new controller under either real-time linus or QNX.

This project bridges computer vision, robotics, rehabilitation engineering and psychophysics on the study of human movement and human interaction with robots. At the U of Alberta precedents for such links already exists in focused projects, proposed or under way, between professors Jagersand (CS), Weber (Kinesiology), Jones (Biomed Engineering) and Pearson (Physiology). Returning to the previous point, even severely handicapped persons (quadroplegics) retain accurate eye movement control. An interesting cross-disciplinary application would be to build an eye-tracker based user interface, where the patient uses his or her gaze direction to control a service robot.