Possible research project topics
Below are several sample topics for research projects. Other
topics are possible as well as long as they are consistant with
our research (se my research pages), and possible to support in
Creating scene and object models is central to content creation for
3-dimensional digital media. As technology is shifting from
traditional analog video and photography to digital visual media, it
becomes possible to not only handle 2D images, but to store, transmit and
view 3D models. However, current techniques for creating 3D models
involve cumbersome and labor intensive manual entry of the 3D
geometry in modeling programs (essentially 3D versions of 2D computer
drawing programs). Modeling currently is a bottleneck to the widespread
adoption of 3D visual media. While the high cost of 3D modeling can be
motivated in some cases, e.g. CAD modeling in mass production and
special effects for major feature films, it currently prevents
the transition from 2D to 3D visualizations in mainstream applications.
In this project we develop an approach to automatic 3D model construction
from multiple 2D views. This enables users to create 3D models based
on standard 2D images and video input. An object can be scanned into
a model by simply rotating it in front of a video camera and a scene can
be scanned using a handheld video camera to capture different viewpoints.
The technical basis of the approach involve recently developed methods
in non-Euclidean geometry which show that reconstruction is possible
without cumbersome calibration.
For this project we already has an lab infrastructure and a set
of programs which perform the capture. (See www.cs.ualberta.ca/~vis/ibmr )
However, to make its use appealing in a wide set of applications we
need to integrate it with existing modeling and rendering systems, as
well as show its usefulness in a test production.
Specifically for the summer we seek to one computer science
person to work on programming a plug-in for our system which integrates
it with "Maya" an industry standard modeling and rendering system.
We seek a second person with talent in both arts as well as
some computer science knowledge to plan and produce a demonstration of
the system. This will involve capturing a number real world objects
and characters, modeling a scene, and then integrating the components
into a virtual 3D world and use this to produce a computer animation.
The proposed topic for the animation would involve recreating a
historical scene from museum objects for educational purposes.
To render realistic images and movies from 3D digital models
texturing is used to endow the surface of the model with fine
scale properties. In conventional modeling both the 3D geometry
and the texture is designed by hand. This is tedious and we instead
work with methods rooted in computer vision and non-Euclidean geometry
where both the 3D geometry and appearance is captured from real world
objects and scenes. A well known challenge with this approach is that
the digital model is never quite as rich and accurate as the
real world object. If not compensates this causes visual artifacts
in the rendered images. We have developed an approach called
"dynamic texturing" which instead of using a static texture image
renders a time varying, view dependent texture, which compensates
for small misalignments in the 3D model and removes these visual
artifacts. Currently the dynamic texture is coded as a global
texture basis depending on the object-to-camera view. However,
and object is represented by numerous facets, each having a different
orientation with respect to the camera. A better approach would be
to code the view dependent texture w.r.t. each facet. This
particularly would improve the visual quality of rendering when
perspective effects are large, e.g. in close up views.
This project involves a student with key experience in graphics
programming to write a shader-based implementation of our dynamic
texturing algorithm, and integrate the shader into our real-time
image-based rendering system.
Telepresence aims to create a high-fidelity simulation of natural human
communication and collaboration, on a network of physically distant
computers. This involves capturing and modeling the geometry and
appearance of remote humans and objects, and unifying these models into
one virtual animated scene bringing the physically remote locations into
virtual proximity. Particularly, this research is on doing the visual part
of telepresence using regular (un-calibrated) cameras, e.g. digital
web-cams, and from the 2D real-time video information only build the
immersive 3D virtual scene. This is in contrast to traditional computer
graphics, which requires a-priori 3D models that are both difficult and
expensive to obtain (e.g. by hand-editing or laser scanning)
Specifically the following will be researched and implemented:
The combination of all three of these aspects will enable modeling of
complex articulated motion from images. With a method for easily acquiring
models and tracking the pose of humans and other complex objects, new
applications in HCI and visual computing become tractable. As a pinacle
application, we will implement a telepresence system where a human is
modeled from video and their three dimensional representation is
transmitted over a network, along with streaming audio as a complete
- 360 degree Tracking In order to capture an object from all sides,
fiducial points on the object must be tracked whenever they are visible.
Our current system is limited to about 30-40 degrees of rotation because
it requires that all tracked points be visible at all times. To overcome
this, trackable points must be detected, added and removed automatically
as they come in and out of view. This involves finding an apropriate
methods for detecting areas which will track well, and detecting if and
when a particular tracker has lost its target.
- Modeling articulated and non-rigid objects To model complex objects (such
as humans), more than simple rigid structure is required. Articulated
motion should be detected and extracted from the tracked image
projections. Methods for geometry extraction from tracked data must be
reformulated to accommodate articulated object. Non-rigid motion can be
represented by the dynamic textures (as discussed in the Thesis section).
- Dealing with occluded textures With support for 360 degree views, and more
complicated, self-occluding objects, the entire texture will not be
visible in every training image. A method must be found to detect the
visibility of all parts of the texture in the training data. Then, only
visible parts of the texture will be extracted during the capture stage
when the sample textures are analyzed.
Service robotics and prothesis for the elderly and handicapped are
emerging areas in sensory-based intelligent robotics. While robotics
has been successfully applied in engineered (e.g. manufacturing)
applications, traditional methods for robot control, calibrated in a
global Euclidean frame, have proven difficult to apply in everyday
(unstructured) human environments. In addition, traditional robots are
inflexible and unnatural for the human to program.
In contrast, humans effortlessly interact physically and visually in
the world -- a human can easily pick up an visible object, and can
also watch a whole task, learn, and transform the visual information
into the necessary motor(muscle) movements. In vision-based robotics,
instead of conventional programming, the human can show the robot what
to do using gestures carrying symbolic (what) and deictic (where)
information. In this implementation, the robot and human share the
same visual frame, and the robot interprets the visual directions to
carry out the task.
A main challenge for hand-eye coordination in uncalibrated
environments is how to transform visual information into the motor
frame and how to use it for motion control to ensure stable and
convergent behavior. In visual feedback motion control this is a
continuous process and the instantaneous coordinate transforms can be
estimated on-line using Broyden methods from optimization
theory. Researchers have solved some example tasks using these ideas,
but complete and coherent principles for the application of
uncalibrated or partially calibrated methods to arbitrary task are
In a summer project one or more of following can be studied:
This project bridges computer vision, robotics, rehabilitation engineering and psychophysics on the study of human movement and human interaction with robots. At the U of Alberta precedents for such links already exists in focused projects, proposed or under way, between professors Jagersand (CS), Weber (Kinesiology), Jones (Biomed Engineering) and Pearson (Physiology). Returning to the previous point, even severely handicapped persons (quadroplegics) retain accurate eye movement control. An interesting cross-disciplinary application would be to build an eye-tracker based user interface, where the patient uses his or her gaze direction to control a service robot.
- How to formulate typical human manipulation tasks using information from uncalibrated or partially calibrated camera-robot systems. Hespana, Dodds and Hager have recently theoretically proved that the level of camera calibration affects what manipulation tasks are solvable, e.g. for a weakly calibrated camera exactly those tasks expressable by a projectively invariant task function are solvable. We intend to study to what the least amount of information needed for classes of normal human tasks by looking for the least restrictive formulation and invariance needed. This tells us if a task is at all solvable from visual information given particular vision system assumptions, but not how to generate the motions to solve the task.
- How to go from visual specifications to actual robot motion. Published vision based control have used simple linear controllers. The stability and convergence of these depend on the task (objective) function properties. We intend to study how to compose geometric task specifications into convex (solvable) task functions, and how to formulate stable, convergent controllers for these.
- To implement and study several experimental tasks, such as a robot that can pick up and manipulate objects visually pointed to by a human supervisor. Human-assistive robotics is an area where experimental evaluation of real tasks often suggests new principles for the robotic solutions. The resulting systems can serve as prototypes for how to implement real robotic assistants in home care and rehabilitation.
- Current robotic hardware, ie industrial arms, need to be modified to
support this kind of control. In particular, low level access directly
to the joint servos need to be provided in corrent, velocity and
position modes. In this project a student talented in real-time
programming and electrical engineering will work
togehter with other researchers in the group on providing the hardware
and software access. In particular the responsibilities of the new
person will be to modify the hardware in an exisiting Unimation PUMA
760 controller to connect the analog controllers and amplifiers with
a PC-card based AD/DA card (Servo-2-go card), and hence bypassing
the built in inflexible digital controller. Nex the new person
will develop the SW drivers for the new HW in collaboration with
the otjher researchers to support the use of the new controller under
either real-time linus or QNX.