Introduction to Virtual/Augmented
Reality and Telepresence
MM-806
(Fall 2020)
|
|
|
Ivan Sutherland`s HMD (1968)
|
Modern HMDs (Today)
|
General Information
Instructor: Pierre Boulanger
Tel: 780-492-3031
Email: pierreb@cs.ualberta.ca
URL: www.cs.ualberta.ca/~pierreb
Office: 411 Athabasca Hall
Office hours: By appointment only.
Lectures: Tuesday and Thursday 7h30 to 8h50
Room: Remote
TAs:
n Garett Hunter garett1@ualberta.ca
Office Hours: Friday 10 am - 11 am (Zoom)
n
Thea Wang ywang3@ualberta.ca
Office Hour: Wednesday 8 pm- 9 pm (Zoom)
n
Gabriel Lugo Bustillo lugobust@ualberta.ca
Office hour: Monday 8 pm- 9 pm (Zoom)
Course
Description
Virtual reality and
augmented reality can provide an immersive environment where many scenarios can
be simulated. For example, manufacturing and engineering tasks, medical
planning and training, art and design, rehabilitation, Physics, Biology and
Chemistry concept exploration, and many others can benefit from a virtual
reality environment. This course focuses on the challenges of setting up a
user-friendly virtual reality scene where users can interact intuitively and
naturally. The use of interactive techniques and sensor-based devices, such as
haptic and head-mount display, in creating a virtual environment for scientific
analysis, visualization exploration, and Tele-presence, as well as how mobile
users can participate in these applications, will be discussed.
Prerequisites
Some knowledge of computer graphics and multimedia
systems
Assignments
Assignments must be submitted electronically to
VRARMM806@gmail.com . Most of the assignments will be using Unity 3D or Unreal Engine.
There will be five problem sets. Don't be misled by the relatively few points assigned to
homework grades in the final grade calculation. While the grade that you get on
your homework is at most a minor component of your final grade, working the
problems is a crucial part of the learning process and will invariably have a
significant impact on your understanding of the material
Course Project
There will be a group
project, culminating in a final ten pages report in IEEE format and a
presentation at a day workshop. Progress and checkpoints before the last due
date will count toward the final grade.
Course Grade
The final grade for the course is based on our
best assessment of your understanding of the material, as well as your
commitment and participation. The problem sets and final projects are combined
to give a final grade:
|
ACTIVITIES |
Weight |
|
Final Project |
50% |
|
Assignments (5 x 10%) |
50% |
Lecture Notes
|
TOPICS |
Slides |
Extras |
|
|
First Class Class overview and Irene presentation |
|||
|
Introduction o Definition
of Virtual Reality, Augmented Reality, Virtualized Reality, and Telepresence o Generic
Configuration of Virtual Reality Systems (image rendering systems, sound
rendering systems, haptic rendering systems, communication systems, physical
modeling systems, etc.) o Brief
overview of the applications of Virtual Reality (in remote robotic control,
in medicine, in e-commerce, in communication, in industrial design) |
|
||
|
History of VR |
|||
|
Short Review
of Computer Graphics o
Overview and Transformations |
CAMERAS AND EFFECTS in Unity 1. Cameras GEOMETRY IN UNITY 1. Meshes 2. Mesh Renderers and Mesh Filters Assignment 1: Due 29
September
|
||
|
o
Lighting
and Shading |
LIGHTING IN UNITY 2. Unity
5 - Lighting and Rendering 3. Lights RENDERING AND SHADING in Unity 2. Materials 3. Textures 4. A Gentle Introduction to Shaders |
||
|
Introduction to Human Vision |
|
Group Description: Due
October 1 |
|
|
Visual Rendering Systems o Essential characteristics of human
visual perception o Stereoscopic display systems (LCD,
CRT, HMD, flat, panoramic, and hemispheric screens, etc.) o Detailed analysis of advanced
visualization systems o New display devices such as real-time
auto-stereogram and retinal writing o Cinematic display |
Assignment
2: Due
13 October |
||
|
o
Display
Systems Continue |
|
||
|
Digitizing the Human Body o
Three-dimensional
position sensors (magnetic, ultrasound, photogrammetric, mechanical, and
inertial sensors) |
VR Term Projects By Monday, Oct.
28, I would like each team to
send me the following information: o A
project titles o The
name of each member of the team o A
short description of the project o A
diagram of the VR interface functionality o An
estimated timeline of the execution of the project, o A
list of the equipment required. Once I get this, I
will meet with each team individually and discuss the logistic of the
project. You can find last
year's teams project proposals at Projects.rar |
||
|
o
Digitizing
body motion and forces (DataGlove, CyberGlove, PowerGlove, DHM
Dexterous Hand Master, etc.) |
Assignment
3: Due
1 November |
||
|
o
Tracking
Continue |
|||
|
o
Review
of student projects |
|||
|
Sound Rendering Systems o Essential characteristics of human
auditory perception o Synthesis of 3D sound (convolvotrons, Beachtrons and Acoustetrons, simple 3D sound) |
Assignment
4: Due 10 November |
||
|
Haptic Rendering Systems o Human requirements for
good haptic perception o Tactile and force
feedback (difference between tactile and force feedback, various tactile
feedback systems, haptic rendering, force feedback systems) o
Combination of tactile and force feedback. |
|
||
|
Computing Architectures
|
|
||
|
Modeling for VR o Geometric
Modeling (geometric primitives, scene hierarchies, constructive geometry,
etc.) o Cinematic
Model (object motion, collision detection, navigation models, motion
hierarchies, etc.) o Physical
Modeling (gravity, collision, deformable model, surface texture, etc.) o Behavioral
Modeling (Artificial life, responsive model, etc.) o Model segmentation (segmentation in cells,
LOD, etc.) o Modeling
real-life from sensors |
|
||
|
Modeling for VR Continue |
|||
|
Fall reading week |
Week of Nov. 9-13 |
|
|
|
VR Programming o Java 3D o Vizard
Toolkit o Unity 3D o MiddleVR |
|||
|
Human Factors in VR and Augmented Reality |
Assignment 5: Due
5 December |
||
|
Augmented Reality |
|
||
|
Tele-presence and Cinematic VR |
|||
|
Applications of VR and AR o Medicine and
rehabilitation (surgery, anatomic simulator, remote surgery, hybrid systems) o VR games o Arts
(virtual actors, virtual museum, virtual music, virtual theatre) o Virtual
product design (CAD display, process simulation, virtual prototyping) o Robotic
(robot and virtual reality, design of robots, robot programming, supervisory
control, the Mars Rover) o Teaching
systems (military training 'SIMNET', NASA training systems, flight
simulators) o Virtual
teleconferencing systems |
|
||
|
Students Presentations and Demos |
|
Starts
week of December 9 |
|
|
Final Report |
|
Due
no later than December 13 |
|
|
Recommended Reference
Books o Handbook of Virtual Environments: Design,
Implementation, and Applications. 2nd Edition. K. S. Hale, K. M. Stanney, Eds. 2014. o 3D User Interfaces: Theory and Practice. D. A.
Bowman, E. Kruijff, J. J. LaViola,
and I. Poupyrev. 2014. o Understanding Virtual Reality: Interface,
Application, and Design. W. R. Sherman and A. B. Craig. 2012. Developer References Research Articles o Stereoscopy and the Human Visual System o Optometric and Perceptual Issues with Head-mounted Displays Other Online Resources o A developer's
perspective on immersive 3D computer graphics o https://www.edx.org/professional-certificate/virtual-reality-vr-app-development
o http://stanford.edu/class/ee267/ o http://moodle.epfl.ch/course/view.php?id=6841 o https://www.evl.uic.edu/aej/528/ |
|||