Introduction to Virtual/Augmented Reality and Telepresence

MM-806

(Fall 2018)


Ivan Sutherland`s HMD (1968)	Modern HMDs (Today)

General Information

Instructor: Pierre Boulanger
Tel: 780-492-3031
Email: pierreb@cs.ualberta.ca

URL: www.cs.ualberta.ca/~pierreb
Office: 411 Athabasca Hall
Office hours: By appointment only.

Lectures: Tuesday and Thursdays 13h00 to 14h20 in Tory Building 2-99

Course Description

Virtual reality and augmented reality can provide an immersive environment where many scenarios can be simulated. For example, manufacturing and engineering tasks, medical planning and training, art and design, rehabilitation, Physics, Biology and Chemistry concept exploration and many others can benefit from a virtual reality environment. This course focuses on the challenges of setting up a user friendly virtual reality scene where users can interact in an intuitive and natural way. The use of interactive techniques and sensor-based devices, such as haptic and head-mount display, in creating a virtual environment for scientific analysis, visualization exploration and Tele-presence, as well as how mobile users can participate in these applications, will be discussed.

Prerequisites

Some knowledge of computer graphics and multi-media systems

Assignments

Assignments must be submitted electronically to VRARMM806@gmail.com . Most of the assignments will be using Unity 3D.

There will be 5 problem sets. Don't be misled by the relatively few points assigned to homework grades in the final grade calculation. While the grade that you get on your homework is at most a minor component of your final grade, working the problems is a crucial part of the learning process and will invariably have a major impact on your understanding of the material

Course Project

There will be a group project, culminating in a final 10 pages report in IEEE format and a presentation at a day workshop. Progress and check points before the final due date will count toward the final grade.

Course Grade

The final grade for the course is based on our best assessment of your understanding of the material, as well as your commitment and participation. The problem sets and final projects are combined to give a final grade:

ACTIVITIES	Weight
Final Project	50%
Assignments (5 x 10%)	50%

Lecture Notes

			Course calendar.
LECTURE DATE	TOPICS	Slides		Extras
W1: Sept. 4	First Class Class overview and Irene presentation	Class Overview VR-Intro
W1: Sept 6	Introduction o Definition of Virtual Reality, Augmented Reality, Virtualized Reality, and Telepresence o Generic Configuration of Virtual Reality Systems (image rendering systems, sound rendering systems, haptic rendering systems, communication systems, physical modeling systems, etc.) o Brief overview of the applications of Virtual Reality (in remote robotic control, in medicine, in e-commerce, in communication, in industrial design)	VR-Intro
W2: Sept. 11	History of VR	VR-History
W2: Sept. 13	Short Review of Computer Graphics o Overview and Transformations	VR-Graphics		Installing Unity 3D CAMERAS AND EFFECTS in Unity 1. Cameras 2. Image Effects: Overview GEOMETRY IN UNITY 1. Meshes 2. Mesh Renderers and Mesh Filters Assignment 1: Due Sept. 29
W3: Sept. 18	o Lighting and Shading	VR-Graphics		LIGHTING IN UNITY 1. Unity 5 Lighting Overview 2. Unity 5 - Lighting and Rendering 3. Lights 4. Using Skyboxes RENDERING AND SHADING in Unity . The Standard Shader 2. Materials 3. Textures 4. A Gentle Introduction to Shaders 5. Using detail textures for extra realism close-up 6. Frame Debugger
W3: Sept. 20	No Class
W4: Sept. 25	No Class
W4: Sept. 27	Introduction to Human Vision	VR-Human Vision		Group Description Due
W5: Oct 2	Visual Rendering Systems o Basic characteristics of human visual perception o Stereoscopic display systems (LCD, CRT, HMD, flat, panoramic, and hemispheric screens, etc.) o Detailed analysis of advanced visualization systems o New display devices such as real-time auto-stereogram and retinal writing o Cinematic display	VR-Displays		Assignment 2: Due October 13
W5: Oct 4	o Display Systems Continue	VR-Stereo-Displays
W6: Oct. 9	Digitizing the Human Body o Three-dimensional position sensors (magnetic, ultrasound, photogrammetric, mechanical, and inertial sensors)	VR-Tracking
W6: Oct. 11	o Digitizing fine body motion and forces (DataGlove, CyberGlove, PowerGlove, DHM Dexterous Hand Master, etc.)	VR-Tracking		Assignment 3: Due October 27
W7: Oct. 16	o Tracking Continue	VR-Tracking		Project Description Due
W7: Oct. 18	o Review of student projects	List of Projects
W8: Oct. 23	Sound Rendering Systems o Basic characteristics of human auditory perception o Synthesis of 3D sound (convolvotrons, Beachtrons and Acoustetrons, simple 3D sound) o Unity 3D Sound Spatializer	VR-Sound		Assignment 4: Due November 17
W8: Oct. 25	Haptic Rendering Systems o Human requirements for good haptic perception o Tactile and force feedback (difference between tactile and force feedback, various tactile feedback systems, haptic rendering, force feedback systems) o Combination of tactile and force feedback.	VR-Haptic
W9: Oct. 30	Computing Architectures Graphic requirement of human perception vs. VR graphic engines Basic architecture of a VR graphic engine (graphic pipeline, graphic cards for PC, etc.) Detail analysis of modern graphic hardware Highly parallel VR graphic engines Distributed virtual reality systems	VR-Hardware
W9: Nov. 1	Visit of the SERVIER Virtual Cardiac Centre	We all meet at the AMMI Lab at 13h00 sharp
W10: Nov. 6	Modeling for VR o Geometric Modeling (geometric primitives, scene hierarchies, constructive geometry, etc.) o Cinematic Model (object motion, collision detection, navigation models, motion hierarchies, etc.) o Physical Modeling (gravity, collision, deformable model, surface texture, etc.) o Behavioral Modeling (Artificial life, responsive model, etc.) o Model segmentation (segmentation in cells, LOD, etc.) o Modeling real-life from sensors	VR-Modeling
W10: Nov. 8	Modelling for VR Continue	VR-Modeling
W11: Nov. 13 to Nov. 15	Fall reading week
W12: Nov. 19 to Nov. 22	VR Programming o Java 3D o Vizard Toolkit o Unity 3D o MiddleVR o Scanning Demo	VR-Modeling VR-Programming MiddleVR KScan3D Tutorials
W13: Nov. 27	Human Factors in VR and Augmented Realty	VR-Human-Factors		Assignment 5: Due Dec. 8
W13: Nov. 29	Augmented Reality	Augmented Reality
W14: Dec. 4	Tele-presence and Cinematic VR	VR-Telepresence
W14: Dec. 6	Applications of VR and AR o Medicine and rehabilitation (surgery, anatomic simulator, remote surgery, hybrid systems) o VR games o Arts (virtual actors, virtual museum, virtual music, virtual theatre) o Virtual product design (CAD display, process simulation, virtual prototyping) o Robotic (robot and virtual reality, design of robots, robot programming, supervisory control, the Mars Rover) o Teaching systems (military training 'SIMNET', NASA training systems, flight simulators) o Virtual teleconferencing systems	VR-Future
W15: Dec. 21	Students Presentations and Demos
W15: Dec. 21	Final Report Due

Additional Recommended Books

Handbook of Virtual Environments: Design, Implementation, and Applications. 2nd Edition. K. S. Hale, K. M. Stanney, Eds. 2014.
3D User Interfaces: Theory and Practice. D. A. Bowman, E. Kruijff, J. J. LaViola, and I. Poupyrev. 2014.
Understanding Virtual Reality: Interface, Application, and Design. W. R. Sherman and A. B. Craig. 2012.

Developer References

Research Articles

Other Online Resources