Incremental 3D Line Segment Extraction for Surface Reconstruction from Semi-dense SLAM

People: Shida He, Xuebin Qin, Zichen Zhang, and Martin Jagersand.

It is challenging to utilize the large scale point clouds of semi-dense SLAM for real-time surface reconstruction. In order to obtain meaningful surfaces and reduce the number of points used in surface reconstruction, we propose to simplify the point clouds generated by semi-dense SLAM using 3D line segments. Specifically, we present a novel incremental approach for real-time 3D line segments extraction. Our experimental results show that the 3D line segments generated by our method are highly accurate compared to other methods. We demonstrate that using the extracted 3D line segments greatly improves the quality of 3D surface compared to using the 3D points directly from SLAM systems.


People: Xuebin Qin, Shida He, Zichen Zhang, Masood Dehghan and Martin Jagersand.

Salient closed boundaries are typical common structures of many objects, such as object contours, cup and bin rims. These closed boundaries are hard to be tracked due to the lack of enough textures. In our project, we address the problem by prior shape constrained line segments and edge fragments perceptual grouping. Related techniques, such as edge fragments detection, tracking measure definition and graph based optimization are proposed to form real-time trackers.


People: Xuebin Qin, Shida He, Xiucheng Yang, Masood Dehghan, Qiming Qin and Martin Jagersand.

We developed a semi-automatic image and video annotation tool. This annotation tool replaces the polygons approximation of boundaries by one-pixel-width pixel chains which are smoother and more accurate. It defines objects as groups of one or multiple boundaries that means not only simple objects, which consist of one closed boundary, but also complex objects, such as objects with holes, objects split by occlusions, can be labeled and annotated easily.

MODNet: Moving Object Detection Network

People: Mennatullah Siam, Heba Mahgoub, Mohamed Zahran, Senthil Yogamani, Martin Jagersand, Ahmad El-Sallab

For autonomous driving, moving objects like vehicles and pedestrians are of critical importance as they primarily influence the maneuvering and braking of the car. Typically, they are detected by motion segmentation of dense optical flow augmented by a CNN based object detector for capturing semantics. In this paper, our aim is to jointly model motion and appearance cues in a single convolutional network. We propose a novel two-stream architecture for joint learning of object detection and motion segmentation. We designed three different flavors of our network to establish systematic comparison.

Convolutional Gated Recurrent Networks for Video Segmentation

People: Mennatullah Siam, Sepehr Valipour, Martin Jagersand, Nilanjan Ray

Semantic segmentation has recently witnessed major progress, where fully convolutional neural networks have shown to perform well. However, most of the previous work focused on improving single image segmentation. To our knowledge, no prior work has made use of temporal video information in a recurrent network. In this paper, we introduce a novel approach to implicitly utilize temporal data in videos for online semantic segmentation. The method relies on a fully convolutional network that is embedded into a gated recurrent architecture. This design receives a sequence of consecutive video frames and outputs the segmentation of the last frame.

Modular Tracking Framework (MTF), A Highly Efficient and Extensible Library for Registration based Visual Tracking

People: Abhineet Singh; Mennatullah Siam; Vincent Zhang; Martin Jagersand

MTF is a modular, extensible and highly efficient open source framework for registration based tracking targeted at robotics applications. It is implemented entirely in C++ and is designed from the ground up to easily integrate with systems that support any of several major vision and robotics libraries including OpenCV, ROS, ViSP and Eigen.

Tracking video benchmark database

People: Ankush Roy, Xi Zhang, Nina Wolleb, Camilo Perez Quintero, Martin Jagersand

Tracking human manipulation tasks is challenging. This benchmark contains 100 videos of ordinary human manipulation tasks (pouring cereal, drinking coffee, moving and reading rigid books and flexible letters etc). The videos are categorized w.r.t. task difficulty, speed of motion and light conditions...

Wavelet light basis

People: Cameron Upright; Dana CobzasMartin Jagersand

A wavelet based light representation used for representing and capturing light...

Range intensity registration

People: Dana CobzasHong ZhangMartin Jagersand

A novel image-based algorithm for registering camera intesity data with range data from a laser rangefinder...

Modeling with dynamic texture

People: Martin JagersandDana Cobzas; Keith Yerex; Neil Birkbeck

Inaccuracies in a coarse geometry obtained using structure-from-motion is compensated by an image-based correction - dynamic texture..

Variational Shape and Reflectance Estimation

People: Neil Birkbeck; Dana Cobzas; Peter Sturm; Martin Jagersand

Variational method implemented as a PDE-driven surface evolution interleaved with reflectance estimation...

3D SSD Tracking

People: Martin JagersandDana Cobzas; Peter Sturm

3D SSD tracking A 3D consistent model is imposed to all tracked SSD regions, thus supporting direct tracking of camera position...

An image-based capture system

People: Keith Yerex; Neil Birkbeck; Cleo Espiritu; Cam Upright; Adam Rachmielowski; Dana CobzasMartin Jagersand

A quick and easy to use image-based modeling system for the acquisition of both 3D geometry and visual appearance of real world objects...

On-line tracking and modeling

People: Adam Rachmielowski; Neil BirkbeckMartin JagersandDana Cobzas

On-line 3D reconstruction from tracked feature points...

Activities of Daily Living: Frequency and Timing of Human Tasks

People: Laura Petrich; Martin Jagersand

Human assistive robotics can help the elderly and those with disabilities with Activities of Daily Living (ADL). Robotics researchers approach this bottom-up publishing on methods for control of different types of movements. Health research on the other hand focuses on hospital clinical assessment and rehabilitation using the International Classification of Functioning (ICF), leaving arguably important differences between each domain. In particular, little is known quantitatively on what ADLs humans perform in their ordinary environment - at home, work etc. This information can guide robotics development and prioritize what technology to deploy for in-home assistive robotics. This study targets several large lifelogging databases, where we compute (i) ADL task frequency from long-term low sampling frequency video and Internet of Things (IoT) sensor data, and (ii) short term arm and hand movement data from 30 fps video data of domestic tasks. Robotics and health care have different terms and taxonomies for representing tasks and motions. From the quantitative ADL task and ICF motion data we derive and discuss a robotics-relevant taxonomy in attempts to ameliorate these taxonomic differences.

Point to Line Visual Servoing

People: Mona Gridseth; Camilo Perez Quintero; Romeo Tatsambon; Martin Jagersand

Uncalibrated visual servoing is used to align the end-effector with the marked line on a patient...

Mobile Manipulation and Telerobotics for Space Exploration

People: Azad Shademan; Alejandro Hernandez-Herdocia; David Lovi; Neil Birkbeckartin; Jagersand

We have a partnership with the Canadian Space Agency (CSA) to develop semi-autonomous supervised control in space telerobotics using uncalibrated vision for traking and modeling...

Visual Task Specification

People: Mona Gridseth

A visual interface where the user can specify tasks for the robot to complete using visual servoing...

Image-based maps for robot localization

People: Dana CobzasHong ZhangMartin Jagersand

Panoramic image mosaic augmented with depth information applied to robot localization...

Predictive display

People: Martin Jagersand; Adam Rachmielowski; Dana Cobzas; Azad Shademan; Neil Birkbeck

A predictive display system provided immediate visual feedback from robot site at the operator site...

Uncalibrated Visual Servoing

People: Azad Shademan; Amir-massoud Farahmand; Martin Jagersand

Vision-based motion control of a WAM arm without using camera calibration or knowledge of camera/robot transformation for use in unstructured environments...

A modular and flexible bimanipulation system for space-analogue

People: Alejandro Hernandez-Herdocia; Azad Shademan; Martin Jagersand

An advanced dual-arm mobile manipulator is prototyped for research in semi-autonomous teleoperation and supervisory control...

Communication mechanisms for cooperative human-robot manipulation tasks in unstructured environments

People: Camilo Perez QuinteroMartin Jagersand

Humans teach each other manipulation tasks through gestures and pointing. We develop an HRI where the robot interprets spatial gestures using comuter vision and carries out manipulation tasks...

Tumor growth prediction

People: Dana Cobzas; Parisa Mosayebi; Albert MurthaMartin Jagersand

Predicting tumor invasion margin using a geodesic distance defined on the Riemannian manifold of white fibers...

Semi-automatic segmentation software

People: Neil Birkbeck; Dana CobzasMartin JagersandAlbert Murtha; Tibor Kesztyues, University of Applied Sciences Ulm, Germany

A semi-automatic software for medical image segmentation...See also project description on Neil's webpage

FEM for fast image registration

People: Karteek Popuri; Dana CobzasMartin Jagersand

A finite element implementation for the diffusion-based non-rigid registration algorithm...

A continuous formulation of Conditional Random Fields (CRFs)

People: Dana Cobzas,University of Alberta Mark Schmidt,UBC

A variational formulation for a discriminative model in image segmentation...

Brain Tumor Segmentation

People: Dana Cobzas;  Karteek Popuri; Neil Birkbeck; Mark SchmidtMartin JagersandAlbert Murtha

A variational segmentation method applied to a high-dimensional feature set computed from the MRI...

Muscle and fat segmentation

People: Howard Chung; Dana Cobzas; Jessica Lieffers, Agricultural, Food and Nutritional Science; Laura Birdsell, Oncology; Vickie Baracos, Oncology

Muscle and adipose tissue segmentation in CT images based on a deformable shape...

RW for deformable image registration

People: Dana CobzasAbhishek Sen

A discrete registration method based on the random walker algorithm...