Mathematical Models in Medical Image Analysis

Reading group

Meetings :

Mondays at noon, CS 3-49

Contact :

Dana Cobzas, cobzas[at]ualberta.ca

The reading group will cover theory and practical applications related to medical image processing. At each meeting 1 paper will be presented and discussed in about 1/2 hour. Anyone who is interested is welcome to attend. Send an e-mail (cobzas[at]ualberta.ca) if you are like to attend and we will include you on the list.

We we mainly discuss papers on image segmentation, registration, shape analysis and specific techniques for brain image analysis. Most of the modern methods are formulated in a continuous space and utilize geometric partial differential equations (PDEs) in conjunction with image processing techniques. A good review paper that covers the main mathematical methods used in medical imaging is presented by [Angenent et al. Bull. AMS 2006].



Date Topic Papers Presenter
Monday Nov 21, 2016 Graph Cuts for image segmentation [Boykov et al ICCV06] Min Tang
Monday Nov 7, 2016 Convex relaxation for binary segmentation [Chan et al. ICIP05] Daniel Cremers (video)
Monday Oct 3, 2016 Levelset segmentation - overlap data prior [Ben Ayed et al ICCV2009] Dana Cobzas
Monday Sept 26, 2016 Levelset segmentation and shape priors [Rousson, Paragios IJCV 2008] Min Tang
Wed Aug 24, 2016 Shape currents - statistics [Mansi et al TMI 2011] Dana Cobzas
Wed July 13, 2016 Shape currents - application to longitudinal prediction [Rekik,Dinggang et al IPMI2015] Dana Cobzas
Wed July 6, 2016 Shape currents [Durrleman et al. MIA 2009]
Gary Christensen video
Wed June 22, 2016 LDDMM registration [Beg et al IJCV2005]
Dana Cobzas
Wed June 8, 2016 Sparse regression [Kandel et al. IPMI 2013]
Li Zhang
Wed June 1, 2016 Shape: Diffusion on surfaces [Chung et al. MIA 2015]
Dana Cobzas
Wed May 25, 2016 Shape: LB matching [Lombaert et al. IPMI 2013]
Min Tang
Wed May 18, 2016 Shape: LB representation [Levy SMI 2006]
Dana Cobzas
Wed Nov 11, 2015 Shape: LB matching [Lombaert et al. IPMI 2013]
Dana Cobzas
Wed Oct 28, 2015 Shape: LB representation [Levy SMI 2006]
Dana Cobzas
Wed Oct 14, 2015 Shape: SH representation [Chung et al. Neuroim 2010]
Dana Cobzas
Wed July 15, 2015 Patch-based segmentation [Coupe et al Neuroim 2011]
Dana Cobzas
Wed July 8, 2015 Personalized atlas [Commowick et al. MICCAI 2009] Adam Harrison
Wed June 17, 2015 Multi atlas fusion [Sabuncu et al. TMI 2010]
Dana Cobzas
Wed Feb 4, 2015 Sparse methods [Schmidt et al.2008]
Dana Cobzas
Wed Jan 21, 2015 Sparse methods [Kandel et al. 2013]
Dana Cobzas
Tu August 6, 2013 Segmentation - Convex formulation [Punithakumar et al. 2012]
Kumar Punithakumar
Tu July 30, 2013 Auto-context in segmentation [Tu and Bai PAMI2010]
Dana Cobzas
Tu July 16, 2013 Power watersheds [Couprie et al. ICCV 2009]
Adam Harrison
Tu July 9, 2013 Learning hierarchical invariant spatio-temporal features [Le et al. 2011]
Ankush Roy
Tu June 25, 2013 Learning feature representations [Coates and Ng 2012]
Matt Low
Tu June 18, 2013 Multi-atlas segmentation : EM formulation [Sabuncu et al. 2010]
Karteek Popuri
Tu June 4, 2013 Segmentation with random forests [Criminisi et al MICCAI 2010]
Matt Low
Thu Dec 13, 2012 Continuous max-flow/min-cut algorithms [Yuan et al. CVPR 2010]
Karteek Popuri
Thu Nov 15, 2012 Segmentation via convex relaxation [Ben Ayed et al. MICCAI 2012]
Kumaradevan Punithakumar
Thu Nov 1, 2012 Joint tumor segmentation and MRI registration [Parisot et al. MICCAI 2012]
Dana Cobzas
Thu Oct 18, 2012 Heart motion abnormality detection [Punithakumar et al. MICCAI 2012]
Kumaradevan Punithakumar
Thu Sept 27, 2012 MRF-based active contours [Xu et al. MICCAI 2010]
Karteek Popuri
Wed Sept 7, 2011 Diffsuion maps [Lafon et al. PAMI 2006]
Dana Cobzas
Wed, Aug 24, 2011 Diffsuion maps [Coifman,Lafon et al. 2005]
Travis Dick
Wed, Aug 10, 2011 Shapes - Spectral embedding [Bronstein et al. 2008]
Karteek Popuri
Tu, July 26, 2011 Shapes - MDS [Bronstein et al. 2008]
Dana Cobzas
Wed, Jun 15, 2011 Shapes - rigid geometry ICP [Bronstein et al. 2008]
Neil Birkbeck
Wed, Jun 8, 2011 Shape matching - signed distance functions [Dedner et al.DAGM 2007]
Matt Gallivan
Wed, May 25, 2011 Region statistics - overlap prior [Ayed et al. IJCV 2009]
Dana Cobzas
Wed, May 11, 2011 Local region statistics [Mori et al. ICCV 2007]
Karteek Popuri
Wed, Feb 9, 2011 Face detection [Viola,Jones IJCV04]]
Karteek Popuri
Wed, Jan 26, 2011 Detection and segmentation using MSL [Ling et al. CVPR08]]
Neil Birkbeck
Wed, Dec 8, 2010 Convex formulation of active contours [Bresol et al. 2007 ]
Dana Cobzas
Wed, Dec 1, 2010 Geodesic distance [Bai and Sapiro ICCV2007]
Karteek Popuri
Wed, Nov 10, 2010 Geocut [Boykov and Kolmogorov ICCV2003]
Dana Cobzas
Wed, Nov 3, 2010 Interactive segmentation - Graph Cut [Boykov and Jolly ICCV2001]
Maddy Wang
Wed, Oct 20, 2010 Interactive segmentation - Random walker with priors [Grady CVPR2005]
Abhishek Sen
Wed, Oct 13, 2010 Interactive segmentation - Random walker [Grady PAMI2006]
Karteek Popuri
Wed, April 8, 2009 Atlas - patient with tumor regsitration by combining a growth and a deformable model [Cuadra et al. TMI04 ]
Dana Cobzas
Wed, April 1, 2009 Using mechanical model for mass effect in the context of patient-atlas registration [Mohamed et al. MIA06 ]
Dana Cobzas
Wed, March 18, 2009 Mechanical model for mass effect in tumor growth [Ayache et al. TMI05]
Parisa Mosayebi
Wed, February 25, 2009 STAPLE - a framework for estimating ground thruth by combining data from several experts [Warfield STAPLE 04]
Bret Hoehn
Wed, February 18, 2009 Coupled registration with multiple atlases [Rousson 06]
Parisa Mosayebi
Wed, January 28, 2009 Coupled locally affine registration framework [Commowick 06]
Dana Cobzas
Wed, January 14, 2009 Bayesian formulation for atlas registration-segmentation [Pohl et al NeuroImage06]
Karteek Popuri
Tu, October 28, 2008 Riemannian structure of white matter [Lenglet et al TR03 ]
Dana Cobzas
Wed, October 1, 2008 Log Euclidean metric for DTI [Arsigny MRM06 IJCV06]
Dana Cobzas
Wed, September 17, 2008 The Riemannian Manifold of DTI [Pennec IJCV06]
[Weickert and Hagen 2006 Ch17]
Dana Cobzas
Wed, September 10, 2008 Spatial tranformations of DTI [Alexander 2001] Dana Cobzas
Wed, September 3, 2008 DTI Reglarization [Weickert and Brox 02] Dana Cobzas
Wed, August 27, 2008 DTI Estimation, Reglarization and Visualization [Tschumperle and Deriche ICCV03]
Dana Cobzas
Wed, June 7, 2008 Intro to DTI [Weickert and Hagen 2006 Ch5]
Parisa Mosayebi
Wed, May 7, 2008 Tumor growth prediction model [Ayache et al. MICCAI06]
Parisa Mosayebi
Wed, April 30, 2008 Tumor growth model [Ayache et al. TR04]
Dana Cobzas
Wed, March 26, 2008 Learning the Diffusion Tensor [Black et al ICCV03]
[Roth and Black ICCV07]
Dana Cobzas
Wed, March 12, 2008 Diffusion [Weickert and Brox 02] Dana Cobzas
Wed, October 3, 2007 MRI basics (II) [Huettel ch4]
[Huettel ch5]
Matt Brown
Wed, September 26, 2007 MRI basics (I) [Huettel ch2]
[Huettel ch3]
Matt Brown
Wed, September 5, 2007 Super-resolution [Freeman 02] Dana Cobzas
Wed, August 29, 2007 Shape priors [Rousson et al. 04] Karteek Popuri
Wed, August 22, 2007 Segmentation: texture [Rousson et al. 03] Dana Cobzas
Wed, August 15, 2007 Segmentation: Continuous methods - active regions [Chan Vese 02] Karteek Popuri
Wed, August 8, 2007 Segmentation: Discrete methods - CRFs, DRFs [Kumar, Hebert 03] Alireza Farhangfar
Wed, August 1, 2007 Segmentation: Discrete methods - graph cuts [Boykov 06] Nilanjan Ray
Wed, July 25, 2007 Registration: Dense deformation field [Stefanescu 04] Martin Jagersand
Wed, July 18, 2007 Registration: Local deformations, BSplines [Rueckert 99] Dana Cobzas
Wed, July 11, 2007 Registration: Rigid body [ch2: Human Brain Function] Matt Brown

MRI basics

Basic and easy to read intro to MRI imaging
[Plewes et al.] D.B. Plewes, W. Kucharczyk: The Physics of MRI - Basic Spin Gymnastics 2003

More in depth description of MRI basics
[Huettel Ch 2,3,4,5] Scott A. Huettel: Functional Magnetic Resonance Imaging (book) 2004

Registration

Registration techniques refer to automatic methods that align two or more data sets with each other. In the context of medical images, the task can involve registration of data coming from the same patient but at different times and/or different modalities (intra-patient registration). Alternatively the registration can involve different patients and atlases (inter-patient registration).

One other important aspect that has to be considered in image registration is the similarity measure. We will discuss different choices including SSD (sum of square differences), NCC (normalized cross correlation), CR (correlation ratio), MI (mutual information). A good overview of different similarity measures in the context of non-rigid deformation is presented in [Hermosillo PhD].

Existing techniques can be classified into two major categories : global registration and local, non-rigid registration. In the first case the two data sets are related by a global linear transformation (e.g. rigid body transformation-Euclidean, similarity, affine) while in the second case a full non-linear deformation field is recovered.

Local deformations can be described by a linear combination of a set of low frequency basis functions (e.g. Discrete Sine or Discrete Cosine Transforms) [Ashburner 99] (method used by the SPM package). One advantage of this approach is that the number of recovered parameters are relatively low, but they are global over the image space.

One other popular approach in computing local deformations uses a grid of control points. The methods recover only deformations at the control points and interpolate the values for all interior points. One examples is to express deformations as linear combinations of BSplines defined on a regular grid [Rueckert 99], [Metaxas 04]. The major disadvantage of this model is the computational complexity. To overcome this problem, [Rohde 03] defined deformations as linear combinations radial symmetric functions (with local support) whose location adapt depending on the residual error of the matching function. An interesting deformation model that uses a set of coupled affine transformations for different anatomical structures in the brain is proposed in [Commowick 06].Their method registers the curent data with a labeled atlas.

One last approach in image registration is to recover a dense deformation field between the two images. The methods regard images as continuous functions and recover a diffeomorphism that represents the mapping that will align then according to a similarity criterion. As the problem is in general ill-posed, an additional regularization prior has to be imposed. An overview and classification of registration algorithm can be found in [Cachier 03]. Initially the problem was formulated using an elastic model that penalizes large deformations. Alternatively, incremental or fluid methods recover the transformation as an evolution process. The approach is closely related to the optic flow recovery. A classical example is the viscoelastic algorithm by [Christensen 96]. A very popular extension is the demons algorithm [Thirion 98] where deformations are first recovered based on image forces which create displacements and then regularized by Gaussian filtering. [Pennec 98] reformulate the demons algorithm in a variational framework. [Cachier 03] later generalized the algorithm by proposing PARSHA, a two step method that uses a mixed elastic-fluid regularization. While most methods use isotropic regularization, anisotropic regularization has also been addressed [Hermosillo PhD]. Recently [Stefanescu 04] designed a data and atlas-driven regularization that accounts for local anatomy variability as well as presence of pathology (e.g. tumor).

Rigid-body registration

[ch2: Human Brain Function] . Ashburner and K. Friston: Human Brain Function, Second Edition, Chapter 2

Local deformations - parametric models

[Rueckert 99] Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Non-rigid registration using free-form deformations: Application to breast MR images, IEEE Trans. Med. Imag., 1999 (BSplines)

[Metaxas 04] X. Huang, D. Metaxas, T.Chen: MetaMorphs: Deformable Shape and Texture Models, CVPR 2004 (BSplines)

[Rohde 03] Rohde, G.K. and Aldroubi, A. and Dawant, B.M. : The adaptive bases algorithm for intensity-based nonrigid image registration, IEEE Transactions on Medical Imaging, 22(11), 2003 (radial symmetric basis)

[Ashburner 99] Ashburner J, Friston KJ.: Nonlinear spatial normalization using basis functions, Human Brain Mapping, 7 (4) 1999 (DCT basis)

[Commowick 06] Commowick , V. Arsigny , J. Costa , N. Ayache , G. Malandain: An Efficient Locally Affine Framework for the Registration of Anatomical Structures, Medical Image Analysis, 12(4):427-441, 2008.

Dense deformation field (diffeomorphism)

[Thirion 98] J.-P. Thirion: Image matching as a diffusion process: an analogy with Maxwell's demons, Med. Image Anal., 1998

[Pennec 98] Xavier Pennec and Pascal Cachier and Nicholas Ayache: Understanding the "Demon's Algorithm": 3D Non-rigid Registration by Gradient Descent, MICCAI '99

[Cachier 03] Pascal Cachier and Eric Bardinet and Didier Dormont and Xavier Pennec and Nicholas Ayache: Iconic feature based nonrigid registration: the PASHA algorithm, Comput. Vis. Image Underst., 2003

[Cachier 04] Pascal Cachier and Nicholas Ayache: Isotropic Energies, Filters and Splines for Vector Field Regularization, J. Math. Imaging Vis., 2004

[Stefanescu 04] R. Stefanescu, X. Pennec , N. Ayache: Grid Powered Nonlinear Image Registration with Locally Adaptive Regularization, Medical Image Analysis, 2004

[Hermosillo PhD] Gerardo Hermosillo: Variational Methods for Multimodal Image Matching, PhD Thesis, University de Nice, 2002

[Hermosillo IJCV02] Gerardo Hermosillo and Christophe Chefd'hotel and Olivier Faugeras: Variational Methods for Multimodal Image Matching, IJCV 2002

Coupled registration with segmentation

[Pohl et al NeuroImage06] K. M. Pohl and J. Fisher and W.E.L. Grimson and R. Kikinis and W.M. Wells: A Bayesian Model for Joint Segmentation and Registration, NeuroImage, 2006

[Pohl ICCV05] K.M. Pohl and J. Fisher and R. Kikinis and W.E.L. Grimson and W.M. Wells: Shape Based Segmentation of Anatomical Structures in Magnetic Resonance Images, ICCV 2005

[Commowick 06] Commowick , V. Arsigny , J. Costa , N. Ayache , G. Malandain: An Efficient Locally Affine Framework for the Registration of Anatomical Structures, Medical Image Analysis, 12(4):427-441, 2008.

[Rousson 06] Mikaekl Rousson and Chenyang Xu: A General Framework for Image Segmentation Using Ordered Spatial Dependency, MICCAI 2006

Registration of brain images with tumor pathology

Using a mechanical model for mass effect in the context of registration of brain images with tumor
[Mohamed et al. MIA 06] Mohamed, A. and Zacharaki, E.I. and Shen, D. and Davatzikos, C.: Deformable registration of brain tumor images via a statistical model of tumor-induced deformation, Medical Image Analysis, 10(5):752-763, 2006

Atlas - patient with tumor registration by combining a growth model with a deformable model
[Cuadra et al. TMI04 ] Meritxell Bach Cuadra, Claudio Pollo, Anton Bardera, Olivier Cuisenaire and Jean-guy Villemure: Atlas-based segmentation of pathological MR brain images using a model of lesion growth, IEEE Transactions on Medical Imaging, 2004

Extension of non-rigid daemon's registration method that uses a confidence field for dealing with abnormal and less reliable image regions.
[Stefanescu 04] R. Stefanescu, X. Pennec , N. Ayache: Grid Powered Nonlinear Image Registration with Locally Adaptive Regularization, Medical Image Analysis, 2004

Segmentation

Image segmentation is a central problem in medical image analysis. Some early techniques like region growing or split-and-merge do not require a clear definition of a cost functional being mostly heuristic and mot very robust. Modern technique minimize a cost functional and can be divided in two main categories : discrete, combinatorial methods and continuous, variational methods (see [Boykov 06] for a nice overview of existing methods). More references for variational methods (referred as active contours and active regions) can be found on the last summer level set reading group page or in the review paper [Cremers 07].

Among segmentation methods some use an explicit representation for segmentation contour. Examples include dynamic programming and path-based graph as discrete methods (that are mostly only 2D approaches). Continuous explicit methods are referred as snakes in the literature (e.g. [Kass et al. IJCV87]).

An alternative to explicit representation is an implicit, level set representation of the contour. Discrete such methods based on graph cuts became quite popular since [Boykov and Jolly 2001] proposed the global minimization s/t cut technique. When training data is available, discriminative methods based on CRFs (Conditional Random Fields) are succefully used for segmentation and classification [Kumar, Hebert 03] [Shotton 06].

Continuous methods based on level sets are more flexible as they can easily incorporate different types of energy terms. The first implicit edge-based active contour was proposed simultaneously by [Caselles et al. IJCV97] and [Kichenassamy et al. 96]. A better solution for medical image segmentation is to consider region information (rather than local edge information - see next paragraph). [Chan Vese 02] proposed a popular active region model based on the Mumford-Shah functional.

One other aspect that differentiate different image segmentation methods are the cues that are used in designing the energy functional. Most algorithms based on explicit contour representation use only static image cues (mostly image edges given by the image gradients). While the edge detector ensures that the information on both sides of the edge is dissimilar it does not deal with the problem that the interior of the region has to be homogeneous. A solution to this shortcoming is to incorporate region statistics in the curve evolution. The initial homogeneous model from [Chan Vese 02] was later extended to more complicated statistics like Gaussian Model or non-parametric Parzen histogram. Also texture information was incorporated trough image features (calculated using Gabor filters or structure tensor) [Rousson et al. 03] . While most region-based segmentation methods are formulated using the likelihood model ([Chan Vese 02],[Cremers 07]) that assumes maximum separation between foreground and background statistics, recently other less restricted formulations based on foreground/background overlap (distance between distributions) have been proposed [Ayed et al. IJCV 2009].

Both edge and region information are image cues. People also considered geometric cues that give information about the desired shape of the contour. examples include - length, flux or shape priors. In medical image segmentation, shape priors are quite important when the segmentation target has a expected anatomical shape [Rousson et al. 04].

Discrete, combinatorial methods

Combinatorial graph cuts
[Boykov 06] Yuri Boykov, Gareth Funka-Lea : Graph Cuts and Efficient N-D Image Segmentation, IJCV 2006

Link between graph cuts and geodesic contours (level sets)
[Boykov and Kolmogorov ICCV 2003] Yuri Boykov, Vladimir Kolmogorov : Computing Geodesics and Minimal Surfaces via Graph Cuts, ICCV 2003

CRF-DRF
[Kumar, Hebert 03] S. Kumar and M. Hebert. Discriminative random fields: A discriminative framework for contextual interaction in classification, CVPR, 2003

CRF for object segmentation and recognition using shape and appearance
[Shotton 06] Jamie Shotton, John Winn, Carsten Rother, Antonio Criminisi: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-Class Object Recognition and Segmentation, ECCV 2006

Continuous, variational methods

[Cremers 07] D. Cremers and M. Rousson and R. Deriche: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape, IJCV 2007

Active contours - snakes
[Kass et al. IJCV87] Kass, M. and Witkin, A. and Terzopolous, D.: Snakes: Active Contour Models, IJCV 1987

Geodesic active contours
[Caselles et al. IJCV97] V. Caselles and R. Kimmel and G. Sapiro: Geodesic Active Contours, IJCV 1997

Active regions
[Chan Vese 02] Tony F. Chan, Luminita A. Vese: Active Contour and Segmentation Models Using Geometric PDE's for Medical Imaging, Geometric Methods in Bio-Medical Image Processing, Series: Mathematics and Visualization, Springer 2002

Convex formulation of active contours
[Breson et al 2007] X. Bresson, S. Esedoglu, P. Vandergheynst, J-P Thiran and S. Osher: Fast Global Minimization of the Active Contour/Snake Model, Journal of Mathematical Imaging and Vision, 28(2), 2007

Models for regions statistics

Local priors
[Mori et al. ICCV 2007] Mory, B., Ardon, R., Thiran, J.-P.; Variational Segmentation using Fuzzy Region Competition and Local Non-Parametric Probability Density Functions, ICCV 2007

Overlap priors
[Ayed et al. IJCV 2009] I. Ben Ayed, S. Li and I. Ross A Statistical Overlap Prior for Variational Image Segmentation, IJCV 2009

Texture features - non-parametric model
[Rousson et al. 03] M. Rousson and T. Brox and R. Deriche: Active unsupervised texture segmentation on a diffusion based feature space, CVPR 2003

Shape priors

[Rousson et al. 04] M. Rousson and N. Paragios and R. Deriche: Implicit Active Shape Models for 3D Segmentation in MR Imaging, MICCAI 2004

Validation

[Warfield STAPLE 04] Simon K. Warfield and Kelly H. Zou and William M. Wells: Simultaneous Truth and Performance Level Estimation {(STAPLE)}: An Algorithm for the Validation of Image Segmentation, IEEE Trans Med Imaging, 2004

Interactive Segmentation

Object segmentation is an important part of scene understanding and interpretation and a crucial task in medical imaging. It involves finding the regions of the image that correspond to a target object or, alternatively the boundary of the object. As the segmentation task is often ambiguous, an interactive technique seams to be the right choice for a successful segmentation. The modern variations of interactive segmentation are primarily build on top of small set of core algorithms - graph cuts, random walker and shortest path. Viewing the image as a graph, these methods minimize an energy functional discretized on the graph to produce a segmentation. User interaction is usually done by specifying foreground/background pixels using scribblings.

The Random Walker algorithm [Grady PAMI2006] assigns each unlabeled pixel to the seed for which there is a minimum diffusion distance. In the original formulation the segmentation is calculated based on localized image data. The extension from [Grady CVPR2005] solves this problem by introducing regional intensity priors. Recently [Zhang et al. CVPR2010] formulated the RW method as anisotropic heat diffusion by considering the seeded pixels as heat sources.

The Graph Cut algorithm [Boykov and Jolly ICCV2001] estimates image segmentation by finding the minimum cut between foreground and background seeds via a max flow computation. An popular iterative version of the algorithm is GrabCut [Rother et al. SIGG2004 ] where the user draws a bounding box to disambiguate the target object instead of scribbling background/foreground seeds.

Shortest path algorithms [Bai and Sapiro ICCV2007] assign each pixel to the foreground if the geodesic distance to a foreground seed (on the weighted graph) is smaller than any geodesic distance to background seeds.

Random walker

[Grady PAMI2006] Leo Grady: Random Walks for Image Segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 28(11) 2006

Extension to include region prior models
[Grady CVPR2005] Leo Grady: Multilabel Random Walker Image Segmentation Using Prior Models, CVPR 2005

Formulation as anisotropic diffusion
[Zhang et al. CVPR2010] Juyong Zhang, Jianmin Zheng, Jianfei Cai: A Diffusion Approach to Seeded Image Segmentation, CVPR 2010

Unifying framework for RW, GC and geodist as continuous-valued MRF
[Singaraju et al. CVPR2009] Dheeraj Singaraju, Leo Grady, Rene Vidal: P-Brush: Continuous Valued MRFs with Normed Pairwise Distributions for Image Segmentation,CVPR 2009

Graph Cuts

[Boykov and Jolly ICCV2001] Yuri Boykov and Marie-Pierre Jolly: Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D images, ICCV 2001

Bonding box interaction
[Rother et al. SIGG2004 ] Carsten Rother and Vladimir Kolmogorov and Andrew Blake: "GrabCut": interactive foreground extraction using iterated graph cuts, ACM Trans. Graph.23(3) - SIGGRAPH 2004

Shortest Path

[Bai and Sapiro ICCV2007] X. Bai and G. Sapiro: A geodesic framework for fast interactive image and video segmentation and matting, ICCV 2007

Texture and Image priors

Texture is an important characteristics of object appearance in natural scenes and a powerful cue in visual perception. One way to analyze texture is to look at the statistics of filter responses. Three important properties have been reveled: (1) non-Gaussian behavior of image statistics, (2) non-independent response of filters and (3) invariance of image statistics to scaling of images [Srivastava et al. 03],[Mumford et al. 98]. See also Dana's short presentation on image priors.

One of the most popular texture models is based on filtering theory. The theory was inspired by the multi-channel filtering mechanism discovered and generally accepted in neurophysiology [Silverman et al. 89]. The filtering theory developed along this direction includes the Gabor filter and wavelet pyramids. The statistics of filter responses are then modeled using analytical densities (ex. Student-t [Roth, Black 05], [Mumford et al. 98].

Another approach in modeling texture is to isolate the lower dimensional manifold of natural images (image manifold) in the space of rectangular arrays of positive numbers. For example a linear subspace can be obtained using PCA, ICA. Another idea is to combine local filter responses into higher level structures that provide a better representation for the image manifold (ex. texons structure from [Malik et al. 01]).

One last approach is to represent texture through example patches. An interesting MRF structure that organizes example patches in a neighborhood system is used for super-resolution [Freeman et al. 02].

[Srivastava et al. 03] A. Srivastava and A. B. Lee and E. P. Simoncelli and S.-C. Zhu: On Advances in Statistical Modeling of Natural Images, J. Math. Imaging Vis., 18(1), 2003

[Mumford et al. 98] Song Chun Zhu and Yingnian Wu and David Mumford: Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling, IJCV 27(2), 1998

[Freeman et al. 02] William T. Freeman and Thouis R. Jones and Egon C. Pasztor: IEEE Computer Graphics and Applications 22 (2), 2002

[Roth, Black 05] Stefan Roth and Michael J. Black: Fields of Experts:A Framework for Learning Image Priors, CVPR 2005

[Malik et al. 01] Jitendra Malik, Serge Belongie, Thomas Leung and Jianbo Shi: Contour and Texture Analysis for Image Segmentation, IJCV 2001

[Malik et al. 01] Jitendra Malik, Serge Belongie, Thomas Leung and Jianbo Shi: Contour and Texture Analysis for Image Segmentation, IJCV 2001

Frequency analysis of texture for differentiating different types of tumor tissue
[Mitchell 05] Brown RA, Zlatescu MC, Cairncross JG, Mitchell JR., Texture Analysis for Non-Invasive Identification of Brain Tumor Genotype from MRI, IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP), 2005

Shape matching and analysis

Nonrigid shape analysis plays an important role in medical imaging and computational anatomy. Shape spaces characterize variability of certain structures/organs between individuals. Example applications include building anatomical atlases, defining priors for image segmentation, or characterizing disease as deviations from normal. The difficulty in shape analysis lies in the highly nonlinear nature of shape variation. As a consequence, the Euclidean metric is not appropriate in describing this variability and other metrics are instead use to characterize the shape manifold (e.g. geodesic or diffusion distances).

Shape analysis can involve defining a distance between shapes (and a shape space) used for example in shape classification and retrieval, clustering and outlier detection (to differentiate diseased vs. normal individuals) or as a regularization term in image segmentation. Another aspect of shape analysis involves detailed matching of shapes and statistical analysis of shape deformations.

One of the most popular intrinsic shape models, used primarily in image segmentation, is the signed distance function that represents shape using a 3D image of the distance from the shape surface. This representation can be used in shape matching [Dedner et al.DAGM 2007] using an approach similar to deformable image registration. Shape variability and a shape distance in can be computed using PCA on the sign distance functions [Leventon et al. 2002]. One problem with this approach is that the resulting space of shapes is not a Riemmanian manifold. Even performing PCA on signed distance functions is problematic as they don't form a vector space. To cope with this, [Charpiat, Faugeras and Keriven 2005] proposed shape statistics based on differentiable approximations of the Hausdorff distance. However, their work is limited to a linearized shape space with small deformation modes around a mean shape. To better characterize the shape manifold, [Etyngier, Segonne and Keriven ICCV07] use manifold learning techniques (diffusion maps) on the same space of signed distance functions to capture shape variability.

Book
(Euclidean geometry, ICP, geodesic embedding MDS, GMDS, Spectral embedding)

[Bronstein et al. 2008] Numerical Geometry of Non-Rigid Shapes by A. Bronstein, M. Bronstein, R. Kimmel, Springer 2008

Signed distance functions

Variational shape registration of signed distance functions
[Dedner et al.DAGM 2007] Andreas Dedner , Marcel Luthi , Thomas Albrecht , Thomas Vetter: Curvature guided level set registration using adaptive finite elements, DAGM, 2007

PCA on signed distance functions for image segmentation
[Leventon et al. CVPR2002] M. Leventon and W. Grimson and O. Faugeras: Statistical Shape Influence in Geodesic Active Contours CVPR 2002

Shape distance based on differentiable approximations of the Hausdorff distance
[Charpiat, Faugeras and Keriven 2005] Guillaume Charpiat and Olivier Faugeras and Renaud Keriven: Approximations of Shape Metrics and Application to Shape Warping and Empirical Shape Statistics, Found. Comput. Math., 2005

Shape distance using diffusion maps
[Etyngier, Segonne and Keriven ICCV07] Etyngier, P. and Segonne, F. and Keriven, R.: Shape priors using Manifold Learning Techniques, ICCV 2007

Spectral embedding and Laplacian eigenmaps

Laplacian eigenmaps
[Belkin, Niyogi, 2003] M. Belkin, P. Niyogi : Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation, Vol. 15/6, pp. 1373-1396, June 2003.

Matching shapes using spectral embedding
[Mateus et al CVPR2008] D. Mateus and R. P. Horaud and D. Knossow and F. Cuzzolin and E. Boyer: Articulated shape matching using laplacian eigenfunctions and unsupervised point registration, CVPR 2008

Matching shapes using spectral embedding
[Dubrovina,Kimmel] Anastasia Dubrovina and Ron Kimmel : Matching shapes by eigendecomposition of the Laplace-Beltrami operator,3DPVT 2010

Diffusion geometry

[Coifman,Lafon et al. 2005] R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, S. Zucker: Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps, Proc. National Academy of Sciences (PNAS), 2005.

[Lafon et al. PAMI 2006] S. Lafon, Y. Keller, R. R. Coifman:Data Fusion and Multicue Data Matching by Diffusion Maps, PAMI 2006

[Langs,Paragios CVPR08] G. Langs and N. Paragios: Modeling the structure of multivariate manifolds: Shape Maps, CVPR 2008

Tumor growth modeling

Complex growth model : diffusion + growth + mass effect
[Ayache et al. 04] Olivier Clatz and Pierre-Yves Bondiau and Herve Delingette and Maxime Sermesant and Simon K.Warfield and Gregoire Malandain and Nicholas Ayache: Brain Tumor Growth Simulation, TR INRIA 2004

Complex growth model : diffusion + growth + mass effect
[Ayache et al. TMI05] Olivier Clatz, Maxime Sermesant, Pierre-Yves Bondiau, Herve Delingette, Simon K Warfield, Gregoire Malandain, Nicholas Ayache: Realistic Simulation of the 3-D Growth of Brain Tumors in MR Images Coupling Diffusion With Biomechanical Deformation, IEEE Trans Med Imaging,24(10), 2005

Diffusion based model for extrapolating tumor invasion
[Ayache et al. MICCAI06] Ender Konukoglu, Olivier Clatz, Pierre-Yves Bondiau, Hervé Delingette, and Nicholas Ayache: Extrapolating Tumor Invasion Margins for Physiologically Determined Radiotherapy Regions, MICCAI 2006

Understanding diffusion - very good mathematical description of isotropic and anisotropic diffusion
[Weickert and Brox 02] J. Weickert and T. Brox: Diffusion and regularization of vector and matrix-valued images, TR Universitat des Saarlandes, 2002

Learning the diffusion tensor from natural images. We can use this idea for learning the parameters of the diffusion tensor in the context of growth prediction.
[Roth and Black ICCV07] Stefan Roth and Michael J. Black: Steerable Random Fields, ICCV 2007

Mechanical model for tumor growth (mass effect)

Mechanical model for mass effect
[Mohamed et al. 05] Ashraf Mohamed, Christos Davatzikos: Finite Element Modeling of Brain Tumor Mass-Effect from 3D Medical Images, MICCAI 2005

Complex growth model : diffusion + growth + mass effect
[Ayache et al. TMI05] Olivier Clatz, Maxime Sermesant, Pierre-Yves Bondiau, Herve Delingette, Simon K Warfield, Gregoire Malandain, Nicholas Ayache: Realistic Simulation of the 3-D Growth of Brain Tumors in MR Images Coupling Diffusion With Biomechanical Deformation, IEEE Trans Med Imaging,24(10), 2005

Complex growth model : diffusion + growth + mass effect (technical report)
[Ayache et al. 04] Olivier Clatz and Pierre-Yves Bondiau and Herve Delingette and Maxime Sermesant and Simon K.Warfield and Gregoire Malandain and Nicholas Ayache: Brain Tumor Growth Simulation, TR INRIA 2004

Using a mechanical model for mass effect in the context of registration of brain images with tumor pathology
[Mohamed et al. MIA 06] Mohamed, A. and Zacharaki, E.I. and Shen, D. and Davatzikos, C.: Deformable registration of brain tumor images via a statistical model of tumor-induced deformation, Medical Image Analysis, 10(5):752-763, 2006

DTI - Diffusion Tensor Images

Catherine Lebel's nice introduction to Diffusion Weighted Imaging and DTI
[Lebel07] Catherine Lebel: Diffusion Tensor Imaging, Biomedical Engineering, University of Alberta

Diffusion Weighted MRI and DTI basics - physics and mathematical modeling
[Weickert and Hagen 2006 Ch5] Joachim Weickert and Hans Hagen (Eds): Visualization and Processing of Tensor Fields, Springer 2006

DT-MRI - variational estimation, regularization and visualization
[Tschumperle and Deriche ICCV03] Tschumperle, D. Deriche, R. : Variational frameworks for DT-MRI estimation, regularization and visualization, ICCV 2003

Tensor regularization

DTI regularization - isotropic and anisotropic (full tensor)
[Weickert and Brox 02] J. Weickert and T. Brox: Diffusion and regularization of vector and matrix-valued images, TR, Universitat des Saarlandesm 2002

DTI regularization - curvature driven methods (mean curvature, self snakes, active contours)
[Weickert et al. IJCV06] Christian Feddern, Joachim Weickert, Bernhard Burgeth and Martin Welk: Curvature-Driven PDE Methods for Matrix-Valued Images, IJCV 69(1), 2006

DTI regularization - isotropic - decoupling tensor's eigenvalues and eigenvectors
[Tschumperle and Deriche CVPR01] David Tschumperle and Rachid Deriche: Diffusion Tensor Regularization with Constraints Preservation, CVPR 2001

Flow on matrix-valued functions - application to DTI = flow on the manifold of symmetric, positive semi-def matrices
[Chefd'Hotel 02 ] Christophe Chefd'Hotel, David Tschumperle, Rachid Deriche, Olivier D. Faugeras: Constrained Flows of Matrix-Valued Functions: Application to Diffusion Tensor Regularization, ECCV 2002

Median filtering of tensors
[Welk 03] Martin Welk, Christian Feddern, Bernhard Burgeth, and Joachim Weickert: Median Filtering of Tensor-Valued Images, Pattern Recognition 2003

The Riemannain space of DTI

Complete discussion of the Riemannian structure of the DT space (applications to interpolation, filtering, regularization)
[Pennec IJCV06] Xavier Pennec, Pierre Fillard and Nicholas Ayache: A Riemannian Framework for Tensor Computing, IJCV 2006

An easier but not so complete presentation of the Riemannian space of DT with application to interpolation. Also a metric based on Kullback-Leibler divergence.
[Weickert and Hagen 2006 Ch17] Maher Moakher and Philipp G. Batchelor: Symmetric Positive-Definite Matrices: from geometry to applications and visualization in Joachim Weickert and Hans Hagen (Eds): Visualization and Processing of Tensor Fields, Springer 2006

Log Euclidean metric - another Riemannian metric not affine invariant but invariant to similarities. Much easier to perform computations.
[Arsigny MRM06 IJCV06] Vincent Arsigny, Pierre Fillard, Xavier Pennec, Nicholas Ayache: Log-Euclidean metrics for fast and simple calculus on diffusion tensors, Magnetic Resonance in Medicine, 56(2) 2006

DTI Registration

Survey on registration techniques
[Weickert and Hagen 2006 Ch20] James Gee and Daniel Alexander: Diffusion Tensor Image Registration in Joachim Weickert and Hans Hagen (Eds): Visualization and Processing of Tensor Fields, Springer 2006

Spatial transformations on tensors: rigid, affine, general
[Alexander 2001] Alexander, D.C., Pierpaoli, C., Basser, P.J., Gee, J.C.: Spatial transformations of diffusion tensor magnetic resonance images, IEEE Transactions on Medical Imaging 20 (11), 2001

Piecewise affine registration - considers tensor reorientation
[Zhang 2006] Zhang H, Yushkevich PA, Alexander DC, Gee JC.: Deformable registration of diffusion tensor MR images with explicit orientation optimization.Med Image Anal. 2006

Diffeomorphic registration - reorientation with finite stain - exact gradient
[Yeo ISBI08] Yeo, B.T.T.,Vercauteren, T.,Fillard, P.,Pennec, X.,Gotland, P.,Ayache, N.,Clatz, O. : DTI registration with exact finite-strain differential, ISBI 2008

Diffeomorphic registration - reorientation with PPD (preservation of principal directions) - exact gradient
[Cao CVPRw 06] Yan Cao Miller, M.I. Mori, S. Winslow, R.L. Younes, L: Diffeomorphic Matching of Diffusion Tensor Images, CVPR workshop 2006

DTI Segmentation

Statistical framework for segmentation on tensor images. Defines statistics between tensors based on 3 metrics Euclidean, Riemannian and based on Kullback-Leibler divergence. Variational formulation (active regions)
[Lenglet et al ISBI 2006] C. Lenglet, M. Rousson, R. Deriche. A Statistical Framework for DTI Segmentation, Proc. IEEE ISBI, 794-797, 2006

Segmentation in the space of multivariate normal distributions - variational framework (active regions)
[Lenglet et al IPMI 2005] C. Lenglet, M. Rousson, R. Deriche, O. Faugeras, S. Lehericy, K. Ugurbil. A Riemannian Approach to Diffusion Tensor Images Segmentation, Proc. Information Processing in Medical Imaging, 2005

White fiber connectivity

Riemannian structure of white matter fibers - application to fiber tacking
[Lenglet et al TR03 ] C. Lenglet, R. Deriche, O. Faugeras. Diffusion Tensor Magnetic Resonance Imaging: Brain Connectivity Mapping, INRIA Research Report 4983, October 2003