Nilanjan Ray

Professor
Department of Computing Science
University of Alberta, Canada


Left Ventricle Segmentation



Fig. 1: End-to-end Convolutional Neural Net and Dynamic Programming
Fig. 2: EDPCNN Results

We combine convolutional neural net (CNN) and dynamic programming (DP) in an end-to-end pipeline to accomplish left ventricle segmentation. While CNN is differentiable, DP implementation is non-differentiable due to its dependence on "argmin" function. We have approximated the gradient across DP by a neural network (see Fig. 1). Similar technique, known as synthetic gradients, has been used for asynchronous training of deep neural networks.

Our results (Fig. 2) show that EDPCNN can train with fewer labeled data. Further EDPCNN is quite resistant to overfitting compared to U-Net (see publication). We will continue to work with this direction, which we refer to as ADEPT: Architecture for Differentiable End-to-end Programming and Training.

Related Publications
  • N.M. Nguyen, N. Ray, "End-to-end Learning of Convolutional Neural Net and Dynamic Programming for Left Ventricle Segmentation," arXiv: https://arxiv.org/abs/1812.00328
  • N. Ray, S.T. Acton, H. Zhang, “Seeing through clutter: Snake computation with dynamic programming for particle segmentation,” ICPR 2012.

Muscle Segmentation

Fig. 3: Leg muscle segmentation

Leg muscle segmentation from MRI is challenging because often there is no strong edge/boundary that delineates a muscle. Thus, we need to resort of supervised learning, where expert segmentations are first created to form a training set of images. Since the segmentation problem is 3D here, we can represent a stack of 2D contours by two 2D images (see details in our publications). Then we can learn a CNN that regresses this representation from 3D MRI. However, we noted that using a pre-trained PCA layer in conjunction with the CNN helps to improve the accuracy of segmentation. We train this system end-to-end.

Related Publications
  • S. Ghosh, P. Boulanger, S.T. Acton, S.S. Blemker, N. Ray, “Automated 3D Muscle Segmentation From MRI Data Using Convolutional Neural Network,” IEEE ICIP 2017.
  • S. Ghosh, N. Ray, P. Boulanger, “A Structured Deep-learning Based Approach for the Automated Segmentation of Human Leg Muscle from 3D MRI,” CRV 2017, Edmonton, Alberta, Canada.
  • N. Ray, S. Mukherjee, K. Kanth, S.T. Acton, S.S. Blemker, “3D-To-2D Mapping for User Interactive Segmentation of Human Leg Muscles from MRI Data,” GlobalSIP14-Data Flow Algorithms and Architecture for Signal Processing Systems, Atlanta, Georgia, United States.