Longitudinal Multiple Sclerosis Lesion Segmentation Challenge

I. INTRODUCTION

The Longitudinal MS Lesion Segmentation Challenge was conducted at the 2015 International Symposium on Biomedical Imaging in New York, NY, April 16-19. Competing teams applied their automatic lesion segmentation algorithms to MR neuroimaging data acquired at multiple time points from MS patients. Algorithms were evaluated against manual segmentations from multiple raters in terms of their segmentation accuracy and ability to track lesion evolution.
34 Teams initially registered for the Challenge coming from 15 different countries, representing 27 different institutions/universities. Congratulations to Team IIT Madras (First Prize), Team PVG_1 (Second Prize), and Team IMI (Third Prize and Efficiency Prize)!

II. DATA

The overall data will be composed of three parts:

• Training data consisting of longitudinal images from 5 patients;
• Test data consisting of longitudinal images from 14 patients;

Only the training data will include manual delineations when it is released to teams.

Each longitudinal dataset will include T1-weighted, T2-weighted, PD-weighted, and T2-weighted FLAIR MRI with 3-5 time points acquired on a 3T MR scanner. T1-weighted images will have approximately a 1mm cubic voxel resolution, while the other scans will be 1mm in plane with 3mm sections. Accounting for the multiple time points, this constitutes approximately 80 individual data sets. To minimize the dependency of the results on registration performance and brain extraction, all images will be provided already rigidly co-registered to the baseline T1-weighted image with automatically computed skull stripping masks that may optionally be used by the teams.

III. GUIDELINES

• Segmentation algorithms must be automated. User provided initial seed points or contours are not allowed.
• Numerical input parameters may be used and should be constant for a particular test data set.
• Algorithms may use some or all of the provided contrasts in the original or preprocessed data.
• Output lesion segmentations should be in the same space as the preprocessed data.
• Other publicly available data sets may be used within the algorithm, but lesion modeling should come primarily from the provided training data or using an unsupervised model.
• There are no restrictions on how the algorithm is implemented in regards to platform, programming language, or dependent software libraries.
• Output segmentations should be saved in NIFTI format with a label of 1 assigned to lesions and 0 otherwise. Segmentations should be in 3-D. For 3D segmentations, filenames should be as follows: "subject_timepoint_teamname.nii" with the timepoints using 2 digits (example: "test01_03_jhu.nii").
• Population-based approaches are allowed.

IV. ADDITIONAL INFORMATION

• Training data Set Description (Feb 11, 2015)
• Evaluation Software Description (March 31, 2015)

V. CHALLENGE RESULTS

• You can access the top 25 submission results here

VI. PUBLISHED PAPER

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge
Neuroimage: 2017, 148;77-102
[PubMed:28087490] [DOI]