Developing predictive imaging biomarkers using whole-brain classifiers : Application to the ABIDE I dataset

Within clinical neuroimaging communities there is considerable optimism that functional magnetic resonance imaging (fMRI) will provide much needed objective biomarkers for diagnosing and tracking the severity of psychiatric and neurodevelopmental disorders (Castellanos et al. 2013). Training classifiers to predict disease state and severity that are robust not only to the considerable heterogeneity present in these disorders, but also to variation in systems and protocols used to collect fMRI data, require very large and diverse training datasets. The Autism Brain Imaging Dataset Exchange (ABIDE) is addressing this need for autism spectrum disorders (ASD) by aggregating data collected from imaging studies collected at 17 different sites (Di Martino et al. 2013). To learn more about applying machine learning methods to develop fMRI-based biomarkers of disease, the goal of our ‡ § | ¶ #


Introduction
Within clinical neuroimaging communities there is considerable optimism that functional magnetic resonance imaging (fMRI) will provide much needed objective biomarkers for diagnosing and tracking the severity of psychiatric and neurodevelopmental disorders (Castellanos et al. 2013).Training classifiers to predict disease state and severity that are robust not only to the considerable heterogeneity present in these disorders, but also to variation in systems and protocols used to collect fMRI data, require very large and diverse training datasets.The Autism Brain Imaging Dataset Exchange (ABIDE) is addressing this need for autism spectrum disorders (ASD) by aggregating data collected from imaging studies collected at 17 different sites (Di Martino et al. 2013)

Description
We implemented a Python based command-line program for training and testing disease classifiers from resting state fMRI data that was designed to be flexible enough to be run on different high performance computing platforms (e.g.distributed computing cluster).We used a modular framework based on the Scikit-learn machine-learning library (Pedregosa et al. 2011) that enables the classifer model to be easily switched between many different algorithms.A variety of voxel and graph -based measures calculated from the data can be used classifier features (Varoquaux and Craddock 2013).To simplify our initial implementation, we focused exclusively on the voxel-based measures and decided to leave the higher dimensional time series based anlayses for a later implementation.

Using the software
Running the program requires several key components: a) input directory: location of 3d NIfTI files; b) pheno_file: csv file in "long" format with subjects as rows and at least two columns containing subject identifiers and labels used for classification; c) model_dir: directory where trained models will be saved and models to be tested are loaded from; d Example Use-Case: ASD Diagnostic Prediction using Regional Homogeneity: To test our software for ASD classification, we used a preprocessed version of the ABIDE I dataset available through the Preprocessed Connectomes Project (http://preprocessed-connectomes-project.org/abide/).We specifically focused on the regional homogeneity (ReHo) fMRI derivative (Zang et al. 2004) from the Configurable Pipeline for the Analysis of Connectomes (CPAC) pipeline (Craddock et al. 2013).FMRI processing involved slice-time correction, motion correction, skull-stripping, global mean signal normalization, 24 parameter nuisance regression including motion correction, bandpass filtering, and registration to a 3mm MNI template.The MNI template was used as a mask to separate gray matter voxels from other tissue types as well as non-brain voxels.All voxels within the gray matter mask were chosen as features i.e. no feature reduction was performed.
Participants and Data: The ABIDE dataset contains 539 individuals with ASD and 573 control subjects.Although most subjects were male, the ratio of males/females in both groups was identical.Gender was not considered as a feature for the classifier.Weights (β-coefficients) for voxel-wise ReHo features from a support vector machine (SVM) classifier mapped on the glass brain to separate individuals with and without Autism Spectrum Disorder Developing predictive imaging biomarkers using whole-brain classifiers: ...
) mask: full path to a mask file applied to each subject volume; e) model: type of algorithm to utilize.Executing the program in training mode (with the --train flag) generates a sklearn (cite) model written to disk as a serialized object, a NIfTI file containing a feature weightmap, as well as csv files containing weights at each feature, training accuracy, and model predictions.During training, users have several options including tuning hyperparameters using a gridsearch implemented via stratified five-fold cross-validation and/or imposing a sparse model solution via L1 regularization.During training the program will automatically invoke the necessary routines to: mask samples to ensure corresponding voxels are the same across subjects, reshape data into a format necessary for algorithm training, and balance label classes across training folds if hyperparameter tuning is requested.Executing the program in testing mode (with the --test flag) requires a previously trained model and saves two csv files containing model predictions and testing accuracy.

Classifier
Training and Testing: First, participants were randomly divided into balanced split-half training and testing sets.During training, feature selection was performed by selecting only voxels falling within a grey matter template mask in MNI152 space.These voxels were subsequently used to train a whole brain support vector machine with L1regularization, to enforce a sparse model solution.The hyper-parameter controlling the margin of the hyper-plane was tuned using a parameter grid-search with 5-fold crossvalidation within the training set.The best performing hyper-parameter was then utilized to train a single model on the entire training set.This modeled was then applied to data from the test set in order to generate subject level predictions about diagnosis (i.e.neuro-typical or ASD).Accuracy scores were computed by comparing classifier predictions with true subject diagnoses.Results:Fig.1 shows one instance of the training model with the weights for each voxelwise feature depicted on the glass brain.The accuracy of our model without any dimensionality reduction or feature selection was ~ 62%.
. To learn more about applying machine learning methods to develop fMRI-based biomarkers of disease, the goal of our ‡