diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..647d688 --- /dev/null +++ b/.gitignore @@ -0,0 +1,8 @@ +*.nii.gz +*.dcm +*.nii +*.json +*.html +*.tsv +.DS_Store + diff --git a/README.md b/README.md new file mode 100644 index 0000000..1b968f9 --- /dev/null +++ b/README.md @@ -0,0 +1,99 @@ +# HW5: fmriprep + +Due 3/15. Members of the fmriprep presentation group are exempt from this homework as a course requirement, but might want to use this as an opportunity to process their own data. + +# Overview + +Use singularity and the fmriprep container at ` /scratch/psyc5171/containers/fmriprep_1.5.9.sif` to preprocess some fMRI data. + +You can apply fmriprep to either + +1. The example dataset at `/scratch/psyc5171/exercises/mbdata/bids` OR +2. The dataset that you have identified for your final presentation + + +# Demo Dataset + +The demo dataset is adapted from [ds001544](https://openneuro.org/datasets/ds001544/versions/1.1.0) and contains a single subject and session with T1w, T2w, bold/sbref, and epi fieldmap modalities. Note that the scans have been renamed from the original dataset and some of the metadata has been modified. + +# Exercise + +1. Fork and clone this repository to the cluster. +2. Edit the `sbatch_fmriprep.sh` script to run fmriprep. You should use the following non default options: + - `--cifti-output` to generate a CIFTI dense timeseries, containing both cortical surface and subcortical volume data +3. Submit your job +4. When fmriprep has successfully run, use git to add, commit, push, and pull request. + + +# Usage Notes + +The minimal fmriprep command is: + +```bash +singularity run --cleanenv \ +--bind ${BIDS_DIR}:/data \ +--bind ${OUT_DIR}:/out \ +/scratch/psyc5171/containers/fmriprep_1.5.9.sif \ +/data /out participant + +``` + +For surface processing using FreeSurfer (default, and required by the `--cifti-output` option), the path (with respect to the container) to a FreeSurfer license file must be provided using the `--fs-license-file` option. You can register for your own free license at [https://surfer.nmr.mgh.harvard.edu/registration.html](https://surfer.nmr.mgh.harvard.edu/registration.html) or use the license file at `/scratch/psyc5171/exercises/mbdata/bids/code/license.txt`. One approach is to create a `code` directory in the bids folder and place the license file there. You can then provide this file as `--fs-license-file /data/code/license.txt`. If you use FreeSurfer/fmriprep outside of class, please register for a license. + +You need to provide two directories for processing: + +- `${BIDS_DIR}` is the full path to the BIDS folder. Since mriqc does not modify the input BIDS folder, you do not need to copy the input data to your own directory. +- `${OUT_DIR}` is the full path to a directory where the output files will go. This directory needs to be under your control and exist prior to running fmriprep (use `mkdir` to create a new directory). For example, you might create the directory `/scratch/${USER}/hw5/fmriprep` to store the output, and would replace the string `${OUT_DIR}` with this path, so the line would read `--bind /scratch/${USER}/hw5/fmriprep:/out`. +- The command above needs to be in a SLURM script. You can adapt a script from your previous homework as a starting point. Don't forget to include the `module load singularity` command in your SLURM script. +- See the slides for notes on resource needs. If you adjust the mriqc `--nprocs` option to use 2 or 3 processors, the script should not take more than 1 hour to run. + +You should also [review the options](https://fmriprep.readthedocs.io/en/stable/usage.html#Options%20to%20handle%20performance) for controlling CPU and memory usage and harmonize these with the resources requested in your SLURM request. + +# Processing your own data + +For your own dataset with multiple subjects, you have at least two options for managing performance: + +1. Process all subjects in one job. This will make use of the parallel processing capabilities of fmriprep, but will generally require a long job runtime. +2. Process one subject in each job. To do this, add the option `--participant_label $1` to your fmriprep command. When submitting your job using `sbatch`, specify the subject ID to process without `sub-`, e.g. + +```bash +sbatch sbatch_fmriprep.sh 01 +``` + +will process `sub-01`. You can submit a job for each subject and up to 8 jobs will run at once. `$1` is a special variable that refers to the first argument given to your `sbatch_fmriprep.sh` script. + +I recommend the second approach since your jobs can execute faster and job failure on a single subject will not affect processing for the others. + + + +# git workarounds + +- To associate your clone with your GitHub username, run the command + + ```bash + git remote set-url origin \ + https://${USER}@github.uconn.edu/${USER}/hw5.git + ``` + in your `hw5` directory. This will avoid the `The requested URL returned error: 403 Forbidden while accessing ` error when pushing. +- Use the command `unset SSH_ASKPASS` prior to pushing to avoid the error `(gnome-ssh-askpass:19484): Gtk-WARNING **: cannot open display:` +- When committing, you can specify a short commit message on the command line with the `-m` option: `git commit -m 'commit message'`. If you do not do this, the default editor will open so you can write your commit message. The default is usually `vi`. + - Press `i` to enter insertion mode and write your message + - Press `ctrl-c` or `esc` to exit insertion mode + - Type `:wq` to save and quit or `:q!` to quite without saving + +- The command `git status` will show files that have changed since the last commit. +- The command `git log` will show the commit history + +# Resources + +- The [BIRC computing guide](https://bircibrain.github.io/computingguide/docs/fmri-preprocessing/fmriprep.html) has some more detailed instructions on running fmriprep. The guide also provides instructions for downloading the container image, but this has already been done for you. +- The [fmriprep](https://fmriprep.readthedocs.io/en/stable/) webpage has additional documentation on the various options, and more detailed descriptions and references for various pipeline steps and outputs. + + +# Troubleshooting + +- Consult your sbatch log files for clues to the cause of errors, e.g. missing files. +- It is sometimes neccessary to remove partial outputs from your mriqc output directory in order to successfully rerun the pipeline. +- fmriprep and many other BIDS Apps will create a `work` directory that stores temporary files. It can be helpful to also remove this directory before rerunning a failed fmriprep job. +- Files can be deleted with the `rm` command. Directories (and the directories/files they contain) can be removed with `rm -r`. Make sure you know what you are deleting before running this command! + diff --git a/sbatch_fmriprep.sh b/sbatch_fmriprep.sh new file mode 100755 index 0000000..94b82fd --- /dev/null +++ b/sbatch_fmriprep.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +# Add your SLURM options here + +# Add commands needed to run fmriprep +