Setting Up A Pipeline Configuration#

Overview#

This section explains how to create a new pipeline or edit an existing one. If you wish to use one of the pre-configured pipelines that come packaged with C-PAC, you can view the current available library of pipelines here.

Neuroparc v1.0: Baseline

On July 20, 2020, Neuroparc released v1.0. In moving from v0 to v1, the paths to several of the Neuroparc atlases changed. The atlases are used by C-PAC in its default and preconfigured pipelines. C-PAC v1.7.0 includes the Neuroparc v1.0 paths, but if you are using a pipeline based on C-PAC 1.6.2a or older, you will need to update any Nueroparc v0 paths in your config

All paths begin with /ndmg_atlases/label/Human/#

Neuroparc v0

Neuroparc v1

aal_space-MNI152NLin6_res-1x1x1.nii.gz

AAL_space-MNI152NLin6_res-1x1x1.nii.gz

aal_space-MNI152NLin6_res-2x2x2.nii.gz

AAL_space-MNI152NLin6_res-2x2x2.nii.gz

AAL2zourioMazoyer2002.nii.gz

AAL_space-MNI152NLin6_res-1x1x1.nii.gz

brodmann_space-MNI152NLin6_res-1x1x1.nii.gz

Brodmann_space-MNI152NLin6_res-1x1x1.nii.gz

brodmann_space-MNI152NLin6_res-2x2x2.nii.gz

Brodmann_space-MNI152NLin6_res-2x2x2.nii.gz

CorticalAreaParcellationfromRestingStateCorrelationsGordon2014.nii.gz

CAPRSC_space-MNI152NLin6_res-1x1x1.nii.gz

desikan_space-MNI152NLin6_res-1x1x1.nii.gz

Desikan_space-MNI152NLin6_res-1x1x1.nii.gz

desikan_space-MNI152NLin6_res-2x2x2.nii.gz

Desikan_space-MNI152NLin6_res-2x2x2.nii.gz

DesikanKlein2012.nii.gz

DesikanKlein_space-MNI152NLin6_res-1x1x1.nii.gz

glasser_space-MNI152NLin6_res-1x1x1.nii.gz

Glasser_space-MNI152NLin6_res-1x1x1.nii.gz

Juelichgmthr252mmEickhoff2005.nii.gz

Juelich_space-MNI152NLin6_res-1x1x1.nii.gz

MICCAI2012MultiAtlasLabelingWorkshopandChallengeNeuromorphometrics.nii.gz

MICCAI_space-MNI152NLin6_res-1x1x1.nii.gz

princetonvisual-top_space-MNI152NLin6_res-2x2x2.nii.gz

Princetonvisual-top_space-MNI152NLin6_res-2x2x2.nii.gz

Schaefer2018-200-node_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer200_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer2018-300-node_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer300_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer2018-400-node_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer400_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer2018-1000-node_space-MNI152NLin6_res-1x1x1.nii.gz

Schaefer1000_space-MNI152NLin6_res-1x1x1.nii.gz

slab907_space-MNI152NLin6_res-1x1x1.nii.gz

Slab907_space-MNI152NLin6_res-1x1x1.nii.gz

yeo-7_space-MNI152NLin6_res-1x1x1.nii.gz

Yeo-7_space-MNI152NLin6_res-1x1x1.nii.gz

yeo-7-liberal_space-MNI152NLin6_res-1x1x1.nii.gz

Yeo-7-liberal_space-MNI152NLin6_res-1x1x1.nii.gz

yeo-17_space-MNI152NLin6_res-1x1x1.nii.gz

Yeo-17_space-MNI152NLin6_res-1x1x1.nii.gz

yeo-17-liberal_space-MNI152NLin6_res-1x1x1.nii.gz

Yeo-17-liberal_space-MNI152NLin6_res-1x1x1.nii.gz

There are two ways of setting up or editing a pipeline configuration for C-PAC:

  • Using the pipeline configuration interface in the C-PAC GUI

  • Using a text editor (useful for remote servers where using the C-PAC GUI is not possible or impractical)

Definitions#

Design A Pipeline#

Note

The C-PAC pipeline configuration was changed to a nested format with import capabilities in v1.8.0.

With this change, the following configuration keys are deprecated:

  • crashLogDirectory

  • output_tree

  • TR

  • fdCalc

  • reGenerateOutputs

  • runMedianAngleCorrection

  • slice_timing_pattern

  • targetAngleDeg

  • runSymbolicLinks

  • configFileTwomm

  • ref_mask_2mm

  • template_skull_for_anat_2mm

  • surface_reconstruction

See Regressors specification to manually update any of the following keys:

  • nComponents

  • nuisanceBandpassFreq

  • numRemovePrecedingFrames

  • numRemoveSubsequentFrames

  • runFrequencyFiltering

  • runFristonModel

  • runMotionSpike

  • spikeThreshold

  • smoothing_order

  • already_skullstripped

  • roiTSOutputs

Mappings for all other C-PAC 1.7 keys can be found here.

C-PAC offers a graphical interface you can use to quickly and easily modify the default pipeline or create your own from scratch: https://fcp-indi.github.io/C-PAC_GUI/

Currently the GUI creates a C-PAC v1.6.0 pipeline configuration file. This syntax persisted through v1.7.2 but is deprecated with the release of v1.8.0.

If given a pipeline file in the older syntax, C-PAC v1.8 will attempt to convert the pipeline configuration file to the new syntax, saving the converted file in your output directory.

An update to the GUI to create v1.8.0 syntax configuration files is underway.

The newer (v1.8) syntax will not work with older versions of C-PAC.

See Using a Text Editor for configuring a custom pipeline without the GUI.

See also

Details about mapping the older syntax to the new.

../../_images/gui_home1.png

Once you save the pipeline configuration YAML file, you can provide it to the C-PAC Docker container like so:

docker run -i --rm \
        -v /Users/You/local_bids_data:/bids_dataset \
        -v /Users/You/some_folder:/outputs \
        -v /tmp:/tmp \
        -v /Users/You/Documents:/configs \
        -v /Users/You/resources:/resources \
        fcpindi/c-pac:latest /bids_dataset /outputs participant --pipeline_file /configs/pipeline_config.yml

Or you can provide it to the C-PAC Singularity container like so:

singularity run \
        -B /Users/You/some_folder:/outputs \
        -B /tmp:/tmp \
        -B /Users/You/Documents:/configs \
        fcpindi_c-pac_latest-{date}-{hash value}.img s3://fcp-indi/data/Projects/ADHD200/RawDataBIDS /outputs participant --pipeline_file /configs/pipeline_config.yml

Reporting errors and getting help

Please report errors on the C-PAC GitHub issue tracker. Please use Neurostars for help using C-PAC and this application.

Using a Text Editor#

If you want to base a pipeline on another pipeline configuration YAML file, you can specify

FROM: /path/to/pipeline.yml

in your pipeline configuration file. You can use the name of a preconfigured pipeline instead of a filepath if you want to base a configuration file on a preconfigured pipeline. If FROM is not specified, the pipeline will be based on the default pipeline.

C-PAC will include all expected keys from the pipeline file specified in FROM (or the default pipeline if none is specified). Any keys specified in a pipeline configuration file will take precedence over the same key in the FROM base configuration, but all omitted keys will retain their values from the FROM base configuration.

From terminal, you can quickly generate a default pipeline configuration YAML file template in the directory you are in:

cpac utils pipe_config new_template

You can then edit the file as needed. For values that you want to leave at the default, you can either leave the key as-is, or you can remove the key, and C-PAC will automatically use value from the default pipeline configuration (or from the pipeline specified in FROM).

If you want to run the analysis from terminal:

cpac run --pipe_config {path to pipeline config} {path to data config}

Pipeline configuration files, like the data settings and data configuration files discussed in the data configuration builder section, are stored as YAML files. Similarly, each of the parameters used by C-PAC to assemble your pipeline can be specified as nested key-value pairs, so a pipeline configuration YAML would have multiple lines of the form key: value like so

An example of a pipeline configuration YAML file can be found here. Tables explaining the keys and their potential values can be found on the individual pages for each of the outputs C-PAC is capable of producing. All pipeline setup configuration files should have the keys in the Output Settings table defined.

String values can include the simplest form of POSIX parameter expansion <https://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_02> (${parameter}). Two special variables are included for these types of parameters:

  • resolution_for_anat will be populated with the value set in registration_workflows['anatomical_registration']['resolution_for_anat'].

  • func_resolution will

    • be populated with the value set in registration_workflows['functional_registration']['func_registration_to_template']['output_resolution']['func_preproc_outputs'] if funcreg is in the value’s key,

    • be populated with the value set in registration_workflows['functional_registration']['func_registration_to_template']['output_resolution']['func_derivative_outputs'] if deriv is in the value’s key, or

    • raise an exception if neither funcreg nor deriv is in the value’s key.

If FROM is defined (see above), any undefined keys will be inferred from the pipeline configuration specified; otherwise, any undefined keys will be inferred from the default pipeline.

Why a list?#

You may notice as you learn about the settings for various outputs that many of the values for C-PAC’s configurable settings are stored in lists (i.e., multiple values are separated by commas and surrounded by square brackets). Such lists containing On``s and ``Off``s (for ``True and False respectively) allow you to toggle on multiple options at the same time, and branch a pipeline into two different analysis strategies. See the developer documentation for more information about how lists are used in C-PAC.

Configurable Settings#

Open image

Data Management and Environment Settings#

Pre- and post-processing#

Derivatives#

References

1

Poldrack, R. A., Mumford, J. A., and Nichols, T. E. 2011 August. Handbook of Functional MRI Data Analysis. New York: Cambridge University Press.