differentialabundance
nf-core/differentialabundance
Differential abundance analysis
Key tools
Run it
nfclaw run differentialabundance --input samplesheet.csv --outdir results --filtering-min-abundance <filtering_min_abundance> -profile dockernextflow run pipelines/differentialabundance/upstream -profile docker --input samplesheet.csv --outdir results --filtering_min_abundance <filtering_min_abundance>nfclaw run differentialabundance --demo --outdir results # adds the upstream test profile (-profile test,docker)Inputs
| column | type | required | allowed values | constraints |
|---|---|---|---|---|
sample | integer or string | no | matches ^\S+$ |
--input must match ^\S+\.(csv|tsv)$.
For tabular CSV/TSV input, use this header; fill each value per the table above and reference.md (no example value is invented here):
sampleFrom data files. nfclaw cannot generate this samplesheet: the samplesheet declares no file type nfclaw could match files against (no column's pattern names a file extension).
Reference genome
No reference genome is set by default: supply your own reference. The schema declares AWS iGenomes at s3://ngi-igenomes/igenomes/; selecting --genome <id> uses the paths in the pipeline's reference configuration. Access to that remote source is required. To supply your own reference, use the reference_genome_options group in reference.md lists the reference options. Set --igenomes-ignore true to disable the lookup entirely. Confirm the selected reference's paths and source in the pinned upstream/nextflow.config and upstream/conf/ before running.
Resources
Resource requests follow the pipeline's configs and selected profiles. nf-core's resource labels often assume a server — a single step can request far more memory than a workstation has, and retry rules can increase requests. If a run fails with Process requirement exceeds available memory (or CPUs), cap every request, and every retry, at what this machine actually has:
nfclaw run differentialabundance --input samplesheet.csv --outdir results --filtering-min-abundance <filtering_min_abundance> -profile docker \ --limit-cpus 4 --limit-memory 15.GBnfclaw turns those into Nextflow's process.resourceLimits and passes them as a -c config — the mechanism nf-core prescribes for exactly this (docs). Set them to the machine's real capacity. --limit-time DURATION also caps how long any one task may run: a task that needs longer is stopped, and so is every retry, so set it only as a deliberate ceiling, never as a guess at how long the analysis takes. The generated config is kept in <outdir>/provenance/, so commands.sh replays the run under the same ceiling.
Nextflow engine
This release declares nextflowVersion = '!>=25.10.4'.
To pin this release's declared minimum engine — useful when a newer Nextflow has incompatible config parsing:
nfclaw run differentialabundance ... --nxf-ver 25.10.4--nxf-ver is recorded in <outdir>/provenance/, so the replay uses the same engine. See known-issues.
Demo
nfclaw run differentialabundance --demo --outdir results # adds the upstream test profile (-profile test,docker)Samplesheet
A CSV with this header (the columns the schema requires). Fill each value per the constraints below.
sample| Column | Type | Required | Allowed / constraints |
|---|---|---|---|
sample | integer or string | optional | matches ^\S+$ |
Required parameters
--study-namestringA string identifier used to name result files in the output directory
--study-typestringInput data format category used for input validation and routing (not for selecting analysis methods).
rnaseqgeneric_matrixaffy_arraymaxquantgeo_soft_file--inputstring (file path)Path to CSV/TSV file containing information about the samples in the experiment.
Constraint · matches ^\S+\.(csv|tsv)$
--outdirstring (directory path)The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
--observations-id-colstringColumn in the sample sheet to be used as the primary sample identifier
--observations-typestringType of observation
--features-id-colstringFeature ID attribute in the abundance table as well as in the GTF file (e.g. the gene_id field)
--features-name-colstringFeature name attribute in the abundance table as well as in the GTF file (e.g. the gene symbol field)
--features-typestringType of feature. Often 'gene'
--filtering-min-abundanceinteger or booleanMinimum abundance value. Set to false to disable abundance filtering.
--exploratory-clustering-methodstringClustering method used in dendrogram creation
--exploratory-cor-methodstringCorrelation method used in dendrogram creation
--exploratory-n-featuresintegerNumber of features selected before certain exploratory analyses. If -1, will use all features.
--exploratory-main-variablestringHow should the main grouping variable be selected? 'auto_pca', 'contrasts', or a valid column name from the observations table.
--exploratory-palette-namestringValid R palette name
--differential-feature-id-columnstringThe feature identifier column in differential results tables
--differential-min-fold-changenumberMinimum fold change used to calculate differential feature numbers. Note that this number will be log2 transformed
--differential-max-pvalnumberMaximum p value used to calculate differential feature numbers
--differential-max-qvalnumberMaximum q value used to calculate differential feature numbers
--differential-palette-namestringValid R palette name
--report-filestring (file path)Qmd/Rmd/ipynb report template(s) from which to create the pipeline report. Supply a single path or a comma-separated list of paths to render multiple reports per paramset.
Constraint · matches ^[^,\s]+\.(Rmd|qmd|ipynb)(\s*,\s*[^,\s]+\.(Rmd|qmd|ipynb))*$
--logo-filestringA logo to display in the report instead of the generic pipeline logo.
--css-filestringCSS to use to style the output, in lieu of the default nf-core styling
Parameter explorer
188 parameters · 25 groupsEvery parameter from the pinned nextflow_schema.json. nfclaw checks explicit values before launch; complex rules use the pinned pipeline's runtime validation. Search, filter by group, and expand any parameter for its default, allowed values and constraints.
169 parameters · 19 hidden
Base Abundance Values4
Alternative to matrix: a compressed CEL files archive such as often found in GEO
(RNA-seq only): optional transcript/gene length matrix with samples and transcript_ids/gene_ids as in the abundance matrix.
TSV/CSV-format abundance matrix
Use SOFT files from GEO by providing the GSE study identifier
Base Features options7
A text file listing technical features (e.g. spikes)
Supply your own feature annotations. Can be derived from the GTF (rnaseq) or from the Bioconductor annotation package (affy arrays).
Feature ID attribute in the abundance table as well as in the GTF file (e.g. the gene_id field)
Comma-separated string, specifies feature metadata columns to be used for exploratory analysis, platform-specific
Feature name attribute in the abundance table as well as in the GTF file (e.g. the gene symbol field)
Type of feature. Often 'gene'
When set, use the control features in scaling/ normalization (currently only supported for differential_method deseq2)
Base Input Output options7
A CSV/TSV/YML/YAML file describing sample contrasts to compare groups.
Path to CSV/TSV file containing information about the samples in the experiment.
The output directory where the results will be saved. You have to use absolute paths to storage on Cloud infrastructure.
To how many digits should numeric output in different modules be rounded? If -1 or null, will not round.
Global seed for stochastic methods
A string identifier used to name result files in the output directory
Input data format category used for input validation and routing (not for selecting analysis methods).
Base Observations E G Samples options3
Column in the sample sheet to be used as the primary sample identifier
Column in the sample sheet to be used as identifier for observations. If unset, the --observations_id_col is used.
Type of observation
Base Paramsheet options2
Name of the paramset to run. In profile mode, set by the analysis profile for output directory naming. In paramsheet mode, selects which paramset(s) to run (comma-separated).
Path to a paramsheet YAML file. Setting this activates multi-run (paramsheet) mode where paramsheet values take priority over CLI flags.
Differential Deseq2 Specific options Rna Seq Only15
alpha parameter passed to results()
altHypothesis parameter passed to results()
fitType parameter passed to DESeq()
independentFiltering parameter passed to results()
lfcThreshold parameter passed to results()
'minReplicatesForReplace' parameter passed to DESeq()
minmu parameter passed to results()
pAdjustMethod parameter passed to results()
sfType parameter passed to DESeq()
Shrink fold changes in results?
test parameter passed to DESeq()
useT parameter passed to DESeq2
blind parameter for rlog() and/ or vst()
variance stabilisation method to use when making a variance stabilised matrix
nsub parameter passed to vst()
Differential Differential Analysis8
The feature identifier column in differential results tables
Advanced option: the suffix associated tabular differential results tables. Will by default use the appropriate suffix according to the study_type.
Maximum p value used to calculate differential feature numbers
Maximum q value used to calculate differential feature numbers
Differential analysis method
Minimum fold change used to calculate differential feature numbers. Note that this number will be log2 transformed
Valid R palette name
In differential analysis (DEseq2 or Limma), subset to the contrast samples before modelling variance?
Differential Dream Specific options12
Method used to adjust p-values for multiple testing (passed to p.adjust).
Turns on and off usage of voomWithDreamWeights() normalization in the DREAM module.
passed to variancePartition::dream(), logical, should 95% confidence intervals be output for logFC? Alternatively, can take a numeric value between zero and one specifying the confidence level required.
Method used to estimate effective degrees of freedom for hypothesis testing in the linear mixed model. Allowed values: adaptive (default), Satterthwaite, Kenward-Roger.
passed to variancePartition::dream() (via eBayes), assumed proportion of genes that are differentially expressed (numeric 0-1).
passed to variancePartition::dream(), logical, use REML estimation for mixed-model variance components (passed through to lmer()).
passed to variancePartition::dream() (via eBayes), logical, should the estimation of df.prior and var.prior be robustified against outlier sample variances?
passed to variancePartition::dream() (via eBayes), comma-separated numeric pair giving the lower and upper limits of the standard deviation of the logFC values used in the prior.
passed to variancePartition::dream() (via eBayes), logical, should an intensity-dependent trend be allowed for the prior variance?
passed to variancePartition::dream() (via eBayes), comma-separated numeric pair giving the proportion of the lower and upper tails to be winsorized before estimating the standard deviation.
Differential Limma Specific options15
passed to topTable(), method used to adjust the p-values for multiple testing.
Sample sheet column to be used to derive a vector or factor specifying a blocking variable on the arrays for limma::lmFit(); however, for random effects models, DREAM is the recommended approach in this pipeline
passed to topTable(), logical, should confidence 95% intervals be output for logFC? Alternatively, can take a numeric value between zero and one specifying the confidence level required.
passed to limma::lmFit(), the inter-duplicate or inter-technical replicate correlation; however for random effects models, DREAM is the recommended approach in this pipeline
passed to topTable(), minimum absolute log2-fold-change required
passed to lmFit(), the fitting method
passed to lmFit(), positive integer giving the number of times each distinct probe is printed on each array.
cutoff value for adjusted p-values. Only genes with lower p-values are listed.
passed to eBayes(), a numeric value between 0 and 1, assumed proportion of genes which are differentially expressed
passed to eBayes(), logical, should the estimation of df.prior and var.prior be robustified against outlier sample variances?
passed to lmFit(), positive integer giving the spacing between duplicate occurrences of the same probe, spacing=1 for consecutive rows.
passed to eBayes, comma separated string of two values, assumed lower and upper limits for the standard deviation of log2-fold-changes for differentially expressed genes
passed to eBayes(), logical, should an intensity-dependent trend be allowed for the prior variance?
Turns on and off usage of voom normalization in the Limma module.
passed to eBayes, comma separated string of length 1 or 2, giving left and right tail proportions of x to Winsorize. Used only when robust=TRUE.
Differential Propd Specific options9
Alpha value for the Box-Cox transformation used by propd. Leave unset to skip the transformation.
False discovery rate threshold used to define significantly proportional pairs.
Use moderated theta values in propd.
Number of theta cutoffs evaluated when estimating FDR by permutation.
Number of permutations for FDR estimation. 0 uses the analytical FDR from F-statistic p-values.
Save the gene-by-gene adjacency matrix. Must be set to true when pairing propd with --functional_method grea, which consumes the adjacency.
Save the table of significant pairwise statistics.
Save the full table of pairwise statistics (very large).
Save the propd object as an RDS file.
Exploratory Exploratory Analysis8
Clustering method used in dendrogram creation
Correlation method used in dendrogram creation
Of which assays to compute the log2 during exploratory analysis. Not necessary for maxquant data as this is controlled by the pipeline.
Threshold on MAD score for outlier identification
How should the main grouping variable be selected? 'auto_pca', 'contrasts', or a valid column name from the observations table.
Number of features selected before certain exploratory analyses. If -1, will use all features.
Valid R palette name
Length of the whiskers in boxplots as multiple of IQR. Defaults to 1.5.
Functional Decoupler Specific options3
Comma-separated list of methods to use (e.g., 'ora,ulm')
Removes sources of a net with less than min_n targets
Path to TSV file containing network file for decoupler
Functional Functional Analysis options2
Functional analysis method. Set to 'none' (default) to disable functional analysis.
Gene sets in GMT or GMX-format; for GSEA: multiple comma-separated input files in either format are possible. For gprofiler2: A single file in GMT format is possible; this has lowest priority and will be overridden by --gprofiler2_token and --gprofiler2_organism.
Functional Gprofiler213
Which column to use as gene IDs in the background matrix.
Path to CSV/TSV/TXT file that should be used as a background list of genes for the query; alternatively, 'auto' (default) or 'false'.
The method that should be used for multiple testing correction.
How to calculate the statistical domain size.
Whether to include evcodes in the results.
Maximum q value used for significance testing.
Should underrepresentation be measured instead of overrepresentation?
How many genes must be differentially expressed in a pathway for it to be considered enriched? Default 1.
Short name of the organism that is analyzed, e.g. hsapiens for homo sapiens.
Valid R palette name
Should only significant enrichment results be considered?
On which source databases to run the gprofiler query
Token that should be used as a query.
Functional Grea3
Number of permutations used by grea to estimate enrichment significance.
Maximum number of features in a gene set.
Minimum number of features in a gene set.
Functional Gsea16
Make detailed geneset report?
Use median for class metrics
Metric for ranking genes
Normalisation mode
Number of permutations
Number of markers
Gene list ordering mode
Permutation type
Plot graphs for the top sets of each phenotype
Randomization mode
Save random ranked lists
Enrichment statistic
Max size: exclude larger sets
Min size: exclude smaller sets
Gene list sorting mode
Make a zipped file with all reports
Generic options6
Email address for completion summary, only when pipeline fails.
Display the help message.
Display the full detailed help message.
Method used to save pipeline results to output directory.
Display hidden parameters in the help message (only works when --help or --help_full are provided).
Boolean whether to validate parameters against the schema at runtime
Preprocessing Affy Input options9
logical value. If set to true, apply background correction using RMA.
integer value indicating which RMA background to use
logical value. If TRUE, a matrix of probe annotations will be derived.
Used to specify the name of an alternative cdf package. If set to NULL, then the usual cdf package based on Affymetrix' mappings will be used.
logical value. If TRUE, then works on the PM matrix in place as much as possible, good for large datasets.
Column of the sample sheet containing the Affymetrix CEL file name
if TRUE, then overrides what is in rm.mask and rm.oultiers.
should the spots marked as 'MASKS' set to NA?
should the spots marked as 'OUTLIERS' set to NA?
Preprocessing Filtering6
An optional grouping variable to be used to calculate a min_samples value
Minimum abundance value. Set to false to disable abundance filtering.
A minimum proportion of observations, given as a number between 0 and 1, that must pass the threshold. Overrides minimum_samples
A minimum proportion of observations, given as a number between 0 and 1, that must have a value (not NA) to retain the row/ feature (e.g. gene).
Minimum observations that must pass the threshold to retain the row/ feature (e.g. gene).
Minimum observations that must have a value (not NA) to retain the row/ feature (e.g. gene). Overrides filtering_min_proportion_not_na.
Preprocessing Gtf Parsing options3
If a GTF file is supplied, which feature type to use
If a GTF file is supplied, which field should go first in the converted output table
Genome annotation file in GTF format
Preprocessing Proteus Input options5
Prefix of the column names of the MaxQuant proteingroups table in which the intensity values are saved; the prefix has to be followed by the sample names that are also found in the samplesheet. Default: 'LFQ intensity'; will search for both the prefix as entered and the prefix followed by one whitespace.
Normalization function to use on the MaxQuant intensities.
Valid R palette name
Should a loess line be added to the plot of mean-variance relationship of the conditions? Default: true.
Which method to use for plotting sample distributions of the MaxQuant intensities; one of 'violin', 'dist', 'box'.
Reference Genome options1
Name of iGenomes reference.
Report Reporting options8
A markdown file containing citations to include in the final report
Email address for completion summary.
An author for reporting outputs
Semicolon-separated string of contributor info that should be listed in the report.
A description for reporting outputs
Qmd/Rmd/ipynb report template(s) from which to create the pipeline report. Supply a single path or a comma-separated list of paths to render multiple reports per paramset.
A title for reporting outputs
Skip generation of reports
Shiny Shiny App Settings4
Should a Shiny app be built?
Should the app be deployed to shinyapps.io?
Your shinyapps.io account name
The name of the app to push to in your shinyapps.io account
Outputs
Results land in --outdir; the files and directory layout depend on the selected workflow, parameters and publication settings. Run metadata is normally placed in <outdir>/pipeline_info/ (execution report, software versions), when those outputs are enabled. nfclaw run also writes <outdir>/provenance/ with the exact params file and the run log, <outdir>/provenance/logs/run.log — the whole launch, whose last line states the outcome (Nextflow's own log is <outdir>/.nextflow.log); unless --no-provenance it adds a run manifest (pinned version, commit and exact command), input/output SHA-256 checksums, and a replayable commands.sh.
The exact output files and directory layout for this release are documented upstream: github.com/nf-core/differentialabundance/blob/2.0.0/docs/output.md
Check a run — in the foreground or the background — with nfclaw status <outdir>: success, still running, how it ended (with the error), or stopped without an outcome; exit 0 success, 3 running, 1 otherwise.
Chaining
Run differentialabundance as one stage of a chain: nfclaw chain run starts each stage only after the one before it succeeded, and prepares its inputs from that stage's outputs. Every rule is checked against both pinned schemas.
Fed by
- rnaseq — rnaseq's merged gene counts and the gene lengths of the same quantification become differentialabundance's --matrix and --feature-length-matrix (DESeq2 models length bias with them; differentialabundance does not recommend raw counts without lengths) — from the alignment-based quantification when rnaseq ran one, otherwise its pseudo-aligner — with the GTF the rnaseq run used. An rnaseq release that published no lengths for that quantification (before 3.15.0; RSEM before 3.23.0) fails the handoff: hand over its length-scaled counts with an inline handoff instead. The experimental design (--input, sample names as in rnaseq) and --contrasts describe your study and go in the stage's own params.
nfclaw chain edges differentialabundance