Reference genomes

Many nf-core pipelines use reference genomes for alignment, annotation, and similar tasks. This page describes available approaches for managing reference genomes.

There are three main ways to use reference genomes with nf-core pipelines:

• Local copies of genomes: user downloaded and self-managed
• AWS iGenomes: Illumina-hosted pre-build reference genomes and indices
• Refgenie: programmatic genome asset management tool

Local copies of genomes

Most genomics nf-core pipelines can start from just a FASTA and GTF file and create downstream reference assets (genome indices, interval files, etc.) as part of pipeline execution.

Using GRCh38 as an example:

1. Download the latest files:

   #!/bin/bash
    
   VERSION=108
   wget -L ftp://ftp.ensembl.org/pub/release-$VERSION/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz
   wget -L ftp://ftp.ensembl.org/pub/release-$VERSION/gtf/homo_sapiens/Homo_sapiens.GRCh38.$VERSION.gtf.gz

2. Run pipeline with --save_reference to generate indices:

   nextflow run \
       nf-core/rnaseq \
       --input samplesheet.csv \
       --fasta Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz \
       --gtf Homo_sapiens.GRCh38.108.gtf.gz \
       --save_reference

   Note

   The pipeline will generate and save reference assets. For example, the STAR index will be stored in <results-dir>/genome/index/star.

3. Move generated assets to a central, persistent storage location for re-use in future runs.

4. Use pre-generated indices in future runs.

   nextflow run \
       nf-core/rnaseq \
       --input samplesheet.csv \
       --fasta Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz \
       --gtf Homo_sapiens.GRCh38.108.gtf.gz \
       --star_index </path/to/moved/star/directory/> \
       --gene_bed </path/to/moved/genes.bed>

AWS iGenomes

AWS iGenomes is Illumina’s centralized resource that organizes commonly used reference genome and pre-built index files in a consistent structure for multiple genomes. It provides the following benefits:

• Hosted on AWS S3 through the Registry of Open Data
• Free to access and download
• Maintained by nf-core as a copy of the original Illumina resource

See the AWS iGenomes documentation for more information.

Use remote AWS iGenomes

To use remote AWS iGenomes in supported nf-core pipelines, supply the --genome flag to your pipeline (e.g., --genome GRCh37). On execution the pipeline will then:

1. Automatically download required reference files.
2. Auto-populated reference genome parameters from conf/igenomes.config.
   • Parameters like FASTA, GTF, and index paths are set automatically.
3. Download only what it requires for that specific workflow.

Use local AWS iGenomes

To use local AWS iGenomes:

1. Download the iGenomes reference files you need to a local directory.

2. Set --igenomes_base to your local iGenomes directory path.

   Warning

   This directory structure must reflect the structure defined in conf/igenomes.config.

3. Pipeline will use local files instead of downloading from AWS.

Check annotation versions

To check the version of annotations used by AWS iGenomes:

1. Download the README file from the iGenomes S3 bucket using AWS CLI:

   aws s3 cp --no-sign-request s3://ngi-igenomes/igenomes/Homo_sapiens/Ensembl/GRCh37/Annotation/README.txt

2. View the README to see annotation details:

   cat README.txt

   Example output:

   The contents of the annotation directories were downloaded from Ensembl on: July 17, 2015.
    
   Gene annotation files were downloaded from Ensembl release 75. SmallRNA annotation files were downloaded from miRBase release 21.

   This confirms the annotations are from Ensembl release 75 (July 2015), which is significantly outdated.