giraffe (vg giraffe + GATK)
giraffe (vg giraffe + GATK)
Beta
Note that the Parabricks GPU-accelerated Giraffe tool is currently in beta.
Generate BAM output given one or a pair of FASTQ files using the pangenome aligner VG Giraffe [^note] [^note].
Refer to the giraffe Reference section for a detailed listing of all available options.
What is giraffe?
VG Giraffe is a short-read mapping tool developed by Dr. Benedict Paten’s lab at the University of California, Santa Cruz (UCSC). This innovative tool can align reads to a graph representation of multiple reference genomes, enhancing the quality of downstream analyses. By accurately mapping reads to thousands of genomes simultaneously, VG Giraffe offers a substantial improvement over traditional single-reference aligners.
Why giraffe?
By utilizing a graph-based approach, VG Giraffe can more effectively handle genetic diversity and structural variations across populations. Here are three key benefits of using VG Giraffe:
- Improved accuracy: VG Giraffe achieves higher precision and recall in read mapping compared to linear genome aligners, especially when dealing with complex genomic regions or populations with significant genetic diversity.
- Reduced reference bias (or mapping bias): By incorporating multiple haplotypes and known variants into its graph structure, VG Giraffe minimizes the reference bias inherent in traditional linear genome aligners. This leads to more comprehensive and unbiased characterization of genetic variation, especially for samples that diverge significantly from the standard reference genome.
- Faster performance: Despite working with more complex graph structures, VG Giraffe is significantly faster than its predecessor VG Map and comparable in speed to popular linear genome mappers. It can map sequencing reads to thousands of human genomes at a speed similar to methods that map to a single reference genome.
How Should I Use giraffe in Parabricks?
VG Giraffe can be used within Parabricks, a software suite designed for accelerated
secondary analysis in genomics. Our wrapper (pbrun giraffe) will run our GPU-accelerated
VG Giraffe and sort the output BAM by coordinate.
While users can build custom reference graphs for VG Giraffe using the VG Autoindex tool, pre-built pangenome graphs are also available. Dr. Paten’s lab and the Human Pangenome Consortium have made these resources publicly accessible, allowing researchers to leverage high-quality, ready-to-use pangenome graphs for their analyses (HPRC data).
Generating Required Index Files
The index files .gbz, .dist, .min, and .zipcodes are required to run Giraffe.
A reference paths file is also needed to define the set of paths used for BAM output.
The index files can be generated from a GBZ graph using vg autoindex. The following
example uses the HPRC v1.1 Minigraph-Cactus pangenome graph aligned to GRCh38:
Quick Start
Before running giraffe, ensure you have generated the required index files. Refer to the index generation section above for instructions.
System Requirements and Useful Options for Performance
To ensure optimal performance with VG Giraffe, consider the following system requirements based on your GPU configuration:
-
A 2 GPU system should have at least 100GB CPU RAM and at least 32 CPU threads.
-
A 4 GPU system should have at least 200GB CPU RAM and at least 64 CPU threads.
-
For GPUs with less than 22 GB of device memory, use
--low-memory.
Auto Mode
By default, --nstreams is set to auto, which enables auto mode. In this
mode, Giraffe automatically configures the number of CUDA streams, batch size, and GPU
acceleration options based on the available GPU device memory. Auto mode is designed to
provide sensible defaults, but may still require further optimization for each specific
system. The following table summarizes the auto mode configuration based on GPU
device memory:
Auto mode also takes host memory into account. If host memory is insufficient,
--minimizers-gpu may be disabled, and batch size and work queue capacity may
be reduced.
Note that --minimizers-gpu is only enabled for single-end (SE) reads.
For paired-end (PE) reads, the number of streams and batch size are configured as
shown above, but --minimizers-gpu is always disabled.
For best performance, auto mode can be overridden by explicitly setting --nstreams
and other options. The following configurations are, for example, recommended for L4, H100
and RTX PRO 6000 Blackwell Server Edition GPUs:
-
L4 (16 GB):
--batch-size 5000 --nstreams 2 -
H100 (80 GB):
--nstreams 5 --num-cpu-threads-per-gpu 24 --minimizers-gpu -
RTX PRO 6000 Blackwell Server Edition (96 GB):
--nstreams 4 --num-cpu-threads-per-gpu 24 --minimizers-gpu
Note: While a fixed base memory allocation exists per device, the number of streams and batch size are the primary factors affecting total device memory consumption.
Using Giraffe in Variant Calling Workflows
To use Giraffe-aligned BAM files for variant calling, you need to extract the appropriate reference file from the Giraffe index files. Run the following commands from the directory containing the Giraffe index files:
These commands will generate a FASTA file (hprc-v1.1-mc-grch38.d9.fa),
and the corresponding index (hprc-v1.1-mc-grch38.d9.fa.fai), that can
be used as the reference for variant calling. Note that these files can be also used
for BQSR (bqsr). We can now run Giraffe to obtain the aligned BAM as follows:
Once you have the Giraffe-aligned BAM file and the extracted reference FASTA, you can proceed with variant calling using HaplotypeCaller, Deepvariant, Pangenome_aware_deepvariant, or the end-to-end pangenome_germline pipeline.
For more detailed instructions on variant calling, refer to the tool-specific documentation (haplotypecaller, deepvariant, pangenome_aware_deepvariant, pangenome_germline).
Using Giraffe with Haplotype Sampling
Giraffe’s haplotype sampling functionality, activated using arguments
--haplotype-name and --kff-name, was introduced to significantly enhance
alignment accuracy by tailoring the reference graph to the specific genetic profile
of a sample. This process begins by analyzing sequencing reads with a kmer counter to
identify patterns of kmer presence and frequency. Using this information, Giraffe
sub-samples the GBWT (using the original .hapl and .gbz files)
to select haplotypes that best represent the sample, creating a customized graph.
From this tailored graph, Giraffe also generates new index files
(.dist, .min, and .zipcodes) that are optimized for the sample to be analyzed.
These steps can be performed using the baseline VG container for graph customization and index generation, followed by Parabricks’ accelerated Giraffe for high-performance alignment, as demonstrated below.
The required .hapl and .gbz files can be downloaded as follows:
You will also need to extract and filter reference paths for this graph:
Compatible CPU-based vg giraffe and GATK4 Commands
The commands below are the vg-1.70.0 and GATK4 counterpart of the Parabricks command above. The output from these commands will be identical to the output from the above command. Refer to the Output Comparison page for comparing the results.
The index files used below are generated in the index generation section.
Source of Mismatches
When comparing output with the CPU counterpart the following can be sources of small differences.
-
Baseline VG Container
-
Single-end (SE) reads: Parabricks matches the baseline
quay.io/vgteam/vg:v1.70.0container exactly. No modifications to the baseline container are needed. -
Paired-end (PE) reads: A bug fix for fragment distance recording is required in the baseline container. You need to cherry-pick the fix and rebuild the container as follows:
-
-
Unmapped reads
- Parabricks
giraffesorts unmapped reads slightly differently than baseline GATK SortSam. Unmapped reads can be filtered with samtools by runningsamtools view -F 4.
- Parabricks
Options
- Jouni Sirén et. al., Pangenomics enables genotyping of known structural variants in 5202 diverse genomes. Science 374, abg 8871 (2021). DOI: 10.1126/science.abg8871
- Baseline VG Giraffe: https://github.com/vgteam/vg