> This page is for version v4.8.0 (default). > For other versions, use one of these documentation indexes: > - v4.8.0 (default): https://docs.nvidia.com/clara/parabricks/v4.8.0/llms.txt > - v4.7.1: https://docs.nvidia.com/clara/parabricks/v4.7.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/clara/parabricks/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/clara/parabricks/_mcp/server. # giraffe (vg giraffe + GATK) Generate BAM or CRAM output given one or a pair of FASTQ files using the pangenome aligner VG Giraffe [^giraffe-siren] [^giraffe-vg]. Refer to the [giraffe Reference](#options) section for a detailed listing of all available options. ## What is giraffe? VG Giraffe is a short-read mapping tool developed by Dr. Benedict Paten's lab at the University of California, Santa Cruz (UCSC). This innovative tool can align reads to a graph representation of multiple reference genomes, enhancing the quality of downstream analyses. By accurately mapping reads to thousands of genomes simultaneously, VG Giraffe offers a substantial improvement over traditional single-reference aligners. ## Why giraffe? By utilizing a graph-based approach, VG Giraffe can more effectively handle genetic diversity and structural variations across populations. Here are three key benefits of using VG Giraffe: 1. Improved accuracy: VG Giraffe achieves higher precision and recall in read mapping compared to linear genome aligners, especially when dealing with complex genomic regions or populations with significant genetic diversity. 2. Reduced reference bias (or mapping bias): By incorporating multiple haplotypes and known variants into its graph structure, VG Giraffe minimizes the reference bias inherent in traditional linear genome aligners. This leads to more comprehensive and unbiased characterization of genetic variation, especially for samples that diverge significantly from the standard reference genome. 3. Faster performance: Despite working with more complex graph structures, VG Giraffe is significantly faster than its predecessor VG Map and comparable in speed to popular linear genome mappers. It can map sequencing reads to thousands of human genomes at a speed similar to methods that map to a single reference genome. ## How Should I Use giraffe in Parabricks? VG Giraffe can be used within Parabricks, a software suite designed for accelerated secondary analysis in genomics. Our wrapper (`pbrun giraffe`) will run our GPU-accelerated VG Giraffe and sort the output BAM by coordinate. While users can build custom reference graphs for VG Giraffe using the VG Autoindex tool, pre-built pangenome graphs are also available. Dr. Paten's lab and the Human Pangenome Consortium have made these resources publicly accessible, allowing researchers to leverage high-quality, ready-to-use pangenome graphs for their analyses ([HPRC data](https://humanpangenome.org/data/)). ## Generating Required Index Files The index files `.gbz`, `.dist`, `.min`, and `.zipcodes` are required to run Giraffe. A reference paths file is also needed to define the set of paths used for BAM output. ### Supported GBZ version Parabricks Giraffe reads **GBZ version 1**. Starting with `vg` 1.72.0, `vg autoindex` and `vg gbwt` write GBZ version 2 by default, which stores the node sequences zstd-compressed. That format is not supported: loading such a graph fails right away with an error naming the conversion command below. An existing version 2 graph can be converted with the same `vg` v1.75.1 image used for index generation: ```sh docker run --rm --volume $(pwd):/workdir \ --workdir /workdir \ --user $(id -u):$(id -g) \ quay.io/vgteam/vg:v1.75.1 \ vg gbwt -Z graph.gbz -g graph-v1.gbz --gbz-v1 ``` `--gbz-v1` is part of the GBWTGraph construction step of `vg gbwt`: `-Z` reads the version 2 graph and `-g` writes the version 1 copy. The option was added in `vg` 1.74.0, so the conversion requires 1.74.0 or later even if the graph itself is older. Refer to [GBZ versions](https://github.com/vgteam/vg/wiki/VG-GBWT-Subcommand#gbz-versions) in the upstream `vg` wiki for more details. Conversion only changes how the node sequences are stored, so existing `.dist`, `.min`, and `.zipcodes` files built from the same graph stay valid. ### Building the index files The index files can be generated from a **GBZ version 1** graph using `vg autoindex`. The following example uses the HPRC v1.1 Minigraph-Cactus pangenome graph aligned to GRCh38. That download is already version 1, and `vg autoindex` reads the graph without rewriting it, so the graph stays version 1 even when indexed with `vg` v1.75.1: ```sh # Download GBZ # https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/freeze1/minigraph-cactus/hprc-v1.1-mc-grch38/hprc-v1.1-mc-grch38.d9.gbz aws s3 cp \ s3://human-pangenomics/pangenomes/freeze/freeze1/minigraph-cactus/hprc-v1.1-mc-grch38/hprc-v1.1-mc-grch38.d9.gbz \ . \ --no-sign-request # Extract index files from GBZ docker run --rm --volume $(pwd):/workdir \ --workdir /workdir \ --user $(id -u):$(id -g) \ quay.io/vgteam/vg:v1.75.1 \ vg autoindex \ -p hprc-v1.1-mc-grch38.d9.autoindex.1.75.1 \ -G hprc-v1.1-mc-grch38.d9.gbz \ -w giraffe # Extract paths from GBZ docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg paths -x hprc-v1.1-mc-grch38.d9.gbz \ -L --paths-by GRCh38 > hprc-v1.1-mc-grch38.d9.paths # As per best practices, remove decoys, unplaced/unlocalized contigs, # and other non-primary paths unnecessary for pangenome graph alignment. grep -v _decoy hprc-v1.1-mc-grch38.d9.paths \ | grep -v _random \ | grep -v chrUn_ \ | grep -v chrEBV \ | grep -v chrM \ | grep -v chain_ > hprc-v1.1-mc-grch38.d9.paths.sub ``` This produces the following files: * `hprc-v1.1-mc-grch38.d9.gbz` -- GBZ graph * `hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.dist` -- distance index * `hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.withzip.min` -- minimizer index * `hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.zipcodes` -- zipcodes * `hprc-v1.1-mc-grch38.d9.paths.sub` -- filtered reference paths ## Quick Start Before running giraffe, ensure you have generated the required index files. Refer to the [index generation](/clara/parabricks/v4.8.0/tool-reference/tools/giraffe#generating-required-index-files) section above for instructions. ```sh # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun giraffe --read-group "sample_rg1" \ --sample "sample-name" --read-group-library "library" \ --read-group-platform "platform" --read-group-pu "pu" \ --gbz-name /workdir/hprc-v1.1-mc-grch38.d9.gbz \ --dist-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.dist \ --minimizer-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.withzip.min \ --zipcodes-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.zipcodes \ --ref-paths /workdir/hprc-v1.1-mc-grch38.d9.paths.sub \ --in-fq /workdir/${INPUT_FASTQ_1} /workdir/${INPUT_FASTQ_2} \ --out-bam /outputdir/${OUTPUT_BAM} ``` ## CRAM Output Giraffe writes CRAM instead of BAM when `--out-bam` is given a path ending in `.cram`; there is no separate output option. CRAM encoding needs a linear reference, so `--ref` is required in this case. The reference must match the graph passed to `--gbz-name` and the paths listed in `--ref-paths`; refer to [Extracting the Reference FASTA](/clara/parabricks/v4.8.0/tool-reference/tools/giraffe#extracting-the-reference-fasta) for how to produce it from the graph. * `--run-ref-verification` runs a pre-flight check of the reference (`.fai` plus the sequences themselves) against the GBZ graph. It requires the FASTA index, which can be created with `samtools faidx`. Without this option, Giraffe warns that an inconsistent reference may produce an invalid CRAM file, so enabling it is recommended whenever a reference is used for the first time. * `--align-only` cannot be combined with CRAM output, because CRAM is written by the coordinate-sorting stage that `--align-only` skips. * `--gpuwrite` uses one GPU to accelerate CRAM compression. A `.crai` index is written alongside the coordinate-sorted CRAM output. ```sh # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun giraffe --read-group "sample_rg1" \ --sample "sample-name" --read-group-library "library" \ --read-group-platform "platform" --read-group-pu "pu" \ --gbz-name /workdir/hprc-v1.1-mc-grch38.d9.gbz \ --dist-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.dist \ --minimizer-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.withzip.min \ --zipcodes-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.zipcodes \ --ref-paths /workdir/hprc-v1.1-mc-grch38.d9.paths.sub \ --ref /workdir/hprc-v1.1-mc-grch38.d9.fa \ --run-ref-verification \ --in-fq /workdir/${INPUT_FASTQ_1} /workdir/${INPUT_FASTQ_2} \ --out-bam /outputdir/${OUTPUT_CRAM} ``` ## System Requirements and Useful Options for Performance To ensure optimal performance with VG Giraffe, consider the following system requirements based on your GPU configuration. Giraffe uses 16 CPU threads per GPU by default (`--num-cpu-threads-per-gpu`): * A 2 GPU system should have at least 100GB CPU RAM and at least 32 CPU threads. * A 4 GPU system should have at least 200GB CPU RAM and at least 64 CPU threads. * An 8 GPU system should have at least 400GB CPU RAM and at least 128 CPU threads. * For GPUs with less than 22 GB of device memory, use `--low-memory`. > **Note** > > Running on a single GPU is supported but not recommended. If using a single GPU, ensure that > your system meets the 2-GPU requirement mentioned above. ## Auto Mode By default, `--nstreams` is set to `auto`, which enables auto mode. In this mode, Giraffe automatically configures the number of CUDA streams, batch size, and GPU acceleration options based on GPU device memory. Auto mode is designed to provide sensible defaults, but may still require further optimization for each specific system. The following table summarizes the auto mode configuration based on the total device memory of the smallest GPU, in MiB, as reported by `nvidia-smi --query-gpu=memory.total`. Each range includes its lower bound and excludes its upper bound: | Total GPU Memory (MiB) | Streams | Batch Size | Work Queue Capacity | minimizers-gpu (SE only) | | ---------------------- | -------------- | ---------- | ------------------- | ------------------------ | | \< 22,000 | 1 (low-memory) | 5000 | 10 | No | | 22,000 to \< 32,000 | 1 | 8000 | 40 | No | | 32,000 to \< 40,000 | 2 | 10000 | 40 | No | | 40,000 to \< 80,000 | 3 | 10000 | 40 | No | | 80,000 to \< 120,000 | 4 | 10000 | 40 | Yes | | >= 120,000 | 5 | 10000 | 40 | Yes | Auto mode also takes total host memory into account: * With less than 80 GiB of host memory, the batch size is reduced to 5000 and the work queue capacity to 10. * `--minimizers-gpu` is disabled if host memory is insufficient for the total number of streams across all GPUs. Note that `--minimizers-gpu` is only enabled for single-end (SE) reads. For paired-end (PE) reads, the number of streams and batch size are configured as shown above, but `--minimizers-gpu` is always disabled. For best performance, auto mode can be overridden by explicitly setting `--nstreams` and other options. The following configurations are, for example, recommended for L4, H100 and RTX PRO 6000 Blackwell Server Edition GPUs: * **L4 (16 GB):** `--batch-size 5000 --nstreams 2` * **H100 (80 GB):** `--nstreams 5 --num-cpu-threads-per-gpu 24 --minimizers-gpu` * **RTX PRO 6000 Blackwell Server Edition (96 GB):** `--nstreams 6 --num-cpu-threads-per-gpu 24 --minimizers-gpu` Note: While a fixed base memory allocation exists per device, the number of streams and batch size are the primary factors affecting total device memory consumption. ## Using Giraffe in Variant Calling Workflows To use Giraffe-aligned BAM files for variant calling, you need to extract the appropriate reference file from the Giraffe index files. ### Extracting the Reference FASTA Run the following commands from the directory containing the Giraffe index files: ```sh # Extract the sequences corresponding to the list of paths to a FASTA file docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg paths -x hprc-v1.1-mc-grch38.d9.gbz \ -p hprc-v1.1-mc-grch38.d9.paths.sub \ -F > hprc-v1.1-mc-grch38.d9.fa # Index the FASTA file docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/biocontainers/samtools:1.17--hd87286a_2 \ samtools faidx hprc-v1.1-mc-grch38.d9.fa ``` These commands generate a FASTA file (`hprc-v1.1-mc-grch38.d9.fa`) and the corresponding index (`hprc-v1.1-mc-grch38.d9.fa.fai`). The FASTA can be used as the reference for variant calling, for [CRAM output](/clara/parabricks/v4.8.0/tool-reference/tools/giraffe#cram-output), and for BQSR ([bqsr](/clara/parabricks/v4.8.0/tool-reference/tools/bqsr)). We can now run Giraffe to obtain the aligned BAM as follows: ```sh # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun giraffe --read-group "sample_rg1" \ --sample "sample-name" --read-group-library "library" \ --read-group-platform "platform" --read-group-pu "pu" \ --gbz-name /workdir/hprc-v1.1-mc-grch38.d9.gbz \ --dist-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.dist \ --minimizer-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.withzip.min \ --zipcodes-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.zipcodes \ --ref-paths /workdir/hprc-v1.1-mc-grch38.d9.paths.sub \ --in-fq /workdir/${INPUT_FASTQ_1} /workdir/${INPUT_FASTQ_2} \ --out-bam /outputdir/${OUTPUT_BAM} ``` Once you have the Giraffe-aligned BAM file and the extracted reference FASTA, you can proceed with variant calling using HaplotypeCaller, Deepvariant, Pangenome_aware_deepvariant, or the end-to-end [pangenome_germline](/clara/parabricks/v4.8.0/tool-reference/tools/pangenome_germline) pipeline. ```sh # Haplotype Caller # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun haplotypecaller \ --ref /workdir/hprc-v1.1-mc-grch38.d9.fa \ --in-bam /workdir/${INPUT_BAM} \ --out-variants /outputdir/${OUTPUT_VCF} # Deepvariant # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun deepvariant \ --ref /workdir/hprc-v1.1-mc-grch38.d9.fa \ --in-bam /workdir/${INPUT_BAM} \ --out-variants /outputdir/${OUTPUT_VCF} # Pangenome_aware_deepvariant # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun pangenome_aware_deepvariant \ --ref /workdir/hprc-v1.1-mc-grch38.d9.fa \ --pangenome /workdir/hprc-v1.1-mc-grch38.d9.gbz \ --in-bam /workdir/${INPUT_BAM} \ --out-variants /outputdir/${OUTPUT_VCF} ``` For more detailed instructions on variant calling, refer to the tool-specific documentation ([haplotypecaller](/clara/parabricks/v4.8.0/tool-reference/tools/haplotypecaller), [deepvariant](/clara/parabricks/v4.8.0/tool-reference/tools/deepvariant), [pangenome_aware_deepvariant](/clara/parabricks/v4.8.0/tool-reference/tools/pangenome_aware_deepvariant), [pangenome_germline](/clara/parabricks/v4.8.0/tool-reference/tools/pangenome_germline)). ## Using Giraffe with Haplotype Sampling Giraffe's haplotype sampling functionality, activated using arguments `--haplotype-name` and `--kff-name`, was introduced to significantly enhance alignment accuracy by tailoring the reference graph to the specific genetic profile of a sample. This process begins by analyzing sequencing reads with a kmer counter to identify patterns of kmer presence and frequency. Using this information, Giraffe sub-samples the GBWT (using the original `.hapl` and `.gbz` files) to select haplotypes that best represent the sample, creating a customized graph. From this tailored graph, Giraffe also generates new index files (`.dist`, `.min`, and `.zipcodes`) that are optimized for the sample to be analyzed. These steps can be performed using the baseline VG container for graph customization and index generation, followed by Parabricks' accelerated Giraffe for high-performance alignment, as demonstrated below. The required `.gbz` file can be downloaded as follows: ```sh aws s3 cp \ s3://human-pangenomics/pangenomes/freeze/freeze1/minigraph-cactus/hprc-v1.1-mc-grch38/hprc-v1.1-mc-grch38.gbz \ . \ --no-sign-request ``` The `.hapl` file published alongside this graph predates the current haplotype information format and is rejected with a version error, so it has to be regenerated locally. `vg haplotypes` builds it from a distance index and an r-index, which it locates by file name, so both must sit next to the `.gbz` file and share its base name: ```sh # Build the distance index docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg index -j hprc-v1.1-mc-grch38.dist \ --no-nested-distance hprc-v1.1-mc-grch38.gbz # Build the r-index docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg gbwt -p --num-threads 16 \ -r hprc-v1.1-mc-grch38.ri \ -Z hprc-v1.1-mc-grch38.gbz # Build the haplotype information docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg haplotypes -v 2 -t 16 \ -H hprc-v1.1-mc-grch38.hapl \ hprc-v1.1-mc-grch38.gbz ``` These three commands write `.dist`, `.ri`, and `.hapl` files rather than a graph, so they do not affect the GBZ version of the input. You will also need to extract and filter reference paths for this graph: ```sh docker run --rm \ --user $(id -u):$(id -g) \ --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg paths -x hprc-v1.1-mc-grch38.gbz \ -L --paths-by GRCh38 > hprc-v1.1-mc-grch38.paths grep -v _decoy hprc-v1.1-mc-grch38.paths \ | grep -v _random \ | grep -v chrUn_ \ | grep -v chrEBV \ | grep -v chrM \ | grep -v chain_ > hprc-v1.1-mc-grch38.paths.sub ``` ```sh # Run KMC on the input reads to obtain the .kff file mkdir kmc_tmp_dir cat > input.fq.paths <<- EOM ${INPUT_FASTQ_1} ${INPUT_FASTQ_2} EOM docker run --rm --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/biocontainers/kmc:3.2.4--haf24da9_3 \ kmc \ -k29 \ -m128 \ -okff \ -t64 \ @input.fq.paths \ input.fq.distr kmc_tmp_dir # Compute the sampled .gbz file using the baseline container docker run --rm --volume $(pwd):/workdir \ --workdir /workdir \ quay.io/vgteam/vg:v1.75.1 \ vg haplotypes \ -v 2 -t 64 \ --include-reference \ --diploid-sampling \ -i hprc-v1.1-mc-grch38.hapl \ -k input.fq.distr.kff \ -g hprc-v1.1-mc-grch38.sampled.gbz \ hprc-v1.1-mc-grch38.gbz ``` `vg haplotypes` always writes the sampled graph as GBZ version 2, and has no option to write version 1, so the sampled graph has to be converted before Parabricks Giraffe can read it. Refer to [Supported GBZ version](/clara/parabricks/v4.8.0/tool-reference/tools/giraffe#supported-gbz-version) for background: ```sh # Convert the sampled graph to GBZ version 1 docker run --rm --volume $(pwd):/workdir \ --workdir /workdir \ --user $(id -u):$(id -g) \ quay.io/vgteam/vg:v1.75.1 \ vg gbwt -Z hprc-v1.1-mc-grch38.sampled.gbz \ -g hprc-v1.1-mc-grch38.sampled.v1.gbz --gbz-v1 ``` The index files are then generated from the converted graph, and the converted graph is also the one passed to `--gbz-name`: ```sh # Generate index files from the sampled graph using autoindex docker run --rm --volume $(pwd):/workdir \ --workdir /workdir \ --user $(id -u):$(id -g) \ quay.io/vgteam/vg:v1.75.1 \ vg autoindex \ -p hprc-v1.1-mc-grch38.sampled.autoindex.1.75.1 \ -G hprc-v1.1-mc-grch38.sampled.v1.gbz \ -w giraffe # Align the reads to the sampled graph using Parabricks Giraffe # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun giraffe --read-group "sample_rg1" \ --sample "sample-name" --read-group-library "library" \ --read-group-platform "platform" --read-group-pu "pu" \ --gbz-name hprc-v1.1-mc-grch38.sampled.v1.gbz \ --dist-name hprc-v1.1-mc-grch38.sampled.autoindex.1.75.1.dist \ --minimizer-name hprc-v1.1-mc-grch38.sampled.autoindex.1.75.1.shortread.withzip.min \ --zipcodes-name hprc-v1.1-mc-grch38.sampled.autoindex.1.75.1.shortread.zipcodes \ --ref-paths hprc-v1.1-mc-grch38.paths.sub \ --in-fq ${INPUT_FASTQ_1} ${INPUT_FASTQ_2} \ --out-bam /outputdir/${OUTPUT_BAM} ``` ## Aligning Ultima CRAM Input Giraffe can align or re-align reads directly from a single-end CRAM file with `--in-se-cram`. Reads are always taken in sequencing orientation, so input that is already aligned is re-aligned from scratch. Paired-end CRAM input is not supported. * `--in-cram-ref` supplies the reference FASTA used to decode the input. It is required only when the input contains aligned reads. * `--cram-read-threads` sets the number of threads used to decode the input (default: 4). Ultima Genomics data carries per-base `tp` and `t0` auxiliary tags. Passing `--ultima-tags` carries these tags from the `--in-se-cram` input through to the output. This is supported for single-end input only and requires `--in-se-cram`. Because alignment can reverse-complement a read, the order of the per-base values matters. Giraffe determines the order of the incoming values from the `@RG` lines of the input header: * `tp:reference` -- the values follow the `SEQ` field, so for reverse-strand records they run opposite to the sequencing direction. * `tp:synthesis` -- the values are in sequencing direction regardless of strand. **If the input contains aligned reads and no `@RG` line declares either `tp:reference` or `tp:synthesis`, Giraffe exits with an error**, because the order of the values cannot be determined. Add the declaration to the read group before re-aligning such a file. The order written to the output is controlled by `--tp-output-order`: * `reference` (default) -- the values match the output `SEQ`, and are reversed for reverse-strand alignments. * `synthesis` -- the values are kept in sequencing direction regardless of strand. ```sh # This command assumes all the inputs are in the current working directory and all the outputs go to the same place. docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \ --workdir /workdir \ nvcr.io/nvidia/clara/clara-parabricks:4.8.0-1 \ pbrun giraffe --read-group "sample_rg1" \ --sample "sample-name" --read-group-library "library" \ --read-group-platform "platform" --read-group-pu "pu" \ --gbz-name /workdir/hprc-v1.1-mc-grch38.d9.gbz \ --dist-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.dist \ --minimizer-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.withzip.min \ --zipcodes-name /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.zipcodes \ --ref-paths /workdir/hprc-v1.1-mc-grch38.d9.paths.sub \ --in-se-cram /workdir/${INPUT_CRAM} \ --in-cram-ref /workdir/${INPUT_CRAM_REFERENCE} \ --ultima-tags \ --tp-output-order reference \ --out-bam /outputdir/${OUTPUT_BAM} ``` Note that `--ultima-tags` and `--tp-output-order` are available on `pbrun giraffe` only. The [pangenome_germline](/clara/parabricks/v4.8.0/tool-reference/tools/pangenome_germline) pipeline accepts CRAM input but passes the alignments to a variant caller that has no use for the per-base tags. ## Compatible CPU-based vg giraffe and GATK4 Commands The commands below are the [vg-1.75.1](https://github.com/vgteam/vg/releases/tag/v1.75.1) and GATK4 counterpart of the Parabricks command above. The output from these commands will be identical to the output from the above command. Refer to the [Output Comparison](/clara/parabricks/v4.8.0/about-parabricks/comparison-with-baseline-tools) page for comparing the results. Note that the baseline needs a few upstream follow-up commits on top of the `v1.75.1` tag; refer to [Source of Mismatches](/clara/parabricks/v4.8.0/tool-reference/tools/giraffe#source-of-mismatches) for the build recipe. The index files used below are generated in the [index generation](/clara/parabricks/v4.8.0/tool-reference/tools/giraffe#generating-required-index-files) section. ```sh # Run giraffe and pipe the output to create a sorted BAM. $ vg giraffe \ -t 16 \ -Z /workdir/hprc-v1.1-mc-grch38.d9.gbz \ -d /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.dist \ -m /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.withzip.min \ -z /workdir/hprc-v1.1-mc-grch38.d9.autoindex.1.75.1.shortread.zipcodes \ --ref-paths /workdir/hprc-v1.1-mc-grch38.d9.paths.sub \ -f /workdir/${INPUT_FASTQ_1} \ -f /workdir/${INPUT_FASTQ_2} \ --output-format bam | \ gatk SortSam \ --java-options -Xmx30g \ --MAX_RECORDS_IN_RAM 5000000 \ -I /dev/stdin \ -O cpu.bam \ --SORT_ORDER coordinate # Mark duplicates. $ gatk MarkDuplicates \ -I cpu.bam \ -O cpu.markdup.bam \ -M metrics.txt ``` ## Source of Mismatches When comparing output with the CPU counterpart the following can be sources of small differences. * **Baseline VG Container** Alignment output is validated against a CPU `vg` binary built from the upstream `v1.75.1` tag with a few upstream follow-up commits cherry-picked on top. All of them were made after the release, so the unmodified `v1.75.1` container produces different output: it cannot report comparable supplementary alignments, and its Dozeu implementation predates the two fixes that Parabricks matches. The commits below are the upstream fixes for both. ```sh # Clone the repo (full history needed for cherry-pick) git clone https://github.com/vgteam/vg.git cd vg # Checkout v1.75.1 tag and create a patch branch git checkout v1.75.1 git checkout -b v1.75.1-suppl-fix-dozeu-fix # The elfutils submodule URL in the tag no longer resolves git config submodule.deps/elfutils.url https://github.com/vgteam/elfutils.git git submodule update --init --recursive # Supplementary-alignment fixes git cherry-pick 4f873b0ccb37789aa53281e7b918bb7fcc5e4e57 # gate the per-read log message git cherry-pick a34eb7d2e494166a7b23c1aa7f2d5949e37b109e # first round of fixes git cherry-pick d7227c4454a13c78b11d26499583dbc09b0468db # follow-up fix git cherry-pick e8389c3955505b11dfff7dce73e7e654ca2c331d # std::move fix # Dozeu fixes git cherry-pick 05d80785dd69561b232c3b099391d9ca5bb16f0a git cherry-pick ea9a9150fa4903e43040f5c3ff298a3d967e856e git submodule update --init --recursive ``` The patched checkout is then built with upstream `vg`'s own `Dockerfile`, which carries the full dependency list and build steps: ```sh # Record the version string compiled into the binary make version # Build the container from the patched checkout docker build --build-arg THREADS=16 -t vg:v1.75.1-suppl-fix-dozeu-fix . # Verify the resulting binary docker run --rm vg:v1.75.1-suppl-fix-dozeu-fix vg version ``` `THREADS` controls build parallelism and is clamped to the number of available cores. Skipping `make version` is possible, but then the version must be supplied with `--build-arg VG_GIT_VERSION=` or the binary reports its version as `unknown`. * **Unmapped reads** * Parabricks `giraffe` sorts unmapped reads slightly differently than baseline GATK SortSam. Unmapped reads can be filtered with samtools by running `samtools view -F 4`. ## Options | Type | Name | Required? | Description | | ----------- | --------------------------------------------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | I/O | `--in-fq IN_FQ IN_FQ` | No | Path to the paired-end FASTQ files. The files must be in fastq or fastq.gz format. Example 1: --in-fq sampleX_1_1.fastq.gz sampleX_1_2.fastq.gz. | | I/O | `--in-fq-list IN_FQ_LIST` | No | Path to a file that contains the locations of pair-ended FASTQ files. Each line must contain the location of the FASTQ files followed by a read group, each separated by a space. Each pair of files (and associated read group) must be on a separate line. Files must be in fastq/fastq.gz format. Line syntax: \ \ \. | | I/O | `--in-se-fq IN_SE_FQ` | No | Path to the single-end FASTQ file. The file must be in fastq or fastq.gz format. | | I/O | `--in-se-fq-list IN_SE_FQ_LIST` | No | Path to a file that contains the locations of single-ended FASTQ files. Each line must contain the location of the FASTQ files followed by a read group, each separated by a space. Each file (and associated read group) must be on a separate line. Files must be in fastq/fastq.gz format. Line syntax: \ \. | | I/O | `--in-se-cram IN_SE_CRAM` | No | Path to a single-end CRAM file to align or re-align. Reads are taken in sequencing orientation, so already aligned input is re-aligned. | | I/O | `--in-cram-ref IN_CRAM_REF` | No | Path to the reference FASTA used to decode --in-se-cram. Required only when the input contains aligned reads. | | I/O | `-d DIST_NAME, --dist-name DIST_NAME` | Yes | Cluster using this distance index. | | I/O | `-m MINIMIZER_NAME, --minimizer-name MINIMIZER_NAME` | Yes | Use this minimizer index. | | I/O | `-Z GBZ_NAME, --gbz-name GBZ_NAME` | Yes | Map to this GBZ graph. | | I/O | `-z ZIPCODES_NAME, --zipcodes-name ZIPCODES_NAME` | Yes | Use this zipcodes file for clustering. | | I/O | `-x XG_NAME, --xg-name XG_NAME` | No | XG graph used for BAM output. | | I/O | `-g GRAPH_NAME, --graph-name GRAPH_NAME` | No | GBWTGraph used for mapping. | | I/O | `-H GBWT_NAME, --gbwt-name GBWT_NAME` | No | GBWT index for mapping. | | I/O | `--out-bam OUT_BAM` | Yes | Path of a BAM/CRAM file for output. CRAM output is selected by using a '.cram' extension and requires --ref. | | I/O | `--ref REF` | No | Path to the reference FASTA. Required when --out-bam is a CRAM file; used to encode (and optionally verify) the CRAM output. Must match the reference used to build the graph and --ref-paths. | | I/O | `--ref-paths REF_PATHS` | No | Path to file containing ordered list of paths in the graph, one per line or HTSlib .dict, for HTSLib @SQ headers. | | I/O | `--run-ref-verification` | No | Run the pre-flight reference verification step that checks the FASTA (.fai + sequences) against the GBZ graph. Requires --ref. | | I/O | `--out-duplicate-metrics OUT_DUPLICATE_METRICS` | No | Path of duplicate metrics file after marking duplicates. | | Tool | `--read-group READ_GROUP` | No | Read group ID for this run. | | Tool | `--sample SAMPLE` | No | Sample (SM) tag for read group in this run. | | Tool | `--read-group-library READ_GROUP_LIBRARY` | No | Library (LB) tag for read group in this run. | | Tool | `--read-group-platform READ_GROUP_PLATFORM` | No | Platform (PL) tag for read group in this run; refers to platform/technology used to produce reads. | | Tool | `--read-group-pu READ_GROUP_PU` | No | Platform unit (PU) tag for read group in this run. | | Tool | `--prune-low-cplx` | No | Prune short and low complexity anchors during linear format realignment. | | Tool | `--max-fragment-length MAX_FRAGMENT_LENGTH` | No | Assume that fragment lengths should be smaller than MAX-FRAGMENT-LENGTH when estimating the fragment length distribution. | | Tool | `--fragment-mean FRAGMENT_MEAN` | No | Force the fragment length distribution to have this mean. | | Tool | `--fragment-stdev FRAGMENT_STDEV` | No | Force the fragment length distribution to have this standard deviation. | | Tool | `--max-multimaps MAX_MULTIMAPS` | No | Produce up to INT alignments for each read (default: 1). | | Tool | `--supplementary` | No | Identify and report supplementary alignments. | | Tool | `--align-only` | No | Generate output BAM after vg-giraffe alignment. The output will not be co-ordinate sorted. | | Tool | `--copy-comment` | No | Append FASTQ comment to BAM output via auxiliary tag. | | Tool | `--no-markdups` | No | Do not perform the Mark Duplicates step. Return BAM after sorting. | | Tool | `--markdups-single-ended-start-end` | No | Mark duplicate on single-ended reads by 5' and 3' end. | | Tool | `--ignore-rg-markdups-single-ended` | No | Ignore read group info in marking duplicates on single-ended reads. This option must be used with `--markdups-single-ended-start-end`. | | Tool | `--markdups-assume-sortorder-queryname` | No | Assume the reads are sorted by queryname for marking duplicates. This will mark secondary, supplementary, and unmapped reads as duplicates as well. This flag will not impact variant calling while increasing processing times. | | Tool | `--markdups-picard-version-2182` | No | Assume marking duplicates to be similar to Picard version 2.18.2. | | Tool | `--optical-duplicate-pixel-distance OPTICAL_DUPLICATE_PIXEL_DISTANCE` | No | The maximum offset between two duplicate clusters in order to consider them optical duplicates. Ignored if --out-duplicate-metrics is not passed. | | Tool | `--monitor-usage` | No | Monitor approximate CPU utilization and host memory usage during execution. | | Tool | `--max-read-length MAX_READ_LENGTH` | No | Maximum read length/size (i.e., sequence length) used for giraffe and filtering FASTQ input. (default: 480) | | Tool | `--min-read-length MIN_READ_LENGTH` | No | Minimum read length/size (i.e., sequence length) used for giraffe and filtering FASTQ input. (default: 1) | | Tool | `--ultima-tags` | No | Carry the Ultima per-base tp and t0 auxiliary tags from the --in-se-cram input through to the output. Single-end only. | | Tool | `--tp-output-order TP_OUTPUT_ORDER` | No | Order of the tp and t0 values written to the output: 'reference' matches the output SEQ, 'synthesis' keeps sequencing direction. Only used with --ultima-tags. (default: reference) | | Performance | `--nstreams NSTREAMS` | No | Number of streams per GPU to use; use 'auto' to set from GPU and host memory (may enable low-memory, dozeu/minimizers for SE). Integer overrides. More streams increases device and host memory usage. (default: auto) | | Performance | `--num-cpu-threads-per-gpu NUM_CPU_THREADS_PER_GPU` | No | Number of primary CPU threads to use per GPU. (default: 16) | | Performance | `--batch-size BATCH_SIZE` | No | Batch size used for processing alignments. (default: 10000) | | Performance | `--write-threads WRITE_THREADS` | No | Number of threads used for writing and pre-sorting output. (default: 4) | | Performance | `--cram-read-threads CRAM_READ_THREADS` | No | Number of threads used to decode the --in-se-cram input. (default: 4) | | Performance | `--gpuwrite` | No | Use one GPU to accelerate writing final BAM/CRAM. | | Performance | `--gpuwrite-deflate-algo GPUWRITE_DEFLATE_ALGO` | No | Choose the nvCOMP DEFLATE algorithm to use with --gpuwrite. Note these options do not correspond to CPU DEFLATE options. Valid options are 1, 2, and 4. Option 1 is fastest, while options 2 and 4 have progressively lower throughput but higher compression ratios. The default value is 1 when the user does not provide an input (i.e., None). | | Performance | `--gpusort` | No | Use GPUs to accelerate sorting and marking. | | Performance | `--use-gds` | No | Use GPUDirect Storage (GDS) to enable a direct data path for direct memory access (DMA) transfers between GPU memory and storage. Must be used concurrently with `--gpuwrite`. Please refer to Parabricks Documentation > Best Performance for information on how to set up and use GPUDirect Storage. | | Performance | `--memory-limit MEMORY_LIMIT` | No | System memory limit in GBs during sorting and postsorting. By default, the limit is 70% of the total system memory. | | Performance | `--low-memory` | No | Use low memory mode; will lower the number of streams per GPU and decrease the batch size. | | Performance | `--minimizers-gpu` | No | (SE only) Use GPU for minimizers and seeds. (default: False) | | Performance | `--work-queue-capacity WORK_QUEUE_CAPACITY` | No | Soft limit for the capacity of the work queues in between stages. (default: 40) | | Runtime | `--verbose` | No | Enable verbose output. | | Runtime | `--x3` | No | Show full command line arguments. | | Runtime | `--dry-run` | No | Check if the current command is valid. | | Runtime | `--logfile LOGFILE` | No | Path to the log file. If not specified, messages will only be written to the standard error output. | | Runtime | `--tmp-dir TMP_DIR` | No | Full path to the directory where temporary files will be stored. (default: .) | | Runtime | `--with-petagene-dir WITH_PETAGENE_DIR` | No | Full path to the PetaGene installation directory. By default, this should have been installed at /opt/petagene. Use of this option also requires that the PetaLink library has been preloaded by setting the LD_PRELOAD environment variable. Optionally set the PETASUITE_REFPATH and PGCLOUD_CREDPATH environment variables that are used for data and credentials. Optionally set the PetaLinkMode environment variable that is used to further configure PetaLink, notably setting it to "+write" to enable outputting compressed BAM and .fastq files. | | Runtime | `--keep-tmp` | No | Do not delete the directory storing temporary files after completion. | | Runtime | `--no-seccomp-override` | No | Do not override seccomp options for docker. | | Runtime | `--version` | No | View compatible software versions. | | Runtime | `--preserve-file-symlinks` | No | Override default behavior to keep file symlinks intact and *not* resolve the symlink. | | Runtime | `--help-json` | No | Print out help message in JSON format. | | Runtime | `--num-gpus NUM_GPUS` | No | Number of GPUs to use for a run. (default: 1) | [^giraffe-siren]: Jouni Sirén et. al., Pangenomics enables genotyping of known structural variants in 5202 diverse genomes. Science 374, abg 8871 (2021). DOI: [10.1126/science.abg8871](https://doi.org/10.1126/science.abg8871) [^giraffe-vg]: Baseline VG Giraffe: [https://github.com/vgteam/vg](https://github.com/vgteam/vg)