fq2bam (BWA-MEM + GATK)

Generate BAM/CRAM output given one or more pairs of FASTQ files. Can also optionally generate a BQSR report.

Refer to the fq2bam Reference section for a detailed listing of all available options.

What is BWA-MEM?

BWA-MEM is a fast, accurate algorithm for mapping DNA sequence reads to a reference genome, performing local alignment and producing alignment for different parts of the query sequence. It is the default algorithm in Burrows-Wheeler Aligner (BWA) for reads that are longer than 70bp and is designed for high-throughput sequencing technologies such as Illumina and Pacific Biosciences.

Why BWA-MEM?

BWA-MEM is capable of handling longer reads and is less sensitive to errors than other alignment algorithms. It is therefore used for a variety of applications, from routine analysis of sequencing data to more advanced applications such as de novo assembly and variant calling.

Some of the advantages of using BWA-MEM over similar tools include:

1. It is faster than many other alignment algorithms, making it the ideal choice for high-throughput sequencing.
2. It has a lower false positive rate than many other alignment algorithms, which means fewer false-positive variants are reported.
3. It is memory-efficient, allowing it to be used on limited resources.
4. It is highly accurate, with a reported accuracy of over 99% on Illumina data.

What is fq2bam?

BWA-MEM can be deployed within Parabricks, a software suite designed for accelerated secondary analysis in genomics, bringing industry standard tools and workflows from CPU to GPU, and delivering the same results at up to 60x faster runtimes. FQ2BAM is the Parabricks wrapper for BWA-MEM, which will sort the output and can mark duplicates and recalibrate base quality scores in line with GATK best practices. A 30x whole genome can be run through FQ2BAM in as little as 6 minutes on an NVIDIA DGX system, compared to 4-9 hours on a CPU instance (m5.24xlarge, 96 x vCPU). If device memory is less than 50GB, one may need to experiment with the --bwa-nstreams or --low-memory.

How Should I Use BWA-MEM in fq2bam?

fq2bam uses an accelerated version of BWA-MEM to generate BAM/CRAM output given one or more pairs of FASTQ files. The user can turn-off marking of duplicates by adding the —no-markdups option. The BQSR step is only performed if the —knownSites input and —out-recal-file output options are provided; doing so will also generate a BQSR report.

Quick Start

1
# This command assumes all the inputs are in the current working directory and all the outputs go to the same place.
2
docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \
3
    --workdir /workdir \
4
    nvcr.io/nvidia/clara/clara-parabricks:4.7.1-1 \
5
    pbrun fq2bam \
6
    --ref /workdir/${REFERENCE_FILE} \
7
    --in-fq /workdir/${INPUT_FASTQ_1} /workdir/${INPUT_FASTQ_2}  \
8
    --knownSites /workdir/${KNOWN_SITES_FILE} \
9
    --out-bam /outputdir/${OUTPUT_BAM} \
10
    --out-recal-file /outputdir/${OUTPUT_RECAL_FILE}

Useful Options for Performance

Suggested parameters to control host memory use:

• The parameter --bwa-normalized-queue-capacity controls the amount of batches that will be in memory to be processed. By default, the value normalized to the number of GPUs in the run is 10, corresponding to <number of GPUs> * <normalized capacity> for the total queue size. Multiple queues are used. Lowering this value would be a good strategy to reduce host memory use. However, a value that is too low may hamper performance.

Parabricks automatically uses an optimal number of streams based on the GPU’s device memory specifications (by default --bwa-nstreams auto). The user may experiment further with the --bwa-nstreams and --bwa-cpu-thread-pool parameters to potentially achieve better performance.

Additionally, for advanced performance tuning and additional control the option --bwa-primary-cpus allows for more fine-grained control of CPU threading. Each primary CPU thread drives P CPU thread pool threads as specifed with the option --bwa-cpu-thread-pool. The total number of CPU threads processing the CPU stages of alignment is the product of the --bwa-primary-cpus and --bwa-cpu-thread-pool parameters. This allows the user to control the ratio of “primary” CPU threads, which act indepently, to thread pool threads, which act in unison. Depending on the specific server configuration, input data, and GPUs available, it may be better to switch from more thread pool threads to more primary CPU threads. Changing the number of primary CPU threads may increase the CPU resources required.

• The parameter --gpuwrite uses the GPU to compress the final BAM or CRAM file for improved performance during the final stage.

  • --gpuwrite-deflate-algo can be used to control the compression ratio. See below for more details.

• The parameter --gpusort uses the GPU to sort the BAM or CRAM file.

• The option --cigar-on-gpu offloads CIGAR generation from the CPU (default) to the GPU. Consider enabling it on CPU-constrained systems, or if benchmarking with and without the option indicates the CPU is a bottleneck; it may improve overall runtime.

Compatible CPU-based BWA-MEM, GATK4 Commands

The commands below are the bwa-0.7.15 and GATK4 counterpart of the Parabricks command above. The output from these commands will be identical to the output from the above command. Refer to the Output Comparison page for comparing the results.

1
# Run bwa-mem and pipe the output to create a sorted BAM.
2
$ bwa mem \
3
    -t 32 \
4
    -K 10000000 \
5
    -R '@RG\tID:sample_rg1\tLB:lib1\tPL:bar\tSM:sample\tPU:sample_rg1' \
6
    <INPUT_DIR>/${REFERENCE_FILE} <INPUT_DIR>/${INPUT_FASTQ_1} <INPUT_DIR>/${INPUT_FASTQ_2} | \
7
  gatk SortSam \
8
    --java-options -Xmx30g \
9
    --MAX_RECORDS_IN_RAM 5000000 \
10
    -I /dev/stdin \
11
    -O cpu.bam \
12
    --SORT_ORDER coordinate
13
14
# Mark duplicates.
15
$ gatk MarkDuplicates \
16
    --java-options -Xmx30g \
17
    -I cpu.bam \
18
    -O mark_dups_cpu.bam \
19
    -M metrics.txt
20
21
# Generate a BQSR report.
22
$ gatk BaseRecalibrator \
23
    --java-options -Xmx30g \
24
    --input mark_dups_cpu.bam \
25
    --output <OUTPUT_DIR>/${OUTPUT_RECAL_FILE} \
26
    --known-sites <INPUT_DIR>/${KNOWN_SITES_FILE} \
27
    --reference <INPUT_DIR>/${REFERENCE_FILE}

Source of Mismatches

While Parabricks fq2bam does not lose any accuracy in functionality when compared with BWA-mem and GATK, still there are several sources that can result in differences in output files.

• BWA-mem -K argument

In pair-ended mode, the chunk size specified by -K can cause small mismatches in the output BAM file. To get rid of the mismatches here, make sure to pass the same number to both baseline BWA-mem and Parabricks fq2bam, e.g. -K 10000000.

• PA aux tag

  • Parabricks fq2bam puts the PA tag last while BWA-mem puts it first.
  • BWA-mem rounds PA tag to 3 digits while Parabricks fq2bam does not. And aux tag can be filtered by samtools view -x <TAG>

• Unmapped reads

  • Parabricks fq2bam sorts unmapped reads slightly differently than baseline GATK SortSam. Unmapped reads can be filtered with samtools by running samtools view -F 4.

Options