ont_germline

View as Markdown

Run the germline variant tool to generate BAM and variants on long read ONT sequences using minimap2 for alignment as well as the DeepVariant variant caller.

Refer to the ont_germline Reference section for a detailed listing of all available options.

Quick Start

1# This command assumes all the inputs are in the current working directory and all the outputs go to the same place.
2docker run --rm --gpus all --volume $(pwd):/workdir --volume $(pwd):/outputdir \
3 --workdir /workdir \
4 nvcr.io/nvidia/clara/clara-parabricks:4.7.1-1 \
5 pbrun ont_germline \
6 --ref /workdir/${REFERENCE_FILE} \
7 --in-fq /workdir/${INPUT_FASTQ} \
8 --out-bam /outputdir/${OUTPUT_BAM} \
9 --out-variants /outputdir/${OUTPUT_VCF}

Compatible CPU-based minimap2, GATK4, and Google DeepVariant Commands

The commands below are the minimap2-v2.30, GATK4, and Google DeepVariant counterpart of the Clara Parabricks command above. The output from these commands will be identical to the output from the above command. Refer to the Output Comparison page for comparing the results.

1# Run minimap2 and pipe the output to create a sorted BAM.
2$ minimap2 -ax map-ont \
3 <INPUT_DIR>/${REFERENCE_FILE} \
4 <INPUT_DIR>/${INPUT_FASTQ} | \
5 gatk SortSam \
6 --java-options -Xmx30g \
7 --MAX_RECORDS_IN_RAM 5000000 \
8 -I /dev/stdin \
9 -O cpu.bam \
10 --SORT_ORDER coordinate
11
12 # Run deepvariant
13BIN_VERSION="1.9.0"
14sudo docker run \
15 -v "${PWD}":"/input" \
16 -v "${PWD}/output":"/output" \
17 -v "${PWD}/Ref":"/reference" \
18 google/deepvariant:"${BIN_VERSION}" \
19 /opt/deepvariant/bin/run_deepvariant \
20 --model_type ONT_R104 \
21 --ref /reference/${REFERENCE_FILE} \
22 --reads cpu.bam \
23 --output_vcf /output/"${OUTPUT_VCF_FILE}" \
24 --num_shards $(nproc) \
25 --disable_small_model=true \
26 --make_examples_extra_args "ws_use_window_selector_model=true"

In Google DeepVariant, the small model is enabled by default and must be explicitly disabled (with --disable_small_model=true). In Parabricks DeepVariant, the small model is opt-in and must be explicitly enabled (with --enable-small-model).

Variants called by the small model will have an additional MID field in their VCF record, which denotes which model called the variant. This is consistent with VCF files produced by Google DeepVariant.

Note that a change must be made to the baseline minimap2 code in order to match the results exactly:

A fix must be made to the baseline KSW2 code to round the loop fission start and end points by changing them to st and en respectively. If the start point (st0) is a number below 16, but greater than 0, its scoring values will not be initialized correctly, but will still be used later when computing the actual alignment. This can be fixed by rounding the start and end points to multiples of 16.

To make this fix, change the following code in ksw2_extd2_sse.c:

1// loop fission: set scores first
2if (!(flag & KSW_EZ_GENERIC_SC)) {
3 for (t = st0; t <= en0; t += 16) {
4 __m128i sq, st, tmp, mask;
5 sq = _mm_loadu_si128((__m128i*)&sf[t]);
6 st = _mm_loadu_si128((__m128i*)&qrr[t]);
7 mask = _mm_or_si128(_mm_cmpeq_epi8(sq, m1_), _mm_cmpeq_epi8(st, m1_));
8 tmp = _mm_cmpeq_epi8(sq, st);
9 #ifdef __SSE4_1__
10 tmp = _mm_blendv_epi8(sc_mis_, sc_mch_, tmp);
11 tmp = _mm_blendv_epi8(tmp, sc_N_, mask);
12 #else
13 tmp = _mm_or_si128(_mm_andnot_si128(tmp, sc_mis_), _mm_and_si128(tmp, sc_mch_));
14 tmp = _mm_or_si128(_mm_andnot_si128(mask, tmp), _mm_and_si128(mask, sc_N_));
15 #endif
16 _mm_storeu_si128((__m128i*)((int8_t*)s + t), tmp);
17 }
18 } else {
19 for (t = st0; t <= en0; ++t)
20 ((uint8_t*)s)[t] = mat[sf[t] * m + qrr[t]];
21 }

Fixed version that uses lf_start and lf_en:

1// loop fission: set scores first
2int lf_start = st, lf_en = en;
3if (!(flag & KSW_EZ_GENERIC_SC)) {
4 for (t = lf_start; t <= lf_en; t += 16) {
5 __m128i sq, st, tmp, mask;
6 sq = _mm_loadu_si128((__m128i*)&sf[t]);
7 st = _mm_loadu_si128((__m128i*)&qrr[t]);
8 mask = _mm_or_si128(_mm_cmpeq_epi8(sq, m1_), _mm_cmpeq_epi8(st, m1_));
9 tmp = _mm_cmpeq_epi8(sq, st);
10 #ifdef __SSE4_1__
11 tmp = _mm_blendv_epi8(sc_mis_, sc_mch_, tmp);
12 tmp = _mm_blendv_epi8(tmp, sc_N_, mask);
13 #else
14 tmp = _mm_or_si128(_mm_andnot_si128(tmp, sc_mis_), _mm_and_si128(tmp, sc_mch_));
15 tmp = _mm_or_si128(_mm_andnot_si128(mask, tmp), _mm_and_si128(mask, sc_N_));
16 #endif
17 _mm_storeu_si128((__m128i*)((int8_t*)s + t), tmp);
18 }
19 } else {
20 for (t = lf_start; t <= lf_en; ++t)
21 ((uint8_t*)s)[t] = mat[sf[t] * m + qrr[t]];
22 }

Models for Additional GPUs

See the DeepVariant Models for additional GPUs section for instructions on downloading and using model files for additional GPUs.

Options

TypeNameRequired?Description
I/O--ref REFYesPath to the reference file.
I/O--index INDEXNoPath to a minimizer index file generated by vanilla minimap2 to reduce indexing time.
I/O--in-fq [IN_FQ ...]NoPath to a query sequence file in fastq or fastq.gz format. This option can be used multiple times.
I/O--in-bam IN_BAMNoPath to the input BAM/CRAM file.
I/O-j JUMP_BED, --jump-bed JUMP_BEDNoPath to a BED file (.bed) used in the splice:sr preset for jumping junctions in the annotation.
I/O--junc-bed JUNC_BEDNoPath to a BED file (.bed) used in adding bonus scores if an aligned junction matches a junction in the annotation.
I/O--knownSites KNOWNSITESNoPath to a known indels file. The file must be in vcf.gz format. This option can be used multiple times.
I/O--interval-file INTERVAL_FILENoPath to an interval file in one of these formats: Picard-style (.interval_list or .picard), GATK-style (.list or .intervals), or BED file (.bed). This option can be used multiple times.
I/O--pb-model-file PB_MODEL_FILENoPath to a non-default parabricks model file for deepvariant.
I/O--pb-small-model-file PB_SMALL_MODEL_FILENoPath to a non-default parabricks model file for the small model.
I/O--out-recal-file OUT_RECAL_FILENoPath of the report file after Base Quality Score Recalibration.
I/O--out-bam OUT_BAMYesPath of BAM file after marking duplicates.
I/O--out-variants OUT_VARIANTSYesPath of the vcf/vcf.gz/gvcf/gvcf.gz file after variant calling.
I/O--out-duplicate-metrics OUT_DUPLICATE_METRICSNoPath of a duplicate metrics file after marking duplicates.
I/O--proposed-variants PROPOSED_VARIANTSNoPath of the VCF file, which has proposed variants for the make examples stage.
Tool-k MINIMIZER_KMER_LEN, --minimizer-kmer-len MINIMIZER_KMER_LENNoMinimizer k-mer length.
Tool-ub, --both-strandsNoForce minimap2 to consider both strands when finding canonical splicing sites GT-AG.
Tool-uf, --forward-transcript-strandNoForce minimap2 to consider the forward transcript strand only when finding canonical splicing sites GT-AG.
Tool--mdNoOutput the MD tag.
Tool--eqxNoWrite =/X CIGAR operators.
Tool-y, --copy-commentNoAppend FASTQ comment to BAM output via auxiliary tag.
Tool-L INTERVAL, --interval INTERVALNoInterval within which to call bqsr from the input reads. All intervals will have a padding of 100 to get read records, and overlapping intervals will be combined. Interval files should be passed using the --interval-file option. This option can be used multiple times (e.g. "-L chr1 -L chr2:10000 -L chr3:20000+ -L chr4:10000-20000").
Tool-ip INTERVAL_PADDING, --interval-padding INTERVAL_PADDINGNoAmount of padding (in base pairs) to add to each interval you are including.
Tool--standalone-bqsrNoRun standalone BQSR after generating sorted BAM. This option requires both --knownSites and --out-recal-file input parameters.
Tool--read-group-sm READ_GROUP_SMNoSM tag for read groups in this run.
Tool--read-group-lb READ_GROUP_LBNoLB tag for read groups in this run.
Tool--read-group-pl READ_GROUP_PLNoPL tag for read groups in this run.
Tool--read-group-id-prefix READ_GROUP_ID_PREFIXNoPrefix for the ID and PU tags for read groups in this run. This prefix will be used for all pairs of FASTQ files in this run. The ID and PU tags will consist of this prefix and an identifier, that will be unique for a pair of FASTQ files.
Tool--disable-use-window-selector-modelNoChange the window selector model from Allele Count Linear to Variant Reads. This option will increase the accuracy and runtime.
Tool--gvcfNoGenerate variant calls in .gvcf format.
Tool--norealign-readsNoDo not locally realign reads before calling variants. Reads longer than 500 bp are never realigned.
Tool--sort-by-haplotypesNoReads are sorted by haplotypes (using HP tag).
Tool--keep-duplicatesNoKeep reads that are duplicate.
Tool--keep-legacy-allele-counter-behaviorNoIf specified, the behavior in this commit is reverted: 'https://github.com/google/deepvariant/commit/fbde0674639a28cb9e8004c7a01bbe25240c7d46'. We do not recommend setting this flag to True.
Tool--vsc-min-count-snps VSC_MIN_COUNT_SNPSNoSNP alleles occurring at least this many times in the AlleleCount will be advanced as candidates. (default: 2)
Tool--vsc-min-count-indels VSC_MIN_COUNT_INDELSNoIndel alleles occurring at least this many times in the AlleleCount will be advanced as candidates. (default: 2)
Tool--vsc-min-fraction-snps VSC_MIN_FRACTION_SNPSNoSNP alleles occurring at least this fraction of all counts in the AlleleCount will be advanced as candidates. (default: 0.12)
Tool--vsc-min-fraction-indels VSC_MIN_FRACTION_INDELSNoIndel alleles occurring at least this fraction of all counts in the AlleleCount will be advanced as candidates.
Tool--min-mapping-quality MIN_MAPPING_QUALITYNoBy default, reads with any mapping quality are kept. Setting this field to a positive integer i will only keep reads that have a MAPQ >= i. Note this only applies to aligned reads. (default: 5)
Tool--min-base-quality MIN_BASE_QUALITYNoMinimum base quality. This option enforces a minimum base quality score for alternate alleles. Alternate alleles will only be considered if all bases in the allele have a quality greater than min_base_quality. (default: 10)
Tool--alt-aligned-pileup ALT_ALIGNED_PILEUPNoValue can be one of [none, diff_channels]. Include alignments of reads against each candidate alternate allele in the pileup image.
Tool--variant-caller VARIANT_CALLERNoValue can be one of [VERY_SENSITIVE_CALLER, VCF_CANDIDATE_IMPORTER]. The caller to use to make examples. If you use VCF_CANDIDATE_IMPORTER, it implies force calling. Default is VERY_SENSITIVE_CALLER.
Tool--add-hp-channelNoAdd another channel to represent HP tags per read.
Tool--parse-sam-aux-fieldsNoAuxiliary fields of the BAM/CRAM records are parsed. If either --sort-by-haplotypes or --add-hp-channel is set, then this option must also be set.
Tool--use-wes-modelNoIf specified, the WES model file will be used. Only used in shortread mode.
Tool--include-med-dpNoIf specified, include MED_DP in the output gVCF records.
Tool--normalize-readsNoIf specified, allele counter left align INDELs for each read.
Tool--pileup-image-width PILEUP_IMAGE_WIDTHNoPileup image width. Only change this if you know your model supports this width. (default: 221)
Tool--channel-insert-sizeNoIf specified, add insert_size channel into the pileup image. By default, this parameter is true in WGS and WES mode.
Tool--no-channel-insert-sizeNoIf specified, don't add insert_size channel into the pileup image.
Tool--max-read-size-512NoAllow deepvariant to run on reads of size 512bp. The default size is 320 bp.
Tool--prealign-helper-threadNoUse an extra thread for the pre-align step. This parameter is more useful when --max-reads-size-512 is set.
Tool--track-ref-readsNoIf specified, allele counter keeps track of reads supporting ref. By default, allele counter keeps a simple count of the number of reads supporting ref.
Tool--phase-readsNoCalculate phases and add HP tag to all reads automatically.
Tool--dbg-min-base-quality DBG_MIN_BASE_QUALITYNoMinimum base quality in a k-mer sequence to consider. (default: 15)
Tool--ws-min-windows-distance WS_MIN_WINDOWS_DISTANCENoMinimum distance between candidate windows for local assembly. (default: 80)
Tool--channel-gc-contentNoIf specified, add gc_content channel into the pileup image.
Tool--channel-hmer-deletion-qualityNoIf specified, add hmer deletion quality channel into the pileup image.
Tool--channel-hmer-insertion-qualityNoIf specified, add hmer insertion quality channel into the pileup image.
Tool--channel-non-hmer-insertion-qualityNoIf specified, add non-hmer insertion quality channel into the pileup image.
Tool--skip-bq-channelNoIf specified, ignore base quality channel.
Tool--aux-fields-to-keep AUX_FIELDS_TO_KEEPNoComma-delimited list of auxiliary BAM fields to keep. Values can be [HP, tp, t0]. (default: HP)
Tool--vsc-min-fraction-hmer-indels VSC_MIN_FRACTION_HMER_INDELSNoHmer Indel alleles occurring at least this be advanced as candidates. Use this threshold if hmer and non-hmer indels should be treated differently (Ultima reads)Default will use the same threshold for hmer and non-hmer indels, as defined in vsc_min_fraction_indels.
Tool--vsc-turn-on-non-hmer-ins-proxy-supportNoAdd read-support from soft-clipped reads and other non-hmer insertion alleles,to the most frequent non-hmer insertion allele.
Tool--consider-strand-biasNoIf specified, expect SB field in calls and write it to the VCF file.
Tool--p-error P_ERRORNoBasecalling error for reference confidence model. (default: 0.001)
Tool--channel-ins-sizeNoIf specified, add another channel to represent size of insertions (good for flow-based sequencing).
Tool--max-ins-size MAX_INS_SIZENoMax insertion size for ins_size_channel, larger insertions will look like max (have max intensity). (default: 10)
Tool--disable-group-variantsNoIf using vcf_candidate_importer and multi-allelic sites are split across multiple lines in VCF, add this flag so that variants are not grouped when transforming CallVariantsOutput to Variants.
Tool--filter-reads-too-longNoIgnore all input BAM reads with size > 512bp.
Tool--haploid-contigs HAPLOID_CONTIGSNoOptional list of non autosomal chromosomes. For all listed chromosomes HET probabilities are not considered.
Tool--enable-small-modelNoIf supplied, enable the small model.
Performance--num-threads NUM_THREADSNoNumber of processing threads. (default: 20)
Performance--nstreams NSTREAMSNoNumber of streams to use per GPU. (default: 2)
Performance--gpuwriteNoUse one GPU to accelerate writing final BAM/CRAM.
Performance--gpuwrite-deflate-algo GPUWRITE_DEFLATE_ALGONoChoose the nvCOMP DEFLATE algorithm to use with --gpuwrite. Note these options do not correspond to CPU DEFLATE options. Valid options are 1, 2, and 4. Option 1 is fastest, while options 2 and 4 have progressively lower throughput but higher compression ratios. The default value is 1 when the user does not provide an input (i.e., None).
Performance--gpusortNoUse GPUs to accelerate sorting.
Performance--use-gdsNoUse GPUDirect Storage (GDS) to enable a direct data path for direct memory access (DMA) transfers between GPU memory and storage. Must be used concurrently with `--gpuwrite`. Please refer to Parabricks Documentation > Best Performance for information on how to set up and use GPUDirect Storage.
Performance--max-queue-chunks MAX_QUEUE_CHUNKSNoMax number of chunks to allow in the pre-alignment processing stage. Increasing this value may result in faster processing, but it will use more host memory. It is not recommended to use this argument for splice presets.
Performance--max-queue-reads MAX_QUEUE_READSNoMax number of reads to allow in the alignment processing stage. Increasing this value may result in faster processing, but it will use more host memory. (default: 500000)
Performance--low-memoryNoUse low memory mode.
Performance--chunk-size CHUNK_SIZENoMax number of reads in a processing chunk. Increasing this value may result in faster processing, but it will use more host memory. (default: 1000)
Performance--num-cpu-threads-per-stream NUM_CPU_THREADS_PER_STREAMNoNumber of CPU threads to use per stream. (default: 6)
Performance--num-streams-per-gpu NUM_STREAMS_PER_GPUNoNumber of streams to use per GPU. Default is 'auto' which will try to use an optimal amount of streams based on the GPU. (default: auto)
Performance--run-partitionNoDivide the whole genome into multiple partitions and run multiple processes at the same time, each on one partition.
Performance--gpu-num-per-partition GPU_NUM_PER_PARTITIONNoNumber of GPUs to use per partition.
Performance--max-reads-per-partition MAX_READS_PER_PARTITIONNoThe maximum number of reads per partition that are considered before following processing such as sampling and realignment. (default: 1500)
Performance--partition-size PARTITION_SIZENoThe maximum number of basepairs allowed in a region before splitting it into multiple smaller subregions. (default: 1000)
Performance--use-tf32NoEnable inference optimization using Tensor Float 32(TF32) on ampere+ gpu. Note that this might introduce a few mismatches in the output VCF.
Performance--read-from-tmp-dirNoRunning variant caller reading from bin files generated by Aligner and sort. Run postsort in parallel. This option will increase device memory usage.
Runtime--verboseNoEnable verbose output.
Runtime--x3NoShow full command line arguments.
Runtime--logfile LOGFILENoPath to the log file. If not specified, messages will only be written to the standard error output.
Runtime--tmp-dir TMP_DIRNoFull path to the directory where temporary files will be stored. (default: .)
Runtime--with-petagene-dir WITH_PETAGENE_DIRNoFull path to the PetaGene installation directory. By default, this should have been installed at /opt/petagene. Use of this option also requires that the PetaLink library has been preloaded by setting the LD_PRELOAD environment variable. Optionally set the PETASUITE_REFPATH and PGCLOUD_CREDPATH environment variables that are used for data and credentials. Optionally set the PetaLinkMode environment variable that is used to further configure PetaLink, notably setting it to "+write" to enable outputting compressed BAM and .fastq files.
Runtime--keep-tmpNoDo not delete the directory storing temporary files after completion.
Runtime--no-seccomp-overrideNoDo not override seccomp options for docker.
Runtime--versionNoView compatible software versions.
Runtime--preserve-file-symlinksNoOverride default behavior to keep file symlinks intact and *not* resolve the symlink.
Runtime--num-gpus NUM_GPUSNoNumber of GPUs to use for a run. (default: 1)