cuEST: Accelerating Quantum Chemistry on NVIDIA GPUs

cuEST is a CUDA-X library providing GPU-accelerated primitives for quantum chemistry, particularly Gaussian-basis electronic structure theory.

Overview Focused on Gaussian-Basis Density Functional Theory (DFT)

Quantum chemistry is an intricate field, with high-performance software packages often featuring hundreds of thousands of lines of code to mix concepts like partial differential equations, nonlinear solvers, and domain-specific embeddings of the molecular problem into a methodology that can automatically predict useful chemical observable properties. One of the most verbose and computationally expensive aspects of this process is the manipulation of the Gaussian basis set, which codifies how the continuous geometry of the molecular electronic structure problem becomes discretized into digitally-representable tensor quantities. These tensor quantities can then be used with computationally efficient linear algebra methods to effectively solve the Schrodinger equation within the support of the Gaussian basis. A primary goal of cuEST is to make manipulations of these Gaussian-basis tensor quantities computationally performant.

Fig. 1 Electronic structure “building blocks”

One of the key targets of cuEST is Gaussian-basis density functional theory (DFT). A modern end-to-end DFT code will often orchestrate a “self-consistent field” (SCF) procedure which attempts to iteratively solve the Kohn-Sham DFT form of the ground-state electronic Schrödinger equation. Within each SCF iteration, the DFT code will start from a guess for the one-particle density matrix for the system (equivalent to the set of occupied molecular orbitals in the Gaussian basis). The DFT code will then form the Kohn-Sham potential matrix (also known as the Fock matrix), which represents essentially the derivative of the electronic energy if electrons are moved from place to place within the one-particle density matrix. When the DFT code diagonalizes this Kohn-Sham matrix, a new set of occupied orbitals is obtained, and the SCF procedure may be advanced to the next step. The Kohn-Sham matrix is made of a number of contributions defined by the “alphabet soup” name of the density functional (e.g., B3LYP or B97M-V), any one of which may be quite verbose to implement or quite computationally expensive to evaluate. These Kohn-Sham matrix contributions are key library API functions provided by cuEST.

What cuEST is not:

• An end-to-end quantum chemistry or density functional theory code: cuEST is intended to provide mid-level domain-specific library primitives that accelerate the domain-specific computational geometry problem of quantum chemistry. We leave the solution of the Kohn-Sham equations, the evaluation of interesting chemical observables, and the assembly of higher-level multiphysical properties to the calling end-to-end quantum chemistry code - cuEST only handles the middle level computational geometry problem. Note that end-to-end DFT and quantum chemistry codes using cuEST may also significantly benefit from use of other CUDA-X libraries such as cuBLAS, cuSOLVER, cuFFT, and others for the surrounding parts of the implementation.

• A J / K builder library: while the Coulomb (J) and exchange (K) matrices and their derivatives are key pieces of DFT and prominent cuEST API functions, experience shows that these terms alone are not sufficient to provide a useful GPU-accelerated electronic structure library. Therefore, cuEST also devotes significant attention to other numerically difficult terms related to the Gaussian basis set, such as Vxc, VV10, ECP, and PCM.

• An engine to obtain individual electron repulsion integrals (ERIs): While the ERI tensor is a beautiful object worth intense study, experience suggests that aggregating ERI contribution to the level of contracted matrix quantities like J and K provides dramatic performance improvements without sacrificing generality. Just as GEMM calls do not directly expose individual fused-multiply-add operations on parts of the involved tensors, cuEST does not directly expose individual ERI contributions to each Fock matrix build.

cuEST aims to provide intuitive C and Python APIs that provide users full access to all the features offered by our libraries. This library implements primitives suitable for calculations using contracted Gaussian-type orbitals and vacuum (non-periodic) boundary conditions. The routines provided within cuEST that compute J and K matrices use a density-fitted representation of the electron repulsion integrals.

Available Matrix Quantities

Table 1 cuEST Matrix and Gradient Capabilities

Name	Symbol	Matrix	Gradient	Notes
Overlap	S	Yes	Yes	—
Kinetic	T	Yes	Yes	—
Potential	V	Yes	Yes	—
Coulomb	J	Yes	Yes	Density Fitted
Exchange	K	Yes	Yes	Density Fitted
Exchange/Correlation	Vxc	Yes	Yes	—
VV10 Correlation	VV10	Yes	Yes	—
Effective Core Pseudopotentials	ECP	Yes	Yes	—
Polarizable Continuum Models	PCM	Yes	Yes	—

Basis Sets

Atom-centered, Gaussian basis sets are supported in cuEST.

• Both pure (spherical) and Cartesian treatments of angular momentum are supported by cuEST.

  • In density-fitted calculations, the auxiliary (or fitting) basis must use the spherical representation.

  • “Mixed” representations, where some shells are Cartesians and some are spherical, are not currently supported.

• The span of angular momentum L values for the full molecular system must be contiguous, and must include all basis functions beginning with L=0. In other words, “holes” in angular momentum in the basis set are not supported.

  • For example, if a molecule has basis functions defined for both L=0 and L=2 (s and d functions), the molecule must also have at least one basis function corresponding to L=1 (p functions).

  • Similarly, if a molecule has basis functions defined for L=1 (p functions), the molecule must also have at least one basis function corresponding to L=0 (s functions).

• Effective core pseudopotential (ECPs) matrices and gradients have been implemented using numerical quadrature.

Integration Grids

cuEST is designed to allow the user to construct Becke-style quadrature integration grids in a flexible, performant way. The user is able to specify the nodes and weights corresponding to the radial quadrature, and the Lebedev spherical grid number to be used at each radial quadrature grid. This yields coverage for both pruned and direct product integrations grids within the same API framework that can then be used for subsequent DFT computations..

Density Functionals

cuEST provides both a low-level and high-level exchange-correlation interface.

For the low-level interface, also referred to as the advanced interface in this documentation, the user has complete control over each step of the computation:

• the user provides a density and integration grid

• cuEST collocates the density onto the grid, a rate-limiting step in most DFT programs

• the user takes this grid-space density and computes the grid-space potential however they see fit

• cuEST takes this potential and collocates it back to basis function space, another rate-limiting step

A high-level interface is also provided for the user’s convenience. While the user still provides a density and integration grid, in this case cuEST uses the LIBXC package to evaluate the functional, returning the AO basis exchange-correlation potential. Only a subset of LIBXC’s functionals are supported via this interface - the list of which is shown below. A link to the LIBXC website and repository can be found at https://libxc.gitlab.io. Publications that incorporate DFT results from cuEST should cite [2].

cuEST depends on an unmodified build of LIBXC 7.0.0, which is distributed under the MPL 2.0 license. An unmodified copy of the LIBXC 7.0.0 source code is freely available at https://edge.urm.nvidia.com/artifactory/sw-math-libxc-generic-local/libxc-7.0.0.tar.gz

Table 2 Supported DFT Functionals

Functional Name	Type
HF	n/a
B3LYP1	Hybrid GGA
B3LYP5	Hybrid GGA
B97	Hybrid GGA
BLYP	GGA
M06L	Meta-GGA
PBE	GGA
PBE0	Hybrid GGA
R2SCAN	Meta-GGA
SVWN5	LDA
B97M-V	Meta-GGA with nonlocal correlation

In addition to the generic citation provided above, users are encouraged to cite references specific to the DFT functional they use. cuEST provides a convenient interface for obtaining the references directly from LIBXC:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <cuest.h>

int main(int argc, char **argv)
{
    /* Declare the cuEST handle. */
    cuestHandle_t handle;

    /* Declare the parameters for cuEST handle creation. */
    cuestHandleParameters_t handle_parameters;

    /* Create the cuEST handle parameters. This sets reasonable defaults. */
    checkCuestErrors(cuestParametersCreate(
        CUEST_HANDLE_PARAMETERS,
        &handle_parameters));

    /* Create the cuEST handle */
    cuestCreate(
        handle_parameters,
        &handle);

    /* The cuEST handle parameters may be destroyed immediately following handle creation. */
    checkCuestErrors(cuestParametersDestroy(
        CUEST_HANDLE_PARAMETERS,
        handle_parameters));

    /* Create workspace descriptors */
    cuestWorkspaceDescriptor_t* persistentWorkspaceDescriptor = (cuestWorkspaceDescriptor_t*) malloc(sizeof(cuestWorkspaceDescriptor_t));
    cuestWorkspaceDescriptor_t* temporaryWorkspaceDescriptor = (cuestWorkspaceDescriptor_t*) malloc(sizeof(cuestWorkspaceDescriptor_t));
    /*
     * Create cuEST objects necessary to form XC integral plan:
     *   - cuEST AO basis        (basis)
     *   - cuEST molecular grid  (molecular_grid)
     *
     * Please review the cuEST samples for details
     */

    /* Create the XC integral plan */
    cuestXCIntPlan_t xcint_plan;

    cuestXCIntPlanParameters_t xcint_plan_parameters;
    cuestParametersCreate(
        CUEST_XCINTPLAN_PARAMETERS,
        &xcint_plan_parameters);

    cuestXCIntPlanCreateWorkspaceQuery(
        handle,
        basis,
        molecular_grid,
        CUEST_XCINTPLAN_PARAMETERS_FUNCTIONAL_B3LYP1,
        xcint_plan_parameters,
        persistentWorkspaceDescriptor,
        temporaryWorkspaceDescriptor,
        &xcint_plan);

    cuestWorkspace_t* persistentXCIntPlanWorkspace = allocateWorkspace(persistentWorkspaceDescriptor);;
    cuestWorkspace_t* temporaryXCIntPlanWorkspace = allocateWorkspace(temporaryWorkspaceDescriptor);;

    cuestXCIntPlanCreate(
        handle,
        basis,
        molecular_grid,
        CUEST_XCINTPLAN_PARAMETERS_FUNCTIONAL_B3LYP1,
        xcint_plan_parameters,
        persistentXCIntPlanWorkspace,
        temporaryXCIntPlanWorkspace,
        &xcint_plan);

    cuestParametersDestroy(
        CUEST_XCINTPLAN_PARAMETERS,
        xcint_plan_parameters);

    char*   engineCitation       = NULL;
    char*   functionalCitation   = NULL;

    cuestQuery(
        handle,
        CUEST_XCINTPLAN,
        xcint_plan,
        CUEST_XCINTPLAN_ENGINE_CITATION,
        &engineCitation,
        sizeof(char*));
    cuestQuery(
        handle,
        CUEST_XCINTPLAN,
        xcint_plan,
        CUEST_XCINTPLAN_FUNCTIONAL_CITATION,
        &functionalCitation,
        sizeof(char*));

    /* Free the memory for the citations (after using them, of course) */
    free(engineCitation);
    free(functionalCitation);

    /* clean up */

    cuestXCIntPlanDestroy(xcint_plan);

    freeWorkspace(temporaryXCIntPlanWorkspace);
    freeWorkspace(persistentXCIntPlanWorkspace);

    free(persistentWorkspaceDescriptor);
    free(temporaryWorkspaceDescriptor);

    /* destroy the handle */
    cuestDestroy(handle);

    return 0;
}

Effective Core Pseudopotentials

Effective core pseudopotentials, or ECPs, enable treating the core electrons of heavy atoms via definition of a potential model. In cuEST this potential is represented on a numerical integration grid to extremely high precision. Both the potential matrix and analytic gradients are available in cuEST.

Polarizable Continuum Models

A polarizable continuum model, or PCM, based on the improved switching Gaussian approach (iSWIG) [1] is available in cuEST. PCM provides a simple, standardized, and accurate way of embedding a molecule inside a medium with a user-defined dielectric constant, representing the molecule inside a uniform solvent. Both PCM potential and analytic gradients may be computed.

Getting Started

To get started with cuEST, take a look at some simple examples on the API Examples page. Refer to our Installation Guide for detailed instructions on the various installation choices available.

[1]

Lange, A.W. and Herbert, J.M. J. Chem. Phys. 133, 244111, 2010.

[2]

Susi Lehtola, Conrad Steigemann, Micael J. T. Oliveira, and Miguel A. L. Marques, Recent developments in Libxc - A comprehensive library of functionals for density functional theory, Software X 7, 1 (2018).