17.10 Updates and Additions

• Integrated full support for the official release of the CUDA Toolkit 9.0; resolved PGI 17.9 limitation involving changes to warp shuffle instructions in CUDA 9.0. CUDA 7.5 remains the default version used by the compilers. To use either CUDA 8.0 or CUDA 9.0 instead, add either the cuda8.0 or cuda9.0 sub-option to the -⁠ta=tesla or -⁠Mcuda compile- and link-time options.
• Implemented initial support for CUDA 9.0 cooperative groups in CUDA Fortran. Refer to the CUDA Fortran Programming Guide for details.

17.9 Updates and Additions

• PGI Accelerator Compilers
  Integrated support for the CUDA Toolkit 9.0 Release Candidate (RC). CUDA 7.5 remains the default version used by the compilers. To use either CUDA 8.0 or CUDA 9.0 instead, add either the cuda8.0 or cuda9.0 sub-option to the -⁠ta=tesla or -⁠Mcuda compile- and link-time options.

  Note: There are known problems using PGI 17.9 compilers with CUDA 9.0 installations. In particular, CUDA 9.0 introduces new synchronizing warp-shuffle instructions. The compilers in this PGI release still use older and now unreliable warp shuffle instructions to share values across threads in a warp for reduction operations and to copy values from vector lane zero to all vector lanes, in routines with the acc routine vector directive. This problem can occur even on Kepler and Maxwell GPUs. This issue is corrected in the PGI 17.10 release.

17.7 Updates and Additions

• PGI Accelerator Compilers
  • Added support for Tesla V100 GPUs in PGI OpenACC and CUDA Fortran. Based on the new NVIDIA Volta GV100 GPU, Tesla V100 offers more memory bandwidth, more streaming multiprocessors, next generation NVLink and new microarchitectural features that add up to better performance and programmability. For OpenACC and CUDA Fortran programmers, Tesla V100 offers improved hardware support and performance for CUDA Unified Memory features on both x86-64 and OpenPOWER processor-based systems. Use the sub-option cc70 with the -⁠ta=tesla or -⁠Mcuda compile- and link-time options to generate code for the Tesla V100 GPUs; the CUDA 9 toolkit must be used when targeting Volta GPUs.
  • Added initial support for the CUDA 9 toolkit. Requirements for using CUDA 9 with the PGI 17.7 compilers and profiler:
    • Install the release candidate (RC) or early access (EA) version of the CUDA 9 toolkit.
    • Specify the location of the CUDA 9 toolkit using the compiler option CUDAROOT; for example, if CUDA 9 is installed in /opt/cuda-9.0, add CUDAROOT=/opt/cuda-9.0 when compiling, linking, or invoking the profiler.
    • Use the sub-option cuda9.0 with the -⁠ta=tesla or -⁠Mcuda compile- and link-time options.
  • Added Beta support for automatic deep copy of Fortran derived types. This feature allows you to port applications with modern deeply nested data structures to Tesla GPUs using OpenACC. The PGI 17.7 compilers allow you to list aggregate Fortran data objects in OpenACC copy, copyin, copyout and update directives to move them between host and device memory including traversal and management of pointer-based objects within the aggregate data object. When enabled, full deep copy ensures that, when moving Fortran variables of derived type from host to device or device to host, the entire data structure, including pointers and allocatable arrays, is copied back and forth between host and device, or device and host, memory. To enable deep copy, use the deepcopy sub-option to -⁠ta=tesla. Two things to note: polymorphic data types are not supported, and the presence of overlapping pointers may cause runtime errors.
• All PGI Compilers
  • Improved inlining with the -⁠Minline option. You may see different functions inlined with this release than you observed in prior releases. In some cases, compilation may take noticeably longer to complete because of an increase in the number of functions inlined. Added the smallsize:number sub-option tomanage -⁠Minline which you can use to always inline functions of size smaller than number regardless of other size limits.
  • Added a new feature for program exploration called Program Analysis Summary Output (PASO). PASO enables the exporting of information gathered from the internal representation (IR) of various compilation stages in an organized, human readable format. This data forms a detailed summary of the program's internal structure, spanning not just one but all the files that make up a project. To enable PASO, compile and link with the -⁠Msummary option. To find out more about how PASO works and how you might leverage it, refer to Program Analysis Using Summary Output. PASO is available on all platforms supported by PGI except macOS.
• Profiler

  The following new profiler features are enabled only when invoking the profiler with CUDA 9. To use CUDA 9 with PGI 17.7, start the profiler with the CUDAROOT option set to the CUDA 9 installation location. For example, if CUDA 9 is installed in /opt/cuda-9.0, add CUDAROOT=/opt/cuda-9.0 when launching the profiler.

  • Enhanced unified memory profiling:
    • Added association between unified memory events/CPU page faults and the source location of memory allocation.
    • Added new unified memory profiling events for page thrashing, throttling and remote map.
    • Added support for switching between segment and non-segment timelines.
    • Added filtering of events based on the virtual address, migration reason or page fault access type.
  • Added an OpenACC details table summarizing each recorded event.
  • Added support for the profiling of cooperative kernel launches.
  • Added NVLink events to the timeline.
  • Added memory throughput to the NVLink topology diagram.
  • Increased the options available for multi-hop remote profiling.
  • Added support for OpenACC profiling on all multicore systems.
• Documentation

  Added HTML-formatted documentation for all PGI compilers and tools online at https://www.pgroup.com. The new format should make it easier to use any Internet-connected system or device to search or reference PGI documentation. PDF versions of all documents also remain available.

17.5 Updates and Additions

• CUDA Toolkit
  • This version includes an updated nvlink linker utility that addresses an issue with the link ordering of object files on Linux.
• Operating Systems
  • Added support for Ubuntu 17.04 and RHEL 6.9 Linux distributions.

17.4 Updates and Additions

• PGI Accelerator Compilers
  • Added support for atomic add and atomic subtract in device code for Fortran's single precision complex data types.
  • Added device versions of isfinite, isnan, isinf, and round.
  • Added support for CUDA Fortran warp-vote operations.
  • Changed the impact of -⁠g on the optimization level used when compiling device code. For host code, -⁠g sets the compiler's optimization level to zero unless another optimization level is specified using a -⁠O option. For device code, the same is now true. For details about generating debug information for device code, refer to the PGI Compiler Reference Guide's section on DWARF Debugging Formats.
  • Updated the version of nvlink to 8.0.73; this version of the device linker allows an arbitrary ordering of object files on its command line.

17.3 Updates and Additions

The 17.3 release was the first PGI Visual Fortran release in 2017; all updates and additions listed for PGI 17.1 apply to 17.3 as well unless otherwise noted.

17.1 Updates and Additions

• All PGI Compilers
  • Updated how floating point divides are computed to guarantee results will be uniform for scalar and vector operations. This change can cause numerical differences between PGI 17.1 and previous PGI releases. In rare cases, this change can cause an increase in execution time.
  • Improved inlining with the -⁠Minline option. You may see different functions inlined with this release than you observed in prior releases. In some cases, compilation may take longer to complete because of an increase in the number of functions inlined. Some of the -⁠Minline sub-options, which you can use to control inlining, have also changed from previous releases:
    • Added totalsize:n to limit inlining to the total size of n where n is the size of the combined program units on a per file basis.
    • Changed size:n to maxsize:n which allows inlining only of functions smaller than approximately n lines. The compilers silently convert the previous size:n to maxsize:n.
    • Dropped levels:n which limited inlining to n levels of functions. The compilers silently ignore levels:n.
  • Added the -⁠cpp option as an alias for the -⁠Mpreprocess option.
• PGI Accelerator OpenACC Compilers
  • Dropped support for CUDA 7.0.
  • Changed the default version of the CUDA Toolkit used by the compilers from CUDA 7.0 to 7.5. The CUDA Toolkit 8.0 can be used instead of 7.5 by adding the sub-option cuda8.0 to the -⁠ta=tesla or -⁠Mcuda compile- and link-time options.
  • Removed compute capability 2.0 (Fermi) from the compilers' default set of compute capabilities. The cc20 sub-option to the -⁠ta=tesla and -⁠Mcuda options is deprecated.
  • Improved data management within the OpenACC cache directive.
  • Added support for additional OpenACC 2.5 features:
    • Changed the behavior of the exit data directive to decrement the dynamic reference count.
    • Added the new optional finalize clause to set the dynamic reference count to zero.
    • Added the if_present clause to the update directive which changes the behavior when data is not present from a runtime error to a no-op.
    • Added new init, shutdown, and set directives.
    • Added new API routines to get and set the default async queue value.
    • Added support for the new definition of routine bind clause.
    • Updated the value of _OPENACC to 201510.
    With the exception of nested parallelism, declare link, and adding restrictions to cache clause variable references to variables within a cached region, OpenACC 2.5 feature support is complete in PGI 2017.
• Other Features and Additions
  • Added support for Microsoft Windows Server 2016.
• Deprecations and Eliminations
  • PGI 2017 supports 64-bit operating systems only. Compiling 32-bit applications for execution on either 32-bit or 64-bit operating systems is no longer supported on any platform.
  • Dropped support for the CUDA 7.0 toolkit.

Targeting a CUDA Toolkit Version

To use a version of the CUDA Toolkit, you must first download and install the appropriate CUDA Driver from NVIDIA at http://www.nvidia.com/cuda. The CUDA Driver version must be at least as new as the version of the CUDA Toolkit with which you compiled your code.

• The PGI tool pgaccelinfo prints the driver version as the first line of output. Use it if you are unsure which version of the CUDA toolkit is installed on your system.
• The CUDA 7.5 Toolkit is set as the default in PGI 17.10.
• You can compile with the CUDA 8.0 Toolkit either by adding the option -⁠ta=tesla:cuda8.0 to the command line or by adding set DEFCUDAVERSION=8.0 to the siterc file. Compiling with the CUDA 8 Toolkit means you must use a CUDA 8.0 or later driver.
• You can compile with the CUDA 9.0 Toolkit either by adding the option -⁠ta=tesla:cuda9.0 to the command line or by adding set DEFCUDAVERSION=9.0 to the siterc file. Compiling with the CUDA 9.0 Toolkit means you must use a CUDA 9.0 or later driver.

Multicore Support

PGI Accelerator OpenACC compilers support the option -⁠ta=multicore, to set the target accelerator for OpenACC programs to the host multicore CPU. This will compile OpenACC compute regions for parallel execution across the cores of the host processor or processors. The host multicore will be treated as a shared-memory accelerator, so the data clauses (copy, copyin, copyout, create) will be ignored and no data copies will be executed.

By default, -⁠ta=multicore will generate code that will use all the available cores of the processor. If the compute region specifies a value in the num_gangs clause, the minimum of the num_gangs value and the number of available cores will be used. At runtime, the number of cores can be limited by setting the environment variable ACC_NUM_CORES to a constant integer value. If an OpenACC compute construct appears lexically within an OpenMP parallel construct, the OpenACC compute region will generate sequential code. If an OpenACC compute region appears dynamically within an OpenMP region or another OpenACC compute region, the program may generate many more threads than there are cores, and may produce poor performance.

The ACC_BIND environment variable is set by default with -⁠ta=multicore; ACC_BIND has similiar behavior to MP_BIND for OpenMP.

The -⁠ta=multicore option differs from the -⁠ta=host option in that -⁠ta=host generates sequential code for the OpenACC compute regions.

Default Compute Capability

The default compute capability list for OpenACC and CUDA Fortran compilation for NVIDIA Tesla targets is cc30, cc35, and cc50. If CUDA 8.0 is specified using the cuda8.0 sub-option, cc60 is added to the default compute capability list. Similarly, if CUDA 9.0 is specified using the cuda9.0 sub-option, cc70 is added to the default compute capability list. The generation of device code can be time consuming, so you may notice an increase in compile time. You can override the default by specifying one or more compute capabilities using either command-line options or an rcfile.

To change the default with a command-line option, provide a comma-separated list of compute capabilities to -⁠ta=tesla: for OpenACC or -⁠Mcuda= for CUDA Fortran.

To change the default with an rcfile, set the DEFCOMPUTECAP value to a blank-separated list of compute capabilities in the siterc file located in your installation's bin directory:

set DEFCOMPUTECAP=30 35 50;

Alternatively, if you don't have permissions to change the siterc file, you can add the DEFCOMPUTECAP definition to a separate .mypgirc file (mypgi_rc on Windows) in your home directory.

OpenACC 2.5

The PGI compilers implement OpenACC 2.5 as defined in The OpenACC Application Programming Interface, Version 2.5, August 2013, http://www.openacc.org, with the exception that these features are not yet supported:

• nested parallelism
• declare link
• enforcement of the new cache clause restriction that all references to listed variables must lie within the region being cached

Support for Profiler/Trace Tool Interface

PGI compilers support the OpenACC 2.5 version of the OpenACC Profiler/Trace Tools Interface. This is the interface used by the PGI profiler to collect performance measurements of OpenACC programs.

OpenMP 3.1

The PGI Fortran, C, and C⁠+⁠+ compilers support OpenMP 3.1 on all platforms.

ACC routine directive limitations

• Fortran assumed-shape arguments are not yet supported.

Clause Support Limitations