DOCA App Shield
This guide provides instructions on using the DOCA App Shield API.
DOCA App Shield API offers a solution for strong intrusion detection capabilities using the DPU services to collect and analyze data from the host's (or a VM on the host ) memory in real time. This solution provides intrusion detection and forensics investigation in a way that is:
Robust against attacks on a host machine
Able to detect a wide range of attacks (including zero-day attacks)
Least disruptive to the execution of host application (where current detection solutions hinder the performance of host applications)
Transparent to the host, such that the host does not need to install anything (other than providing some files obtained from the doca_apsh_config.py tool)
App Shield uses a DMA device to access the host's memory and analyze it.
The App Shield API provides multiple functions that help with gathering data extracted from system's memory (e.g., processes list, modules list, connections). This data helps with detecting attacks on critical services or processes in a system (e.g., services that enforce integrity or privacy of the execution of different applications).
Make sure to run App Shield from the DPU as a root user.
The library requires firmware version 24.32.1010 or higher.
For the library API reference, refer to the DOCA APSH API documentation in the NVIDIA DOCA Library APIs.
The pkg-config (*.pc file) for the APSH library is included in DOCA's regular definitions (i.e., doca).
The following sections provide additional details about the library API.
doca_apsh_dma_dev_set
To attach a DOCA DMA device to App Shield, calling this function is mandatory and must be done before calling doca_apsh_start.
doca_apsh_dma_dev_set(doca_apsh_ctx, doca_dev)
doca_apsh_ctx [in] – App Shield opaque context struct
doca_dev [in] – struct for DOCA Device with DMA capabilities
Capabilities Per System
For each initialized system, App Shield retrieves an array of the requested object according to the getter's name:
Getter Function Name |
Functions Information |
Functions Signature |
Return Type |
|||||||||||||||||
Get modules |
Returns an array with information about the system modules (drivers) loaded into the kernel of the OS. |
|
|
|||||||||||||||||
Get processes |
Returns an array with information about each process running on the system. |
|
|
|||||||||||||||||
Get library |
For a specified process, this function returns an array with information about each library loaded into this process. |
|
|
|||||||||||||||||
Get threads |
For a specified process, this function returns an array with information about each thread running within this process. |
|
|
|||||||||||||||||
Get virtual memory areas/virtual address description |
For a specified process, this function returns an array with information about each virtual memory area within this process. |
|
|
|||||||||||||||||
Get privileges |
For a specified process, this function returns an array with information about each possible privilege for this process, as described here. Warning
Available on a Windows host only.
|
|
|
|||||||||||||||||
Get environment variables |
For a specified process, this function returns an array with information about each environment variable within this process. Warning
Available on a Windows host only.
|
|
|
|||||||||||||||||
Get handles |
For a specified process, this function returns an array with information about each handle this process holds. Warning
Available on a Windows host only.
|
|
|
|||||||||||||||||
Get LDR modules |
For a specified process, this function returns an array with information about each loaded module within this process. Warning
Available on a Windows host only.
|
|
|
|||||||||||||||||
Process attestation |
For a specified process, this function attests the memory pages of the process according to a precomputed golden hash file given as an input. Warning
Single-threaded processes are supported at beta level.
|
|
|
|||||||||||||||||
Attestation refresh |
Refreshes a single attestation handler of a process with a new snapshot. |
|
|
|||||||||||||||||
Get NetScan |
This function scans the system's physical memory and returns an array with information about each socket that resides in the memory. Warning
Only available on hosts with one of the following Windows 10 OS builds:
Warning
This feature is currently supported at beta level.
|
|
|
|||||||||||||||||
Get process parameters |
For a specified process, this function returns a struct object (not an array) with information about the process' parameters (ones not included in the "get processes" capability). Warning
Available on a Windows host only.
Warning
This feature is currently supported at beta level.
|
|
|
|||||||||||||||||
Get security identifier (SID) |
For a specified process, this function returns an array with information about each SID (security identifier) included in the process's security context. Warning
Available on a Windows host only.
|
|
|
|||||||||||||||||
Perform Yara scan |
For a specified process, this function returns an array with information about each Yara rule match found in the process's memory. Warning
Available on a Windows host and Ubuntu 22.04 DPU.
|
Warning
To get a better understanding of the arguments, refer to documentation in doca_apsh.h. |
|
The following attribute getters return a specific attribute of an object, obtained from the array returned from the getter functions listed above, depending on the requested attribute:
doca_apsh_process_info_get(struct doca_apsh_proccess *process, enum
doca_apsh_process_attr attr);
doca_apsh_module_info_get(struct doca_apsh_module *module, enum
doca_apsh_module_attr attr);
doca_apsh_lib_info_get(struct doca_apsh_lib *lib, enum
doca_apsh_lib_attr attr);
doca_apsh_thread_info_get(struct doca_apsh_thread *thread, enum
doca_apsh_lib_attr attr);
doca_apsh_vad_info_get(struct doca_apsh_vad *vad, enum
doca_apsh_vad_attr attr);
doca_apsh_privilege_info_get(struct doca_apsh_privilege *privilege, enum
doca_apsh_privilege_attr attr);
doca_apsh_envar_info_get(struct doca_apsh_envar *envar, enum
doca_apsh_envar_attr attr);
doca_apsh_handle_info_get(struct doca_apsh_handle *handle, enum
doca_apsh_handle_attr attr);
doca_apsh_ldrmodule_info_get(struct doca_apsh_ldrmodule *ldrmodule, enum
doca_apsh_ldrmodule_attr attr);
doca_apsh_attst_info_get(struct doca_apsh_attestation *attestation, enum
doca_apsh_attestation_attr attr);
doca_apsh_netscan_info_get(struct doca_apsh_netscan *connection, enum
doca_apsh_netscan_attr attr)
doca_apsh_process_parameters_info_get(struct doca_apsh_process_parameters *process_parameters, enum
doca_apsh_process_parameters_attr attr);
doca_apsh_sid_info_get(struct doca_apsh_sid *sid, enum
doca_apsh_sid_attr attr);
doca_apsh_yara_info_get(struct doca_apsh_yara *yara, enum
doca_apsh_yara_attr attr);
The return type of the attribute getter can be found in doca_apsh_attr.h.
Usage example:
const
uint pid = doca_apsh_process_info_get(processes[i], DOCA_APSH_PROCESS_PID);
const
char
*proc_name = doca_apsh_process_info_get(processes[i], DOCA_APSH_PROCESS_COMM);
To use App Shield, users must initialize and configure two main structs. This section presents these structs and explains how to interact with them.
doca_apsh_ctx
doca_apsh_ctx is the basic struct used by App Shield which defines the DMA device used to perform the memory forensics techniques required to run App Shield.
The same doca_apsh_ctx struct may be used to run multiple App Shield instances over different systems (e.g., two different VMs on the host).
To acquire an instance of the doca_apsh_ctx struct, use the following function:
struct doca_apsh_ctx *doca_apsh_create(void);
To configure the doca_apsh_ctx instance with DMA device to use:
doca_error_t doca_apsh_dma_dev_set(struct doca_apsh_ctx *ctx, struct doca_dev *dma_dev);
To start the doca_apsh_ctx instance, call the following function:
doca_error_t doca_apsh_start(struct doca_apsh_ctx *ctx);
To destroy the doca_apsh_ctx instance when it is no longer needed, call:
void doca_apsh_destroy(struct doca_apsh_ctx *ctx);
doca_apsh_system
The doca_apsh_system struct is built on the doca_apsh_ctx instance. This struct is created per system running App Shield. doca_apsh_system defines multiple attributes used by App Shield to perform memory analysis over the specific system successfully.
To acquire an instance of the doca_apsh_system struct, use the following function:
const uint pid = doca_apsh_process_info_get(processes[i], DOCA_APSH_PROCESS_PID); const char *proc_name = doca_apsh_process_info_get(processes[i], DOCA_APSH_PROCESS_COMM);
To configure different attributes for the system instance:
OS type – specifies the system's OS type.
doca_error_t doca_apsh_sys_os_type_set(struct doca_apsh_system *ctx, enum doca_apsh_system_os os_type);
WarningCurrently supported types: Windows or Linux.
System representor – specifies the representor of the device connected to the system for App Shield to run on (which can be a representor of VF/PF). For information on querying the DOCA device, refer to the DOCA Core.
After acquiring the DOCA device, use the following function to configure it into the system instance:
doca_error_t doca_apsh_sys_dev_set(struct doca_apsh_system *system, struct doca_dev_rep *dev);
System symbols map – includes information about the OS that App Shield is attempting to run on (e.g., Window 10 Build 18363) and the size and fields of the OS structures, which helps App Shield with the memory forensic techniques it uses to access and analyze these structures in the system's memory. This can be obtained by running the doca_apsh_config.py on the system machine.
After obtaining it, run:
doca_error_t doca_apsh_sys_os_symbol_map_set(struct doca_apsh_system *system, const char *system_os_symbol_map_path);
Memory regions – includes the physical addresses of the memory regions which are mapped for system memory RAM. This is needed to prevent App Shield from accessing other memory regions, such as memory mapped I/O regions. This can be obtained by running the doca_apsh_config.py tool on the system machine.
After obtaining it, run:
doca_error_t doca_apsh_sys_mem_region_set(struct doca_apsh_system *system, const char *system_mem_region_path);
KPGD file (optional and relevant only for Linux OS) – contains the KPGD physical address and the virtual address of init_task. This information is required since App Shield extracts data from the kernel struct in the physical memory. Thus, the kernel page directory table must translate the virtual addresses of these structs. This can be obtained by running the doca_apsh_config.py tool on the system machine with the flag find_kpgd=1. Since setting this attribute is optional, App Shield can work without it, but providing it speeds up App Shield's initialization process.
After obtaining it, run:
doca_error_t doca_apsh_sys_kpgd_file_set(struct doca_apsh_system *system, const char *system_kpgd_file_path);
To start the doca_apsh_system:
doca_error_t doca_apsh_system_start(struct doca_apsh_system *system);
To destroy the doca_apsh_system instance when it is no longer needed, call:
void
doca_apsh_system_destroy(struct doca_apsh_system *system);
doca_apsh_config.py Tool
The doca_apsh_config.py tool is a python3 script which can be used to obtain all the attributes needed to run doca_apsh_system instance.
The following parameters are necessary to use the tool:
Parameter |
Description |
pid (optional) |
The process ID of the process we want to run attestation capability on |
os (mandatory) |
The OS type of the machine (i.e., Linux or Windows) |
find_kpgd (optional) |
Relevant for Linux OS only, AS flag to enable/disable creating kpgd_file.conf. Default 0. |
files (mandatory) |
A list of files for the tool to create. File options: hash, symbols, memregions, kpgd_file (only relevant for Linux). Warning
Make sure that the value set is appropriate for your setup.
|
path (mandatory) |
Warning
Make sure that the value set is appropriate for your setup.
|
The tool creates the following files:
Symbol map – this file changes once the system kernel is updated or a kernel module is installed. The file does not change on system reboot.
Memory regions – this file changes when adding or removing hardware or drivers that affect the system's memory map (e.g., when adding register addresses). The file does not change on system reboot.
hash.zip – this file is required for attestation but is unnecessary for all other capabilities. The ZIP file contains the required data to attest to a single process. The file changes on library or executable update.
kpgd_file.conf (relevant for Linux OS only) – helps with faster initialization of the library. The file changes on system reboot.
This section provides DOCA App Shield library sample implementations on top of BlueField DPU.
Sample Prerequisites
Follow the guidelines in section "Prerequisites" then copy the generated JSON files, symbols.json and mem_regions.json , to the /tmp/ directory.
Running the Sample
Refer to the following documents:
NVIDIA DOCA Installation Guide for Linux for details on how to install BlueField-related software.
NVIDIA DOCA Troubleshooting Guide for any issue you may encounter with the installation, compilation, or execution of DOCA samples.
To build a given sample:
cd /opt/mellanox/doca/samples/doca_apsh/<sample_name> meson /tmp/build ninja -C /tmp/build
WarningThe binary doca_<sample_name> will be created under /tmp/build/.
Sample (e.g., apsh_libs_get) usage:
Usage: doca_apsh_libs_get [DOCA Flags] [Program Flags] DOCA Flags: -h, --help Print a help synopsis -v, --version Print program version information -l, --log-level Set the (numeric) log level
for
the program <10
=DISABLE,20
=CRITICAL,30
=ERROR,40
=WARNING,50
=INFO,60
=DEBUG,70
=TRACE> --sdk-log-level Set the SDK (numeric) log levelfor
the program <10
=DISABLE,20
=CRITICAL,30
=ERROR,40
=WARNING,50
=INFO,60
=DEBUG,70
=TRACE> -j, --json <path> Parse all command flags from an input json file Program Flags: -p, --pid Process ID of process to be analyzed -f, --vuid VUID of the System device -d, --dma DMA device name -s, --osty <windows|linux> System OS type - windows/linuxFor additional information per sample, use the -h option:
/tmp/build/doca_<sample_name> -h
Samples
Apsh Libs Get
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of loadable libraries of a specific process.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and starting the Apsh context.
Opening DOCA remote PCI device via given vendor unique identifier (VUID).
Creating DOCA Apsh system handler.
Setting fields and starting Apsh system handler.
Getting the list of system process using Apsh API and searching for a specific process with the given PID.
Getting the list of process-loadable libraries using doca_apsh_libs_get Apsh API call.
Querying the libraries for 3 selected fields using doca_apsh_lib_info_get Apsh API call.
Printing libraries' attributes to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_libs_get/apsh_libs_get_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_libs_get/apsh_libs_get_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_libs_get/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h
Apsh Modules Get
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of installed modules on a monitored system.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and starting the Apsh context.
Opening DOCA remote PCI device via given VUID.
Creating DOCA Apsh system handler.
Setting fields and start Apsh system handler.
Getting the the list of system-installed modules using doca_apsh_modules_get Apsh API call.
Querying the names of modules using doca_apsh_module_info_get Apsh API call.
Printing the attributes of up to 5 modules attributes to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_modules_get/apsh_modules_get_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_modules_get/apsh_modules_get_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_modules_get/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h
Apsh Pslist
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of running processes on a monitored system.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and starting the Apsh context.
Opening DOCA remote PCI device via given VUID.
Creating DOCA Apsh system handler.
Setting fields and starting Apsh system handler.
Getting the list of processes running on the system using doca_apsh_processes_get Apsh API call.
Querying the processes for 4 chosen attributes using doca_apsh_proc_info_get Apsh API call.
Printing the attributes of up to 5 processes to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_pslist/apsh_pslist_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_pslist/apsh_pslist_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_pslist/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h
Apsh Threads Get
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of threads of a specific process.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and starting the Apsh context.
Opening DOCA remote PCI device via given VUID.
Creating DOCA Apsh system handler.
Setting fields and starting Apsh system handler.
Getting the list of system processes using Apsh API and searching for a specific process with the given PID.
Getting the list of process threads using doca_apsh_threads_get Apsh API call.
Querying the threads for up to 3 selected fields using doca_apsh_thread_info_get Apsh API call.
Printing thread attributes to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_threads_get/apsh_threads_get_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_threads_get/apsh_threads_get_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_threads_get/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h
Apsh Vads Get
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of virtual address descriptors (VADs) of a specific process.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and start the Apsh context.
Opening DOCA remote PCI device via given VUID.
Creating DOCA Apsh system handler.
Setting fields and starting Apsh system handler.
Getting the list of system processes using Apsh API and searching for a specific process with the given PID.
Getting the list of process VADs using doca_apsh_vads_get Apsh API call.
Querying the VADs for 3 selected fields using doca_apsh_vad_info_get Apsh API call.
Printing the attributes of up to 5 VADs to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_vads_get/apsh_vads_get_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_vads_get/apsh_vads_get_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_vads_get/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h
Apsh Envars Get
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of environment variables of a specific process.
This sample works only on target systems with Windows OS.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and starting the Apsh context.
Opening DOCA remote PCIe device via given VUID.
Creating DOCA Apsh system handler.
Setting fields and starting Apsh system handler.
Getting the list of system processes using Apsh API and searching for a specific process with the given PID.
Getting the list of process envars using doca_apsh_envars_get Apsh API call.
Querying the envars for 2 selected fields using doca_apsh_envar_info_get Apsh API call.
Printing the envars attributes to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_envars_get/apsh_envars_get_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_envars_get/apsh_envars_get_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_envars_get/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h
Apsh Privileges Get
This sample illustrates how to properly initialize DOCA App Shield and use its API to get the list of privileges of a specific process.
This sample works only on target systems with Windows OS.
The sample logic includes:
Opening DOCA device with DMA ability.
Creating DOCA Apsh context.
Setting and starting the Apsh context.
Opening DOCA remote PCIe device via given VUID.
Creating DOCA Apsh system handler.
Setting fields and starting Apsh system handler.
Getting the list of system processes using Apsh API and searching for a specific process with the given PID.
Getting the list of process privileges using the doca_apsh_privileges_get Apsh API call.
Querying the privileges for 5 selected fields using the doca_apsh_privilege_info_get Apsh API call.
Printing the privileges attributes to the terminal.
Cleaning up.
References:
/opt/mellanox/doca/samples/doca_apsh/apsh_privileges_get/apsh_privileges_get_sample.c
/opt/mellanox/doca/samples/doca_apsh/apsh_privileges_get/apsh_privileges_get_main.c
/opt/mellanox/doca/samples/doca_apsh/apsh_privileges_get/meson.build
/opt/mellanox/doca/samples/doca_apsh/apsh_common.c; /opt/mellanox/doca/samples/doca_apsh/apsh_common.h