Build Your Own Thin Client

Overview

The thin client and cli provide ease of access to the cuOpt service, but these are simply references on how to communicate with the cuOpt service. The underlying details of the API are discussed below if you wish to create your own thin client.

API Authorization

Once you have the NVIDIA Identity Federation API Key, you can pass it in the Authorization header on subsequent requests to the cuOpt Service.

List cuOpt Versions

cuOpt admins create functions on behalf of the client, and these versions can be listed so that the latest/appropriate version can be chosen.

Example:

1curl --request GET https://api.nvcf.nvidia.com/v2/nvcf/functions
2--header 'Content-Type: application/json' \
3--header 'Authorization: Bearer <NVIDIA Identity Federation API Key>'

This would fetch a list of versions of cuOpt that are available, depending on creation date clients can choose cuOpt version.

The response would look something like the following:

 1{
 2 "functions": [
 3     {
 4         "id": "649b46a7-f65d-4943-873f-920bfe37ae66",
 5         "versionId": "b9957f66-4e1d-4957-8ab8-cf472e3a0f5b",
 6         ...
 7         ...
 8         ...
 9         "createdAt": "2023-08-14T12:16:32.116Z"
10     }
11   ]
12 }

id and versionId would be used to invoke a cuOpt function.

Invoking cuOpt Service

Sending cuOpt Request when payload is <= 250KB

This method is used to execute the cuOpt function. The “requestBody” contains a JSON object which will be processed by cuOpt.

You must include a function ID and versionId in the URL as shown below; this identifies the specific function to invoke.

Example with Function Version ID:

 1curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/pexec/functions/{id}/versions/{versionId}' \
 2--header 'Content-Type: application/json' \
 3--header 'Authorization: Bearer <NVIDIA Identity Federation API Key>' \
 4--data '{
 5        "data": {
 6          "cost_matrix_data": {"data": {"0": [[0, 1], [1, 0]]}},
 7          "task_data": {"task_locations": [1], "demand": [[1]], "task_time_windows": [[0, 10]], "service_times": [1]},
 8          "fleet_data": {"vehicle_locations":[[0, 0]], "capacities": [[2]], "vehicle_time_windows":[[0, 20]] },
 9          "solver_config": {"time_limit": 2}
10        },
11        "client_version": "custom"
12    }
13}'

JSON_DATA should follow Open-API spec described for cuOpt input.

Sending cuOpt Request when payload is > 250KB

The maximum size of the requestBody is 250KB. If the size of the JSON_DATA is larger than 250KB, it first needs to be uploaded separately, and the asset id which is returned needs to be referenced in the request. cuOpt will fetch the asset based on the ID and process it.

Example:

1curl -X 'POST' \
2'https://api.nvcf.nvidia.com/v2/nvcf/assets' \
3-H 'Authorization: Bearer <NVIDIA Identity Federation API Key>' \
4-H 'accept: application/json' \
5-H 'Content-Type: application/json' \
6-d '{
7"contentType": "application/octet-stream",
8"description": "Optimization-data"
9}'

This will result in a response like this, containing the “assetId” and a “uploadUrl” to push the data to:

1{
2    "assetId":"b5b841c3-11c2-4c34-b057-9475e82c5369",
3    "uploadUrl":"<pre-signed-upload-url>",
4    "contentType":"application/octet-stream",
5    "description":"Optimization-data"
6}

Compress the JSON data.

1import msgpack
2
3pickled_cuopt_data = msgpack.dump(cuopt_problem_json_data, open('problem_data.mpk', 'wb'))

Upload the compressed asset to the cloud.

1 curl -X PUT -T problem_data.mpk "<pre-signed-URL>" -H "Content-Type: application/octet-stream" -H "x-amz-meta-nvcf-asset-description: Optimization-data"

Listing Assets

1curl -X GET https://api.nvcf.nvidia.com/v2/nvcf/assets -H "Authorization: Bearer <NVIDIA Identity Federation API Key>"

Referencing Assets when Invoking a Function

To use an asset in a function invocation, add a requestHeader field as shown below. Note that the value of inputAssetReferences can technically be a list, but cuOpt expects only a single assetId. Since the cuOpt data has been uploaded as an asset, set the requestBody to {}.

1 curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/pexec/functions/{id}/versions/{versionId}' \
2 --header 'Content-Type: application/json' \
3 --header 'Authorization: Bearer <NVIDIA Identity Federation API Key>' \
4 --header 'NVCF-INPUT-ASSET-REFERENCES: <assetID>' \
5 --data '{
6     "data": null,
7     "client_version": "custom"
8 }'

Deleting Assets

Assets should be deleted after you are done processing them; however, they will be automatically garbage collected.

1curl -X DELETE https://api.nvcf.nvidia.com/v2/nvcf/assets/{assetId} -H "Authorization: Bearer <NVIDIA Identity Federation API Key>"

cuOpt Result Retrieval

The cuOpt service employs long polling for invocation and result retrieval.

When you make an invocation request, the system will hold your request open for a period of time before returning with either:

> HTTP Status 200 completed result

> HTTP Status 202 polling response

On receipt of a polling response, your client should immediately poll the cuOpt service to retrieve your result. Polling once a second is recommended.

A polling scenario would have the following response for an invocation:

1{
2 "reqId": "ef4c1967-d543-467c-af7e-8c7899d75be8",
3 "status": "pending-evaluation"
4}

Polling for a response:

1curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/pexec/status/{request-id}' \
2--header 'Authorization: Bearer <JWT>'

For best performance, it is expected that clients will not close connections until it is determined that no further communication with a server is necessary.

cuOpt Large Response Retrieval

In some cases, the result of an invocation may return a URL for downloading a large response. You will receive a reference in your results to download the payload. A GET request on the URL will return a zipfile containing the result. The URL’s TTL is 24 hours.

1{
2 "reqId":"abc123",
3 "status":"fulfilled",
4 "responseReference": "<pre-signed download url for large result>",
5 "percentComplete":100,
6 "errorCode":0
7}