Build your own Thin Client

The thin client and cli provide ease of access to the cuOpt service, but these are just references on how to communicate with the cuOpt service. Underlying details of the API have been discussed below if the user wants to create their own thin client.

A signed JWT issued by SSA authorizes against the Cloud Functions API. This token is obtained by posting the “Client Id” and “Secret” along with the scopes to the Authorization service.

URL to request authorization : https://tbyyhdy8-opimayg5nq78mx1wblbi8enaifkmlqrm8m.ssa.nvidia.com

Example:

            

            
curl --include \
  --user <client_id>:<client_secret> \
  --request POST https://tbyyhdy8-opimayg5nq78mx1wblbi8enaifkmlqrm8m.ssa.nvidia.com \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode 'grant_type=client_credentials' \
  --data-urlencode 'scope=invoke_function,list_functions'
        

Once you have the token, you can pass it in the ‘Authorization’ header on subsequent requests to cuOpt Service.

You will need to check the validity of the JWT TTL, and when the time is close to expiring, you should request a new one.

cuOpt admins create functions on behalf of the client, and these versions can be listed so that the latest/appropriate version can be chosen.

Example:

            

            
curl --request GET https://api.nvcf.nvidia.com/v2/nvcf/functions
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <Token>'
        

This would fetch a list of versions of cuOpt that are available, depending on creation date clients can choose cuOpt version.

Response would look something like this,

            

            
{
 "functions": [
     {
         "id": "649b46a7-f65d-4943-873f-920bfe37ae66",
         "versionId": "b9957f66-4e1d-4957-8ab8-cf472e3a0f5b",
         ...
         ...
         ...
         "createdAt": "2023-08-14T12:16:32.116Z"
     }
   ]
 }
        

id and versionId would be used to invoke a cuOpt function.

Sending cuOpt Request when payload is <= 250KB

This method is used to execute the cuOpt function. The “requestBody” contains a JSON object which will be processed by cuOpt.

You must include a function id and versionId in the url as shown below; this identifies the specific function to invoke.

Example with Function Version ID:

            

            
curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/exec/functions/{id}/versions/{versionId}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <Token>' \
--data '{
"requestBody": {
"data": JSON_DATA,
"client_version": "custom"
}
}'
        

JSON_DATA should follow Open-API spec described for cuOpt input.

Sending cuOpt Request when payload is > 250KB

The maximum size of the requestBody is 250KB. If the size of the JSON_DATA is larger than 250k, it first needs to be uploaded separately and the asset id which is returned needs to be referenced in the request. cuOpt will fetch the asset based on the id and process it.

Example:

            

            
curl -X 'POST' \
'https://api.nvcf.nvidia.com/v2/nvcf/assets' \
-H 'Authorization: Bearer <JWT Token>' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"contentType": "application/octet-stream",
"description": "Optimization-data"
}'
        

This will result in a response like this, containing the “assetId” and a “uploadUrl” to push the data to

            

            
{
    "assetId":"b5b841c3-11c2-4c34-b057-9475e82c5369",
    "uploadUrl":"<pre-signed-upload-url>",
    "contentType":"application/octet-stream",
    "description":"Optimization-data"
}
        

Pickle the Json data

            

            
import pickle

pickled_cuopt_data = pickle.dump(cuopt_problem_json_data, open('problem_data.pkl', 'wb'))
        

Upload the pickled asset to cloud

            

            
 curl -X PUT -T problem_data.pkl <pre-signed-URL> -H "Content-Type: application/octet-stream" -H "x-amz-meta-nvcf-asset-description: Optimization-data"
        

Listing Assets

            

            
curl -X GET https://api.nvcf.nvidia.com/v2/nvcf/assets -H "Authorization: Bearer <JWT Token>"
        

Referencing Assets when invoking a Function

To use an asset in a function invocation, add a requestHeader field as shown below. Note that the value of inputAssetReferences can technically be a list, but cuOpt expects only a single assetId. Since the cuOpt data has been uploaded as an asset, set the requestBody to {}.

            

            
 curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/exec/functions/{id}/versions/{versionId}' \
 --header 'Content-Type: application/json' \
 --header 'Authorization: Bearer <Token>' \
 --data '{
"requestHeader": {
"inputAssetReferences": [assetId]
}
"requestBody": {
}
}'
        

Deleting assets

Assets should be deleted after you are done processing them, however they will be automatically garbage collected.

            

            
curl -X DELETE https://api.nvcf.nvidia.com/v2/nvcf/assets/{assetId} -H "Authorization: Bearer <JWT Token>"
        

The cuOpt service employs long polling <https://www.pubnub.com/blog/http-long-polling/> for invocation and result retrieval.

When you make an invocation request, the system will hold your request open for a period of time before returning with either:

On receipt of a polling response your client should immediately poll the cuOpt service to retrieve your result. Polling once a second is recommended.

A polling scenario would have the following response for an invocation

            

            
{
 "reqId": "ef4c1967-d543-467c-af7e-8c7899d75be8",
 "status": "pending-evaluation"
}
        

Polling for a response

            

            
curl --location 'https://api.nvcf.nvidia.com/v2/nvcf/exec/status/{request-id}' \
--header 'Authorization: Bearer <JWT>'
        

For best performance, it is expected that clients will not close connections until it is determined that no further communication with a server is necessary.

In some cases the result of an invocation may return a URL for downloading a large response. You will receive a reference in your results to download the payload. A GET request on the URL will return a zipfile containing the result. The URL’s TTL is 24 hours.

            

            
{
 "reqId":"abc123",
 "status":"fulfilled",
 "responseReference": "<pre-signed download url for large result>",
 "percentComplete":100,
 "errorCode":0
}