Function Management#

Function Versioning#

Each time you create a new function a function ID will be created along with a function version ID. You can create additional versions of this function by specifying other models/containers/helm charts to use and additional configurations etc. Here is a sample API call:

 1curl -X 'POST' \
 2"https://api.nvcf.nvidia.com/v2/nvcf/functions/$FUNCTION_ID/versions" \
 3  -H 'accept: application/json' \
 4  -H "Authorization: Bearer $API_KEY" \
 5  -d '{
 6  "name": "echo_function",
 7  "inferenceUrl": "/echo",
 8  "containerImage": "nvcr.io/'$ORG_NAME'/echo:latest",
 9  "apiBodyFormat": "CUSTOM"
10}'

Multiple function versions allow for different deployment configurations for each version while still being accessible through a single function endpoint. Multiple function versions can also be used to support A/B testing.

Warning

Function versioning should only be used if the APIs between the various versions are compatible. Different APIs should be created as new functions.

Function Versioning Best Practices#

Leverage function versioning when making transparent production updates to a function. See also Container Versioning.

Prerequisites#

First, ensure that your client application is invoking your function by function ID, not function version ID. This allows the spread of requests to be even across all deployed versions of your function.

  • Invocation API endpoint should be https://api.nvcf.nvidia.com/v2/nvcf/pexec/functions/{functionID}

Function Update Flow (A/B Testing)#

  1. To initiate an update, create a new version of your function, through API, UI or CLI.

  2. Deploy the new version of your function. Ideally, replicate the min and max instance counts to be the same as the existing version of your function. This ensures it will successfully handle the same amount of traffic.

Note

Invocations to this function ID will now be routing to both the new and old versions of the function after this deploy based on function instance availability. If you do not want this to occur, for example in order to perform some testing first, deploy a separate development function.

  1. Verify that the endpoint deployed successfully by invoking it a few times. Do any additional verification such as load testing or comparing your third party monitoring and analytics across both versions.

  2. Once satisfied, to complete your update, undeploy the old version of the function. All invocations will now successfully route to the new version of the function.

On this page