APEX uses a framework called NxParameterized for storing asset and actor data. NxParameterized objects provide reflection on their parameters making it effective for serialization and auto-generation of user interfaces for tools.
NxParameterized stores data in C++ classes which are auto-generated from their descriptions written in a special DSL. Description files contain information on internal layout and various high-level metadata of corresponding classes and their members. All generated classes implement a special interface for run time reflection and modification of data.
All NxParameterized classes are declared in the NxParameterized namespace. Necessary include files are in NxParameterized/public.
Most of methods return status code of type NxParameterized::ErrorType (or Serializer::ErrorType). Success is denoted with NxParameterized::ERROR_NONE (or NxParameterized::Serializer::ERROR_NONE).
The NxParameterized::Interface class is a reflection interface provided by the NxParameterized class. An instance of Interface represents an object of some NxParameterized class. Interface allows the application to access the object’s metadata (class name and object name, version and checksum, number and types of parameters, etc.) and to read/modify its parameters.
The NxParameterized::Definition class provides information about an individual parameter of a class:
- name
- type
- array information
- struct information (including access to Definition of root, parent and child parameters)
- enum information
- reference information
- hints
(see Doxygen documentation for detailed description). This information is mostly used by various authoring tools.
The NxParameterized::Handle class provides access to an individual parameter within an NxParameterized object. E.g., to access the third element in an array parameter you can do this:
NxParameterized::Handle rootHandle(iface);
iface->getParameterHandle("myArray", rootHandle); // Set the handle to point to root struct
NxParameterized::Handle arrayHandle(iface);
rootHandle.getChildHandle(0, arrayHandle); // Set the handle to point to array field in root struct
NxParameterized::Handle elemHandle(iface);
arrayHandle.getChildHandle(3, elemHandle); // Set the handle to point to the third element in array
elemHandle.getParamF32(val);
or this (much faster):
iface->getParameterHandle("myArray", handle); // Set the handle to point to "myArray" element in root struct
handle.set(3); // Set the handle to point to index 3 in "myArray"
handle.getParamF32(val);
You may access the NxParameterized::Definition object associated with a parameter using Handle::parameterDefinition method:
const NxParameterized::Definition* paramDef = handle.parameterDefinition();
You may convert parameter values to and from strings using Handle::valueToStr and Handle::strToValue e.g.:
char buf[128];
const char* p;
// Result is returned in p which may or may not point to buf
// (e.g. in case of zero valueToStr may simply return "0")
handle.valueToStr(buf, sizeof(buf), p); // p may point to buf or to some static string literal
Method Interface::areParamsOK checks whether object parameters satisfy its constraints (min/max bounds, etc.).
The NxParameterized::Hint class provides information about parameter hints: its name, type, and value, which may be string, 64-bit integer or 64-bit floating point value (see Hints).
The NxParameterized::Serializer class is a base class for existing serializers. It provides methods for serialization/deserialization of array of NxParameterized objects (see Serialization).
The NxParameterized::Traits is an interface class for many of the functions provided by an application to the NxParameterized framework. It includes methods for memory allocation, registering NxParameterized class factories and legacy class conversions, NxParameterized object creation and various application callbacks. You may parameterize different aspects of NxParameterized behavior by overloading methods in Traits class:
- memory handling during inplace deserialization
- memory allocation
- etc.
APEX provides its own Traits implementation accessible through NxApexSDK::getParameterizedTraits method. To override existing methods in APEX Traits you will have to write your own implementation of Traits. To simplify this work NxParameterized provides a special implementation of Traits which delegates all calls to another Traits object (specified during construction). You can derive your class from this implementation and override only the methods you need (other methods will be handled by APEX Traits). Here is an example:
// Override handling of warnings
class MyCustomTraits: NxParameterized::WrappedTraits
{
public:
MyCustomTraits(NxParameterized::Traits* traits): NxParameterized::WrappedTraits(traits) {}
void warn(const char* msg)
{
std::cerr << "Warning from NxParameterized: " << msg << std::endl;
getWrappedTraits()->warn(msg); // Also warn through core traits
}
// All other methods are delegated to APEX Traits
};
// myTraits must persist until all objects using it are released
MyCustomTraits myTraits(sdk->getParameterizedTraits());
Note that in the current version of the APEX SDK the application is only allowed to override the following methods:
- onAllInplaceObjectsDestroyed
- onInplaceObjectDestroyed
- warn
See Inplace deserialization for more examples.
The current version of NxParameterized is capable of representing nested structures of basic C types (PxU8, PxF32, etc.), references, static/dynamic arrays and various combinations thereof. Future versions may add other C++ constructs (unions, inheritance, etc.).
All NxParameterized classes are composed from these basic types:
enum DataType
{
TYPE_UNDEFINED = 0,
TYPE_ARRAY,
TYPE_STRUCT,
TYPE_BOOL,
TYPE_STRING,
TYPE_ENUM,
TYPE_REF,
TYPE_I8,
TYPE_I16,
TYPE_I32,
TYPE_I64,
TYPE_U8,
TYPE_U16,
TYPE_U32,
TYPE_U64,
TYPE_F32,
TYPE_F64,
TYPE_VEC2,
TYPE_VEC3,
TYPE_VEC4,
TYPE_QUAT,
TYPE_MAT33,
TYPE_MAT34,
TYPE_BOUNDS3,
TYPE_MAT44,
TYPE_POINTER,
TYPE_TRANSFORM
};
DataType values may be converted to and from strings using NxParameterized::strToType and NxParameterized::typeToStr.
Most parameters directly correspond to plain C++ types e.g. PxU8, PxVec3, etc. Here are specific details for some of the types.
There are helper methods provided for accessing individual parameters in NxParamUtils.h e.g.:
NxParameterized::getParamU32( NxParameterized::Interface& obj, const char* parameterName, physx::PxU32& value);
NxParameterized::setParamU32( NxParameterized::Interface& obj, const char* parameterName, physx::PxU32 value);
You can also use methods of NxParameterized::Handle which are convenient for traversing objects recursively e.g.:
NxParameterized::Handle::getParamU32( physx::PxU32& value );
NxParameterized::Handle::setParamU32( physx::PxU32 value );
Struct is a plain C structure, i.e., a list of parameters. It may contain other structs as well. Parameters within a struct must be accessed individually, as there are no get/set operations for struct as a whole:
// myStruct has TYPE_STRUCT
NxParameterized::setParamU32(iface, "myStruct.intField", 12U);
You may access fields of struct using handles:
NxParameterized::Handle handle(iface);
iface->getParameterHandle("myStruct", handle);
handle.set(1); // Second field of struct (enumeration starts from 0)
handle.setParamU32(12U);
which is usually much faster than constructing field names by hand.
An array size may be dynamic or static (determined by NxParameterized::Definition::arraySizeIsFixed method). Static arrays correspond to plain C arrays, dynamic arrays are represented with special C structures which track pointers to dynamic buffers and current number of elements and allow dynamic insertion / removal of elements. Elements of array may be accessed either by full indexed name:
// myArray is TYPE_ARRAY
NxParameterized::setParamU32(iface, "myArray[3]", 12U);
or by using faster NxParameterized::Handle::set method:
NxParameterized::Handle handle(iface);
iface->getParameterHandle("myArray", handle);
handle.set(3);
handle.setParamU32(12U);
A dynamically sized array can be resized. During resize existing entries are copied to a new memory location and newly allocated entries are filled with zeros or (for dynamic arrays of enums) with the enum value which has a zero index. Example:
// myArray is TYPE_ARRAY, let's make it grow
NxParameterized::Handle handle(iface, "myArray");
physx::PxI32 arraySize;
handle.getArraySize(arraySize);
handle.resizeArray(arraySize + 1); // Will return error if array is static
// myArray is TYPE_ARRAY, let's shrink it (remove the element at index 3, assume there's like 10 elements in the array)
NxParameterized::Handle handle(iface, "myArray");
physx::PxI32 arraySize;
physx::PxU32 indexToRemove = 3U;
handle.getArraySize(arraySize);
handle.swapArrayElements(indexToRemove, arraySize-1);
handle.resizeArray(arraySize - 1);
All array elements may be accessed one-by-one in a for loop:
physx::PxI32 size;
arrayHandle.getArraySize(size);
for(physx::PxI32 i = 0; i < size; ++i)
{
arrayHandle.set(i); // Handle to i-th element of array
physx::PxU32 val;
arrayHandle.getParamU32(val);
arrayHandle.popIndex(); // Handle of array again
}
or using fast array accessors:
NxParameterized::Handle::getParamU32Array( physx::PxU32* array, physx::PxI32 n, physx::PxI32 offset = 0 );
NxParameterized::Handle::setParamU32Array( const physx::PxU32* array, physx::PxI32 n, physx::PxI32 offset = 0 );
e.g.:
std::vector vals;
NxParameterized::Handle handle(iface, "myArray");
handle.resizeArray(vals.size());
handle.setParamU32Array(&vals[0], vals.size());
Note that dynamic arrays are not automatically resized so you need to call resize before appending new values.
Pointers represent user data pointers which are handled internally by APEX. NxParameterized does not provide get/set methods to access data in pointer fields.
Strings are plain C zero-terminated strings. A string owns memory allocated for its value so setting a string involves freeing the old value, allocating memory for the new one and copying the data.
You may access the string value with set/getParamString:
const char* s = "New string value";
// Set new string value
iface->setParamString(s);
// Check that it was properly set
const char* val = 0;
iface->getParamString(val);
PX_ASSERT(0 == strcmp(s, val));
Enums have a set of predefined string values. ParameterDefinition::enumVal returns an enum value from its number and ParameterDefinition::numEnumVals returns the total number of enum values.
Memory occupied by an enum value is not owned by the enum instances so changing the value of an enum (using setParamEnum) results in a simple pointer copy and does not involve any memory allocations.
You may access an enum value with set/getParamEnum or with set/getParamString:
// Get handle of enum value
NxParameterized::Handle enumHandle(iface);
iface->getParameterHandle("myStruct.myArray[0].myEnumValue", enumHandle);
// Set new enum value with exact string
const char* enumVal = "ENUM_VALUE_0";
enumHandle.setParamEnum(enumVal);
// Check that it was properly set
const char* val = 0;
enumHandle.getParamEnum(val);
PX_ASSERT(enumVal == val); // It's safe to compare pointers
// Try to set invalid enum value (will return error)
enumHandle.setParamEnum("some invalid string");
iface->setParamEnum();
// Set enum value using its index
const NxParameterized::Definition* enumDef = enumHandle->parameterDefinition();
enumHandle.setParamEnum(enumDef->enumVal(0));
// Get enum value index
PX_ASSERT(0 == enumDef->enumValIndex(enumVal));
// Get number of possible enum values
physx::PxI32 numEnumVals = enumDef->numEnumVals();
References are pointers to objects of other NxParameterized classes, i.e., have type NxParameterized::Interface*.
In addition to ordinary getParamRef and setParamRef which are similar to other types you can also initialize a reference field with an object of a given class using NxParameterized::Interface::initParamRef or NxParameterized::Handle::initParamRef:
// Will return error if className is invalid
refHandle.initParamRef("className");
You can access a list of valid class names using methods of NxParameterized::Definition:
const Definition* refDef = refHandle.parameterDefinition();
// Number of possible classes
physx::PxI32 numClassNames = refDef->numRefVariants();
// Get class name by index
const char* className = refDef->refVariantVal(0);
The object will destroy all of its references on destruction but it will not destroy an old reference when setParamRef is called so be sure to destroy this reference manually to avoid memory leaks:
NxParameterized::Interface* oldRef = 0;
refHandle.getParamRef(oldRef);
refHandle.setParamRef(newRef); // oldRef is overwritten by newRef
oldRef->destroy();
References can be either “included” or “named”, but either way they are always of the type NxParameterized::Interface.
An included reference is used for including an NxParameterized class within another. This is sometimes done for the purpose of nesting (for convenience), like when the APEX Render Mesh is an included reference within a Destructible or Clothing asset. The included reference may also be used when a union of some types is desired e.g. the geometry parameter in the APEX shaped emitter is actually an included reference that can represent a cylinder, sphere, sphere shell, box, or explicit geometry. Example of working with included references:
// Setup handle
NxParameterized::Handle handle(iface);
iface->getParameterHandle("myArray[0].myStruct.myRef", handle);
// Get reference
NxParameterized::Interface* ref = 0;
handle.getParamRef(ref);
// Work with reference (set a field)
NxParameterized::Handle refHandle(ref);
ref->getParameterHandle("myU32", refHandle);
refHandle.setParamU32(12U);
A named reference is used for referencing other APEX assets by name within an asset. For instance, in the APEX shaped emitter, IOS and IOFX asset names must be provided. The named reference is still of type NxParameterized::Interface, but it only contains a name and class name. The class name may be obvious, but in the case of the IOS, where either an NxFluidIOS, or a BasicIOS is possible, it is useful to have it. E.g. parameter name: “iofxAsset”, class name: “IOFX”, name: “greenLeafIofx” or parameter name: “iosAsset”, class name: “NxFluidIOS”, name: “flutteringIOS”. The class names in named references always matches one of the Named Resource Provider namespaces (“IOFX”, “ApexRenderMesh”, “NxFluidIosAsset”, “NxBasicIosAsset”).
The NxParameterized::Interface::isIncludedRef() method should be used to determine if a reference parameter is included or named.
In addition to getting and setting references NxParameterized provides NxParameterized::Interface::initParamRef method for their initialization. This methods creates an object of the specified class and initializes it with default values.
Each parameter of an NxParameterized class may have a number of hints, i.e., name-value pairs which may be queried at run time. Hints usually provides authoring, serialization and other useful info about a parameter. A hint must have a name and may be represented by a 64-bit unsigned integer, 64-bit floating point, or a string.
Here are some of the standard APEX hints:
- INCLUDED - used only for parameters of type TYPE_REF; 0 means, the pointer refers to named reference, 1 - actual NxParameterized object
- defaultValue - default value for parameter
- shortDescription, longDescription, programmerDescription - various documentation hints used by authoring tools and for documentation generation, not included in the APEX “release” builds
- min, max, powerOf, multipleOf - parameter constraints.
Most parameters contain at least “shortDescription” and “longDescription” hints.
NxParameterized provides generic serialization support for generated classes. There are two primary file formats supported for all APEX assets: APX (APEX XML) and APB (APEX binary format). XML is used during the authoring stage for debugging, manual/automatic editing, conversion, etc. Binary format is usually much more efficient than XML in terms of disk space and parsing time. Note that data may also be serialized to a memory buffer which allows postprocessing data (encryption, etc.) and embedding it in application-specific file formats. If you wish to use inplace deserialization on embedded APEX assets, make sure that they are properly aligned in your file. Alignment of the asset should be the maximum of alignments of NxParameterized objects that are stored in the asset (alignment of the NxParameterized object may be obtained by calling NxParameterized::Interface::alignment).
To serialize an NxParameterized object you will need to create an NxParameterized::Serializer object using method NxApexSDK::createSerializer and call method Serializer::serialize (you can serialize more than one object in a stream). Prior to this you may want to change target platform using Serializer::setTargetPlatform method (currently supported platforms include Win32, Win64, Xbox 360, Playstation 3, and Android). If target platform was not set explicitly the current platform is used.
Layout of a binary file (endianess, alignments, etc.) depends on the target platform that was specified during file creation allowing extremely fast inplace deserialization on native platform (deserialized objects are fread() into memory and in-place constructed minimizing copy and memory allocation overhead).
To deserialize an NxParameterized object you will usually call the Serializer::deserialize method which will automatically create all objects and store results in a Serializer::DeserializedData object. Note that the binary file may be deserialized on any platform (independently of whether current platform is their native platform or not). Example of deserialization:
// Create serializer
NxParameterized::Serializer* ser = sdk->createSerializer(Serializer::NST_BINARY);
//Create file stream (use NxApexSDK::createMemoryWriteStream to serialize to memory buffer)
physx::PxFileBuf* stream = sdk->createStream("someAsset.apb", physx::PxFileBuf::OPEN_READ_ONLY);
// Do the deserialization
NxParameterized::Serializer::DeserializedData res;
ser->deserialize(*stream, res);
// Extract results
PX_ASSERT(2 == res.size());
NxParameterized::Interface* first = res[0], * second = res[1];
// Release
ser->release();
stream->release();
Note that deserializing directly from a file stream may currently cause performance penalties on some platforms so you may want to pass data through a memory buffer stream (see NxApexSDK::createMemoryReadStream).
You may find the type of serializer suitable for a given asset file using NxApexSDK::getSerializeType method:
NxParameterized::Serializer* ser = sdk->createSerializer(sdk->getSerializeType(*stream));
APEX comes with a program called ParamTool which allows conversions of various asset formats to each other. You can use it to convert between xml and binary formats and to update older assets to the latest version. See “ParamTool.exe -h” for more details.
It may occasionally be useful to hand-edit xml assets in your favourite XML editor. Note that you should respect the asset version stored in the file e.g. by not adding fields that were introduced only in later versions. Otherwise the modified asset will probably fail to load.
If a binary file matches your current platform you may read your data to a memory buffer and perform fast inplace deserialization using the method Serializer::deserializeInplace (if binary files were prepared for different platform method will return an error). There is no difference in using inplace-deserialized objects. Here is an example of inplace deserialization:
NxParameterized::Traits* traits = sdk->getParameterizedTraits();
const void* buf = sdk->getMemoryWriteBuffer(*stream, buflen);
PX_ASSERT(buf);
NxParameterized::Serializer::DeserializedData res;
// Will return Serializer::ERROR_INVALID_PLATFORM if platform is not native
ser->deserializeInplace(buf, buflen, res);
To detect whether a binary file matches your current platform you can use NxApexSDK::getSerializePlatform:
NxParameterized::SerializePlatform curPlatform;
sdk->getCurrentPlatform(curPlatform);
NxParameterized::SerializePlatform filePlatform;
sdk->getSerializePlatform(*stream, platform);
if(platform == curPlatform)
...
During deserialization if APEX finds that some inplace object has an older version it is converted: the new version is created on the heap and the registered updater (which copies legacy data to this new object) is called. Note that this introduces a certain performance penalty (see Versioning). Note that in order for APEX to convert old asset versions, you have the appropriate legacy module loaded (see Versioning).
By default after inplace deserialization objects take ownership of the memory buffer and automatically free it when the last NxParameterized object in this buffer is destroyed. You may create a buffer for the default implementation by using NxApexSdk::getMemoryWriteBuffer method.
Default behavior may be altered by overriding inplace callbacks in your own implementation of Traits class and using it to construct your Serializer.
For example if your application needs manual memory control you can use the following custom Traits class:
// This implementation ignores destruction of inplace objects; memory will be freed by application
class UserTraits: public NxParameterized::WrappedTraits
{
public:
UserTraits(NxParameterized::Traits* traits): NxParameterized::WrappedTraits(traits) {}
// Ignore destruction of inplace object
void onInplaceObjectDestroyed(void*, NxParameterized::Interface*) {}
// Ignore destruction of all inplace objects in memory buffer
void onAllInplaceObjectsDestroyed(void*) {}
};
// myTraits must persist until all objects using it are released
void* buf = ::malloc(sizeof(UserTraits));
UserTraits* myTraits = new(buf) UserTraits(sdk->getParameterizedTraits());
You can use it to create a custom serializer:
NxParameterized::Serializer* mySer = sdk->createSerializer(NxParameterized::Serializer::NST_BINARY, myTraits);
and deserialize your objects:
physx::PxU32 dlen = stream->getFileLength();
const void* data = ::malloc(dlen);
PX_ASSERT(data);
physx::PxU32 nread = stream->read(data, dlen);
PX_ASSERT(nread == dlen);
NxParameterized::Serializer::DeserializedData res;
mySer->deserializeInplace(data, dlen, res);
When the application decides to release a buffer it simply calls free():
::free(data); // App must make sure that all inplace objects are released
APEX provides optional support for legacy NxParameterized classes. E.g. to use legacy clothing assets you have to load a special module:
clothingLegacyModule = sdk->createModule("APEX_Legacy");
After this APEX will automatically do an object update whenever it encounters a legacy clothing asset during deserialization. It is guaranteed that update of the included references is done prior to the parent update.
It is also possible to make manual construction and update of legacy object using methods of NxParameterized::Traits:
NxParameterized::Interface* createNxParameterized(const char* className, physx::PxU32 classVer);
bool updateLegacyNxParameterized(NxParameterized::Interface& legacyObj, NxParameterized::Interface& iface);
Note that updating legacy objects will have a significant performance penalty especially in case of inplace deserialization so you should consider updating your assets offline to improve speed. You may use ParamTool for that (see Serialization).