30. Device-Generated Commands

This chapter discusses the generation of command buffer content on the device. These principle steps are to be taken to generate commands on the device:

  • Make resource bindings accessible for the device via registering in a VkObjectTableNVX.

  • Define via VkIndirectCommandsLayoutNVX the sequence of commands which should be generated.

  • Fill one or more VkBuffer with the appropriate content that gets interpreted by VkIndirectCommandsLayoutNVX.

  • Reserve command space via vkCmdReserveSpaceForCommandsNVX in a secondary VkCommandBuffer where the generated commands should be recorded.

  • Generate the actual commands via vkCmdProcessCommandsNVX passing all required data.

Execution of such generated commands can either be triggered directly with the generation process, or by executing the secondary VkCommandBuffer that was chosen as optional target. The latter allows re-using generated commands as well. Similar to VkDescriptorSet, special care should be taken for the lifetime of resources referenced in VkObjectTableNVX, which may be accessed at either generation or execution time.

vkCmdProcessCommandsNVX executes in a separate logical pipeline from either graphics or compute. When generating commands into a secondary command buffer, the command generation must be explicitly synchronized against the secondary command buffer’s execution. When not using a secondary command buffer, the command generation is automatically synchronized against the command execution.

30.1. Features and Limitations

To query the support of related features and limitations, call:

void vkGetPhysicalDeviceGeneratedCommandsPropertiesNVX(
    VkPhysicalDevice                            physicalDevice,
    VkDeviceGeneratedCommandsFeaturesNVX*       pFeatures,
    VkDeviceGeneratedCommandsLimitsNVX*         pLimits);

  • physicalDevice is the handle to the physical device whose properties will be queried.

  • pFeatures points to an instance of the VkDeviceGeneratedCommandsFeaturesNVX structure, that will be filled with returned information.

  • pLimits points to an instance of the VkDeviceGeneratedCommandsLimitsNVX structure, that will be filled with returned information.

Valid Usage (Implicit)

  • physicalDevice must be a valid VkPhysicalDevice handle

  • pFeatures must be a valid pointer to a VkDeviceGeneratedCommandsFeaturesNVX structure

  • pLimits must be a valid pointer to a VkDeviceGeneratedCommandsLimitsNVX structure

The VkDeviceGeneratedCommandsFeaturesNVX structure is defined as:

typedef struct VkDeviceGeneratedCommandsFeaturesNVX {
    VkStructureType    sType;
    const void*        pNext;
    VkBool32           computeBindingPointSupport;
} VkDeviceGeneratedCommandsFeaturesNVX;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to an extension-specific structure.

  • computeBindingPointSupport specifies whether the VkObjectTableNVX supports entries with VK_OBJECT_ENTRY_USAGE_GRAPHICS_BIT_NVX bit set and VkIndirectCommandsLayoutNVX supports VK_PIPELINE_BIND_POINT_COMPUTE.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_DEVICE_GENERATED_COMMANDS_FEATURES_NVX

  • pNext must be NULL

The VkDeviceGeneratedCommandsLimitsNVX structure is defined as:

typedef struct VkDeviceGeneratedCommandsLimitsNVX {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           maxIndirectCommandsLayoutTokenCount;
    uint32_t           maxObjectEntryCounts;
    uint32_t           minSequenceCountBufferOffsetAlignment;
    uint32_t           minSequenceIndexBufferOffsetAlignment;
    uint32_t           minCommandsTokenBufferOffsetAlignment;
} VkDeviceGeneratedCommandsLimitsNVX;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to an extension-specific structure.

  • maxIndirectCommandsLayoutTokenCount the maximum number of tokens in VkIndirectCommandsLayoutNVX.

  • maxObjectEntryCounts the maximum number of entries per resource type in VkObjectTableNVX.

  • minSequenceCountBufferOffsetAlignment the minimum alignment for memory addresses optionally used in vkCmdProcessCommandsNVX.

  • minSequenceIndexBufferOffsetAlignment the minimum alignment for memory addresses optionally used in vkCmdProcessCommandsNVX.

  • minCommandsTokenBufferOffsetAlignment the minimum alignment for memory addresses optionally used in vkCmdProcessCommandsNVX.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_DEVICE_GENERATED_COMMANDS_LIMITS_NVX

  • pNext must be NULL

30.2. Binding Object Table

The device-side bindings are registered inside a table:

VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkObjectTableNVX)

This is required as the CPU-side object pointers, for example when binding a VkPipeline or VkDescriptorSet, cannot be used by the device. The combination of VkObjectTableNVX and uint32_t table indices stored inside a VkBuffer serve that purpose during device command generation.

At creation time the table is defined with a fixed amount of registration slots for the individual resource types. A detailed resource binding can then later be registered via vkRegisterObjectsNVX at any uint32_t index below the allocated maximum.

30.2.1. Table Creation

To create object tables, call:

VkResult vkCreateObjectTableNVX(
    VkDevice                                    device,
    const VkObjectTableCreateInfoNVX*           pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkObjectTableNVX*                           pObjectTable);

  • device is the logical device that creates the object table.

  • pCreateInfo is a pointer to an instance of the VkObjectTableCreateInfoNVX structure containing parameters affecting creation of the table.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pObjectTable points to a VkObjectTableNVX handle in which the resulting object table is returned.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • pCreateInfo must be a valid pointer to a valid VkObjectTableCreateInfoNVX structure

  • If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • pObjectTable must be a valid pointer to a VkObjectTableNVX handle

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

The VkObjectTableCreateInfoNVX structure is defined as:

typedef struct VkObjectTableCreateInfoNVX {
    VkStructureType                      sType;
    const void*                          pNext;
    uint32_t                             objectCount;
    const VkObjectEntryTypeNVX*          pObjectEntryTypes;
    const uint32_t*                      pObjectEntryCounts;
    const VkObjectEntryUsageFlagsNVX*    pObjectEntryUsageFlags;
    uint32_t                             maxUniformBuffersPerDescriptor;
    uint32_t                             maxStorageBuffersPerDescriptor;
    uint32_t                             maxStorageImagesPerDescriptor;
    uint32_t                             maxSampledImagesPerDescriptor;
    uint32_t                             maxPipelineLayouts;
} VkObjectTableCreateInfoNVX;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to an extension-specific structure.

  • objectCount is the number of entry configurations that the object table supports.

  • pObjectEntryTypes is an array of VkObjectEntryTypeNVX values providing the entry type of a given configuration.

  • pObjectEntryCounts is an array of counts of how many objects can be registered in the table.

  • pObjectEntryUsageFlags is an array of bitmasks of VkObjectEntryUsageFlagBitsNVX specifying the binding usage of the entry.

  • maxUniformBuffersPerDescriptor is the maximum number of VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER or VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER_DYNAMIC used by any single registered VkDescriptorSet in this table.

  • maxStorageBuffersPerDescriptor is the maximum number of VK_DESCRIPTOR_TYPE_STORAGE_BUFFER or VK_DESCRIPTOR_TYPE_STORAGE_BUFFER_DYNAMIC used by any single registered VkDescriptorSet in this table.

  • maxStorageImagesPerDescriptor is the maximum number of VK_DESCRIPTOR_TYPE_STORAGE_IMAGE or VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER used by any single registered VkDescriptorSet in this table.

  • maxSampledImagesPerDescriptor is the maximum number of VK_DESCRIPTOR_TYPE_SAMPLER, VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER or VK_DESCRIPTOR_TYPE_INPUT_ATTACHMENT used by any single registered VkDescriptorSet in this table.

  • maxPipelineLayouts is the maximum number of unique VkPipelineLayout used by any registered VkDescriptorSet or VkPipeline in this table.

Valid Usage

  • If the VkDeviceGeneratedCommandsFeaturesNVX::computeBindingPointSupport feature is not enabled, pObjectEntryUsageFlags must not contain VK_OBJECT_ENTRY_USAGE_COMPUTE_BIT_NVX

  • Any value within pObjectEntryCounts must not exceed VkDeviceGeneratedCommandsLimitsNVX::maxObjectEntryCounts

  • maxUniformBuffersPerDescriptor must be within the limits supported by the device.

  • maxStorageBuffersPerDescriptor must be within the limits supported by the device.

  • maxStorageImagesPerDescriptor must be within the limits supported by the device.

  • maxSampledImagesPerDescriptor must be within the limits supported by the device.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_OBJECT_TABLE_CREATE_INFO_NVX

  • pNext must be NULL

  • pObjectEntryTypes must be a valid pointer to an array of objectCount valid VkObjectEntryTypeNVX values

  • pObjectEntryCounts must be a valid pointer to an array of objectCount uint32_t values

  • pObjectEntryUsageFlags must be a valid pointer to an array of objectCount valid combinations of VkObjectEntryUsageFlagBitsNVX values

  • Each element of pObjectEntryUsageFlags must not be 0

  • objectCount must be greater than 0

Possible values of elements of the VkObjectTableCreateInfoNVX::pObjectEntryTypes array, specifying the entry type of a configuration, are:

typedef enum VkObjectEntryTypeNVX {
    VK_OBJECT_ENTRY_TYPE_DESCRIPTOR_SET_NVX = 0,
    VK_OBJECT_ENTRY_TYPE_PIPELINE_NVX = 1,
    VK_OBJECT_ENTRY_TYPE_INDEX_BUFFER_NVX = 2,
    VK_OBJECT_ENTRY_TYPE_VERTEX_BUFFER_NVX = 3,
    VK_OBJECT_ENTRY_TYPE_PUSH_CONSTANT_NVX = 4,
    VK_OBJECT_ENTRY_TYPE_MAX_ENUM_NVX = 0x7FFFFFFF
} VkObjectEntryTypeNVX;

  • VK_OBJECT_ENTRY_TYPE_DESCRIPTOR_SET_NVX specifies a VkDescriptorSet resource entry that is registered via VkObjectTableDescriptorSetEntryNVX.

  • VK_OBJECT_ENTRY_TYPE_PIPELINE_NVX specifies a VkPipeline resource entry that is registered via VkObjectTablePipelineEntryNVX.

  • VK_OBJECT_ENTRY_TYPE_INDEX_BUFFER_NVX specifies a VkBuffer resource entry that is registered via VkObjectTableIndexBufferEntryNVX.

  • VK_OBJECT_ENTRY_TYPE_VERTEX_BUFFER_NVX specifies a VkBuffer resource entry that is registered via VkObjectTableVertexBufferEntryNVX.

  • VK_OBJECT_ENTRY_TYPE_PUSH_CONSTANT_NVX specifies the resource entry is registered via VkObjectTablePushConstantEntryNVX.

Bits which can be set in elements of the VkObjectTableCreateInfoNVX::pObjectEntryUsageFlags array, specifying binding usage of an entry, are:

typedef enum VkObjectEntryUsageFlagBitsNVX {
    VK_OBJECT_ENTRY_USAGE_GRAPHICS_BIT_NVX = 0x00000001,
    VK_OBJECT_ENTRY_USAGE_COMPUTE_BIT_NVX = 0x00000002,
    VK_OBJECT_ENTRY_USAGE_FLAG_BITS_MAX_ENUM_NVX = 0x7FFFFFFF
} VkObjectEntryUsageFlagBitsNVX;

  • VK_OBJECT_ENTRY_USAGE_GRAPHICS_BIT_NVX specifies that the resource is bound to VK_PIPELINE_BIND_POINT_GRAPHICS

  • VK_OBJECT_ENTRY_USAGE_COMPUTE_BIT_NVX specifies that the resource is bound to VK_PIPELINE_BIND_POINT_COMPUTE

typedef VkFlags VkObjectEntryUsageFlagsNVX;

VkObjectEntryUsageFlagsNVX is a bitmask type for setting a mask of zero or more VkObjectEntryUsageFlagBitsNVX.

To destroy an object table, call:

void vkDestroyObjectTableNVX(
    VkDevice                                    device,
    VkObjectTableNVX                            objectTable,
    const VkAllocationCallbacks*                pAllocator);

  • device is the logical device that destroys the table.

  • objectTable is the table to destroy.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage

  • All submitted commands that refer to objectTable must have completed execution.

  • If VkAllocationCallbacks were provided when objectTable was created, a compatible set of callbacks must be provided here.

  • If no VkAllocationCallbacks were provided when objectTable was created, pAllocator must be NULL.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • objectTable must be a valid VkObjectTableNVX handle

  • If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • objectTable must have been created, allocated, or retrieved from device

Host Synchronization

  • Host access to objectTable must be externally synchronized

30.2.2. Registering Objects

Resource bindings of Vulkan objects are registered at an arbitrary uint32_t index within an object table. As long as the object table references such objects, they must not be deleted.

VkResult vkRegisterObjectsNVX(
    VkDevice                                    device,
    VkObjectTableNVX                            objectTable,
    uint32_t                                    objectCount,
    const VkObjectTableEntryNVX* const*         ppObjectTableEntries,
    const uint32_t*                             pObjectIndices);

  • device is the logical device that creates the object table.

  • objectTable is the table for which the resources are registered.

  • objectCount is the number of resources to register.

  • ppObjectTableEntries provides an array for detailed binding informations, each array element is a pointer to a struct of type VkObjectTablePipelineEntryNVX, VkObjectTableDescriptorSetEntryNVX, VkObjectTableVertexBufferEntryNVX, VkObjectTableIndexBufferEntryNVX or VkObjectTablePushConstantEntryNVX (see below for details).

  • pObjectIndices are the indices at which each resource is registered.

Valid Usage

  • The contents of pObjectTableEntry must yield plausible bindings supported by the device.

  • At any pObjectIndices there must not be a registered resource already.

  • Any value inside pObjectIndices must be below the appropriate VkObjectTableCreateInfoNVX::pObjectEntryCounts limits provided at objectTable creation time.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • objectTable must be a valid VkObjectTableNVX handle

  • ppObjectTableEntries must be a valid pointer to an array of objectCount valid VkObjectTableEntryNVX structures

  • pObjectIndices must be a valid pointer to an array of objectCount uint32_t values

  • objectCount must be greater than 0

  • objectTable must have been created, allocated, or retrieved from device

Host Synchronization

  • Host access to objectTable must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

Common to all resource entries are:

typedef struct VkObjectTableEntryNVX {
    VkObjectEntryTypeNVX          type;
    VkObjectEntryUsageFlagsNVX    flags;
} VkObjectTableEntryNVX;

  • type defines the entry type

  • flags defines which VkPipelineBindPoint the resource can be used with. Some entry types allow only a single flag to be set.

Valid Usage

  • If the VkDeviceGeneratedCommandsFeaturesNVX::computeBindingPointSupport feature is not enabled, flags must not contain VK_OBJECT_ENTRY_USAGE_COMPUTE_BIT_NVX

Valid Usage (Implicit)
typedef struct VkObjectTablePipelineEntryNVX {
    VkObjectEntryTypeNVX          type;
    VkObjectEntryUsageFlagsNVX    flags;
    VkPipeline                    pipeline;
} VkObjectTablePipelineEntryNVX;

  • pipeline specifies the VkPipeline that this resource entry references.

Valid Usage

  • type must be VK_OBJECT_ENTRY_TYPE_PIPELINE_NVX

Valid Usage (Implicit)
typedef struct VkObjectTableDescriptorSetEntryNVX {
    VkObjectEntryTypeNVX          type;
    VkObjectEntryUsageFlagsNVX    flags;
    VkPipelineLayout              pipelineLayout;
    VkDescriptorSet               descriptorSet;
} VkObjectTableDescriptorSetEntryNVX;

  • pipelineLayout specifies the VkPipelineLayout that the descriptorSet is used with.

  • descriptorSet specifies the VkDescriptorSet that can be bound with this entry.

Valid Usage

  • type must be VK_OBJECT_ENTRY_TYPE_DESCRIPTOR_SET_NVX

Valid Usage (Implicit)

  • type must be a valid VkObjectEntryTypeNVX value

  • flags must be a valid combination of VkObjectEntryUsageFlagBitsNVX values

  • flags must not be 0

  • pipelineLayout must be a valid VkPipelineLayout handle

  • descriptorSet must be a valid VkDescriptorSet handle

  • Both of descriptorSet, and pipelineLayout must have been created, allocated, or retrieved from the same VkDevice

typedef struct VkObjectTableVertexBufferEntryNVX {
    VkObjectEntryTypeNVX          type;
    VkObjectEntryUsageFlagsNVX    flags;
    VkBuffer                      buffer;
} VkObjectTableVertexBufferEntryNVX;

  • buffer specifies the VkBuffer that can be bound as vertex bufer

Valid Usage

  • type must be VK_OBJECT_ENTRY_TYPE_VERTEX_BUFFER_NVX

Valid Usage (Implicit)
typedef struct VkObjectTableIndexBufferEntryNVX {
    VkObjectEntryTypeNVX          type;
    VkObjectEntryUsageFlagsNVX    flags;
    VkBuffer                      buffer;
    VkIndexType                   indexType;
} VkObjectTableIndexBufferEntryNVX;

  • buffer specifies the VkBuffer that can be bound as index buffer

  • indexType specifies the VkIndexType used with this index buffer

Valid Usage

  • type must be VK_OBJECT_ENTRY_TYPE_INDEX_BUFFER_NVX

Valid Usage (Implicit)
typedef struct VkObjectTablePushConstantEntryNVX {
    VkObjectEntryTypeNVX          type;
    VkObjectEntryUsageFlagsNVX    flags;
    VkPipelineLayout              pipelineLayout;
    VkShaderStageFlags            stageFlags;
} VkObjectTablePushConstantEntryNVX;
Valid Usage

  • type must be VK_OBJECT_ENTRY_TYPE_PUSH_CONSTANT_NVX

Valid Usage (Implicit)

Use the following command to unregister resources from an object table:

VkResult vkUnregisterObjectsNVX(
    VkDevice                                    device,
    VkObjectTableNVX                            objectTable,
    uint32_t                                    objectCount,
    const VkObjectEntryTypeNVX*                 pObjectEntryTypes,
    const uint32_t*                             pObjectIndices);

  • device is the logical device that creates the object table.

  • objectTable is the table from which the resources are unregistered.

  • objectCount is the number of resources being removed from the object table.

  • pObjectEntryType provides an array of VkObjectEntryTypeNVX for the resources being removed.

  • pObjectIndices provides the array of object indices to be removed.

Valid Usage

  • At any pObjectIndices there must be a registered resource already.

  • The pObjectEntryTypes of the resource at pObjectIndices must match.

  • All operations on the device using the registered resource must have been completed.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • objectTable must be a valid VkObjectTableNVX handle

  • pObjectEntryTypes must be a valid pointer to an array of objectCount valid VkObjectEntryTypeNVX values

  • pObjectIndices must be a valid pointer to an array of objectCount uint32_t values

  • objectCount must be greater than 0

  • objectTable must have been created, allocated, or retrieved from device

Host Synchronization

  • Host access to objectTable must be externally synchronized

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

30.3. Indirect Commands Layout

The device-side command generation happens through an iterative processing of an atomic sequence comprised of command tokens, which are represented by:

VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkIndirectCommandsLayoutNVX)

30.3.1. Tokenized Command Processing

The processing is in principle illustrated below:

void cmdProcessSequence(cmd, objectTable, indirectCommandsLayout, pIndirectCommandsTokens, s)
{
  for (c = 0; c < indirectCommandsLayout.tokenCount; c++)
  {
    indirectCommandsLayout.pTokens[c].command (cmd, objectTable, pIndirectCommandsTokens[c], s);
  }
}

void cmdProcessAllSequences(cmd, objectTable, indirectCommandsLayout, pIndirectCommandsTokens, sequencesCount)
{
  for (s = 0; s < sequencesCount; s++)
  {
    cmdProcessSequence(cmd, objectTable, indirectCommandsLayout, pIndirectCommandsTokens, s);
  }
}

The processing of each sequence is considered stateless, therefore all state changes must occur prior work provoking commands within the sequence. A single sequence is either strictly targeting VK_PIPELINE_BIND_POINT_GRAPHICS or VK_PIPELINE_BIND_POINT_COMPUTE.

The primary input data for each token is provided through VkBuffer content at command generation time using vkCmdProcessCommandsNVX, however some functional arguments, for example binding sets, are specified at layout creation time. The input size is different for each token.

Possible values of those elements of the VkIndirectCommandsLayoutCreateInfoNVX::pTokens array which specify command tokens (other elements of the array specify command parameters) are:

typedef enum VkIndirectCommandsTokenTypeNVX {
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_PIPELINE_NVX = 0,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_DESCRIPTOR_SET_NVX = 1,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_INDEX_BUFFER_NVX = 2,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_VERTEX_BUFFER_NVX = 3,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_PUSH_CONSTANT_NVX = 4,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_INDEXED_NVX = 5,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_NVX = 6,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_DISPATCH_NVX = 7,
    VK_INDIRECT_COMMANDS_TOKEN_TYPE_MAX_ENUM_NVX = 0x7FFFFFFF
} VkIndirectCommandsTokenTypeNVX;
Table 40. Supported indirect command tokens
Token type Equivalent command

VK_INDIRECT_COMMANDS_TOKEN_TYPE_PIPELINE_NVX

vkCmdBindPipeline

VK_INDIRECT_COMMANDS_TOKEN_TYPE_DESCRIPTOR_SET_NVX

vkCmdBindDescriptorSets

VK_INDIRECT_COMMANDS_TOKEN_TYPE_INDEX_BUFFER_NVX

vkCmdBindIndexBuffer

VK_INDIRECT_COMMANDS_TOKEN_TYPE_VERTEX_BUFFER_NVX

vkCmdBindVertexBuffers

VK_INDIRECT_COMMANDS_TOKEN_TYPE_PUSH_CONSTANT_NVX

vkCmdPushConstants

VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_INDEXED_NVX

vkCmdDrawIndexedIndirect

VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_NVX

vkCmdDrawIndirect

VK_INDIRECT_COMMANDS_TOKEN_TYPE_DISPATCH_NVX

vkCmdDispatchIndirect

The VkIndirectCommandsLayoutTokenNVX structure specifies details to the function arguments that need to be known at layout creation time:

typedef struct VkIndirectCommandsLayoutTokenNVX {
    VkIndirectCommandsTokenTypeNVX    tokenType;
    uint32_t                          bindingUnit;
    uint32_t                          dynamicCount;
    uint32_t                          divisor;
} VkIndirectCommandsLayoutTokenNVX;

  • type specifies the token command type.

  • bindingUnit has a different meaning depending on the type, please refer pseudo code further down for details.

  • dynamicCount has a different meaning depending on the type, please refer pseudo code further down for details.

  • divisor defines the rate at which the input data buffers are accessed.

Valid Usage

  • bindingUnit must stay within device supported limits for the appropriate commands.

  • dynamicCount must stay within device supported limits for the appropriate commands.

  • divisor must be greater than 0 and a power of two.

Valid Usage (Implicit)

The VkIndirectCommandsTokenNVX structure specifies the input data for a token at processing time.

typedef struct VkIndirectCommandsTokenNVX {
    VkIndirectCommandsTokenTypeNVX    tokenType;
    VkBuffer                          buffer;
    VkDeviceSize                      offset;
} VkIndirectCommandsTokenNVX;

  • tokenType specifies the token command type.

  • buffer specifies the VkBuffer storing the functional arguments for each squence. These argumetns can be written by the device.

  • offset specified an offset into buffer where the arguments start.

Valid Usage

  • The buffer’s usage flag must have the VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT bit set.

  • The offset must be aligned to VkDeviceGeneratedCommandsLimitsNVX::minCommandsTokenBufferOffsetAlignment.

Valid Usage (Implicit)

The following code provides detailed information on how an individual sequence is processed:

void cmdProcessSequence(cmd, objectTable, indirectCommandsLayout, pIndirectCommandsTokens, s)
{
  for (uint32_t c = 0; c < indirectCommandsLayout.tokenCount; c++){
    input   = pIndirectCommandsTokens[c];
    i       = s / indirectCommandsLayout.pTokens[c].divisor;

    switch(input.type){
      VK_INDIRECT_COMMANDS_TOKEN_TYPE_PIPELINE_NVX:
        size_t    stride  = sizeof(uint32_t);
        uint32_t* data    = input.buffer.pointer( input.offset + stride * i );
        uint32_t  object  = data[0];

        vkCmdBindPipeline(cmd, indirectCommandsLayout.pipelineBindPoint,
          objectTable.pipelines[ object ].pipeline);
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_DESCRIPTOR_SET_NVX:
        size_t    stride  = sizeof(uint32_t) + sizeof(uint32_t) * indirectCommandsLayout.pTokens[c].dynamicCount;
        uint32_t* data    = input.buffer.pointer( input.offset + stride * i);
        uint32_t  object  = data[0];

        vkCmdBindDescriptorSets(cmd, indirectCommandsLayout.pipelineBindPoint,
          objectTable.descriptorsets[ object ].layout,
          indirectCommandsLayout.pTokens[ c ].bindingUnit,
          1, &objectTable.descriptorsets[ object ].descriptorSet,
          indirectCommandsLayout.pTokens[ c ].dynamicCount, data + 1);
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_PUSH_CONSTANT_NVX:
        size_t    stride  = sizeof(uint32_t) + indirectCommandsLayout.pTokens[c].dynamicCount;
        uint32_t* data    = input.buffer.pointer( input.offset + stride * i );
        uint32_t  object  = data[0];

        vkCmdPushConstants(cmd,
          objectTable.pushconstants[ object ].layout,
          objectTable.pushconstants[ object ].stageFlags,
          indirectCommandsLayout.pTokens[ c ].bindingUnit, indirectCommandsLayout.pTokens[c].dynamicCount, data + 1);
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_INDEX_BUFFER_NVX:
        size_t   s tride  = sizeof(uint32_t) + sizeof(uint32_t) * indirectCommandsLayout.pTokens[c].dynamicCount;
        uint32_t* data    = input.buffer.pointer( input.offset + stride * i );
        uint32_t  object  = data[0];

        vkCmdBindIndexBuffer(cmd,
          objectTable.vertexbuffers[ object ].buffer,
          indirectCommandsLayout.pTokens[ c ].dynamicCount ? data[1] : 0,
          objectTable.vertexbuffers[ object ].indexType);
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_VERTEX_BUFFER_NVX:
        size_t    stride  = sizeof(uint32_t) + sizeof(uint32_t) * indirectCommandsLayout.pTokens[c].dynamicCount;
        uint32_t* data    = input.buffer.pointer( input.offset + stride * i );
        uint32_t  object  = data[0];

        vkCmdBindVertexBuffers(cmd,
          indirectCommandsLayout.pTokens[ c ].bindingUnit, 1,
          &objectTable.vertexbuffers[ object ].buffer,
          indirectCommandsLayout.pTokens[ c ].dynamicCount ? data + 1 : {0}); // device size handled as uint32_t
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_INDEXED_NVX:
        vkCmdDrawIndexedIndirect(cmd,
          input.buffer,
          sizeof(VkDrawIndexedIndirectCommand) * i + input.offset, 1, 0);
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_NVX:
        vkCmdDrawIndirect(cmd,
          input.buffer,
          sizeof(VkDrawIndirectCommand) * i  + input.offset, 1, 0);
      break;

      VK_INDIRECT_COMMANDS_TOKEN_TYPE_DISPATCH_NVX:
        vkCmdDispatchIndirect(cmd,
          input.buffer,
          sizeof(VkDispatchIndirectCommand) * i  + input.offset);
      break;
    }
  }
}

30.3.2. Creation and Deletion

Indirect command layouts are created by:

VkResult vkCreateIndirectCommandsLayoutNVX(
    VkDevice                                    device,
    const VkIndirectCommandsLayoutCreateInfoNVX* pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkIndirectCommandsLayoutNVX*                pIndirectCommandsLayout);

  • device is the logical device that creates the indirect command layout.

  • pCreateInfo is a pointer to an instance of the VkIndirectCommandsLayoutCreateInfoNVX structure containing parameters affecting creation of the indirect command layout.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pIndirectCommandsLayout points to a VkIndirectCommandsLayoutNVX handle in which the resulting indirect command layout is returned.

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • pCreateInfo must be a valid pointer to a valid VkIndirectCommandsLayoutCreateInfoNVX structure

  • If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • pIndirectCommandsLayout must be a valid pointer to a VkIndirectCommandsLayoutNVX handle

Return Codes
Success

  • VK_SUCCESS

Failure

  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

The VkIndirectCommandsLayoutCreateInfoNVX structure is defined as:

typedef struct VkIndirectCommandsLayoutCreateInfoNVX {
    VkStructureType                            sType;
    const void*                                pNext;
    VkPipelineBindPoint                        pipelineBindPoint;
    VkIndirectCommandsLayoutUsageFlagsNVX      flags;
    uint32_t                                   tokenCount;
    const VkIndirectCommandsLayoutTokenNVX*    pTokens;
} VkIndirectCommandsLayoutCreateInfoNVX;

The following code illustrates some of the key flags:

void cmdProcessAllSequences(cmd, objectTable, indirectCommandsLayout, pIndirectCommandsTokens, sequencesCount, indexbuffer, indexbufferoffset)
{
  for (s = 0; s < sequencesCount; s++)
  {
    sequence = s;

    if (indirectCommandsLayout.flags & VK_INDIRECT_COMMANDS_LAYOUT_USAGE_UNORDERED_SEQUENCES_BIT_NVX) {
      sequence = incoherent_implementation_dependent_permutation[ sequence ];
    }
    if (indirectCommandsLayout.flags & VK_INDIRECT_COMMANDS_LAYOUT_USAGE_INDEXED_SEQUENCES_BIT_NVX) {
      sequence = indexbuffer.load_uint32( sequence * sizeof(uint32_t) + indexbufferoffset);
    }

    cmdProcessSequence( cmd, objectTable, indirectCommandsLayout, pIndirectCommandsTokens, sequence );
  }
}
Valid Usage

  • tokenCount must be greater than 0 and below VkDeviceGeneratedCommandsLimitsNVX::maxIndirectCommandsLayoutTokenCount

  • If the VkDeviceGeneratedCommandsFeaturesNVX::computeBindingPointSupport feature is not enabled, then pipelineBindPoint must not be VK_PIPELINE_BIND_POINT_COMPUTE

  • If pTokens contains an entry of VK_INDIRECT_COMMANDS_TOKEN_TYPE_PIPELINE_NVX it must be the first element of the array and there must be only a single element of such token type.

  • All state binding tokens in pTokens must occur prior work provoking tokens (VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_NVX, VK_INDIRECT_COMMANDS_TOKEN_TYPE_DRAW_INDEXED_NVX, VK_INDIRECT_COMMANDS_TOKEN_TYPE_DISPATCH_NVX).

  • The content of pTokens must include one single work provoking token that is compatible with the pipelineBindPoint.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_INDIRECT_COMMANDS_LAYOUT_CREATE_INFO_NVX

  • pNext must be NULL

  • pipelineBindPoint must be a valid VkPipelineBindPoint value

  • flags must be a valid combination of VkIndirectCommandsLayoutUsageFlagBitsNVX values

  • flags must not be 0

  • pTokens must be a valid pointer to an array of tokenCount valid VkIndirectCommandsLayoutTokenNVX structures

  • tokenCount must be greater than 0

Bits which can be set in VkIndirectCommandsLayoutCreateInfoNVX::flags, specifying usage hints of an indirect command layout, are:

typedef enum VkIndirectCommandsLayoutUsageFlagBitsNVX {
    VK_INDIRECT_COMMANDS_LAYOUT_USAGE_UNORDERED_SEQUENCES_BIT_NVX = 0x00000001,
    VK_INDIRECT_COMMANDS_LAYOUT_USAGE_SPARSE_SEQUENCES_BIT_NVX = 0x00000002,
    VK_INDIRECT_COMMANDS_LAYOUT_USAGE_EMPTY_EXECUTIONS_BIT_NVX = 0x00000004,
    VK_INDIRECT_COMMANDS_LAYOUT_USAGE_INDEXED_SEQUENCES_BIT_NVX = 0x00000008,
    VK_INDIRECT_COMMANDS_LAYOUT_USAGE_FLAG_BITS_MAX_ENUM_NVX = 0x7FFFFFFF
} VkIndirectCommandsLayoutUsageFlagBitsNVX;

  • VK_INDIRECT_COMMANDS_LAYOUT_USAGE_UNORDERED_SEQUENCES_BIT_NVX specifies that the processing of sequences can happen at an implementation-dependent order, which is not guaranteed to be coherent across multiple invocations.

  • VK_INDIRECT_COMMANDS_LAYOUT_USAGE_SPARSE_SEQUENCES_BIT_NVX specifies that there is likely a high difference between allocated number of sequences and actually used.

  • VK_INDIRECT_COMMANDS_LAYOUT_USAGE_EMPTY_EXECUTIONS_BIT_NVX specifies that there are likely many draw or dispatch calls that are zero-sized (zero grid dimension, no primitives to render).

  • VK_INDIRECT_COMMANDS_LAYOUT_USAGE_INDEXED_SEQUENCES_BIT_NVX specifies that the input data for the sequences is not implicitly indexed from 0..sequencesUsed but a user provided VkBuffer encoding the index is provided.

typedef VkFlags VkIndirectCommandsLayoutUsageFlagsNVX;

VkIndirectCommandsLayoutUsageFlagsNVX is a bitmask type for setting a mask of zero or more VkIndirectCommandsLayoutUsageFlagBitsNVX.

Indirect command layouts are destroyed by:

void vkDestroyIndirectCommandsLayoutNVX(
    VkDevice                                    device,
    VkIndirectCommandsLayoutNVX                 indirectCommandsLayout,
    const VkAllocationCallbacks*                pAllocator);

  • device is the logical device that destroys the layout.

  • indirectCommandsLayout is the table to destroy.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage

  • All submitted commands that refer to indirectCommandsLayout must have completed execution

  • If VkAllocationCallbacks were provided when objectTable was created, a compatible set of callbacks must be provided here

  • If no VkAllocationCallbacks were provided when objectTable was created, pAllocator must be NULL

Valid Usage (Implicit)

  • device must be a valid VkDevice handle

  • indirectCommandsLayout must be a valid VkIndirectCommandsLayoutNVX handle

  • If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • indirectCommandsLayout must have been created, allocated, or retrieved from device

30.4. Indirect Commands Generation

Command space for generated commands recorded into a secondary command buffer must be reserved by calling:

void vkCmdReserveSpaceForCommandsNVX(
    VkCommandBuffer                             commandBuffer,
    const VkCmdReserveSpaceForCommandsInfoNVX*  pReserveSpaceInfo);

  • commandBuffer is the secondary command buffer in which the space for device-generated commands is reserved.

  • pProcessCommandsInfo is a pointer to an instance of the VkCmdReserveSpaceForCommandsInfoNVX structure containing parameters affecting the reservation of command buffer space.

Valid Usage

  • The provided commandBuffer must not have had a prior space reservation since its creation or the last reset.

  • The state of the commandBuffer must be legal to execute all commands within the sequence provided by the indirectCommandsLayout member of pProcessCommandsInfo.

Valid Usage (Implicit)

  • commandBuffer must be a valid VkCommandBuffer handle

  • pReserveSpaceInfo must be a valid pointer to a valid VkCmdReserveSpaceForCommandsInfoNVX structure

  • commandBuffer must be in the recording state

  • The VkCommandPool that commandBuffer was allocated from must support graphics, or compute operations

  • This command must only be called inside of a render pass instance

  • commandBuffer must be a secondary VkCommandBuffer

Host Synchronization

  • Host access to commandBuffer must be externally synchronized

  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Secondary

Inside

Graphics
Compute

 
typedef struct VkCmdReserveSpaceForCommandsInfoNVX {
    VkStructureType                sType;
    const void*                    pNext;
    VkObjectTableNVX               objectTable;
    VkIndirectCommandsLayoutNVX    indirectCommandsLayout;
    uint32_t                       maxSequencesCount;
} VkCmdReserveSpaceForCommandsInfoNVX;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to an extension-specific structure.

  • objectTable is the VkObjectTableNVX to be used for the generation process. Only registered objects at the time vkCmdReserveSpaceForCommandsNVX is called, will be taken into account for the reservation.

  • indirectCommandsLayout is the VkIndirectCommandsLayoutNVX that must also be used at generation time.

  • maxSequencesCount is the maximum number of sequences for which command buffer space will be reserved.

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_CMD_RESERVE_SPACE_FOR_COMMANDS_INFO_NVX

  • pNext must be NULL

  • objectTable must be a valid VkObjectTableNVX handle

  • indirectCommandsLayout must be a valid VkIndirectCommandsLayoutNVX handle

  • Both of indirectCommandsLayout, and objectTable must have been created, allocated, or retrieved from the same VkDevice

Host Synchronization

  • Host access to objectTable must be externally synchronized

The generated commands will behave as if they were recorded within the call to vkCmdReserveSpaceForCommandsNVX, that means they can inherit state defined in the command buffer prior this call. However, given the stateless nature of the generated sequences, they will not affect commands after the reserved space. Treat the state that can be affected by the provided VkIndirectCommandsLayoutNVX as undefined.

The actual generation on the device is handled with:

void vkCmdProcessCommandsNVX(
    VkCommandBuffer                             commandBuffer,
    const VkCmdProcessCommandsInfoNVX*          pProcessCommandsInfo);

  • commandBuffer is the primary command buffer in which the generation process takes space.

  • pProcessCommandsInfo is a pointer to an instance of the VkCmdProcessCommandsInfoNVX structure containing parameters affecting the processing of commands.

Valid Usage (Implicit)

  • commandBuffer must be a valid VkCommandBuffer handle

  • pProcessCommandsInfo must be a valid pointer to a valid VkCmdProcessCommandsInfoNVX structure

  • commandBuffer must be in the recording state

  • The VkCommandPool that commandBuffer was allocated from must support graphics, or compute operations

  • This command must only be called inside of a render pass instance

Host Synchronization

  • Host access to commandBuffer must be externally synchronized

  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Primary
Secondary

Inside

Graphics
Compute

 
typedef struct VkCmdProcessCommandsInfoNVX {
    VkStructureType                      sType;
    const void*                          pNext;
    VkObjectTableNVX                     objectTable;
    VkIndirectCommandsLayoutNVX          indirectCommandsLayout;
    uint32_t                             indirectCommandsTokenCount;
    const VkIndirectCommandsTokenNVX*    pIndirectCommandsTokens;
    uint32_t                             maxSequencesCount;
    VkCommandBuffer                      targetCommandBuffer;
    VkBuffer                             sequencesCountBuffer;
    VkDeviceSize                         sequencesCountOffset;
    VkBuffer                             sequencesIndexBuffer;
    VkDeviceSize                         sequencesIndexOffset;
} VkCmdProcessCommandsInfoNVX;

  • sType is the type of this structure.

  • pNext is NULL or a pointer to an extension-specific structure.

  • objectTable is the VkObjectTableNVX to be used for the generation process. Only registered objects at the time vkCmdReserveSpaceForCommandsNVX is called, will be taken into account for the reservation.

  • indirectCommandsLayout is the VkIndirectCommandsLayoutNVX that provides the command sequence to generate.

  • indirectCommandsTokenCount defines the number of input tokens used.

  • pIndirectCommandsTokens provides an array of VkIndirectCommandsTokenNVX that reference the input data for each token command.

  • maxSequencesCount is the maximum number of sequences for which command buffer space will be reserved. If sequencesCountBuffer is VK_NULL_HANDLE, this is also the actual number of sequences generated.

  • targetCommandBuffer can be the secondary VkCommandBuffer in which the commands should be recorded. If targetCommandBuffer is NULL an implicit reservation as well as execution takes place on the processing VkCommandBuffer.

  • sequencesCountBuffer can be VkBuffer from which the actual amount of sequences is sourced from as uint32_t value.

  • sequencesCountOffset is the byte offset into sequencesCountBuffer where the count value is stored.

  • sequencesIndexBuffer must be set if indirectCommandsLayout’s VK_INDIRECT_COMMANDS_LAYOUT_USAGE_INDEXED_SEQUENCES_BIT_NVX is set and provides the used sequence indices as uint32_t array. Otherwise it must be VK_NULL_HANDLE.

  • sequencesIndexOffset is the byte offset into sequencesIndexBuffer where the index values start.

Valid Usage

  • The provided objectTable must include all objects referenced by the generation process

  • indirectCommandsTokenCount must match the indirectCommandsLayout’s tokenCount

  • The tokenType member of each entry in the pIndirectCommandsTokens array must match the values used at creation time of indirectCommandsLayout

  • If targetCommandBuffer is provided, it must have reserved command space

  • If targetCommandBuffer is provided, the objectTable must match the reservation’s objectTable and must have had all referenced objects registered at reservation time

  • If targetCommandBuffer is provided, the indirectCommandsLayout must match the reservation’s indirectCommandsLayout

  • If targetCommandBuffer is provided, the maxSequencesCount must not exceed the reservation’s maxSequencesCount

  • If sequencesCountBuffer is used, its usage flag must have the VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT bit set

  • If sequencesCountBuffer is used, sequencesCountOffset must be aligned to VkDeviceGeneratedCommandsLimitsNVX::minSequenceCountBufferOffsetAlignment

  • If sequencesIndexBuffer is used, its usage flag must have the VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT bit set

  • If sequencesIndexBuffer is used, sequencesIndexOffset must be aligned to VkDeviceGeneratedCommandsLimitsNVX::minSequenceIndexBufferOffsetAlignment

Valid Usage (Implicit)

  • sType must be VK_STRUCTURE_TYPE_CMD_PROCESS_COMMANDS_INFO_NVX

  • pNext must be NULL

  • objectTable must be a valid VkObjectTableNVX handle

  • indirectCommandsLayout must be a valid VkIndirectCommandsLayoutNVX handle

  • pIndirectCommandsTokens must be a valid pointer to an array of indirectCommandsTokenCount valid VkIndirectCommandsTokenNVX structures

  • If targetCommandBuffer is not NULL, targetCommandBuffer must be a valid VkCommandBuffer handle

  • If sequencesCountBuffer is not VK_NULL_HANDLE, sequencesCountBuffer must be a valid VkBuffer handle

  • If sequencesIndexBuffer is not VK_NULL_HANDLE, sequencesIndexBuffer must be a valid VkBuffer handle

  • indirectCommandsTokenCount must be greater than 0

  • Each of indirectCommandsLayout, objectTable, sequencesCountBuffer, sequencesIndexBuffer, and targetCommandBuffer that are valid handles must have been created, allocated, or retrieved from the same VkDevice

Host Synchronization

  • Host access to objectTable must be externally synchronized

  • Host access to targetCommandBuffer must be externally synchronized

Referencing the functions defined in Indirect Commands Layout, vkCmdProcessCommandsNVX behaves as:

// For targetCommandBuffers the existing reservedSpace is reset & overwritten.

VkCommandBuffer cmd = targetCommandBuffer ?
      targetCommandBuffer.reservedSpace :
      commandBuffer;

uint32_t sequencesCount = sequencesCountBuffer ?
      min(maxSequencesCount, sequencesCountBuffer.load_uint32(sequencesCountOffset) :
      maxSequencesCount;


cmdProcessAllSequences(cmd, objectTable,
                       indirectCommandsLayout, pIndirectCommandsTokens,
                       sequencesCount,
                       sequencesIndexBuffer, sequencesIndexOffset);

// The stateful commands within indirectCommandsLayout will not
// affect the state of subsequent commands in the target
// command buffer (cmd)
Note

It is important to note that the state that may be affected through generated commands must be considered undefined for the commands following them. It is not possible to setup generated state and provoking work that uses this state outside of the generated sequence.