VK_NV_cooperative_vector

Other Extension Metadata

Last Modified Date

2024-05-23

Interactions and External Dependencies

• This extension requires SPV_NV_cooperative_vector
• This extension provides API support for GL_NV_cooperative_vector

Contributors

• Jeff Bolz, NVIDIA

Description

This extension adds support for using cooperative vector types in SPIR-V. Unlike cooperative matrix types, a variable with a cooperative vector type is logically stored in the invocation it belongs to, but they can cooperate behind the scenes when performing matrix-vector multiplies. Cooperative vectors do not require a fully occupied subgroup or uniform control flow like cooperative matrices, although these do increase the likelihood of being on the fast path. And unlike normal vector types, they have arbitrary length and support a relatively limited set of operations. These types are intended to help accelerate the evaluation of small neural networks, where each invocation is performing its own independent evaluation of the network.

Cooperative vector types are defined by the SPV_NV_cooperative_vector SPIR-V extension and can be used with the GL_NV_cooperative_vector GLSL extension.

This extension includes support for enumerating the combinations of types that are supported by the implementation, and for converting matrix data to and from an optimized opaque layout.

New Commands

• vkCmdConvertCooperativeVectorMatrixNV
• vkConvertCooperativeVectorMatrixNV
• vkGetPhysicalDeviceCooperativeVectorPropertiesNV

New Structures

• VkConvertCooperativeVectorMatrixInfoNV
• VkCooperativeVectorPropertiesNV
• Extending VkPhysicalDeviceFeatures2, VkDeviceCreateInfo:
  • VkPhysicalDeviceCooperativeVectorFeaturesNV
• Extending VkPhysicalDeviceProperties2:
  • VkPhysicalDeviceCooperativeVectorPropertiesNV

New Unions

• VkDeviceOrHostAddressConstKHR
• VkDeviceOrHostAddressKHR

New Enums

• VkComponentTypeKHR
• VkCooperativeVectorMatrixLayoutNV

New Enum Constants

• VK_NV_COOPERATIVE_VECTOR_EXTENSION_NAME
• VK_NV_COOPERATIVE_VECTOR_SPEC_VERSION
• Extending VkComponentTypeKHR:
  • VK_COMPONENT_TYPE_FLOAT_E4M3_NV
  • VK_COMPONENT_TYPE_FLOAT_E5M2_NV
  • VK_COMPONENT_TYPE_SINT8_PACKED_NV
  • VK_COMPONENT_TYPE_UINT8_PACKED_NV
• Extending VkPipelineStageFlagBits2:
  • VK_PIPELINE_STAGE_2_CONVERT_COOPERATIVE_VECTOR_MATRIX_BIT_NV
• Extending VkStructureType:
  • VK_STRUCTURE_TYPE_CONVERT_COOPERATIVE_VECTOR_MATRIX_INFO_NV
  • VK_STRUCTURE_TYPE_COOPERATIVE_VECTOR_PROPERTIES_NV
  • VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_VECTOR_FEATURES_NV
  • VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_VECTOR_PROPERTIES_NV

New SPIR-V Capabilities

• CooperativeVectorNV

Issues

Version History

• Revision 4, 2024-05-23 (Jeff Bolz)
  • Add maxCooperativeVectorComponents
• Revision 3, 2024-05-23 (Jeff Bolz)
  • Add training functions
• Revision 2, 2024-02-10 (Jeff Bolz)
  • Add device-side matrix conversion
• Revision 1, 2023-12-13 (Jeff Bolz)
  • Initial revisions