Sparse Resources

As documented in Resource Memory Association, VkBuffer and VkImage resources in Vulkan must be bound completely and contiguously to a single VkDeviceMemory object. This binding must be done before the resource is used, and the binding is immutable for the lifetime of the resource.

Sparse resources relax these restrictions and provide these additional features:

Sparse resources can be bound non-contiguously to one or more VkDeviceMemory allocations.
Sparse resources can be re-bound to different memory allocations over the lifetime of the resource.
Sparse resources can have descriptors generated and used orthogonally with memory binding commands.

Sparse Resource Features

Sparse resources have several features that must be enabled explicitly at resource creation time. The features are enabled by including bits in the flags parameter of VkImageCreateInfo or VkBufferCreateInfo. Each feature also has one or more corresponding feature enables specified in VkPhysicalDeviceFeatures.

The sparseBinding feature is the base, and provides the following capabilities:
- Resources can be bound at some defined (sparse block) granularity.
- The entire resource must be bound to memory before use regardless of regions actually accessed.
- No specific mapping of image region to memory offset is defined, i.e. the location that each texel corresponds to in memory is implementation-dependent.
- Sparse buffers have a well-defined mapping of buffer range to memory range, where an offset into a range of the buffer that is bound to a single contiguous range of memory corresponds to an identical offset within that range of memory.
- Requested via the VK_IMAGE_CREATE_SPARSE_BINDING_BIT and VK_BUFFER_CREATE_SPARSE_BINDING_BIT bits.
- A sparse image created using VK_IMAGE_CREATE_SPARSE_BINDING_BIT (but not VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT) supports all formats that non-sparse usage supports, and supports both VK_IMAGE_TILING_OPTIMAL and VK_IMAGE_TILING_LINEAR tiling.
Sparse Residency builds on (and requires) the sparseBinding feature. It includes the following capabilities:
- Resources do not have to be completely bound to memory before use on the device.
- Images have a prescribed sparse image block layout, allowing specific rectangular regions of the image to be bound to specific offsets in memory allocations.
- Consistency of access to unbound regions of the resource is defined by the absence or presence of VkPhysicalDeviceSparseProperties::residencyNonResidentStrict. If this property is present, accesses to unbound regions of the resource are well defined and behave as if the data bound is populated with all zeros; writes are discarded. When this property is absent, accesses are considered safe, but reads will return undefined: values.
- Requested via the VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT and VK_BUFFER_CREATE_SPARSE_RESIDENCY_BIT bits.
- Sparse residency support is advertised on a finer grain via the following features:
  - The sparseResidencyBuffer feature provides support for creating VkBuffer objects with the VK_BUFFER_CREATE_SPARSE_RESIDENCY_BIT.
  - The sparseResidencyImage2D feature provides support for creating 2D single-sampled VkImage objects with VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
  - The sparseResidencyImage3D feature provides support for creating 3D VkImage objects with VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
  - The sparseResidency2Samples feature provides support for creating 2D VkImage objects with 2 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
  - The sparseResidency4Samples feature provides support for creating 2D VkImage objects with 4 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
  - The sparseResidency8Samples feature provides support for creating 2D VkImage objects with 8 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
  - The sparseResidency16Samples feature provides support for creating 2D VkImage objects with 16 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
  Implementations supporting sparseResidencyImage2D are only required to support sparse 2D, single-sampled images. Support for sparse 3D and MSAA images is optional and can be enabled via sparseResidencyImage3D, sparseResidency2Samples, sparseResidency4Samples, sparseResidency8Samples, and sparseResidency16Samples.
- A sparse image created using VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT supports all non-compressed color formats with power-of-two element size that non-sparse usage supports. Additional formats may also be supported and can be queried via vkGetPhysicalDeviceSparseImageFormatProperties. VK_IMAGE_TILING_LINEAR tiling is not supported.
The sparseResidencyAliased feature provides the following capability that can be enabled per resource:
Allows physical memory ranges to be shared between multiple locations in the same sparse resource or between multiple sparse resources, with each binding of a memory location observing a consistent interpretation of the memory contents.
See Sparse Memory Aliasing for more information.

Sparse Buffers and Fully-Resident Images

Both VkBuffer and VkImage objects created with the VK_IMAGE_CREATE_SPARSE_BINDING_BIT or VK_BUFFER_CREATE_SPARSE_BINDING_BIT bits can be thought of as a linear region of address space. In the VkImage case if VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT is not used, this linear region is entirely opaque, meaning that there is no application-visible mapping between texel location and memory offset.

Unless VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT or VK_BUFFER_CREATE_SPARSE_RESIDENCY_BIT are also used, the entire resource must be bound to one or more VkDeviceMemory objects before use.

Sparse Buffer and Fully-Resident Image Block Size

The sparse block size in bytes for sparse buffers and fully-resident images is reported as VkMemoryRequirements::alignment. alignment represents both the memory alignment requirement and the binding granularity (in bytes) for sparse resources.

Sparse Partially-Resident Buffers

VkBuffer objects created with the VK_BUFFER_CREATE_SPARSE_RESIDENCY_BIT bit allow the buffer to be made only partially resident. Partially resident VkBuffer objects are allocated and bound identically to VkBuffer objects using only the VK_BUFFER_CREATE_SPARSE_BINDING_BIT feature. The only difference is the ability for some regions of the buffer to be unbound during device use.

Sparse Partially-Resident Images

VkImage objects created with the VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT bit allow specific rectangular regions of the image called sparse image blocks to be bound to specific ranges of memory. This allows the application to manage residency at either image subresource or sparse image block granularity. Each image subresource (outside of the mip tail) starts on a sparse block boundary and has dimensions that are integer multiples of the corresponding dimensions of the sparse image block.

Applications can use these types of images to control LOD based on total memory consumption. If memory pressure becomes an issue the application can unbind and disable specific mipmap levels of images without having to recreate resources or modify texel data of unaffected levels.

The application can also use this functionality to access subregions of the image in a megatexture fashion. The application can create a large image and only populate the region of the image that is currently being used in the scene.

Accessing Unbound Regions

The following member of VkPhysicalDeviceSparseProperties affects how data in unbound regions of sparse resources are handled by the implementation:

residencyNonResidentStrict

If this property is not present, reads of unbound regions of the image will return undefined: values. Both reads and writes are still considered safe and will not affect other resources or populated regions of the image.

If this property is present, all reads of unbound regions of the image will behave as if the region was bound to memory populated with all zeros; writes will be discarded.

Image operations performed on unbound memory may still alter some component values in the natural way for those accesses, e.g. substituting a value of one for alpha in formats that do not have an alpha component.

Example: Reading the alpha component of an unbacked VK_FORMAT_R8_UNORM image will return a value of 1.0f.

See Physical Device Enumeration for instructions for retrieving physical device properties.

Mip Tail Regions

Sparse images created using VK_IMAGE_CREATE_SPARSE_BINDING_BIT (without also using VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT) have no specific mapping of image region or image subresource to memory offset defined, so the entire image can be thought of as a linear opaque address region. However, images created with VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT do have a prescribed sparse image block layout, and hence each image subresource must start on a sparse block boundary. Within each array layer, the set of mip levels that have a smaller size than the sparse block size in bytes are grouped together into a mip tail region.

If the VK_SPARSE_IMAGE_FORMAT_ALIGNED_MIP_SIZE_BIT flag is present in the flags member of VkSparseImageFormatProperties, for the image’s format, then any mip level which has dimensions that are not integer multiples of the corresponding dimensions of the sparse image block, and all subsequent mip levels, are also included in the mip tail region.

The following member of VkPhysicalDeviceSparseProperties may affect how the implementation places mip levels in the mip tail region:

residencyAlignedMipSize

Each mip tail region is bound to memory as an opaque region (i.e. must be bound using a VkSparseImageOpaqueMemoryBindInfo structure) and may be of a size greater than or equal to the sparse block size in bytes. This size is guaranteed to be an integer multiple of the sparse block size in bytes.

An implementation may choose to allow each array-layer’s mip tail region to be bound to memory independently or require that all array-layer’s mip tail regions be treated as one. This is dictated by VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT in VkSparseImageMemoryRequirements::flags.

The following diagrams depict how VK_SPARSE_IMAGE_FORMAT_ALIGNED_MIP_SIZE_BIT and VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT alter memory usage and requirements.

In the absence of VK_SPARSE_IMAGE_FORMAT_ALIGNED_MIP_SIZE_BIT and VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT, each array layer contains a mip tail region containing texel data for all mip levels smaller than the sparse image block in any dimension.

Mip levels that are as large or larger than a sparse image block in all dimensions can be bound individually. Right-edges and bottom-edges of each level are allowed to have partially used sparse blocks. Any bound partially-used-sparse-blocks must still have their full sparse block size in bytes allocated in memory.

When VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT is present all array layers will share a single mip tail region.

The mip tail regions are presented here in 2D arrays simply for figure size reasons. Each mip tail is logically a single array of sparse blocks with an implementation-dependent mapping of texels or compressed texel blocks to sparse blocks.

When VK_SPARSE_IMAGE_FORMAT_ALIGNED_MIP_SIZE_BIT is present the first mip level that would contain partially used sparse blocks begins the mip tail region. This level and all subsequent levels are placed in the mip tail. Only the first N mip levels whose dimensions are an exact multiple of the sparse image block dimensions can be bound and unbound on a sparse block basis.

sparseimage alignedmipsize singlemiptail

The mip tail region is presented here in a 2D array simply for figure size reasons. It is logically a single array of sparse blocks with an implementation-dependent mapping of texels or compressed texel blocks to sparse blocks.

When both VK_SPARSE_IMAGE_FORMAT_ALIGNED_MIP_SIZE_BIT and VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT are present the constraints from each of these flags are in effect.

Standard Sparse Image Block Shapes

Standard sparse image block shapes define a standard set of dimensions for sparse image blocks that depend on the format of the image. Layout of texels or compressed texel blocks within a sparse image block is implementation-dependent. All currently defined standard sparse image block shapes are 64 KB in size.

For block-compressed formats (e.g. VK_FORMAT_BC5_UNORM_BLOCK), the texel size is the size of the compressed texel block (e.g. 128-bit for BC5) thus the dimensions of the standard sparse image block shapes apply in terms of compressed texel blocks.

For block-compressed formats, the dimensions of a sparse image block in terms of texels can be calculated by multiplying the sparse image block dimensions by the compressed texel block dimensions.

Table 49. Standard Sparse Image Block Shapes (Single Sample)
TEXEL SIZE (bits)	Block Shape (2D)	Block Shape (3D)
8-Bit	256 {times} 256 {times} 1	64 {times} 32 {times} 32
16-Bit	256 {times} 128 {times} 1	32 {times} 32 {times} 32
32-Bit	128 {times} 128 {times} 1	32 {times} 32 {times} 16
64-Bit	128 {times} 64 {times} 1	32 {times} 16 {times} 16
128-Bit	64 {times} 64 {times} 1	16 {times} 16 {times} 16

Table 50. Standard Sparse Image Block Shapes (MSAA)
TEXEL SIZE (bits)	Block Shape (2X)	Block Shape (4X)	Block Shape (8X)	Block Shape (16X)
8-Bit	128 {times} 256 {times} 1	128 {times} 128 {times} 1	64 {times} 128 {times} 1	64 {times} 64 {times} 1
16-Bit	128 {times} 128 {times} 1	128 {times} 64 {times} 1	64 {times} 64 {times} 1	64 {times} 32 {times} 1
32-Bit	64 {times} 128 {times} 1	64 {times} 64 {times} 1	32 {times} 64 {times} 1	32 {times} 32 {times} 1
64-Bit	64 {times} 64 {times} 1	64 {times} 32 {times} 1	32 {times} 32 {times} 1	32 {times} 16 {times} 1
128-Bit	32 {times} 64 {times} 1	32 {times} 32 {times} 1	16 {times} 32 {times} 1	16 {times} 16 {times} 1

Implementations that support the standard sparse image block shape for all formats listed in the Table 49. Standard Sparse Image Block Shapes (Single Sample) and Table 50. Standard Sparse Image Block Shapes (MSAA) tables may advertise the following VkPhysicalDeviceSparseProperties:

residencyStandard2DBlockShape
residencyStandard2DMultisampleBlockShape
residencyStandard3DBlockShape

Reporting each of these features does not imply that all possible image types are supported as sparse. Instead, this indicates that no supported sparse image of the corresponding type will use custom sparse image block dimensions for any formats that have a corresponding standard sparse image block shape.

Custom Sparse Image Block Shapes

An implementation that does not support a standard image block shape for a particular sparse partially-resident image may choose to support a custom sparse image block shape for it instead. The dimensions of such a custom sparse image block shape are reported in VkSparseImageFormatProperties::imageGranularity. As with standard sparse image block shapes, the size in bytes of the custom sparse image block shape will be reported in VkMemoryRequirements::alignment.

Custom sparse image block dimensions are reported through vkGetPhysicalDeviceSparseImageFormatProperties and vkGetImageSparseMemoryRequirements.

An implementation must not support both the standard sparse image block shape and a custom sparse image block shape for the same image. The standard sparse image block shape must be used if it is supported.

Multiple Aspects

Partially resident images are allowed to report separate sparse properties for different aspects of the image. One example is for depth/stencil images where the implementation separates the depth and stencil data into separate planes. Another reason for multiple aspects is to allow the application to manage memory allocation for implementation-private metadata associated with the image. See the figure below:

In the figure above the depth, stencil, and metadata aspects all have unique sparse properties. The per-texel stencil data is ¼ the size of the depth data, hence the stencil sparse blocks include 4 × the number of texels. The sparse block size in bytes for all of the aspects is identical and defined by VkMemoryRequirements::alignment.

Metadata

The metadata aspect of an image has the following constraints:

All metadata is reported in the mip tail region of the metadata aspect.
All metadata must be bound prior to device use of the sparse image.

Sparse Memory Aliasing

By default sparse resources have the same aliasing rules as non-sparse resources. See Memory Aliasing for more information.

VkDevice objects that have the sparseResidencyAliased feature enabled are able to use the VK_BUFFER_CREATE_SPARSE_ALIASED_BIT and VK_IMAGE_CREATE_SPARSE_ALIASED_BIT flags for resource creation. These flags allow resources to access physical memory bound into multiple locations within one or more sparse resources in a data consistent fashion. This means that reading physical memory from multiple aliased locations will return the same value.

Care must be taken when performing a write operation to aliased physical memory. Memory dependencies must be used to separate writes to one alias from reads or writes to another alias. Writes to aliased memory that are not properly guarded against accesses to different aliases will have undefined: results for all accesses to the aliased memory.

Applications that wish to make use of data consistent sparse memory aliasing must abide by the following guidelines:

All sparse resources that are bound to aliased physical memory must be created with the VK_BUFFER_CREATE_SPARSE_ALIASED_BIT / VK_IMAGE_CREATE_SPARSE_ALIASED_BIT flag.
All resources that access aliased physical memory must interpret the memory in the same way. This implies the following:
- Buffers and images cannot alias the same physical memory in a data consistent fashion. The physical memory ranges must be used exclusively by buffers or used exclusively by images for data consistency to be guaranteed.
- Memory in sparse image mip tail regions cannot access aliased memory in a data consistent fashion.
- Sparse images that alias the same physical memory must have compatible formats and be using the same sparse image block shape in order to access aliased memory in a data consistent fashion.

Failure to follow any of the above guidelines will require the application to abide by the normal, non-sparse resource aliasing rules. In this case memory cannot be accessed in a data consistent fashion.

Enabling sparse resource memory aliasing can be a way to lower physical memory use, but it may reduce performance on some implementations. An application developer can test on their target HW and balance the memory / performance trade-offs measured.

Sparse Resource API

The APIs related to sparse resources are grouped into the following categories:

Physical Device Features

Some sparse-resource related features are reported and enabled in VkPhysicalDeviceFeatures. These features must be supported and enabled on the VkDevice object before applications can use them. See Physical Device Features for information on how to get and set enabled device features, and for more detailed explanations of these features.

Sparse Physical Device Features

sparseBinding: Support for creating VkBuffer and VkImage objects with the VK_BUFFER_CREATE_SPARSE_BINDING_BIT and VK_IMAGE_CREATE_SPARSE_BINDING_BIT flags, respectively.
sparseResidencyBuffer: Support for creating VkBuffer objects with the VK_BUFFER_CREATE_SPARSE_RESIDENCY_BIT flag.
sparseResidencyImage2D: Support for creating 2D single-sampled VkImage objects with VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
sparseResidencyImage3D: Support for creating 3D VkImage objects with VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
sparseResidency2Samples: Support for creating 2D VkImage objects with 2 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
sparseResidency4Samples: Support for creating 2D VkImage objects with 4 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
sparseResidency8Samples: Support for creating 2D VkImage objects with 8 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
sparseResidency16Samples: Support for creating 2D VkImage objects with 16 samples and VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT.
sparseResidencyAliased: Support for creating VkBuffer and VkImage objects with the VK_BUFFER_CREATE_SPARSE_ALIASED_BIT and VK_IMAGE_CREATE_SPARSE_ALIASED_BIT flags, respectively.

Physical Device Sparse Properties

Some features of the implementation are not possible to disable, and are reported to allow applications to alter their sparse resource usage accordingly. These read-only capabilities are reported in the VkPhysicalDeviceProperties::sparseProperties member, which is a VkPhysicalDeviceSparseProperties structure.

Sparse Image Format Properties

Given that certain aspects of sparse image support, including the sparse image block dimensions, may be implementation-dependent, vkGetPhysicalDeviceSparseImageFormatProperties can be used to query for sparse image format properties prior to resource creation. This command is used to check whether a given set of sparse image parameters is supported and what the sparse image block shape will be.

Sparse Image Format Properties API

Sparse Resource Creation

Sparse resources require that one or more sparse feature flags be specified (as part of the VkPhysicalDeviceFeatures structure described previously in the Physical Device Features section) when calling vkCreateDevice. When the appropriate device features are enabled, the VK_BUFFER_CREATE_SPARSE_* and VK_IMAGE_CREATE_SPARSE_* flags can be used. See vkCreateBuffer and vkCreateImage for details of the resource creation APIs.

Specifying VK_BUFFER_CREATE_SPARSE_RESIDENCY_BIT or VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT requires specifying VK_BUFFER_CREATE_SPARSE_BINDING_BIT or VK_IMAGE_CREATE_SPARSE_BINDING_BIT, respectively, as well. This means that resources must be created with the appropriate *_SPARSE_BINDING_BIT to be used with the sparse binding command (vkQueueBindSparse).

Sparse Resource Memory Requirements

Sparse resources have specific memory requirements related to binding sparse memory. These memory requirements are reported differently for VkBuffer objects and VkImage objects.

Buffer and Fully-Resident Images

Buffers (both fully and partially resident) and fully-resident images can be bound to memory using only the data from VkMemoryRequirements. For all sparse resources the VkMemoryRequirements::alignment member specifies both the binding granularity in bytes and the required alignment of VkDeviceMemory.

Partially Resident Images

Partially resident images have a different method for binding memory. As with buffers and fully resident images, the VkMemoryRequirements::alignment field specifies the binding granularity in bytes for the image.

Requesting sparse memory requirements for VkImage objects using vkGetImageSparseMemoryRequirements will return an array of one or more VkSparseImageMemoryRequirements structures. Each structure describes the sparse memory requirements for a group of aspects of the image.

The sparse image must have been created using the VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT flag to retrieve valid sparse image memory requirements.

Sparse Image Memory Requirements

Binding Resource Memory

Non-sparse resources are backed by a single physical allocation prior to device use (via vkBindImageMemory or vkBindBufferMemory), and their backing must not be changed. On the other hand, sparse resources can be bound to memory non-contiguously and these bindings can be altered during the lifetime of the resource.

It is important to note that freeing a VkDeviceMemory object with vkFreeMemory will not cause resources (or resource regions) bound to the memory object to become unbound. Applications must not access resources bound to memory that has been freed.

Sparse memory bindings execute on a queue that includes the VK_QUEUE_SPARSE_BINDING_BIT bit. Applications must use synchronization primitives to guarantee that other queues do not access ranges of memory concurrently with a binding change. Applications can access other ranges of the same resource while a bind operation is executing.

Implementations must provide a guarantee that simultaneously binding sparse blocks while another queue accesses those same sparse blocks via a sparse resource must not access memory owned by another process or otherwise corrupt the system.

While some implementations may include VK_QUEUE_SPARSE_BINDING_BIT support in queue families that also include graphics and compute support, other implementations may only expose a VK_QUEUE_SPARSE_BINDING_BIT-only queue family. In either case, applications must use synchronization primitives to explicitly request any ordering dependencies between sparse memory binding operations and other graphics/compute/transfer operations, as sparse binding operations are not automatically ordered against command buffer execution, even within a single queue.

When binding memory explicitly for the VK_IMAGE_ASPECT_METADATA_BIT the application must use the VK_SPARSE_MEMORY_BIND_METADATA_BIT in the VkSparseMemoryBind::flags field when binding memory. Binding memory for metadata is done the same way as binding memory for the mip tail, with the addition of the VK_SPARSE_MEMORY_BIND_METADATA_BIT flag.

Binding the mip tail for any aspect must only be performed using VkSparseImageOpaqueMemoryBindInfo. If formatProperties.flags contains VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT, then it can be bound with a single VkSparseMemoryBind structure, with resourceOffset = imageMipTailOffset and size = imageMipTailSize.

If formatProperties.flags does not contain VK_SPARSE_IMAGE_FORMAT_SINGLE_MIPTAIL_BIT then the offset for the mip tail in each array layer is given as:

arrayMipTailOffset = imageMipTailOffset + arrayLayer * imageMipTailStride;

and the mip tail can be bound with layerCount VkSparseMemoryBind structures, each using size = imageMipTailSize and resourceOffset = arrayMipTailOffset as defined above.

Sparse memory binding is handled by the following APIs and related data structures.

Sparse Memory Binding Functions

To specify the values to use when waiting for and signaling semaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE, add a VkTimelineSemaphoreSubmitInfo structure to the pNext chain of the VkBindSparseInfo structure.