VulkanHub

Fragment Operations

Fragments produced by rasterization go through a number of operations to determine whether or how values produced by fragment shading are written to the framebuffer.

The following fragment operations adhere to rasterization order, and are typically performed in this order:

  1. Discard rectangles test
  2. Scissor test
  3. Exclusive scissor test
  4. Sample mask test
  5. Certain Fragment shading operations:
  6. Multisample coverage
  7. Depth bounds test
  8. Stencil test
  9. Depth test
  10. Representative fragment test
  11. Sample counting
  12. Coverage to color
  13. Coverage reduction
  14. Coverage modulation

The coverage mask generated by rasterization describes the initial coverage of each sample covered by the fragment. Fragment operations will update the coverage mask to add or subtract coverage where appropriate. If a fragment operation results in all bits of the coverage mask being 0, the fragment is discarded, and no further operations are performed. Fragments can also be programmatically discarded in a fragment shader by executing one of

  • OpTerminateInvocation
  • OpDemoteToHelperInvocationEXT
  • OpKill.

When one of the fragment operations in this chapter is described as replacing a fragment shader output, that output is replaced unconditionally, even if no fragment shader previously wrote to that output.

If there is a fragment shader and it declares the PostDepthCoverage execution mode, the sample mask test is instead performed after the depth test.

If VkPhysicalDeviceMaintenance5PropertiesKHR::earlyFragmentMultisampleCoverageAfterSampleCounting is set to VK_TRUE and there is a fragment shader which declares the EarlyFragmentTests execution mode, fragment shading and multisample coverage operations must be performed after sample counting.

Otherwise, if VkPhysicalDeviceMaintenance5PropertiesKHR::earlyFragmentMultisampleCoverageAfterSampleCounting is set to VK_FALSE and there is a fragment shader which declares the EarlyFragmentTests execution mode, fragment shading and multisample coverage operations should instead be performed after sample counting, but may be performed before sample counting.

If VkPhysicalDeviceMaintenance5PropertiesKHR::earlyFragmentSampleMaskTestBeforeSampleCounting is set to VK_TRUE and there is a fragment shader which declares the EarlyFragmentTests execution mode sample mask test operations must follow the order of fragment operations from above.

Otherwise, if VkPhysicalDeviceMaintenance5PropertiesKHR::earlyFragmentSampleMaskTestBeforeSampleCounting is set to VK_FALSE and there is a fragment shader which declares the EarlyFragmentTests execution mode, sample mask test operations should follow the order of fragment operations from above but may instead be performed after sample counting.

If there is a fragment shader which declares the EarlyAndLateFragmentTestsAMD execution mode, and it does not declare the DepthReplacing or StencilRefReplacingEXT execution mode, fragment shading and multisample coverage operations are instead be performed after sample counting.

For a pipeline with the following properties:

  • a fragment shader is specified
  • the fragment shader either specifies EarlyAndLateFragmentTestsAMD or does not write to storage resources;
  • the fragment shader specifies the StencilRefReplacingEXT execution mode;
  • either
    • the fragment shader specifies the StencilRefUnchangedFrontAMD execution mode;
    • the fragment shader specifies the StencilRefLessFrontAMD execution mode and the pipeline uses a VkPipelineDepthStencilStateCreateInfo::front.compareOp of VK_COMPARE_OP_GREATER or VK_COMPARE_OP_GREATER_OR_EQUAL; or
    • the fragment shader specifies the StencilRefGreaterFrontAMD execution mode and the pipeline uses a VkPipelineDepthStencilStateCreateInfo::front.compareOp of VK_COMPARE_OP_LESS or VK_COMPARE_OP_LESS_OR_EQUAL; and
  • either
    • the fragment shader specifies the StencilRefUnchangedBackAMD execution mode;
    • the fragment shader specifies the StencilRefLessBackAMD execution mode and the pipeline uses a VkPipelineDepthStencilStateCreateInfo::back.compareOp of VK_COMPARE_OP_GREATER or VK_COMPARE_OP_GREATER_OR_EQUAL; or
    • the fragment shader specifies the StencilRefGreaterBackAMD execution mode and the pipeline uses a VkPipelineDepthStencilStateCreateInfo::back.compareOp of VK_COMPARE_OP_LESS or VK_COMPARE_OP_LESS_OR_EQUAL

an additional stencil test may be performed before fragment shading, using the stencil reference value specified by VkPipelineDepthStencilStateCreateInfo::front.reference or VkPipelineDepthStencilStateCreateInfo::back.reference.

For a pipeline with the following properties:

  • a fragment shader is specified
  • the fragment shader either specifies EarlyAndLateFragmentTestsAMD or does not write to storage resources;
  • the fragment shader specifies the DepthReplacing execution mode; and
  • either
    • the fragment shader specifies the DepthUnchanged execution mode;
    • the fragment shader specifies the DepthLess execution mode and the pipeline uses a VkPipelineDepthStencilStateCreateInfo::depthCompareOp of VK_COMPARE_OP_GREATER or VK_COMPARE_OP_GREATER_OR_EQUAL; or
    • the fragment shader specifies the DepthGreater execution mode and the pipeline uses a VkPipelineDepthStencilStateCreateInfo::depthCompareOp of VK_COMPARE_OP_LESS or VK_COMPARE_OP_LESS_OR_EQUAL

the implementation may perform depth bounds test before fragment shading and perform an additional depth test immediately after that using the interpolated depth value generated by rasterization.

Once all fragment operations have completed, fragment shader outputs for covered color attachment samples pass through framebuffer operations.

Discard Rectangles Test

The discard rectangle test compares the framebuffer coordinates (xf,yf) of each sample covered by a fragment against a set of discard rectangles.

Each discard rectangle is defined by a VkRect2D. These values are either set by the VkPipelineDiscardRectangleStateCreateInfoEXT structure during pipeline creation, or dynamically by the vkCmdSetDiscardRectangleEXT command.

A given sample is considered inside a discard rectangle if the xf is in the range [VkRect2D::offset.x, VkRect2D::offset.x + VkRect2D::extent.x), and yf is in the range [VkRect2D::offset.y, VkRect2D::offset.y + VkRect2D::extent.y). If the test is set to be inclusive, samples that are not inside any of the discard rectangles will have their coverage set to 0. If the test is set to be exclusive, samples that are inside any of the discard rectangles will have their coverage set to 0.

If no discard rectangles are specified, the coverage mask is unmodified by this operation.

VkPipelineDiscardRectangleStateCreateInfoEXTStructure specifying discard rectangle
VkPipelineDiscardRectangleStateCreateFlagsEXTReserved for future use
VkDiscardRectangleModeEXTSpecify the discard rectangle mode
vkCmdSetDiscardRectangleEXTSet discard rectangles dynamically for a command buffer
vkCmdSetDiscardRectangleEnableEXTEnable discard rectangles dynamically for a command buffer
vkCmdSetDiscardRectangleModeEXTSets the discard rectangle mode dynamically for a command buffer

Scissor Test

The scissor test compares the framebuffer coordinates (xf,yf) of each sample covered by a fragment against a scissor rectangle at the index equal to the fragment’s ViewportIndex.

Each scissor rectangle is defined by a VkRect2D. These values are either set by the VkPipelineViewportStateCreateInfo structure during pipeline creation, or dynamically by the vkCmdSetScissor command.

A given sample is considered inside a scissor rectangle if xf is in the range [VkRect2D::offset.x, VkRect2D::offset.x + VkRect2D::extent.x), and yf is in the range [VkRect2D::offset.y, VkRect2D::offset.y + VkRect2D::extent.y). Samples with coordinates outside the scissor rectangle at the corresponding ViewportIndex will have their coverage set to 0.

If a render pass transform is enabled, the (offset.x and offset.y) and (extent.width and extent.height) values are transformed as described in render pass transform before participating in the scissor test.

vkCmdSetScissorSet scissor rectangles dynamically for a command buffer

Exclusive Scissor Test

The exclusive scissor test compares the framebuffer coordinates (xf,yf) of each sample covered by a fragment against an exclusive scissor rectangle at the index equal to the fragment’s ViewportIndex.

Each exclusive scissor rectangle is defined by a VkRect2D. These values are either set by the VkPipelineViewportExclusiveScissorStateCreateInfoNV structure during pipeline creation, or dynamically by the vkCmdSetExclusiveScissorNV command.

A given sample is considered inside an exclusive scissor rectangle if xf is in the range [VkRect2D::offset.x, VkRect2D::offset.x + VkRect2D::extent.x), and yf is in the range [VkRect2D::offset.y, VkRect2D::offset.y + VkRect2D::extent.y). Samples with coordinates inside the exclusive scissor rectangle at the corresponding ViewportIndex will have their coverage set to 0.

If no exclusive scissor rectangles are specified, the coverage mask is unmodified by this operation.

VkPipelineViewportExclusiveScissorStateCreateInfoNVStructure specifying parameters controlling exclusive scissor testing
vkCmdSetExclusiveScissorNVSet exclusive scissor rectangles dynamically for a command buffer
vkCmdSetExclusiveScissorEnableNVDynamically enable each exclusive scissor for a command buffer

Sample Mask Test

The sample mask test compares the coverage mask for a fragment with the sample mask defined by VkPipelineMultisampleStateCreateInfo::pSampleMask.

vkCmdSetSampleMaskEXTSpecify the sample mask dynamically for a command buffer

Each bit of the coverage mask is associated with a sample index as described in the rasterization chapter. If the bit in VkPipelineMultisampleStateCreateInfo::pSampleMask which is associated with that same sample index is set to 0, the coverage mask bit is set to 0.

Fragment Shading

Fragment shaders are invoked for each fragment, or as helper invocations.

Most operations in the fragment shader are not performed in rasterization order, with exceptions called out in the following sections.

For fragment shaders invoked by fragments, the following rules apply:

  • A fragment shader must not be executed if a fragment operation that executes before fragment shading discards the fragment.
  • A fragment shader may not be executed if:
    • An implementation determines that another fragment shader, invoked by a subsequent primitive in primitive order, overwrites all results computed by the shader (including writes to storage resources).
    • Any other fragment operation discards the fragment, and the shader does not write to any storage resources.
    • If a fragment shader statically computes the same values for different framebuffer locations, and does not write to any storage resources, multiple fragments may be shaded by one fragment shader invocation. This may affect VK_QUERY_PIPELINE_STATISTIC_FRAGMENT_SHADER_INVOCATIONS_BIT results, but must otherwise not be visible behavior to applications.
  • Otherwise, at least one fragment shader must be executed.
    • If sample shading is enabled and multiple invocations per fragment are required, additional invocations must be executed as specified.
    • If a shading rate image is used and multiple invocations per fragment are required, additional invocations must be executed as specified.
    • Each covered sample must be included in at least one fragment shader invocation.

If no fragment shader is included in the pipeline, no fragment shader is executed, and undefined: values may be written to all color attachment outputs during this fragment operation.

Multiple fragment shader invocations may be executed for the same fragment for any number of implementation-dependent reasons. When there is more than one fragment shader invocation per fragment, the association of samples to invocations is implementation-dependent. Stores and atomics performed by these additional invocations have the normal effect.

For example, if the subpass includes multiple views in its view mask, a fragment shader may be invoked separately for each view.

Similarly, if the render pass has a fragment density map attachment, more than one fragment shader invocation may be invoked for each covered sample. Such additional invocations are only produced if VkPhysicalDeviceFragmentDensityMapPropertiesEXT::fragmentDensityInvocations is VK_TRUE. Implementations may generate these additional fragment shader invocations in order to make transitions between fragment areas with different fragment densities more smooth.

Sample Mask

Reading from the SampleMask built-in in the Input storage class will return the coverage mask for the current fragment as calculated by fragment operations that executed prior to fragment shading.

If sample shading is enabled, fragment shaders will only see values of 1 for samples being shaded - other bits will be 0.

Each bit of the coverage mask is associated with a sample index as described in the rasterization chapter. If the bit in SampleMask which is associated with that same sample index is set to 0, that coverage mask bit is set to 0.

Values written to the SampleMask built-in in the Output storage class will be used by the multisample coverage operation, with the same encoding as the input built-in.

Fragment Shader Tile Image Reads

If the VK_EXT_shader_tile_image extension is enabled, implementations divide the framebuffer into a grid of tiles. A tile image is a view of a framebuffer attachment tile for fragments with locations within the tile.

Within a render pass instance initiated by vkCmdBeginRenderingKHR, fragment shader invocations can read the framebuffer color, depth, and stencil values at the fragment location via tile images.

Even though fragment shader invocation can only read from the corresponding fragment location, the abstraction of a tile image is introduced for the following reasons:

  • Tile dimensions will be exposed in a future extension
  • Future functionality such as executing compute dispatches within render passes via tile shaders can leverage tile images.

Enabling shaderTileImageColorReadAccess, shaderTileImageDepthReadAccess, shaderTileImageStencilReadAccess enables fragment shader invocations to read from color, depth, and stencil, respectively.

Color values are read from tile image variables with OpColorAttachmentReadEXT. Tile image variables are linked to specific color attachments using Location decoration. See Fragment Tile Image Interface for more details.

Depth values are read with OpDepthAttachmentReadEXT.

Stencil values are read with OpStencilAttachmentReadEXT.

The sample to read is specified by a sample index value specified as the Sample operand to OpColorAttachmentReadEXT, OpDepthAttachmentReadEXT, or OpStencilAttachmentReadEXT.

If sample shading is disabled, a fragment invocation can read from all sample locations associated with the fragment regardless of the fragment’s coverage. This functionality is supported for VkPipelineMultisampleStateCreateInfo::rasterizationSamples > 1 when VkPhysicalDeviceShaderTileImagePropertiesEXT::shaderTileImageReadSampleFromPixelRateInvocation is VK_TRUE.

If sample shading is enabled, and minSampleShading is 1.0, a fragment invocation must only read from the coverage index sample. Tile image access must not be used if the value of minSampleShading is not 1.0.

If the fragment shader declares the EarlyFragmentTests execution mode, depth reads are allowed only if depth writes are disabled and stencil reads are allowed only if stencil writes are disabled.

If VkPhysicalDeviceShaderTileImagePropertiesEXT::shaderTileImageReadFromHelperInvocation is VK_FALSE, values read from helper invocations are undefined: otherwise the values read are subject to the coherency guarantees described below.

OpDepthAttachmentReadEXT returns an undefined: value if no depth attachment is present. OpStencilAttachmentReadEXT returns an undefined: value if no stencil attachment is present.

Tile image reads from color, depth and stencil attachments are said to be coherent when the accesses happen in raster order and without data race with respect to accesses to the attachments from framebuffer-space pipeline stages. The samples which qualify for coherent access and the enabling conditions are described below.

  • Let Rc be the set of components being read from an attachment A in a draw call
  • Let Wc be the set of components being written to A by the draw call

The samples which qualify for coherent tile image reads from an attachment A are:

  • All samples in a pixel when Rc is disjoint with Wc.
  • The samples with coverage in a fragment when Rc is not disjoint with Wc. The samples with coverage are determined by the coverage mask for the fragment as calculated by fragment operations that executed prior to fragment shading, including early fragment tests if enabled for the draw call.

A fragment shader can declare NonCoherentColorAttachmentReadEXT, NonCoherentDepthAttachmentReadEXT, or NonCoherentStencilAttachmentReadEXT execution modes to enable non-coherent tile image reads which require an explicit vkCmdPipelineBarrier2 call for the writes to an attachment to be made visible via tile image reads.

When VkPhysicalDeviceShaderTileImagePropertiesEXT::shaderTileImageCoherentReadAccelerated is VK_TRUE, the implementation prefers that coherent tile image reads are used, otherwise the implementation prefers that non-coherent tile image reads are used.

In practice, the most common tile image reads usage patterns fall under one of the following:

  • Programmable blending - each fragment reads from a single sample (SampleID) at its location. Per-sample shading is typically enabled when multisampled rendertargets are used.
  • G-buffer generation and shading in one render pass - in the shading phase a fragment reads from a single sample at its location.
  • Programmable resolve - a fragment reads from all samples at its location (per-sample shading is disabled). This requires the use of a "full-screen triangle" instead of a rectangle composed of two triangles in order to avoid data races along the shared edge of the triangles.
  • 1:1 texturing with LOD - in use cases such a deferred screen space decals a fragment reads a single sample (SampleID) from depth buffer, but requires being able to read from helper threads to derive the texture LOD. This use case is supported as long as the attachment components being read are not overwritten by color, depth, or stencil attachment writes.

All of the above use cases are supported by coherent tile image reads, but only the latter three are supported when non-coherent reads are used as there is no mechanism to synchronize non-coherent reads with writes within a draw call.

Depth Replacement

Writing to the FragDepth built-in will replace the fragment’s calculated depth values for each sample in the input SampleMask. Depth testing performed after the fragment shader for this fragment will use this new value as zf.

Stencil Reference Replacement

Writing to the FragStencilRefEXT built-in will replace the fragment’s stencil reference value for each sample in the input SampleMask. Stencil testing performed after the fragment shader for this fragment will use this new value as sr.

Interlocked Operations

OpBeginInvocationInterlockEXT and OpEndInvocationInterlockEXT define a section of a fragment shader which imposes additional ordering constraints on operations performed within them. These operations are defined as interlocked operations. How interlocked operations are ordered against other fragment shader invocations depends on the specified execution modes.

If the ShadingRateInterlockOrderedEXT execution mode is specified, any interlocked operations in a fragment shader must happen before interlocked operations in fragment shader invocations that execute later in rasterization order and cover at least one sample in the same fragment area, and must happen after interlocked operations in a fragment shader that executes earlier in rasterization order and cover at least one sample in the same fragment area.

If the ShadingRateInterlockUnorderedEXT execution mode is specified, any interlocked operations in a fragment shader must happen before or after interlocked operations in fragment shader invocations that execute earlier or later in rasterization order and cover at least one sample in the same fragment area.

If the PixelInterlockOrderedEXT execution mode is specified, any interlocked operations in a fragment shader must happen before interlocked operations in fragment shader invocations that execute later in rasterization order and cover at least one sample in the same pixel, and must happen after interlocked operations in a fragment shader that executes earlier in rasterization order and cover at least one sample in the same pixel.

If the PixelInterlockUnorderedEXT execution mode is specified, any interlocked operations in a fragment shader must happen before or after interlocked operations in fragment shader invocations that execute earlier or later in rasterization order and cover at least one sample in the same pixel.

If the SampleInterlockOrderedEXT execution mode is specified, any interlocked operations in a fragment shader must happen before interlocked operations in fragment shader invocations that execute later in rasterization order and cover at least one of the same samples, and must happen after interlocked operations in a fragment shader that executes earlier in rasterization order and cover at least one of the same samples.

If the SampleInterlockUnorderedEXT execution mode is specified, any interlocked operations in a fragment shader must happen before or after interlocked operations in fragment shader invocations that execute earlier or later in rasterization order and cover at least one of the same samples.

Multisample Coverage

If a fragment shader is active and its entry point’s interface includes a built-in output variable decorated with SampleMask, but not OverrideCoverageNV, the coverage mask is ANDed with the bits of the SampleMask built-in to generate a new coverage mask. If the SampleMask built-in is also decorated with OverrideCoverageNV, the coverage mask is replaced with the mask bits set in the shader. If sample shading is enabled, bits written to SampleMask corresponding to samples that are not being shaded by the fragment shader invocation are ignored. If no fragment shader is active, or if the active fragment shader does not include SampleMask in its interface, the coverage mask is not modified.

Next, the fragment alpha value and coverage mask are modified based on the line coverage factor if the lineRasterizationMode member of the VkPipelineRasterizationStateCreateInfo structure is VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_KHR, and the alphaToCoverageEnable and alphaToOneEnable members of the VkPipelineMultisampleStateCreateInfo structure.

vkCmdSetAlphaToCoverageEnableEXTSpecify the alpha to coverage enable state dynamically for a command buffer
vkCmdSetAlphaToOneEnableEXTSpecify the alpha to one enable state dynamically for a command buffer

All alpha values in this section refer only to the alpha component of the fragment shader output that has a Location and Index decoration of zero (see the Fragment Output Interface section). If that shader output has an integer or unsigned integer type, then these operations are skipped.

If the lineRasterizationMode member of the VkPipelineRasterizationStateCreateInfo structure is VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_KHR and the fragment came from a line segment, then the alpha value is replaced by multiplying it by the coverage factor for the fragment computed during smooth line rasterization.

If alphaToCoverageEnable is enabled, a temporary coverage mask is generated where each bit is determined by the fragment’s alpha value, which is ANDed with the fragment coverage mask.

No specific algorithm is specified for converting the alpha value to a temporary coverage mask. It is intended that the number of 1’s in this value be proportional to the alpha value (clamped to [0,1]), with all 1’s corresponding to a value of 1.0 and all 0’s corresponding to 0.0. The algorithm may be different at different framebuffer coordinates.

Using different algorithms at different framebuffer coordinates may help to avoid artifacts caused by regular coverage sample locations.

Finally, if alphaToOneEnable is enabled, each alpha value is replaced by the maximum representable alpha value for fixed-point color attachments, or by 1.0 for floating-point attachments. Otherwise, the alpha values are not changed.

Depth and Stencil Operations

Pipeline state controlling the depth bounds tests, stencil test, and depth test is specified through the members of the VkPipelineDepthStencilStateCreateInfo structure.

VkPipelineDepthStencilStateCreateInfoStructure specifying parameters of a newly created pipeline depth stencil state
VkPipelineDepthStencilStateCreateFlagsBitmask of VkPipelineDepthStencilStateCreateFlagBits
VkPipelineDepthStencilStateCreateFlagBitsBitmask specifying additional depth/stencil state information.

Depth Bounds Test

The depth bounds test compares the depth value za in the depth/stencil attachment at each sample’s framebuffer coordinates (xf,yf) and sample index i against a set of depth bounds.

The depth bounds are determined by two floating point values defining a minimum (minDepthBounds) and maximum (maxDepthBounds) depth value. These values are either set by the VkPipelineDepthStencilStateCreateInfo structure during pipeline creation, or dynamically by vkCmdSetDepthBoundsTestEnable and vkCmdSetDepthBounds.

A given sample is considered within the depth bounds if za is in the range [minDepthBounds,maxDepthBounds]. Samples with depth attachment values outside of the depth bounds will have their coverage set to 0.

If the depth bounds test is disabled, or if there is no depth attachment, the coverage mask is unmodified by this operation.

vkCmdSetDepthBoundsTestEnableSet depth bounds test enable dynamically for a command buffer
vkCmdSetDepthBoundsSet depth bounds range dynamically for a command buffer

Stencil Test

The stencil test compares the stencil attachment value sa in the depth/stencil attachment at each sample’s framebuffer coordinates (xf,yf) and sample index i against a stencil reference value.

If the render pass has a fragment density map attachment and the fragment covers multiple pixels, there is an implementation-dependent association of coverage samples to stencil attachment samples within the fragment. However, if all samples in the fragment are covered, and the stencil attachment value is updated as a result of this test, all stencil attachment samples will be updated.

If the stencil test is not enabled, as specified by vkCmdSetStencilTestEnable or VkPipelineDepthStencilStateCreateInfo::stencilTestEnable, or if there is no stencil attachment, the coverage mask is unmodified by this operation.

The stencil test is controlled by one of two sets of stencil-related state, the front stencil state and the back stencil state. Stencil tests and writes use the back stencil state when processing fragments generated by back-facing

polygons, and the front stencil state when processing fragments generated by front-facing polygons or any other primitives.

The comparison operation performed is determined by the VkCompareOp value set by vkCmdSetStencilOp::compareOp, or by VkStencilOpState::compareOp during pipeline creation.

The compare mask sc and stencil reference value sr of the front or the back stencil state set determine arguments of the comparison operation. sc is set by the VkPipelineDepthStencilStateCreateInfo structure during pipeline creation, or by the vkCmdSetStencilCompareMask command. sr is set by VkPipelineDepthStencilStateCreateInfo or by vkCmdSetStencilReference.

sr and sa are each independently combined with sc using a bitwise AND operation to create masked reference and attachment values s'r and s'a. s'r and s'a are used as the reference and test values, respectively, in the operation specified by the VkCompareOp.

If the comparison evaluates to false, the coverage for the sample is set to 0.

A new stencil value sg is generated according to a stencil operation defined by VkStencilOp parameters set by vkCmdSetStencilOp or VkPipelineDepthStencilStateCreateInfo. If the stencil test fails, failOp defines the stencil operation used. If the stencil test passes however, the stencil op used is based on the depth test - if it passes, VkPipelineDepthStencilStateCreateInfo::passOp is used, otherwise VkPipelineDepthStencilStateCreateInfo::depthFailOp is used.

The stencil attachment value sa is then updated with the generated stencil value sg according to the write mask sw defined by writeMask in VkPipelineDepthStencilStateCreateInfo::front and VkPipelineDepthStencilStateCreateInfo::back as:

sa = (sa & ¬sw) | (sg & sw)

vkCmdSetStencilTestEnableSet stencil test enable dynamically for a command buffer
vkCmdSetStencilOpSet stencil operation dynamically for a command buffer
VkStencilOpStateStructure specifying stencil operation state
vkCmdSetStencilCompareMaskSet stencil compare mask dynamically for a command buffer
VkStencilFaceFlagBitsBitmask specifying sets of stencil state for which to update the compare mask
VkStencilFaceFlagsBitmask of VkStencilFaceFlagBits
vkCmdSetStencilWriteMaskSet stencil write mask dynamically for a command buffer
vkCmdSetStencilReferenceSet stencil reference value dynamically for a command buffer
VkStencilOpStencil comparison function

Depth Test

The depth test compares the depth value za in the depth/stencil attachment at each sample’s framebuffer coordinates (xf,yf) and sample index i against the sample’s depth value zf. If there is no depth attachment then the depth test is skipped.

If the render pass has a fragment density map attachment and the fragment covers multiple pixels, there is an implementation-dependent association of rasterization samples to depth attachment samples within the fragment. However, if all samples in the fragment are covered, and the depth attachment value is updated as a result of this test, all depth attachment samples will be updated.

The depth test occurs in three stages, as detailed in the following sections.

Depth Clamping and Range Adjustment

If VkPipelineRasterizationStateCreateInfo::depthClampEnable is enabled, zf is clamped to [zmin, zmax], where zmin = min(n,f), zmax = max(n,f)], and n and f are the minDepth and maxDepth depth range values of the viewport used by this fragment, respectively.

If VkPhysicalDeviceDepthClampZeroOneFeaturesEXT::depthClampZeroOne is enabled:

  • If the depth attachment has a floating-point format and VK_EXT_depth_range_unrestricted is enabled then zf is unchanged.
  • Otherwise, zf is clamped to the range [0, 1].

Otherwise:

  • If zf is not in the range [zmin, zmax], then zf is undefined: following this step.
  • If the depth attachment has a fixed-point format and zf is not in the range [0, 1], then zf is undefined: following this step.

Depth Comparison

If the depth test is not enabled, as specified by vkCmdSetDepthTestEnable or VkPipelineDepthStencilStateCreateInfo::depthTestEnable, then this step is skipped.

The comparison operation performed is determined by the VkCompareOp value set by vkCmdSetDepthCompareOp, or by VkPipelineDepthStencilStateCreateInfo::depthCompareOp during pipeline creation. zf and za are used as the reference and test values, respectively, in the operation specified by the VkCompareOp.

If the comparison evaluates to false, the coverage for the sample is set to 0.

Depth Attachment Writes

If depth writes are enabled, as specified by vkCmdSetDepthWriteEnable or VkPipelineDepthStencilStateCreateInfo::depthWriteEnable, and the comparison evaluated to true, the depth attachment value za is set to the sample’s depth value zf. If there is no depth attachment, no value is written.

vkCmdSetDepthTestEnableSet depth test enable dynamically for a command buffer
vkCmdSetDepthCompareOpSet depth comparison operator dynamically for a command buffer
vkCmdSetDepthWriteEnableSet depth write enable dynamically for a command buffer

Representative Fragment Test

The representative fragment test allows implementations to reduce the amount of rasterization and fragment processing work performed for each point, line, or triangle primitive. For any primitive that produces one or more fragments that pass all prior early fragment tests, the implementation may choose one or more representative fragments for processing and discard all other fragments. For draw calls rendering multiple points, lines, or triangles arranged in lists, strips, or fans, the representative fragment test is performed independently for each of those primitives. The set of fragments discarded by the representative fragment test is implementation-dependent. In some cases, the representative fragment test may not discard any fragments for a given primitive.

VkPipelineRepresentativeFragmentTestStateCreateInfoNVStructure specifying representative fragment test
vkCmdSetRepresentativeFragmentTestEnableNVSpecify the representative fragment test enable dynamically for a command buffer

Sample Counting

Occlusion queries use query pool entries to track the number of samples that pass all the per-fragment tests. The mechanism of collecting an occlusion query value is described in Occlusion Queries.

The occlusion query sample counter increments by one for each sample with a coverage value of 1 in each fragment that survives all the per-fragment tests, including scissor, exclusive scissor, sample mask, alpha to coverage, stencil, and depth tests.

Fragment Coverage to Color

VkPipelineCoverageToColorStateCreateInfoNVStructure specifying whether fragment coverage replaces a color
VkPipelineCoverageToColorStateCreateFlagsNVReserved for future use
vkCmdSetCoverageToColorEnableNVSpecify the coverage to color enable state dynamically for a command buffer
vkCmdSetCoverageToColorLocationNVSpecify the coverage to color location dynamically for a command buffer

Coverage Reduction

Coverage reduction takes the coverage information for a fragment and converts that to a boolean coverage value for each color sample in each pixel covered by the fragment.

Pixel Coverage

Coverage for each pixel is first extracted from the total fragment coverage mask. This consists of rasterizationSamples unique coverage samples for each pixel in the fragment area, each with a unique sample index. If the fragment only contains a single pixel, coverage for the pixel is equivalent to the fragment coverage.

If the render pass has a fragment density map attachment and the fragment covers multiple pixels, pixel coverage is generated in an implementation-dependent manner. If all samples in the fragment are covered, all samples will be covered in each pixel coverage.

If a shading rate image is used, and the fragment covers multiple pixels, each pixel’s coverage consists of the coverage samples corresponding to that pixel, and each sample retains its unique sample index i.

If the fragment shading rate is set, and the fragment covers multiple pixels, each pixel’s coverage consists of the coverage samples with a pixel index matching that pixel, and each sample retains its unique sample index i.

Color Sample Coverage

Once pixel coverage is determined, coverage for each individual color sample corresponding to that pixel is determined.

If the number of rasterizationSamples is identical to the number of samples in the color attachments, a color sample is covered if the pixel coverage sample with the same sample index i is covered.

Otherwise, the coverage for each color sample is computed from the pixel coverage as follows.

If the VK_AMD_mixed_attachment_samples extension is enabled, for color samples present in the color attachments, a color sample is covered if the pixel coverage sample with the same sample index i is covered; additional pixel coverage samples are discarded.

If the pNext chain of VkSubpassDescription2 or VkRenderingInfo includes a VkMultisampledRenderToSingleSampledInfoEXT structure with the multisampledRenderToSingleSampledEnable field equal to VK_TRUE, sample coverage is calculated as if the attachment has VkMultisampledRenderToSingleSampledInfoEXT::rasterizationSamples samples.

When the VK_NV_coverage_reduction_mode extension is enabled, the pipeline state controlling coverage reduction is specified through the members of the VkPipelineCoverageReductionStateCreateInfoNV structure.

VkPipelineCoverageReductionStateCreateInfoNVStructure specifying parameters controlling coverage reduction
VkPipelineCoverageReductionStateCreateFlagsNVReserved for future use
VkCoverageReductionModeNVSpecify the coverage reduction mode
vkCmdSetCoverageReductionModeNVSpecify the coverage reduction mode dynamically for a command buffer
vkGetPhysicalDeviceSupportedFramebufferMixedSamplesCombinationsNVQuery supported sample count combinations
VkFramebufferMixedSamplesCombinationNVStructure specifying a supported sample count combination

Coverage Modulation

VkPipelineCoverageModulationStateCreateInfoNVStructure specifying parameters controlling coverage modulation
VkPipelineCoverageModulationStateCreateFlagsNVReserved for future use
VkCoverageModulationModeNVSpecify the coverage modulation mode
vkCmdSetCoverageModulationModeNVSpecify the coverage modulation mode dynamically for a command buffer
vkCmdSetCoverageModulationTableEnableNVSpecify the coverage modulation table enable state dynamically for a command buffer
vkCmdSetCoverageModulationTableNVSpecify the coverage modulation table dynamically for a command buffer