Vulkan/DX/Metal do provide a gpu-side build/update operations for TLAS & BLAS.

Ideally engine should support most, if not all of them, but there are complications from synchronization and metal.

TLAS

Require update, if any of BLAS was changed, or if new data provided via instance buffer.
Build flag: VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_UPDATE_BIT_KHR

matrix format (vk, dx): matrix is a 3x4 row-major affine transformation matrix. (transposed)
matrix format (metal): matrix is a 4x3 column-major affine transformation matrix.

struct definitions:

// Vulkan
struct VkAccelerationStructureInstanceKHR {
  VkTransformMatrixKHR          transform;
  uint32_t                      instanceCustomIndex:24;
  uint32_t                      mask:8;
  uint32_t                      instanceShaderBindingTableRecordOffset:24;
  VkGeometryInstanceFlagsKHR    flags:8;
  uint64_t                      accelerationStructureReference;
  };

// DX12 (same as vulkan)
struct D3D12_RAYTRACING_INSTANCE_DESC {
  FLOAT                     Transform[3][4];
  UINT                      InstanceID : 24;
  UINT                      InstanceMask : 8;
  UINT                      InstanceContributionToHitGroupIndex : 24;
  UINT                      Flags : 8;
  D3D12_GPU_VIRTUAL_ADDRESS AccelerationStructure;
  };

// MSL; 6.18.5 Indirect Instance Descriptors
struct MTLAccelerationStructureUserIDInstanceDescriptor
{
  MTLPackedFloat4x3 transformationMatrix;
  MTLAccelerationStructureInstanceOptions options;
  uint mask;
  uint intersectionFunctionTableOffset;
  uint accelerationStructureIndex;
  uint userID;
};

BLAS

VkAccelerationStructureGeometryTrianglesDataKHR::transformData is a device or host address to memory containing an optional reference to a VkTransformMatrixKHR

MTLAccelerationStructureTriangleGeometryDescriptor::transformationMatrixBuffer - unspecifyed

Also transposed matrix; described by pointers.

Update command requirements:

• Need to be same:
  • activeness of primitives
  • geometry.triangles.vertexFormat
  • geometry.triangles.maxVertex
  • geometry.triangles.indexType
  • all indices-values of index-buffer
  • presence(!=nullptr) of geometry.triangles.transformData