Together AI ports ThunderKittens kernels to NVIDIA Vera Rubin NVL72

Together AI said it has extended ThunderKittens to write NVFP4 and FP8 matrix-multiplication kernels for NVIDIA Vera Rubin. The work provides an early technical account of how existing ThunderKittens kernels can be tuned for Rubin, although the company has not identified a public release, tag or commit containing the support.

ThunderKittens is a CUDA-embedded, header-only framework for building GPU kernels with tile-level operations. Its repository and paper describe 16-by-16 tile abstractions, PyTorch-like operations, asynchronous thread-block templates and grid-level scheduling support.

According to Together AI, Rubin retains the Blackwell programming model, allowing existing general matrix multiplication, or GEMM, kernels to run before architecture-specific tuning. The company said its Rubin-specific work widened the matrix-multiply instruction, changed output tiling to reuse B operands, deepened shared-memory pipelines, used Rubin’s B-side collector, separated A and B pipeline rings to release A earlier, and added L2-cache eviction hints.

Hardware details also shaped the port. The announcement cites a 64-byte K step, as many as 576 tensor-memory columns and an oversized 328-KiB shared-memory mode. Together AI said all measurements behind its announcement used CUDA 13.4 on a qualification-sample GPU. No independent replication of its performance results was identified.

For context on the wider Vera Rubin platform rollout, NVIDIA’s specifications describe Vera Rubin NVL72 as a rack configuration with 72 Rubin GPUs and 36 Vera CPUs. Together AI’s post covers kernel work on a qualification-sample GPU, not rack-scale or multi-GPU validation across an NVL72 system. The opened ThunderKittens repository still describes the framework as mainly built and tested for Hopper and Blackwell.