Open-Box: Punching Above the VRAM Weight Class

A reproducible research program for running larger effective models on small GPUs.

Read the paper (PDF)

We present a clean-room, reproducible implementation of native sparse attention (NSA) with custom Triton kernels, gated for correctness against a reference at every step and validated from Ampere (sm_86) to Hopper (sm_90). We report a +53% end-to-end training throughput optimization, a −45% model-VRAM reduction, and an honest performance analysis showing the selection branch dominates the training step — a finding independently corroborated by concurrent NSA-kernel work. The paper situates NSA as the first validated component of a broader program: decouple a model's effective capability from what must reside in GPU VRAM.

Everything needed to rebuild the results is public on GitHub.