When Nvidia began to disclose details about its new 4-bit floating point format — NVFP4 — earlier this year, it stated that while it is mainly designed for inference, it could also be used for AI training without significant loss in accuracy. Recently, the company released a paper describing how it managed to train a 12-billion-parameter model on a 10-trillion-token dataset using the NVFP4 format, with several supporting techniques, and achieved results that closely match those of an FP8 baseline.