NVIDIA Making Radical Changes to CUDA After Nearly 20 Years

NVIDIA announced it’s making wholesale changes to the way CUDA operates. It will also transform the way programmers conceive and write AI programs for the company’s red-hot GPUs.

The past was about creating runtimes for standalone GPUs, but now NVIDIA wants to create a single runtime for hundreds of thousands of GPUs.

“We’ve been talking about programming a GPU — how do I get my program on one GPU. And the reality is no one runs on one GPU anymore,” said Stephen Jones, a CUDA architect at NVIDIA, during a technical session at the…