At a media question and answer session, Nvidia CEO Jensen Huang told reporters that one of the most important things that the company announced Monday was Cosmos — a foundation model that understands the physical world.
Just as GPT was a foundation model that understood language and Stable Diffusion was a foundation model that understood images, Nvidia created a foundation model that understands things like “friction, inertia, gravity, object presence and permanence, geometric and spatial understanding,” Huang said.
“Children know, they understand the physical world, in a way that language models today don’t,” He continued. “We believe that there needs to be a foundation model that understands the physical world. Now, once we create that, all the things that you could do with GPTs and all the things that you can do with Stable Diffusion’s, you can now do with Cosmos.”
A world model like Cosmos is necessary, he said, because “if you want AI to be able to operate, interact in the physical world, sensibly, you’re going to have to have an AI that understands that.”
“Self-driving cars need to understand the physical world. Robotics need to understand the physical world,” he said. “These models are the starting point of enabling all of that.”