Nvidia is boosting the power of its synthetic lucidity chip venture with the announcement on Monday of its Blackwell GPU architecture at its first in-person GPU Technology Conference (GTC) in five years.

From a deal with Nvidia, the chip, designed for use in large data centers – the kind that powers companies like AWS, Azure and Google – delivers 20 PetaFLOPS of AI performance, which is 4x faster in AI training workloads. , 30x faster AI-inference workloads, and up to 25x more energy efficient than its predecessor .

Compared to its predecessor , the H100 “Hopper,” the B200 Blackwell is more powerful and energy efficient, Nvidia said. To train an AI standard the size of GPT-4, for example, 8,000 H100 chips and 15 megawatts of power would be needed. This same task would require just 2,000 B200 chips and four megawatts of power.

“ This is the company’s first major advancement in chip design since the debut of the Hopper architecture two years ago,” wrote Bob O’Donnell, founder and chief analyst at Technalysis Research , in his weekly LinkedIn newsletter.

Repacking Training

However, Sebastien Jean, CTO of Phison Electronics , a Taiwanese electronics company, called the chip “a repackaging experiment .”

“It’s good, but it’s not groundbreaking,” he told TechNewsWorld. “It will run faster, use less robustness and allow more computation in a smaller dimension , but from a technologist’s point of view, they just reduced it without actually changing anything fundamental.”

“This means your results are easily replicated by your competitors,” he said. “Although there is value in being first because while the competition catches up, you move on to the next thing .”

“When you force your competition into a permanent catch-up game, unless they have a very potent lead , they will fall into a ‘fast follower’ mentality without realizing it,” he said.

“By being invasive and pioneering,” he continued, “Nvidia can solidify the theory that they are the only true innovators, which further drives demand for their products.”

While Blackwell may be a repackaging trick , he added, he has a real net benefit . “In practical terms, people using Blackwell will be able to perform more calculations faster and with the same power and space budget,” he noted. “This will enable Blackwell-based solutions to outpace and outperform the competition.”

Past tense plug-compatible

O’Donnell said the Blackwell architecture’s second-generation transformer engine is significant progress because it reduces AI floating-point calculations from eight bits to four bits. “In practice, by reducing these 8-bit calculations in previous generations, they can twin the computational performance and model sizes they can support on Blackwell with this single change,” he said.

The new chips are also compatible with their predecessors. “If you already have Nvidia systems with the H100, Blackwell is compatible with plug-ins,” noted Jack E. Gold, founder and chief commentator at J.Gold Associates , an IT consulting firm in Northborough, Massachusetts.

“In theory, you could just unplug the H100s and plug in the Blackwells,” he told TechNewsWorld. “While you can do it theoretically, you may not be able to do it financially.” For example, Nvidia’s H100 chip costs between $30,000 and $40,000 each. While Nvidia hasn’t revealed the price of its new AI chip venture , it will likely be priced in that direction.

Gold added that Blackwell chips could help developers produce better AI applications. “The more data points you can ascertain , the better the AI ​​will be,” he explained. “What Nvidia is talking about with Blackwell is that instead of being able to ascertain billions of data points, you can ascertain trillions.”

Also announced at GTC were Nvidia Inference Microservices (NIM). “NIM tools are built on top of Nvidia’s CUDA platform and will allow companies to bring custom applications and pre-trained AI models into production environments, which should help these companies bring new AI products to market,” Brian Colello, an equity strategist at Morningstar Research Services in Chicago, wrote in a comment note on Tuesday.

Helping to deploy AI

“Large companies with data centers can use new technologies quickly and implement them more quickly, but most humans work in small and medium-sized companies that do not have the resources to purchase, customize and implement new technologies. Anything like NIM that can help them adopt new technologies and deploy them more easily will be a boon to them,” explained Shane Rau, semiconductor commentator at IDC , a global market research firm.

“With NIM, you will find specific templates for what you want to do,” he told TechNewsWorld. “Not everyone wants to do AI universally . They want to do AI that is specifically relevant to their company or enterprise.”

While NIM isn’t as exciting as newer hardware designs, O’Donnell noted that it is significantly more important in the long term for several reasons.

“First,” he wrote, “it should make it faster and more efficient for companies to move from GenAI experiments and POCs (proofs of concepts) to real-world production. There simply aren’t enough data scientists and GenAI programming experts, so many companies that were eager to deploy GenAI were limited by technical challenges. As a result, it’s great to see Nvidia helping facilitate this process.”

“Second,” he continued, “these new microservices enable the generation of a whole new revenue stream and business strategy for Nvidia because they can be licensed per GPU/per hour ( as well as other variations). This could be an important, evergreen, and more diversified means of generating revenue for Nvidia, so even though it’s still early days , it will be important to watch.”

Entrenched leader

Rau predicted that Nvidia will remain entrenched as the AI ​​processing platform of choice for the foreseeable future . “But competitors like AMD and Intel may occupy modest portions of the GPU market,” he said. And because there are different chips you can use for AI – microprocessors, FPGAs and ASICs – these competing technologies will compete for market share and grow.”

“There are very few threats to Nvidia’s dominance in this market,” added Abdullah Anwer Ahmed, founder of Serene Data Ops, a data management company in San Francisco.

“In addition to its superior hardware, its CUDA software solution has been the foundation of the underlying AI threads for more than a dozen ,” he told TechNewsWorld.

“The main threat is that Amazon, Google and Microsoft/OpenAI are working on building their own chips optimized around these models,” he continued. “Google already has its ‘TPU’ chip in production. Amazon and OpenAI have suggested similar projects.”

“In any case, building your own GPUs is an option available only to the largest companies,” he added. “The largest section of the LLM industry will continue to purchase Nvidia GPUs.”