Inspur Info has unveiled its YuanNao HC1000 hyperscale AI server, claiming it has diminished large-model inference prices to as little as ¥1 (about USD 0.14) per million tokens—a milestone the corporate says removes a key barrier to large-scale AI agent deployment.
In line with Inspur’s Chief AI Technique Officer Liu Jun, GPU utilization throughout inference sometimes reaches solely 5–10%, far under the 50%+ utilization seen in coaching workloads. The HC1000 addresses this inefficiency by way of a completely symmetric DirectCom ultra-high-speed structure and a hyperscale design that decomposes computing workflows and optimizes useful resource allocation.
Liu stated the brand new structure can enhance single-card MFU (Mannequin FLOPs Utilization) by as much as 5.7×, considerably decreasing inference prices. He pressured that as token consumption grows exponentially, incremental price optimizations will now not suffice. Elementary modifications to computing architectures are required, and value effectivity will develop into a “license to outlive” for AI firms within the coming period.
Supply: liangziwei
Elevate your perspective with NextTech Information, the place innovation meets perception.
Uncover the most recent breakthroughs, get unique updates, and join with a world community of future-focused thinkers.
Unlock tomorrow’s developments at present: learn extra, subscribe to our publication, and develop into a part of the NextTech neighborhood at NextTech-news.com
