“DeepSeek甚至绕过了CUDA”,论文细节再引热议，工程师灵魂提问：英伟达护城河还在吗？关注前沿科技华尔街见闻2025-01-30205324转自公众号：量子位htp://mp.weixn.qq.com/s?bi-MzlzNjcl NzUzMw=&mid-2247775209&dx=1&sn-0d36e2f770c815086947ddlbcb77bbel来源：量子位(ID:QbA)作者梦晨西风英伟达刚刚从DeepSeek-R1引发的4万亿元暴跌中缓过劲来，又面临新的压力？硬件媒体Tom's Hardware带来开年最新热议：DeepSeek甚至绕过了CUDA使用更底层的编程语言做优化。Tom's Hardware@tomshardware·6小时DeepSeek's Al breakthrough bypasses industry-standard CUDA,usesassembly-like PTX programming insteadDeepSeek的AI突破绕过行业标准CUDA,改用类似汇编的PTX编程DeepSeek's Al breakthrough bypasses industry-standard CUDA,uses assemb...来自tomshardware,comt7119439这一次是DeepSeek-V3论文中的更多细节，被人挖掘出来。来自Mirae Asset Securities Research(韩国未来资产证券)的分析称，V3的硬件效率之所以能比Meta等高出10倍，可以总结为“他们从头开始重建了一切”。在使用英伟达的H800cPU训练DeepSeek-V3时，他们针对自己的需求把132个流式多处理器(S临)中的20个修改成负责服务器间的通信，而不是计算任务。变相绕过了硬件对通信速度的限制。selects only 8 routed experts in practice,it can scale up this number to a maximum of 13 experts(4 nodes x 3.2 experts/node)while preserving the same communication cost.Overall,undersuch a communication strategy,only 20 SMs are sufficient to fully utilize the bandwidths of IBand NVLink.In detail,we employ the warp specialization technique(Bauer et al.,2014)and partition20 SMs into 10 communication channels.During the dispatching process,(1)IB sending,(2)IB-to-NVLink forwarding,and(3)NVLink receiving are handled by respective warps.Thenumber of warps allocated to each communication task is dynamically adjusted according to theactual workload across all SMs.Similarly,during the combining process,(1)NVLink sending,(2)NVLink-to-IB forwarding and accumulation,and(3)IB receiving and accumulation are alsohandled by dynamically adjusted warps.In addition,both dispatching and combining kernelsoverlap with the computation stream,so we also consider their impact on other SM computationkernels.Specifically,we employ customized PTX(Parallel Thread Execution)instructions andauto-tune the communication chunk size,which significantly reduces the use of the L2 cacheand the interference to other SMs.A DeepSeek-V3 Technical Report这种操作是用英伟达的PTX(Parallel Thread Execution)语言实现的，而不是CUDA。PTX在接近汇编语言的层级运行，允许进行细粒度的优化，如寄存器分配和Thread/Warp级别的调整。这种编程非常复杂且难以维护，所以行业通用的做法是使用CUDA这样的高级编程语言。换句话说，他们把优化做到了极致。有网友表示，如果有一群人嫌CUDA太慢而使用PTX,那一定是前量化交易员。Michael Bacarella@mbacarella·1月28日Today's Stratechery on DeepSeek is amazing.If there's any group of people in the world that are degenerate enough tosay"CUDA too slow!write PTX!"'it's former quant traders公众号·量字位一位亚马逊工程师提出灵魂质问：CUD是否还是护城河？这种顶尖实验室可以有效利用任何GPU。