개요

원천: NVIDIA Technical Blog, Mastering LLM Techniques 날짜: 2026-05-08 지역: US 계층: L2 신뢰도: 0.92

핵심 내용

Memory Bandwidth Saturation: Why Inference is Memory-Bound, Not Compute-Bound

출처: NVIDIA Technical Blog, Mastering LLM Techniques 날짜: 2026-05-08 지역: US 계층: L2 | 깊이: detailed 신뢰도: 0.92 | 논제 정합: 0.92

핵심 지표

H100 GPU: 3TB/s HBM bandwidth vs 1.4PB/s compute (matrix multiply). Inference: 1 multiply per 2 bytes loaded = 1.5TB/s effective demand. GPU idle 50%+ waiting for memory

요약

Roofline model proves inference at low batch size completely memory-bandwidth constrained; GPU compute sits idle while memory system saturated, wasting electrical power

Vibe Coding Economy 정합성

Inference arithmetic intensity (flops/byte) = 1 multiply per 8 bytes (FP32 weights + activations), hitting HBM bandwidth roof at 200-300 TFLOPS vs theoretical 1400 TFLOPS

마스터 논제 점수: 0.92

---

원본: P7_US_005 | 출처 URL: https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/

Vibe Coding Economy 정합성

마스터 논제 점수: 0.92

---

원본 ID: P7_P7_US_005