Groq LPU (Language Processing Unit)
Overview
Groq LPU (Language Processing Unit) is Groq's proprietary deterministic LLM inference processor, renowned for its extremely low inference latency (LLM token generation measured in milliseconds). It employs a compiler-defined hardware architecture: no cache misses, no out-of-order execution, all latency predictable. GroqCloud provides API services.
Core Specifications (GroqChip v1)
| Item | Spec |
|---|
| Architecture | Tensor Streaming Processor (TSP) |
| Process | TSMC 14nm |
| On-chip SRAM | 228 MB |
| INT8 Compute (est.) | 1,000 TOPS |
| FP16 | 188 TFLOPS (official) |
| Interconnect | GroqSync (deterministic interconnect) |
| TDP | 300 W (system-level) |
| Form Factor | GroqRack (8 GroqChips) |
Key Features
- Deterministic architecture: All execution predictable
- Ultra-low latency: LLM inference 5-10x faster than GPU
- Compiler-driven: Software-defined hardware
- Streaming tensor architecture
Use Cases
- Real-time LLM inference (Chatbot, Code assistant)
- Low-latency AI services
- GroqCloud API (per-token billing)