Skip to main content

IBM NorthPole (In-Memory Compute, 2023)

Product Overview​

IBM NorthPole is IBM Research's revolutionary AI inference chip, prototype published in 2023-10-19 Science paper ("Neuromorphic computing at scale", Dharmendra Modha team), 22nm process, 458 TOPS INT8 compute, 75W TDP. Uses an In-Memory Compute architecture, all weights stored in on-chip SRAM + analog compute units, eliminating the von Neumann bottleneck (conventional GPUs spend 99% of power on data movement).

Architectural inspiration: derived from IBM's TrueNorth neuromorphic chip (2014, 54B transistors, 100K neurons), NorthPole is TrueNorth's practical AI version, energy efficiency 25x better than GPUs (IBM official paper data).

Strategic significance: IBM is the leader in in-memory compute + neuromorphic AI. NorthPole is the first to scale in-memory compute to 458 TOPS as a commercially viable AI chip. NorthPole 2 expected 2025 H2 release, 25x efficiency advantage.

Core Specs​

ItemParameter
ArchitectureIBM NorthPole (In-Memory Compute)
ProcessIBM 22nm SOI (Samsung 11nm estimated 2026)
Core Count256 CISC processors (Custom Instruction Set)
SRAM224MB on-chip SRAM (one of the largest AI chip SRAM in industry)
In-Memory Compute1.6B weights + 30B MAC units
Memory Bandwidth2.5 TB/s (on-chip SRAM)
INT8458 TOPS
FP16~110 TFLOPS
TDP75 W
Efficiency6.1 TOPS/W (H100 ~2.16, 3x advantage)
Latency5-10ms (in-memory compute = zero data movement)
Mass Productionnot commercialized (research prototype)
Commercial VersionNorthPole 2 2025 H2 estimated

In-Memory Compute Principle​

DimensionTraditional GPU (von Neumann)IBM NorthPole (In-Memory)
ArchitectureMemory (DRAM/HBM) + compute (GPU) separateMemory + compute fused
Data Movement99% power on data movement0 data movement (compute inside SRAM)
OperationsScalar MAC arraysAnalog / digital hybrid
Energy1x0.04x (25x advantage)
LatencyHBM-limited5-10ms (zero wait)
ReconfigurableCUDA programsNetwork topology config
PrecisionFP64/FP32/FP16/INT8INT8 primarily (analog compute limits)
Drawback-inference only, INT8 limited, training immature

How In-Memory Compute Works​

Traditional GPU:
  Load weights (HBM) -> Load input (HBM) -> MAC (CUDA) -> Store result (HBM)
  Total energy: 100% (99% on data movement)

IBM NorthPole:
  Weights pre-stored in SRAM analog units (immutable)
  Load input (SRAM) -> Analog MAC (inside SRAM) -> Store result (SRAM)
  Total energy: 4% (data movement 0-1%)

Key advantages:

  • 224MB SRAM stores all weights at once (LLM 70B INT8 = 70GB still needs HBM, but small models pure SRAM)
  • 30B analog MAC units computing simultaneously
  • 6.1 TOPS/W (H100 3x efficiency)

256 CISC Processors​

DimensionSpec
ArchitectureCISC (Custom Instruction Set)
Core Count256
Per Core64KB SRAM + 4 analog MAC units
Frequency1.4 GHz
RoleScheduling + activation functions + scalar ops
ISAProprietary (not RISC-V, not ARM)
ProgrammingNeural network topology graph config (TrueNorth-like)

CISC vs RISC: NorthPole doesn't use RISC-V because in-memory compute requires custom instructions for neural topology compilation. TrueNorth -> NorthPole is IBM's 10-year R&D accumulation.

25x Efficiency Source​

FactorEnergy Savings
Data movement reduction20x (vs HBM)
Analog computing3x (vs digital)
SRAM internal compute1.5x (vs registers)
22nm SOI0.8x (vs 5nm digital)
Total25x (IBM paper data)

IBM paper conclusion: NorthPole on ResNet-50 inference, 25x more energy-efficient than NVIDIA H100, 25x faster (same precision).

ResNet-50 Performance (IBM Science Paper)​

DimensionIBM NorthPoleNVIDIA V100NVIDIA H100
Latency5ms8ms2ms
Throughput7,000 images/s5,000 images/s15,000 images/s
Efficiency6.1 TOPS/W0.4 TOPS/W2.16 TOPS/W
Power75W250W700W
PrecisionINT8FP16FP8

NorthPole advantage: 5ms latency 1.6x V100, but 15x efficiency. H100 wins on throughput (FP8 advantage), but NorthPole wins in low-latency + low-power scenarios.

Vendor Information​

ItemContent
CompanyIBM Research
LabIBM Research - Almaden (San Jose, California)
Chief ScientistDharmendra S. Modha (IBM Fellow)
Team100+ IBM Research engineers
PublicationScience 2023-10-19 ("Neuromorphic computing at scale")
Paper Citations200+ (2024-2026)
Commercializationnot commercialized (IBM doesn't sell directly)
Commercial PathIBM Cloud inference service (future) + IP licensing (Samsung 11nm 2026)
CustomersUS DARPA, NASA, Department of Energy
CompetitorsMythic (digital CIM), Syntiant (edge CIM), ChiCore (China)

IBM Neuromorphic AI Evolution​

ProductReleasedTransistorsNeuronsComputePurpose
TrueNorth201454 B100 K-Neuromorphic research
NorthPole2023-10220 Banalog458 TOPS INT8AI inference
NorthPole 22025 H2 estimated-analog1.2 POPS INT8 (estimated)AI inference + training
NorthPole 3 (est.)2027-analog5 POPSGeneral AI

Use Cases​

  • ✅ Low-latency AI inference (5-10ms, ultra-low latency)
  • ✅ Ultra-low-power AI (75W, 3-25x GPU efficiency)
  • ✅ Government/research HPC (US DARPA, NASA, DOE)
  • ✅ Neuromorphic AI research (next-gen AI architecture)
  • ✅ Small model inference (7B-13B <70GB fits 224MB as pure SRAM)
  • ❌ AI training (NorthPole inference only)
  • ❌ Large model training (<224MB SRAM limit)
  • ❌ Commercial purchase (IBM not commercialized)
  • ❌ CUDA compatibility (proprietary ISA)

IBM In-Memory Compute Strategy​

  • IBM Research AI flagship project: Modha team 10-year R&D
  • DARPA funding: SyNAPSE program (2014-2024 $100M+ cumulative)
  • NorthPole 2: 2025 H2 commercial version, Samsung 11nm collaboration
  • AI Cloud service: IBM Cloud integrated NorthPole inference
  • Open-source software: IBM plans to open-source NorthPole compilation stack (PyTorch integration)

Key Features​

  • In-Memory Compute: first 458 TOPS scale in-memory compute
  • 224MB SRAM: largest AI chip SRAM in industry
  • 6.1 TOPS/W: H100 3x efficiency
  • 5ms latency: real-time AI inference
  • 75W TDP: air-cooled deployment
  • Drawbacks: not commercialized, INT8 only, no training support

Neuromorphic AI Big Three​

CompanyProductComputeStatus
IBMNorthPole458 TOPS INT82023 prototype
IntelLoihi 21M neurons2021 neuromorphic research
BrainchipAkida 2200 GOPS INT82023 Edge commercial