2D Reshard Architecture

📡 2D Reshard architecture animation: Physical GPU layout + CP by layer × DCP by page → interleaved transfer

PREFILL NODE
8 GPUs (TP=8) → 8 CP Ranks

KV Layout: Layers (rows) × Pages (cols)

Horizontal bands = CP by layer
Each CP rank holds all pages

        filter
p%4=d
→ ⌊p/4⌋
      

→

RDMA
64 tok/page

DECODE NODE
8 GPUs (TP=8) → 4 DCP Ranks

KV Layout: Layers (rows) × Pages (cols)

Vertical bands = DCP by page
Each DCP rank holds all layers

CP0/DCP0

CP1/DCP1

CP2/DCP2

CP3/DCP3

CP4

CP5

CP6

CP7

Animation: 32 paths cycling (8 CP × 4 DCP), highlighting current 2D cross-block + corresponding physical GPUs