PAC-NoC: A Hierarchical Network-on-Chip for Efficient Parallel Aggregated Communication in Transformer Accelerators

In the era of Transformers and large language models (LLMs), communication bandwidth has superseded compute capability as the dominant architectural bottleneck. These workloads require frequent aggregated multicast, requiring the simultaneous delivery of distinct data streams to multiple processing element (PE) clusters. Existing network-on-chip (NoC) multicast solutions, however, are largely restricted to identical-data broadcasts, necessitating serialized transfers for heterogeneous data and severely degrading efficiency. We present parallel aggregated connection NoC (PAC-NoC), a novel bandwidth-tapered fat tree NoC that introduces aggregated multicast as its core communication primitive. By codesigning three key components, namely source-side operand aggregation (SSOA), deterministic aggregated communication (DAC), and hardwired slicing (HS), PAC-NoC improves bandwidth utilization and bypasses the complex routing and arbitration logic of conventional NoCs. Our results on Transformer benchmarks demonstrate that PAC-NoC reduces latency by up to 52% and energy by up to 52% over traditional baselines. Even against bandwidth-equivalent competitors with optimized dataflow mapping, PAC-NoC provides significant gains, reducing latency by up to 49% and energy by up to 57%.