Digital IC · RTL · Model compilation

INT8 NPU

4×4 systolic array and ResNet-50 integer inference

I independently designed and implemented the NPU and its TFLite/ONNX compiler. The compiler turns a model into tasks that the NPU fetches and executes. Development is ongoing. The current RTL runs the full ResNet-50 workload in simulation and matches the integer reference at every layer.

4×4
INT8 systolic array
72
hardware tasks, all numerically verified
2.48×
speedup at equal clock frequency vs. serial-loading baseline

System architecture

The host supplies a descriptor base address and count; the NPU fetches and schedules the tasks. GEMM, convolution and fully connected layers share the matrix engine, while pooling and residual addition use dedicated engines. Separate tasks exchange data through DDR.

NPU system architecture: read interconnect, task dispatch, compute engines and shared writeback
Solid lines show data; dashed lines show control and configuration. Select the image for full detail.

Compute and memory

Designed the INT8 MAC array, INT32 accumulation and quantized postprocessing. Tiled scheduling, A/B double buffering and data reuse coordinate transfers, computation and writeback.

Model compilation

Implemented operator mapping, weight layouts, quantization conversion and DDR allocation. ResNet-50 compiles into 72 hardware tasks for autonomous execution by the NPU.

Validation and optimization

Built an independent integer reference model to check each layer and the complete DDR contents. Matched-condition benchmarks measure the gains from double buffering and subsequent optimizations.

ResNet-50 full-network execution cycles

  1. Burst · Serial loading

    1,212,904,532 cycles
  2. Burst · Double buffering

    948,269,129 cycles
  3. Current design

    489,351,345 cycles

Compared with the serial-loading baseline with Burst enabled, the current design uses 59.65% fewer cycles, equivalent to a 2.48× speedup at equal clock frequency.

The tests use ONNX model weights and synthetic INT8 inputs to check integer results and compare RTL execution cycles under matched conditions.

Technical details and validation

Metrics and comparison setup

  • Double buffering and prefetch scheduling reduce cycles by 21.82% relative to the serial-loading baseline.
  • Subsequent optimizations together reduce cycles by a further 48.40% relative to the double-buffered design.
  • Effective throughput is 8.0683 MAC/cycle, with 50.43% utilization over the full run, including data transfers and auxiliary operators.
  • DDR reads fall from 2,093,158,512 to 1,278,406,224 bytes, a 38.92% reduction.

Full-network numerical validation

  • The input is 224×224×3. The 72 tasks comprise 53 Conv, 16 Add, one MaxPool, one GAP and one FC.
  • All three runs pass every layer comparison; the complete 29,404,672-byte DDR image matches the independent NumPy integer reference.
  • All three runs enable Burst and share the model, input, quantization parameters and DDR timing: 64-bit reads, 32-bit writes and a 20-cycle first-beat read delay.
  • The serial and double-buffered baselines were reconstructed from historical code with the functional backports needed for this comparison.

Current compute and memory paths

The current design retains native 64-bit operand reads, A/B double banks, output-stationary accumulation, cross-K input reuse and partial-sum slots. The convolution read cache has been removed. The hardware supports local convolution/pooling fusion; this ResNet-50 workload executes its 72 tasks separately, exchanging data through DDR.

Earlier compute and memory diagram

This diagram records the design before cleanup, including the 2 KiB convolution read cache removed from the current implementation. SRAM depths, widths and partial-sum paths are retained here as an earlier architecture record.

Earlier compute and memory diagram including the convolution read cache removed from the current design
Earlier architecture record · Select for full detail.
Historical benchmarks and DMA debugging

MNIST CNN

An earlier design reduced cycles from 531,563 to 74,527 (85.98%) through input-tile reuse, AXI Burst and word assembly. All five layer outputs matched the software reference for four test images.

CIFAR-10 ResNet-8

Earlier native 64-bit and convolution-cache optimizations reduced cycles from 1,486,468 to 1,255,817 (15.52%). A separate fixed configuration passed 140 layer checks across ten images.

Missed DMA completion pulse

After input reuse was added, an early DMA completion could leave the fully connected layer waiting for a one-cycle completion pulse it had already missed. Latching completion from the request phase until both inputs were ready resolved the timeout; output checks and full-network regression passed.

DFT experiments · Earlier synthesized netlist

Inserted 16 scan chains covering 75,137 flip-flops and ran stuck-at ATPG, achieving 78.58% fault coverage. A separate single-SRAM MBIST experiment passed normal-operation and fault-injection checks.

The run produced 16,502 patterns with 92.19% test coverage. Uncollapsed fault statistics give 50% credit to possibly detected faults: fault coverage uses all 2,747,638 faults, while test coverage excludes 405,530 undetectable faults.

MBIST connects DW_mbist_ctrl and DW_mbist_apg to a 32×64 single-port SRAM. The fault-free memory passes; forcing read-data bit 0 to zero produces a detected fault, with reads and writes to all 32 addresses.

Source on GitHub ↗

ONE QUICK QUESTION · OPTIONAL

What brings you here?

Knowing who visits helps me make this portfolio more useful.

What brings you here?

No name or contact details requested. Answers are counted in aggregate. This browser remembers your choice to avoid asking again.