Benchmarks
Catalyst vs the Competition
Standard neuromorphic benchmark suite. Fully reproducible. Every result deployable to Catalyst FPGA hardware.
Catalyst N3 Latest
All N3 models use adaptive LIF neurons with surrogate gradient BPTT and cosine LR scheduling.
| Benchmark | Classes | Architecture | Neuron | Float Acc | Params |
|---|---|---|---|---|---|
| SHD | 20 | 700→1536→20 (rec) | adLIF | 91.0% | 3.47M |
| SSC | 35 | 700→1024→512→35 (rec) | adLIF | 76.4% | 2.31M |
| N-MNIST | 10 | Conv2D+LIF→10 | LIF | 99.1% | 691K |
| GSC-12 | 12 | 40→512→12 (rec, S2S) | adLIF | 88.0% | 291K |
| DVS Gesture | 11 | Deep conv+rec | adLIF | 89.0% | ~1.2M |
Catalyst N2
All N2 models deploy to Catalyst N2 FPGA hardware via int16 weight quantization.
| Benchmark | Classes | Architecture | Neuron | Float Acc | Params |
|---|---|---|---|---|---|
| SHD | 20 | 700→512→20 (rec) | adLIF | 84.5% | 759K |
| SSC | 35 | 700→1024→512→35 (rec) | adLIF | 72.1% | 2.31M |
| N-MNIST | 10 | Conv2D+LIF→10 | adLIF | 97.8% | 466K |
| GSC KWS | 12 | 40→512→12 (rec, S2S) | adLIF | 88.0% | 291K |
| MIT-BIH ECG | 5 | 187→128→5 (rec) | adLIF | 90.9% | ~35K |
Catalyst N1
N1 uses basic LIF neurons only (no adaptation). Demonstrates competitive performance through model capacity alone.
| Benchmark | Classes | Architecture | Neuron | Float Acc | Params |
|---|---|---|---|---|---|
| SHD | 20 | 700→1024→20 (rec) | LIF | 90.6% | 1.79M |
| N-MNIST | 10 | Conv2D+LIF→10 | LIF | 99.2% | 466K |
| DVS Gesture | 11 | Deep conv+rec | LIF | 69.7% | ~1.2M |
| GSC-12 | 12 | 40→512→12 (rec, S2S) | LIF | 86.4% | 291K |
Competitive Context
| Benchmark | Catalyst N3 | Catalyst N2 | Catalyst N1 | Loihi 2 |
|---|---|---|---|---|
| SHD | 91.0% | 84.5% | 90.6% | 90.9% |
| SSC | 76.4% | 72.1% | — | 69.8% |
| N-MNIST | 99.1% | 97.8% | 99.2% | — |
| GSC-12 | 88.0% | 88.0% | — | — |
Loihi 2 results from Mészáros et al. 2025 (Table I, best per-dataset).
FPGA Hardware Characterisation
Kria K26 Edge (xczu5ev, 2-core variants, 100 MHz target)
| Processor | LUTs | LUT% | FFs | BRAM | DSP | Fmax | Power |
|---|---|---|---|---|---|---|---|
| N1 | 20,109 | 17.2% | 30,847 | 52.5 (36.5%) | 14 (1.1%) | 100 MHz | 0.642W |
| N2 | 26,431 | 22.6% | 38,666 | 52.5 (36.5%) | 16 (1.3%) | ~97 MHz | 0.688W |
| N3 | 53,420 | 45.6% | 80,395 | 24 (16.7%) | 20 (1.6%) | ~58.5 MHz | 0.867W |
AWS F2 Cloud FPGA (Xilinx VU47P)
| Processor | Tests | Pass Rate | Throughput | Frequency |
|---|---|---|---|---|
| N1 | — | PASS | — | 62.5 MHz |
| N2 | 28/28 | 100% | 8,690 ts/sec | 62.5 MHz |
| N3 | 19/19 | 100% | 14,512 ts/sec | 62.5 MHz |
Methodology
Training
Surrogate gradient BPTT with fast-sigmoid surrogate. AdamW optimizer, cosine annealing LR, gradient clipping at 1.0. Event-drop augmentation for regularization.
Quantization
Float32 → int16 weight quantization. Membrane decay mapped to 12-bit fixed-point. Typical accuracy loss: <1%. Scale: w_hw = round(w_float * threshold_hw / threshold_float).
Deployment
Quantized weights loaded into Neurocore SDK Network. Deployed to Catalyst FPGA via USB/UART or AWS F2 cloud instance. Same test set evaluated on hardware.
Reproduce Everything
git clone https://github.com/catalyst-neuromorphic/catalyst-benchmarks.git
cd catalyst-benchmarks
pip install -e .
# Train any benchmark (auto-downloads data)
python shd/train.py --neuron adlif --hidden 1536 --epochs 200 --device cuda:0 --amp
python ssc/train.py --hidden1 1024 --hidden2 768 --recurrent2 --epochs 70 --device cuda:0 --amp
# Evaluate quantized accuracy
python shd/deploy.py --checkpoint shd_model.pt
# Full hyperparameter sweep
python sweep.py --benchmark shd --device cuda:0