Experimental deep smol lm trained on 23BT using a tinyllama tokenizer
Benchmark Results
Final Summary
| Benchmark | Primary Metric | Score | Accuracy | Normalized Accuracy | Notes |
|---|---|---|---|---|---|
| BananaMind Base Bench 1.1 | Overall Elo | 875.00 | 33.43% | — | Weighted Accuracy: 32.52% |
| ArithMark 3.0 | acc_norm | 33.10% | 32.90% | 33.10% | Official AxiomicLabs runner |
| ARC Easy | acc_norm | 37.21% | 40.07% | 37.21% | lm-eval 0.4.12 · zero-shot |
| ARC Challenge | acc_norm | 22.87% | 17.92% | 22.87% | lm-eval 0.4.12 · zero-shot |
| HellaSwag | acc_norm | 28.38% | 27.60% | 28.38% | lm-eval 0.4.12 · zero-shot |
| PIQA | acc_norm | 60.23% | 59.79% | 60.23% | lm-eval 0.4.12 · zero-shot |
Intelligence Index
INT Index: 10.603
Highlights
- Overall Elo: 875
- BananaMind Weighted Accuracy: 32.52%
- Best benchmark: PIQA — 60.23% normalized accuracy
- ARC Easy: 37.21%
- ArithMark 3.0: 33.10%
- HellaSwag: 28.38%
- ARC Challenge: 22.87%