Hugging Peers

AuroraGPT-Math

☆ Save
by SmallAICreator text-generation transformers apache-2.0
28 downloads 0 likes on Hugging Face updated

Model card

AuroraGPT-Math (700M)

A 707M-parameter chat model by UltraLabs, continuing from AuroraGPT-Qwen-Distill with a dedicated push on real arithmetic and multi-turn context use, while keeping the same chat feel and tool-calling.

What's new vs AuroraGPT-Qwen-Distill

Continued full-parameter SFT (2 epochs, sequence-packed) adding:

  • ~250k procedurally generated, correct-by-construction chain-of-thought math examples (addition/subtraction/multiplication/division/percentages/fractions/order-of-operations/word problems) — every answer computed in Python, so labels are guaranteed correct, teaching the model to follow the algorithm (e.g. distributive-breakdown multiplication) rather than memorize answers.
  • ~8k multi-turn context-use conversations: summarize-that, explain-simpler, compare-two-things, recall-your-own-first-question, incremental list continuation, name/topic correction, topic-switch-and-return.
  • The full original distilled-chat + tool-call + identity/greeting-fix dataset mixed back in for balance, so the new skills don't come at the cost of chat feel.

Measured results (local benchmark vs the prior flagship, 16 closed-book arithmetic Qs)

  • Math: 5/16 → 10/16 (roughly doubled)
  • Tool-calling: 3/3 → 3/3 (zero regression — web_search / calculator / fetch_url all still fire correctly)
  • Chat feel / identity / instruction-following: essentially unchanged

Honest limitations

  • Harder arithmetic is still fragile — multi-digit subtraction with borrowing and 2-digit×2-digit multiplication are still error-prone. Use the calculator tool for anything you need to be exactly right.
  • False-premise correction is unchanged (not targeted this round) — it can still confidently agree with a popular myth rather than catching it.
  • 700M-scale limits still apply: closed-book knowledge is limited by design — pair with web_search/fetch_url for facts outside its training.

Chat format (NOT ChatML)

<|system|>{system}<|end|><|user|>{user}<|end|><|assistant|>{reply}<|end|>

Tool call (model emits): <tool_call>\n{"name": "...", "arguments": {...}}\n</tool_call> Tool result (feed back as a user turn): <|user|><tool_response>\n{result}\n</tool_response><|end|>

Usage (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("SmallAICreator/AuroraGPT-Math")
model = AutoModelForCausalLM.from_pretrained("SmallAICreator/AuroraGPT-Math")
msgs = [{"role": "user", "content": "What is 47 times 23? Show your work."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=100)[0][ids.shape[1]:], skip_special_tokens=True))

On-device (llama.cpp / GGUF)

A ready-to-run Q8_0 GGUF (AuroraGPT-Math.Q8_0.gguf, 753MB) is included, with a tool-capable chat template embedded so mobile GGUF apps show the tool picker.

Made by UltraLabs. EOS token is <|end|>.