mlx-community/Fara1.5-9B-8bit

microsoft/Fara1.5-9B converted to MLX and quantized to 8-bit, for inference on Apple Silicon.

Fara1.5 is a computer-use / web-agent vision-language model built on Qwen3.5 — it reads screenshots and acts on interfaces. The vision path is preserved in this conversion, because for this model class it is the point.

Quantization

Effective bits exceed the requested value because mlx-vlm quantizes only the language model and leaves the vision tower in bf16 by design — 333 vision tensors, none of them quantized. The vision encoder is a small share of the weights but disproportionately sensitive to quantization error.

language_model : quantized (8-bit, group size 64)
vision_tower   : 333 tensors, bf16    <- unquantized

Measured results

All figures measured against the unquantized bf16 source, greedy decoding, on an M2 Pro / 32 GB. At 9B the bf16 control fits in memory, so this is a behavioural comparison, not a weight-level proxy.

Top-1 agreement is 1.000 over 204 teacher-forced tokens — the 8-bit model picked the same argmax token as bf16 on every one. KL divergence is 5.9e-4 nats/token. At this fidelity, quantization is not the limiting factor in output quality.

At 8-bit the model runs 1.9x faster using 1.6x less memory than bf16.

Generation agreement (reported, but not a quality metric)

BLEU against bf16 is not a quality measure. It treats bf16 output as ground truth, so it penalises valid paraphrase. The divergences here are exactly that — both models answer correctly, then differ in trailing filler:

prompt: "Name the capital of Japan in one word."
bf16 : "Tokyo ... I will answer the question directly without any extra content."
8-bit: "Tokyo ... The user asked for the capital of Japan in one word."

Same answer, different tail. That is why task accuracy is measured separately — and it is 8/8 for both.

Vision path

Verified working, not merely present. Given a synthetic UI screenshot with two buttons and a total, the 8-bit model returned:

"two large rectangular buttons side by side: On the left: a "SubmitOrder" button. On the right: a "Cancel" button. Below them, the total amount displayed is 42.50 USD."

Both button labels and the amount read correctly.

What was not measured

No standard benchmarks — no MMLU, GSM8K, ScreenSpot, WebArena, or any agentic evaluation. The accuracy layer is 8 short verifiable prompts, not a benchmark. No blind LLM-judge quality grading was run for this variant. The vision path was verified for correctness on a single synthetic screenshot, not scored on a dataset. If your use case is the full computer-use loop, evaluate on your own tasks.

Usage

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("mlx-community/Fara1.5-9B-8bit")

prompt = apply_chat_template(
    processor, model.config,
    "Describe this screenshot. What buttons do you see?",
    num_images=1,
)
out = generate(model, processor, prompt, image=["screenshot.png"], max_tokens=256)
print(out.text)

Text-only works too — pass num_images=0 and omit image.

Note that stock mlx-lm loads the text path only; use mlx-vlm for image input.

Credits

All credit for the model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. Licensed MIT, as the original. See the original card for intended use and limitations.