clef on Tenstorrent p150x2

Serves Cloudflare/clef by Cloudflare at revision 2f3de3dd, post-trained from Qwen/Qwen3.8-27B. See the original model card for training and evaluation details.

Clef is a 27B-parameter multimodal decision model from Cloudflare: a Qwen3.8-27B backbone (64 layers, 48 Gated DeltaNet and 16 full-attention, hidden size 5120) with its 27-block vision tower, and a 65M-parameter joint schema head. It reads one state (text, JSON, images or video) and a schema of typed questions (choice, score, noul) and returns a probability for every allowed option of every question in one forward pass. It generates no text. This package serves it through the model's own FastAPI server, which implements the Jev/SystemOne API (POST /v1/systemone), so SystemOne clients work against it unchanged. The weights pointer covers everything the server loads: the backbone shards, the vision encoder, the joint schema head (joint_head.safetensors), the tokenizer and processor configuration, and the author's joint_schema_model.py.

Runs on Tenstorrent p150x2 (mesh P150x2).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

Typed decisions over a state of up to 16,384 tokens, with or without images (classification, routing, triage, intent detection, extraction choices, policy and eligibility checks, multiple-choice reading comprehension, judging an answer against stated criteria, matching a caption or label to an image), in workflows that act on confidence with thresholds set on a labelled sample of your own data.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150x2

Quickstart

tt serve tt-hous/clef

tt serve downloads the Docker image and the Cloudflare/clef at 2f3de3dd85f379784083b0814d997ab627200f0c weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve tt-hous/clef.

A browser demo and a command-line client ship in code/models/autoports/cloudflare_clef/demo/ of this repo (inside the image at /opt/tt-metal/models/autoports/cloudflare_clef/demo). With the server up on the port your serve command reported:

hf download tt-hous/clef --include "code/models/autoports/cloudflare_clef/demo/*" --local-dir clef-pkg
cd clef-pkg/code/models/autoports/cloudflare_clef/demo
DEMO_BIND=0.0.0.0 ./run.sh http://<server-host>:<port>

Open http://<demo-host>:8080/#server=http://<server-host>:<port>. The page sends POST /v1/systemone from the browser and draws every option probability: a support-triage preset, the invoice JSON state from the upstream card, a repeated-state preset that shows the prefix cache, two New Yorker cartoons and a synthetic receipt as image requests, and a replay of 24 labelled records (ARC-Challenge, BANKING77, New Yorker) with running accuracy. Against this package every preset answers (8 of 8) and the replay scores 22 of 24.

The CLI needs only Python 3: python3 quickstart.py --all sends every preset, --replay runs the labelled replay, --preset "<title>" sends one, --show-request prints the request bodies without sending them, and --server points it at another host. Presets live in presets.json, shared by the page and the CLI.

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

Request shape

A request has model, state (any string or JSON value), questions (question id to question), and optional images and videos. A question has type (noul, choice or score), optional instructions, and criteria (a mapping of option id to description for choice, a list of ordered option descriptions for score, optional true and false descriptions for noul). Each image is base64-encoded image bytes (a bare string or a data:image/...;base64, URL) or an http(s) URL; each video is a list of frames in the same encodings. No API key is checked unless the server was started with CLEF_API_KEY.

Text example (the Jev/SystemOne example of the upstream model card), with <port> the port your serve command reported:

curl -s http://127.0.0.1:<port>/v1/systemone -H 'content-type: application/json' -d '{
  "model": "clef",
  "state": "Our checkout started returning errors and orders are blocked.",
  "questions": {
    "department": {"type": "choice", "instructions": "Which team should handle the message?",
                   "criteria": {"billing": "Payments or invoices", "technical": "Bugs or outages"}},
    "urgency": {"type": "score", "criteria": ["Can wait", "This week", "Today"]},
    "outage": {"type": "noul", "instructions": "Is a service down?"}
  }
}'

The same request, and an image request, from Python:

import base64
import requests

URL = "http://127.0.0.1:<port>/v1/systemone"

text = requests.post(URL, timeout=300, json={
    "model": "clef",
    "state": "Our checkout started returning errors and orders are blocked.",
    "questions": {
        "department": {"type": "choice", "instructions": "Which team should handle the message?",
                       "criteria": {"billing": "Payments or invoices", "technical": "Bugs or outages"}},
        "urgency": {"type": "score", "criteria": ["Can wait", "This week", "Today"]},
        "outage": {"type": "noul", "instructions": "Is a service down?"},
    },
}).json()
print(text["answers"]["department"]["choice"], text["answers"]["urgency"]["score"], text["answers"]["outage"]["noul"])

with open("receipt.jpg", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()
image = requests.post(URL, timeout=300, json={
    "model": "clef",
    "state": {"task": "Review the attached receipt."},
    "images": [image_b64],
    "questions": {"legible": {"type": "noul", "instructions": "Is the receipt total legible?"}},
}).json()
print(image["answers"]["legible"]["noul"], image["usage"]["input_tokens"], image["latency_ms"])

Response shape

The response body is the one the author's joint_schema_model.systemone() returns (model, answers, usage) plus latency_ms. Answer shapes follow the upstream systemone(): choice has choice, confidence and probabilities; noul has noul (the probability of true); score has score, confidence, legend and probabilities. For the text example above this package answers department = technical (probability 0.9138), urgency score 1.83 (Today at 0.8706) and outage noul 0.9011, with 300 input tokens and about 270 ms of model time; the author's code on CPU (bf16) answers technical (0.9155), 1.82 (Today at 0.8651) and 0.8995 with the same 300 input tokens.

Routes

A state over CLEF_MAX_STATE tokens, or a schema over CLEF_MAX_TAIL tokens, returns 422 with the token count. A request whose questions have no criteria (other than noul) returns 422, as the upstream systemone() raises. In the traced opt-in mode (below) an image grid outside the warm list, or a request with more than one image or video, returns 422.

Server environment

The defaults below are the shipped configuration (eager engine, prefix planner on), the one every number under Expected performance was measured with. tt-model serve has no flag for extra environment variables; to change one, run the image with docker run --env.

Expected performance

Evaluations

Public benchmarks rendered once to SystemOne requests (no prompt tuning, no option dropping) and run against this server on profile p150x2; coverage is complete on every file (0 rejected, 0 truncated, 0 unanswered). The reference figures are the Decision Index results on the Cloudflare/clef model card, whose request renderings are not public.

The same rendering on a stratified 100-item sample of each benchmark, this package against the author's own joint_schema_model.py on CPU in bf16 (same metric as above):

On ARC one argmax differs between the two (reference top-2 margin 0.37); on BANKING77 none (2 near ties).

Kev's labelled suites (hard-v1, devtools-v1 and documents-v1 test splits, SystemOne requests, kev.benchmark --remote, clean block): accuracy, Brier, and ECE (expected calibration error, 10 bins on the top probability); coverage complete. The Cloudflare/clef card publishes no figure for these suites.

The same requests against tt-hous/kev-9b on one p150 score 0.826 / 0.787 / 0.896 accuracy (ECE 0.053 / 0.101 / 0.015); these suites are Kev-9B's own development distribution.

Latency

Profile p150x2 (the only profile). Latency is the server's latency_ms: the model section of the request on its worker thread (state prefill or cache restore, schema continuation, vision tower when images are present, joint schema head on the host), without queue wait and HTTP. "New state" is a prefix-cache miss; "repeated state" is a hit (the KV pages and the Gated DeltaNet state of the slot are restored, then the schema tail runs). A state under 128 tokens has no cached prefix, so the two are equal by construction for short states.

A request costs about 220 ms of fixed work plus about 0.35 s per 1,024-token chunk of prefill; a 16,384-token state (the limit) takes 7,471.8 ms on a miss and 1,600.0 ms on a hit. Over the 8 reference image records (320 to 2,000 patches) the median is 551 ms. The largest image run end to end is 57,344 patches ((1, 224, 256), 3584 x 4096 pixels, 14,336 image tokens): 33.2 s per request warm; a 2048 x 2048 image (16,384 patches, 4,096 tokens) takes 5.7 s. Above 4,096 patches the vision tower time grows about quadratically.

Throughput: 2.36 requests/s at 64 concurrent clients for the 6-question short-state case (256 requests over 64 distinct states per client level at 1, 8, 32 and 64 clients, two passes, the second timed); it is flat in the client count (2.35 to 2.36 requests/s) because requests are served one at a time, and the queue wait grows linearly with the clients. 64 distinct new 2,200-token states give 0.81 requests/s.

Measured on 2026 Oct 05 on two Blackhole chips of one p300 board (profile p150x2, the shipped configuration: eager engine, prefix planner on) with the port's scripts/serving_bench_remote.py over HTTP; latency rows are medians of 20 requests after 2 warm-up requests. Precision: bfp8 weights for the MLP gate and up projections and for the attention and Gated DeltaNet projections, bf16 MLP down weights, bf16 KV cache and embeddings, fp32 Gated DeltaNet recurrent state and decay gate, vision tower in bf16 with fp32 accumulation, joint schema head in fp32 on the host.

Parity against the author's CPU reference

Over HTTP against the served package. Max dp is the largest absolute difference of an option probability; flips are argmax changes at a reference top-2 margin of at least 0.05.

Vision tower parity against the HF model.visual on CPU on the 4 reference images: merged-output PCC 0.99880, 0.99822, 0.99909 and 0.99898; the 4-frame video record 0.99839.

Limitations

• Tested on two Blackhole chips of one p300 board (profile p150x2, the only profile) only. English only: the benchmarks above are English and other languages were not evaluated. The Decision Index of the upstream card was not reproduced; its request data is not public.
• Not for text generation, chat, summarisation or open-ended question answering: the model only scores the options it is given. Not for fully automated decisions with legal, medical, financial, employment or similar consequences for people without human review. The SystemOne permute and separate routes are not served.
• Requests are served one at a time: 2.36 requests/s for short states and 0.81 for new 2,200-token states at any client count, and the queue wait grows linearly with the number of clients (27 s at 64 clients).
• Startup takes 106 s on every start (mesh open 4 s, weights converted from the safetensors and the model built in 100.6 s, one warmup request); no tensor cache is written.
• A state over CLEF_MAX_STATE tokens (16,384, the upstream encode_record limit) or a schema over CLEF_MAX_TAIL (4,096) returns 422 instead of the upstream silent truncation (CLEF_TRUNCATE_STATES=1 opts into truncation). The 262,144-token context the HF config advertises is not served.
• A few confident decisions per thousand move by up to 0.3 in probability against the author's CPU reference: on a 200-record development set 274 of 278 argmaxes agree and accuracy is within 0.36 pp, but two questions move by 0.33 and 0.47 while the reference is confident; over all parity sets there is one argmax flip at a reference margin of at least 0.05 (margin 0.17). One development image record moves by 0.175 with the argmax kept.
• Images run through the on-device vision tower in bf16. The largest image run end to end is 57,344 patches (3584 x 4096 pixels, 14,336 image tokens, 33 s per request); above 4,096 patches the vision tower time grows about quadratically. The first image of a patch grid the box has not seen compiles once (1 to 2 s at up to 2,048 padded rows, 6 to 9 s at 4,096, 26 s at 57,344); the kernel cache under ~/.cache/tt-model/clef/cache persists across container restarts. Images and videos are not accepted together in one request (one media kind per request).
• Traced mode (CLEF_TRACED=1) needs CLEF_VISION_WARM_GRID, refuses image grids outside that list and more than one image or video per request (422), reserves a 1 GiB trace region, and gains 0 to 3 percent.
• The prefix cache holds CLEF_PREFIX_CACHE states (4 at the 16,384-token limit); only the 128-aligned state prefix is cached, the schema always runs, and a state under 128 tokens is never cached. The joint schema head runs on the host CPU over every row of the record, so the host part grows with the state length (a 16,384-token cache hit takes 1,600 ms).
• Option order can change an answer. Knowledge is set by the base model.
• Calibrated probabilities can create unwarranted trust: measure accuracy and calibration on a labelled sample of your own data before setting thresholds, and monitor error rates in production.
• The server is open unless CLEF_API_KEY is set and fetches http(s) image URLs unless CLEF_ALLOW_REMOTE_IMAGES=0; states, images and videos may contain personal or confidential data, so put the server behind your own access control and do not expose it to untrusted networks without a proxy.

Licensing

Cloudflare/clef (backbone, vision encoder, joint schema head and joint_schema_model.py) is released under Apache-2.0; its model card states the base model Qwen/Qwen3.8-27B is Apache-2.0. The serving code under code/ is tt-metal (Apache-2.0) plus the Clef port under models/autoports/cloudflare_clef (Apache-2.0, see its NOTICE), which vendors the joint schema head classes from joint_schema_model.py and reuses serving code from the Apache-2.0 kev project. The browser demo (in code/models/autoports/cloudflare_clef/demo, also inside the image) ships ten New Yorker caption contest cartoons from the jmhessel/newyorker_caption_contest dataset under CC BY 4.0, as its NOTICE states.

Related packages

Upstream model card: https://huggingface.co/Cloudflare/clef. Announcement: https://blog.cloudflare.com/clef-decision-models. Decision Index leaderboard: https://clef-evals.workers-ai-mle.workers.dev. The smaller variant Cloudflare/clef-flash is not packaged. The same serving pattern on one Blackhole chip: tt-hous/kev-9b.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/clef/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: