Qualified on a Real GPU: How weightroom Proves a Model Before It Serves One
October 22, 2026
A model that works in a notebook hasn't shown that it works as a service. Before weightroom serves an open-weight model to anyone's API key, it has to pass a gate on the real hardware it will run on. This week one did: Qwen3-4B-Instruct-2507 on an A100 80GB, from allocation to a customer key answered through the gateway, and then a clean delete.

One run, seven checks, all passed.
What “qualified” means here
Qualified is a measured claim, not a label. The gate has four parts, and a model has to clear all of them in the same run.
| Check | What it requires |
|---|---|
| Cold start | Boot from an empty cache. Every model, tokenizer and chat-template file is hashed against the pinned revision before the server starts. |
| Warm restart | Restart the serving process in the same container and come back on a fresh boot, without a second GPU. |
| Measured pass | Replay a fixed dataset through the real private path: at least 12 requests, at least 95% field accuracy, zero errors and zero validation mismatches, p95 first token under 2 s, p95 request under 15 s, GPU memory under 95%. A second request sent while one is streaming must be refused with HTTP 429. |
| Telemetry | The GPU and memory samples collected during startup and the test are revalidated against the saved report after the run. |
Only then does an operator publish the qualification. The controller verifies the worker's identity before it publishes a route, and the gateway serves nothing that isn't published.
The run

Allocation to deletion took 19 minutes and 31 seconds of pod time.
The cold start took 132 seconds, including downloading and hash-checking the model. The warm restart took 60 seconds. The measured pass took 73 seconds for 20 requests. Twenty-one seconds after the operator published, the controller had verified the worker and the route was live.
What the A100 measured

Qwen3-4B-Instruct-2507, vLLM 0.22.1, bf16, one GPU, one request at a time.
The test is invoice extraction: four synthetic invoices (a US software invoice, a German supplier invoice, a noisy scanned print-shop invoice and a consulting invoice with a deliberate error), five measured runs each.
- Accuracy: 100% of fields correct, 0 errors across 20 requests.
- Latency: p50 2,334 ms and p95 2,644 ms per request, with the first token at a p95 of 69 ms.
- Throughput: about 140 output tokens per second.
- GPU memory: peak 84% of the card.
- Cost: the measured window cost $0.024, or about $1.21 per 1,000 requests at one request in flight on a $1.80/hour pod. That is a floor-case measurement for this test, not a price.
Nulls are an answer
The scanned print-shop invoice prints no currency anywhere. A small model will happily guess one from the vendor's address. weightroom's extraction contract says a value has to appear in the document, so a currency the document never prints, as a symbol, an ISO code or a currency name, comes back as null, and the reviewer is asked to confirm it. If your workflow posts invoices to a ledger, a null you can see beats a USD you can't trust.
A customer key, through the gateway

Every response was the designed one.
A project key with the generation:invoke scope got an answer from the live GPU. A missing key, an invalid key and a revoked key were all rejected with a 401, and a request for more output than the qualified limit was refused with a 400. When the owner deleted the endpoint, the pod, its secret, its network key and its network node were gone in 43 seconds, and the provider showed no live pods.
The public endpoint is https://api.weightroom.co. Cloudflare fronts it, the origin accepts only Cloudflare, and only the documented customer paths reach the gateway.
What this is, and what it isn't
This was a bounded pilot, and we'd rather be exact about it:
- One model, one GPU, one provider (RunPod, Secure Cloud), and four synthetic documents. It shows the gate works end to end, not that the model fits your documents. We'd run your representative set before recommending it.
- The gateway test was sent over the gateway's local listener on the origin host with the same code that serves
api.weightroom.co.
Getting to a clean run took a dozen GPU allocations and some unglamorous fixes, including a wrong pinned checksum and a receipt-hashing mismatch. Those belong in our engineering notes, not in the pitch. What we publish here is what the final run measured.
What's next
If you have a workflow in mind, such as invoice fields, ticket routing or note summaries, the next step is to measure it. We'll run your documents through the same gate and tell you what it does and doesn't do before you commit.
See what weightroom does for private AI, or get setup help.
Update · October 22, 2026
The managed Qwen3 4B US plan is available at $1,899/month per endpoint on one NVIDIA A100 SXM4 80 GB in US-MD-1, United States, subject to current qualified capacity. Review the plan and deploy from your team dashboard.
Explore GPU Workspaces.
Choose a pre-configured environment for development, training, or inference.