Models / glm-5-3
GLM-5.3
Z.ai's model, with open weights. Forcebench has run it in 0 configurations 2 on the lite subset. API is models reached through a hosted service; Local, models served on Forcebench’s own hardware.
By reasoning effort
The same configuration at several efforts, each against its lowest reasoning effort on the same tasks (a paired 95% interval), and charted against the effort tier and against the output tokens it costs. These scores are pass@1, the first answer alone: a paired comparison needs both efforts' answers to the same tasks, and not every effort has had its retries. Lite-subset sweeps (4 tasks per suite) are never plotted with the full benchmark. The effort study has every model.
| Configuration · effort · lite subset | pass@1 95% CI | Against the lowest reasoning effort same tasks, paired | Tokens |
|---|---|---|---|
| EXL3 2.75bpw experts, 4-bit dense · TensorFold · lowbaseline | 32%95% CI 22–42 | — | 1.2k |
| EXL3 2.75bpw experts, 4-bit dense · TensorFold · high | 43%95% CI 33–53 | clearly higher (+11 pts, 95% CI +3 pts to +20 pts) | 7.8k |
Show the numbers
| Model | Effort (tier) | pass@1 | 95% CI | Tasks | No answer | Tokens: mean | median | Reasoning tokens | Latency: median per answer |
|---|---|---|---|---|---|---|---|---|---|
| GLM-5.3 · EXL3 2.75bpw experts, 4-bit dense | low | 32% | 22–42 | 60 | 0% | 1.2k | 535 | 698 | 19 s |
| GLM-5.3 · EXL3 2.75bpw experts, 4-bit dense | high | 43% | 33–53 | 60 | 8% | 7.8k | 2.1k | 4.4k | 72 s |
In the studies
- Reasoning effort: its score against effort and against output tokens.
Private evaluations
Want this for your team?
Azul Labs runs Forcebench privately for two kinds of Salesforce team.
Choosing a model to run yourself?
A private benchmark of your shortlist
Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.
- Private tasks written from your work only ever run on local models; they are never sent to an API.
- Re-testing after any change of model, quantisation or engine is available.
- Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
Building or buying a Salesforce AI tool?
A private evaluation of your tool
Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.
- It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
- Repeat evaluations after your releases are available.
- Typical turnaround is 2–3 weeks.
Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.
Or email leo@azl.au · azl.au