Studies
Holding everything else still
The leaderboard compares everything with everything. Studies compare runs of the same model that differ in one setting, on the same tasks, and say when something else had to change with it (in the quantisation study, the engine).
Quantisation
The same model in different quantisations (10 formats, down to 2.9 bits per weight; no BF16 runs), each on an engine that supports it. What changes when a model runs in fewer bits?
Reasoning effort
Score against effort tier and against output tokens. Where does extra thinking stop paying for itself?
Max effort
The same tasks at max and one level down, under a 32,768-token output budget: how often max runs out of room before answering, and how it does on the tasks it answers.
Pushback
Ask for SOQL in a loop. Which models build the safe version and tell you why, and which just write the loop?
Local vs API
Where the best local model, served on Forcebench’s own hardware, lands among API models, reached through a hosted service: compared at the same reasoning state, with the tokens each spent.
Coding agent
The same tasks in one message, in a coding agent without a Salesforce org, and in the agent with Salesforce's own skills, for 3 open-weight models.
Harness
The same model on the same task, many times, in Claude Code, opencode, pi, with and without Salesforce's skills.
Private evaluations
Want this for your team?
Azul Labs runs Forcebench privately for two kinds of Salesforce team.
Choosing a model to run yourself?
A private benchmark of your shortlist
Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.
- Private tasks written from your work only ever run on local models; they are never sent to an API.
- Re-testing after any change of model, quantisation or engine is available.
- Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
Building or buying a Salesforce AI tool?
A private evaluation of your tool
Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.
- It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
- Repeat evaluations after your releases are available.
- Typical turnaround is 2–3 weeks.
Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.
Or email leo@azl.au · azl.au