Private evaluations
Forcebench, run privately for your team
Azul Labs runs Forcebench privately for two kinds of Salesforce team: teams choosing a model to run themselves, and teams building or buying a Salesforce AI tool, with the same harness as the public leaderboard.
Choosing a model to run yourself?
A private benchmark of your shortlist
Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation. It's the offer behind Sovereign AI.
- 1
Shortlist
You and Azul Labs agree a shortlist of models you could realistically own and run, typically three to five, at the quantisations and reasoning efforts that fit your hardware and budget. An API model can join as a baseline on the public tasks.
- 2
Test on your kind of work
Azul Labs runs the Forcebench suites that match what your team does, and can add private tasks written from examples of your real work. Same harness as the public leaderboard: check-only deploys to scratch orgs created for the benchmark, hidden tests, error bars.
- 3
Report back
A written report you can share internally: scores with confidence intervals per suite, speed and token cost, examples of where each model fails, and a clear recommendation of which model, quantisation and effort to run.
- Private tasks written from your work only ever run on local models; they are never sent to an API.
- Re-testing after any change of model, quantisation or engine is available.
- Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
- Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.
Building or buying a Salesforce AI tool?
A private evaluation of your tool
Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.
- It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
- Repeat evaluations after your releases are available.
- Typical turnaround is 2–3 weeks.
- Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.
Or email leo@azl.au with what you're choosing or building, roughly how many models or tools, and your Salesforce areas.