Leaderboard

Which models can actually do Salesforce work?

Two boards, ranked separately: API, models reached through a hosted service, and Local, models served on Forcebench’s own hardware. This is the API board. It opens on each model at its best configuration; All configurations has one row per configuration: a model, the service it was reached through (the vendor’s own API or a third party’s) and the reasoning effort it ran at. Scores are the share of tasks solved, with the uncertainty drawn in.

Benchmark v0.2.0272 tasks in 15 suites27 models41 API configurations4 lite effort-sweep runs not ranked hereUpdated 12 October 2026Download JSON

API: models reached through a hosted service. Local: models served on Forcebench’s own hardware.

Score by model

Each bar is the share of tasks solved within three attempts (c@3), with its 95% interval: after a failed answer, the model was shown what the deploy, tests or tools reported. Pick pass@1, the first answer alone, or one suite under Score. How it's scored

Within 3 attempts (c@3), by model

Bars whose intervals overlap are not clearly different: point at or tap one to highlight which. Ranks marked = are ties: the same score to the whole percent.

95% interval

1Claude Opus 5.5 lowbest of 3 configurations

96%

2GPT-6 Astra mediumbest of 2 configurations

95%

=3GPT-6.1 Sol high tie1 configuration

93%

=3Claude Sonnet 5.5 high tiebest of 3 configurations

93%

5Grok 4.7 high1 configuration

92%

=6Grok 4.6 high tie1 configuration

89%

=6Gemini 3.8 Flash medium tie1 configuration

89%

=8GPT-5.6 Sol medium tie1 configuration

88%

=8Claude Fable 5.1 high tie1 configuration

88%

=8Claude Opus 4.8 medium tiebest of 4 configurations

88%

11GPT-6 Luna medium1 configuration

82%

12DeepSeek V4.1 Flash high1 configuration

81%

13Claude Sonnet 5 highbest of 4 configurations

77%

14Claude Haiku 5.5 mediumbest of 3 configurations

76%

15MiMo V2.6 Pro thinking on1 configuration

71%

16DeepSeek V4 Pro high1 configuration

69%

17GLM-5.3 Flash high1 configuration

65%

=18Inkling Small high tie1 configuration

60%

=18Mistral Large 4 high tie1 configuration

60%

20Nemotron 3 Ultra thinking on1 configuration

59%

21MiMo V2.6 Flash thinking on1 configuration

55%

22Claude Haiku 4.5 thinking onbest of 2 configurations

50%

23Qwen3.8 Flash-Next thinking on1 configuration

44%

24Laguna S 2.1 thinking on1 configuration

37%

25Ling 3.0 Flash thinking on1 configuration

36%

26Nemotron 3.5 Lightning thinking on1 configuration

20%

Not ranked yet, waiting for their retries: Nemotron 3 Super (thinking on). Each is added once its retries finish and it has a score within three attempts.

Show the numbers

Model	Score	95% CI
Claude Opus 5.5 low · best of 3 configurations	96%	93–98
GPT-6 Astra medium · best of 2 configurations	95%	92–97
GPT-6.1 Sol high · 1 configuration	93%	90–95
Claude Sonnet 5.5 high · best of 3 configurations	93%	90–96
Grok 4.7 high · 1 configuration	92%	89–95
Grok 4.6 high · 1 configuration	89%	86–92
Gemini 3.8 Flash medium · 1 configuration	89%	85–92
GPT-5.6 Sol medium · 1 configuration	88%	85–92
Claude Fable 5.1 high · 1 configuration	88%	85–91
Claude Opus 4.8 medium · best of 4 configurations	88%	84–91
GPT-6 Luna medium · 1 configuration	82%	77–86
DeepSeek V4.1 Flash high · 1 configuration	81%	77–85
Claude Sonnet 5 high · best of 4 configurations	77%	73–81
Claude Haiku 5.5 medium · best of 3 configurations	76%	72–81
MiMo V2.6 Pro thinking on · 1 configuration	71%	66–75
DeepSeek V4 Pro high · 1 configuration	69%	65–74
GLM-5.3 Flash high · 1 configuration	65%	60–69
Inkling Small high · 1 configuration	60%	55–66
Mistral Large 4 high · 1 configuration	60%	55–65
Nemotron 3 Ultra thinking on · 1 configuration	59%	54–64
MiMo V2.6 Flash thinking on · 1 configuration	55%	50–60
Claude Haiku 4.5 thinking on · best of 2 configurations	50%	45–55
Qwen3.8 Flash-Next thinking on · 1 configuration	44%	39–49
Laguna S 2.1 thinking on · 1 configuration	37%	32–42
Ling 3.0 Flash thinking on · 1 configuration	36%	32–41
Nemotron 3.5 Lightning thinking on · 1 configuration	20%	16–24

Breakdown by suite

Each suite's score on the page's metric, the same rows in the same order as the chart above: within three attempts (c@3) while the page is on c@3, the first answer (pass@1) when you pick pass@1 under Score. Pick a suite's name to rank the page by it.

c@3 by suite

Darker is higher on light backgrounds, brighter is higher on dark ones; every cell prints its score.

c@3 on each suite, in percent, for each model at its best configuration, in the order of the chart above. A dash: no complete score on that suite.
Model
Claude Opus 5.5low	100	83	94	100	82	100	100	100	100	94	94	100	94	100	94
GPT-6 Astramedium	100	89	100	100	82	94	100	100	100	94	100	100	67	100	94
GPT-6.1 Solhigh	100	89	94	100	76	100	100	100	100	78	94	100	61	100	100
Claude Sonnet 5.5high	100	89	100	100	76	89	100	95	100	94	83	93	78	100	94
Grok 4.7high	100	89	88	100	65	94	100	95	100	94	83	93	83	100	94
Grok 4.6high	100	78	94	100	53	94	100	89	100	100	83	93	61	100	89
Gemini 3.8 Flashmedium	100	89	88	100	47	100	89	95	100	100	78	87	61	100	94
GPT-5.6 Solmedium	95	78	88	100	53	61	100	95	100	94	89	93	78	100	100
Claude Fable 5.1high	100	39	94	85	59	100	100	100	100	94	83	93	83	95	94
Claude Opus 4.8medium	100	94	100	100	53	94	100	95	94	89	83	93	50	100	72
GPT-6 Lunamedium	85	72	88	100	47	61	94	84	100	83	78	87	56	100	89
DeepSeek V4.1 Flashhigh	95	83	88	95	29	67	61	89	100	89	78	87	61	95	94
Claude Sonnet 5high	90	89	76	90	24	67	100	79	100	83	72	80	50	100	56
Claude Haiku 5.5medium	90	89	94	85	29	61	78	95	94	83	67	87	50	95	50
MiMo V2.6 Prothinking on	90	72	71	90	35	39	28	95	94	78	61	80	39	100	89
DeepSeek V4 Prohigh	80	67	88	90	18	56	17	89	100	72	50	87	39	100	89
GLM-5.3 Flashhigh	90	72	82	75	12	39	22	79	94	67	44	80	44	90	78
Inkling Smallhigh	80	56	65	65	24	17	44	84	100	56	56	67	33	95	67
Mistral Large 4high	75	39	82	70	18	50	28	74	94	67	39	67	39	100	61
Nemotron 3 Ultrathinking on	75	61	71	60	24	33	22	79	94	56	39	67	28	95	83
MiMo V2.6 Flashthinking on	90	44	65	55	24	22	17	68	94	56	44	73	11	100	56
Claude Haiku 4.5thinking on	75	44	65	60	0	44	17	79	89	56	33	73	11	70	28
Qwen3.8 Flash-Nextthinking on	30	33	59	40	0	17	11	58	78	44	44	60	11	95	83
Laguna S 2.1thinking on	55	33	41	30	6	17	17	53	67	39	22	47	0	90	39
Ling 3.0 Flashthinking on	75	33	35	15	0	22	11	74	72	44	17	33	11	75	28
Nemotron 3.5 Lightningthinking on	15	33	18	10	0	6	11	47	11	39	11	7	6	65	22

Every number

How to read this. Best per model shows each model's highest-scoring configuration. Models run in more configurations get more chances at a high score, so check the N badge. API configurations only, ranked among themselves. Scores are within three attempts (c@3): after a failed answer, the model is shown what the deploy, tests or tools reported and answers again. Each has a 95% confidence interval. A configuration still waiting for its retries is listed but not ranked yet, and pass@1, the first answer alone, is a choice away under Score. The dot is the score and the bar the interval. Where two configurations’ bars overlap, they are not clearly different; where they don’t, one is clearly above the other. Quantisation is shown where the provider declares it, as for open-weight models, and is Unknown otherwise; providers don’t disclose their engine. Latency includes the provider’s network and queueing. Methodology

Claude Opus 5.5Anthropic · low

low

9693–98

100

1.2k936 · 2.7k

9.8 s

Every configuration of Claude Opus 5.5 on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
Anthropic · low	96%95% CI 93–98	1.2k	best, shown above
Anthropic · high	95%95% CI 93–98	2.4k	not clearly different
Anthropic · medium	95%95% CI 92–97	1.8k	not clearly different

Claude Opus 5.5: every configuration, on every board →

GPT-6 AstraOpenRouter · medium

medium

9592–97

100

846502 · 2k

7.4 s

Every configuration of GPT-6 Astra on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
OpenRouter · medium	95%95% CI 92–97	846	best, shown above
OpenRouter · high	92%95% CI 90–95	1.8k	not clearly different

GPT-6 Astra: every configuration, on every board →

GPT-6.1 SolOpenRouter · high1 configuration1 configuration

high

9390–95

100

1.5k749 · 3.9k

20 s

Claude Sonnet 5.5Anthropic · high

high

9390–96

100

1.8k1.2k · 4.1k

9.3 s

Every configuration of Claude Sonnet 5.5 on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
Anthropic · high	93%95% CI 90–96	1.8k	best, shown above
Anthropic · low	90%95% CI 86–93	1k	not clearly different
Anthropic · medium	88%95% CI 85–92	1.1k	not clearly different

Claude Sonnet 5.5: every configuration, on every board →

Grok 4.7OpenRouter · high1 configuration1 configuration

high

9289–95

100

13k7.4k · 35k

111 s

Grok 4.6OpenRouter · high1 configuration1 configuration

high

8986–92

100

6.2k4.3k · 14k

69 s

Gemini 3.8 FlashOpenRouter · medium1 configuration1 configuration

medium

8985–92

100

7.4k3.9k · 22k

41 s

GPT-5.6 SolOpenRouter · medium1 configuration1 configuration

medium

8885–92

100

1.4k943 · 3.7k

10 s

Claude Fable 5.1Anthropic · high1 configuration1 configuration

high

8885–91

100

2.1k1.2k · 4.9k

18 s

Claude Opus 4.8Anthropic · medium

medium

8884–91

100

1.6k828 · 4.6k

11 s

Every configuration of Claude Opus 4.8 on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
Anthropic · medium	88%95% CI 84–91	1.6k	best, shown above
Anthropic · high	87%95% CI 83–91	2k	not clearly different
Anthropic · low	85%95% CI 81–89	940	not clearly different
Anthropic · thinking off	82%95% CI 78–86	882	not clearly different

Claude Opus 4.8: every configuration, on every board →

GPT-6 LunaOpenRouter · medium1 configuration1 configuration

medium

8277–86

100

1.5k1.1k · 3.4k

18 s

DeepSeek V4.1 FlashOpenRouter · FP8 · high1 configuration1 configurationopen weights

high

8177–85

100

9.6k6.8k · 27k

136 s

Claude Sonnet 5Anthropic · high

high

7773–81

100

2.4k978 · 7.2k

11 s

Every configuration of Claude Sonnet 5 on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
Anthropic · high	77%95% CI 73–81	2.4k	best, shown above
Anthropic · medium	75%95% CI 71–79	1.2k	not clearly different
Anthropic · thinking off	71%95% CI 67–76	919	not clearly different
Anthropic · low	69%95% CI 64–74	730	not clearly different

Claude Sonnet 5: every configuration, on every board →

Claude Haiku 5.5Anthropic · medium

medium

7672–81

2.8k1.7k · 7k

8.7 s

Every configuration of Claude Haiku 5.5 on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
Anthropic · medium	76%95% CI 72–81	2.8k	best, shown above
Anthropic · high	76%95% CI 72–80	4.5k	not clearly different
Anthropic · low	74%95% CI 69–78	2.2k	not clearly different

Claude Haiku 5.5: every configuration, on every board →

MiMo V2.6 ProOpenRouter · FP8 · thinking on1 configuration1 configurationopen weights

thinking on

7166–75

100

14k9.5k · 33k

285 s

DeepSeek V4 ProOpenRouter · FP8 · high1 configuration1 configurationopen weights

high

6965–74

100

11k7.8k · 27k

127 s

GLM-5.3 FlashOpenRouter · FP8 · high1 configuration1 configurationopen weights

high

6560–69

3.5k1.1k · 9k

30 s

=18

Inkling SmallOpenRouter · FP8 · high1 configuration1 configurationopen weights

high

6055–66

100

5.4k3.9k · 12k

58 s

=18

Mistral Large 4OpenRouter · high1 configuration1 configuration

high

6055–65

100

15k10k · 33k

219 s

Nemotron 3 UltraOpenRouter · thinking on1 configuration1 configurationopen weights

thinking on

5954–64

1.8k1k · 3.7k

55 s

MiMo V2.6 FlashOpenRouter · FP8 · thinking on1 configuration1 configurationopen weights

thinking on

5550–60

100

13k6.7k · 33k

244 s

Claude Haiku 4.5Anthropic · thinking on

thinking on

5045–55

5.1k4.6k · 10k

41 s

Every configuration of Claude Haiku 4.5 on this board, against its best
Configuration	Within 3 attempts c@3 · 95% CI	Tokens	Against the best 95% intervals
Anthropic · thinking on	50%95% CI 45–55	5.1k	best, shown above
Anthropic · thinking off	46%95% CI 41–52	1.1k	not clearly different

Claude Haiku 4.5: every configuration, on every board →

Qwen3.8 Flash-NextOpenRouter · thinking on1 configuration1 configurationopen weights

thinking on

4439–49

21k22k · 33k

610 s

Laguna S 2.1OpenRouter · FP4 · thinking on1 configuration1 configurationopen weights

thinking on

3732–42

3.4k522 · 10k

30 s

Ling 3.0 FlashOpenRouter · BF16 · thinking on1 configuration1 configurationopen weights

thinking on

3632–41

4.9k1.9k · 11k

101 s

Nemotron 3.5 LightningOpenRouter · NVFP4 · thinking on1 configuration1 configurationopen weights

thinking on

2016–24

6.6k5k · 14k

447 s

–

Nemotron 3 SuperOpenRouter · thinking onopen weightsretries pending

thinking on

—

2.6k1.6k · 5.3k

—

score and 95% interval suite cells tinted by scoreTokens: mean output tokens per task, with the median and 90th percentile under it · Latency: median seconds per task, every attempt the score countsc@3: solved within three attempts, each retry shown what the deploy, tests or tools reported · Fixed: of the answers that failed the first attempt, how many a later attempt fixed (exactly how)partial not every suite run yet, so not ranked

Why intervals matter

A few tasks going one way or the other can move a score by several points. The interval shows how much, so a two-point gap between configurations isn't sold as a win.

Methodology →

Does quantisation hurt?

The same model in different quantisations (10 formats, down to 2.9 bits per weight; no BF16 runs), side by side, with the interval on each.

Quantisation study →

Is more thinking worth it?

Score against reasoning effort, and against the output tokens that effort costs.

Effort study →

Private evaluations

Want this for your team?

Azul Labs runs Forcebench privately for two kinds of Salesforce team.

Choosing a model to run yourself?

A private benchmark of your shortlist

Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

Private tasks written from your work only ever run on local models; they are never sent to an API.
Re-testing after any change of model, quantisation or engine is available.
Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.

Get a private benchmark How it works

Building or buying a Salesforce AI tool?

A private evaluation of your tool

Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
Repeat evaluations after your releases are available.
Typical turnaround is 2–3 weeks.

Ask about an evaluation How it works

Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

Or email leo@azl.au · azl.au