Suites / docs

Salesforce docs

Navigating the official Salesforce documentation: limits, allocations and platform behaviour an engineer has to look up while designing a solution. Each answer is a short, exact fact and must cite the developer.salesforce.com page that states it; both the fact and the cited page are checked deterministically.

tasks

Deterministic validators

grading

88%

highest c@3 · Claude Opus 5.5

configurations with results

How it's graded

Each answer is a short, exact fact and must cite the developer.salesforce.com page that states it. Both the fact and the cited page are checked deterministically.

Difficulty mix

5 easy7 medium5 hard

Leaderboard

Who's best at Salesforce docs

Within three attempts (c@3) on this suite only, with 95% confidence intervals. With 17 tasks the intervals are wide: where two overlap, those configurations are not clearly different. 1 finished configuration isn’t shown yet: its retries haven’t finished. Open in the full leaderboard →

Salesforce docs: c@3 by configuration with 95% confidence intervals
#	Model and config	Score c@3 · 95% CI	Tasks
1	Claude Opus 5.5 effort high	88%95% CI 66–97	17
2	Claude Opus 5.5 effort low	82%95% CI 59–94	17
2	GPT-6 Astra effort medium	82%95% CI 59–94	17
4	Claude Opus 5.5 effort medium	76%95% CI 53–90	17
4	Claude Sonnet 5.5 effort high	76%95% CI 53–90	17
4	GPT-6.1 Sol effort high	76%95% CI 53–90	17
7	GPT-6 Astra effort high	65%95% CI 41–83	17
7	Grok 4.7 effort high	65%95% CI 41–83	17
9	Claude Fable 5.1 effort high	59%95% CI 36–78	17
9	Claude Opus 4.8 effort high	59%95% CI 36–78	17
9	Claude Sonnet 5.5 effort low	59%95% CI 36–78	17
12	Claude Opus 4.8 effort medium	53%95% CI 31–74	17
12	GPT-5.6 Sol effort medium	53%95% CI 31–74	17
12	Grok 4.6 effort high	53%95% CI 31–74	17
15	Claude Opus 4.8 effort low	47%95% CI 26–69	17
15	Claude Opus 4.8 thinking off	47%95% CI 26–69	17
15	Gemini 3.8 Flash effort medium	47%95% CI 26–69	17
15	GPT-6 Luna effort medium	47%95% CI 26–69	17
19	Claude Sonnet 5.5 effort medium	41%95% CI 22–64	17
20	MiMo V2.6 Pro FP8 · thinking on	35%95% CI 17–59	17
21	Claude Haiku 5.5 effort medium	29%95% CI 13–53	17
21	DeepSeek V4.1 Flash FP8 · effort high	29%95% CI 13–53	17
23	Claude Haiku 5.5 effort low	24%95% CI 10–47	17
23	Claude Sonnet 5 effort high	24%95% CI 10–47	17
23	Claude Sonnet 5 thinking off	24%95% CI 10–47	17
23	Claude Sonnet 5 effort low	24%95% CI 10–47	17
23	DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high	24%95% CI 10–47	17
23	DeepSeek V4.1 Flash Native FP8, FP4 experts · SGLang · effort high	24%95% CI 10–47	17
23	GLM-5.3 Flash EXL3 4.0bpw experts, 4-bit dense · TensorFold · effort high	24%95% CI 10–47	17
23	GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort low	24%95% CI 10–47	17
23	Inkling Small FP8 · effort high	24%95% CI 10–47	17
23	MiMo V2.6 Flash FP8 · thinking on	24%95% CI 10–47	17
23	Nemotron 3 Ultra thinking on	24%95% CI 10–47	17
23	Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium	24%95% CI 10–47	17
35	Claude Haiku 5.5 effort high	18%95% CI 6–41	17
35	Claude Sonnet 5 effort medium	18%95% CI 6–41	17
35	DeepSeek V4 Pro FP8 · effort high	18%95% CI 6–41	17
35	Mistral Large 4 effort high	18%95% CI 6–41	17
35	Qwen3.8 Flash-Next MLX 4-bit · TensorFold · effort medium	18%95% CI 6–41	17
40	Claude Haiku 4.5 thinking off	12%95% CI 3–34	17
40	DeepSeek V4 Flash Vision experimental FP8 + FP4 experts · vLLM · effort high	12%95% CI 3–34	17
40	GLM-5.3 Flash FP8 · effort high	12%95% CI 3–34	17
40	GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort high	12%95% CI 3–34	17
40	GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off	12%95% CI 3–34	17
40	MiMo V2.6 Flash FP8 + MXFP4 experts · SGLang · thinking on	12%95% CI 3–34	17
40	Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium	12%95% CI 3–34	17
40	Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium	12%95% CI 3–34	17
48	Laguna S 2.1 FP4 · thinking on	6%95% CI 1–27	17
48	Qwen3.8 27B Splash 4-bit · Splash · effort low	6%95% CI 1–27	17
50	Qwen3.8 27B AWQ-INT4 · vLLM · effort medium	3%95% CI 0–23	17
51	Claude Haiku 4.5 thinking on	0%95% CI 0–18	17
51	Gemma 4 26B-A4B NVFP4 · vLLM · thinking on	0%95% CI 0–18	17
51	Gemma 4 31B QAT W4A16 · vLLM · thinking on	0%95% CI 0–18	17
51	Ling 3.0 Flash BF16 · thinking on	0%95% CI 0–18	17
51	Nemotron 3.5 Lightning NVFP4 · thinking on	0%95% CI 0–18	17
51	Qwen3.6 35B-A3B NVFP4 · vLLM · thinking on	0%95% CI 0–18	17
51	Qwen3.8 27B MLX 8-bit · MTPLX · effort low	0%95% CI 0–18	17
51	Qwen3.8 27B MLX 4-bit · MTPLX · effort low	0%95% CI 0–18	17
51	Qwen3.8 27B AWQ-INT4 · vLLM · effort low	0%95% CI 0–18	17
51	Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh	0%95% CI 0–18	17
51	Qwen3.8 Flash-Next thinking on	0%95% CI 0–18	17

Tasks

What's in the suite

Solve rate is the share of finished runs whose first answer to the task passed (pass@1), a rough guide to how hard models find it. Difficulty is the task author’s label; from results is what the solve rate says: easy when two thirds or more of the finished runs pass it, hard at a third or fewer, once five have graded it.

Salesforce docs tasks with difficulty and solve rate
Task	Difficulty	From results	Solve rate
Concurrent long-running API request limitdocs-api-concurrent-long-running	easy	medium	39%
Default Apex callout timeoutdocs-callout-default-timeout	easy	hard	8%
Maximum scratch org durationdocs-scratch-org-max-duration	easy	medium	56%
REST response header with API usagedocs-rest-limit-info-header	easy	hard	18%
Where to enable Lightning Web Securitydocs-lws-enable-setting	easy	hard	23%
After trigger vs after-save flow orderdocs-order-exec-after-trigger-vs-flow	medium	medium	58%
Batch Apex scope maximum with a QueryLocatordocs-batch-querylocator-scope-max	medium	medium	60%
Callout response size limit in async Apexdocs-callout-async-response-size	medium	medium	50%
Default platform event trigger batch sizedocs-pe-trigger-batch-size	medium	hard	26%
Jobs a running Queueable can enqueuedocs-queueable-async-enqueue-limit	medium	medium	50%
Raw CSV size per Bulk API 2.0 ingest jobdocs-bulk2-ingest-upload-size	medium	hard	13%
What LWS uses instead of secure wrappersdocs-lws-distortions	medium	hard	5%
Compute an org's daily API request allocationdocs-api-daily-allocation-calc	hard	hard	15%
Concurrent long-running Apex transactions for a large orgdocs-long-running-apex-concurrency	hard	hard	11%
Cumulative cross-namespace SOQL limitdocs-cross-namespace-soql-limit	hard	hard	27%
Platform event delivery allocation with several subscribersdocs-pe-delivery-allocation-calc	hard	hard	15%
Text length budget in a big object indexdocs-big-object-index-text-length	hard	hard	10%

Private evaluations

Want this for your team?

Azul Labs runs Forcebench privately for two kinds of Salesforce team.

Choosing a model to run yourself?

A private benchmark of your shortlist

Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

Private tasks written from your work only ever run on local models; they are never sent to an API.
Re-testing after any change of model, quantisation or engine is available.
Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.

Get a private benchmark How it works

Building or buying a Salesforce AI tool?

A private evaluation of your tool

Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
Repeat evaluations after your releases are available.
Typical turnaround is 2–3 weeks.

Ask about an evaluation How it works

Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

Or email leo@azl.au · azl.au