Suites / flow
Flow
Salesforce Flow engineering: record-triggered flows (before-save fast field updates, after-save related-record automation, before-delete), entry conditions and $Record__Prior, decisions and formulas, Custom Error, fault paths, subflows, autolaunched flows called from Apex, invocable Apex actions, scheduled paths, and choosing between flows and Apex triggers. Answers are Flow metadata files (API 67.0) deployed check-only to a clean scratch org, where hidden Apex tests do DML (including 200-record bulk tests that assert on Limits) and check the outcome.
How it's graded
Flow metadata is deployed as a check-only validation to a clean scratch org, where hidden Apex tests do DML (including 200-record bulk tests that assert on governor limits) and check the outcome. Decision-guide questions (flow or Apex?) are multiple choice.
Difficulty mix
Leaderboard
Who's best at Flow
Within three attempts (c@3) on this suite only, with 95% confidence intervals. With 18 tasks the intervals are wide: where two overlap, those configurations are not clearly different. 1 finished configuration isn’t shown yet: its retries haven’t finished. Open in the full leaderboard →
| # | Model and config | Score c@3 · 95% CI | Tasks |
|---|---|---|---|
| 1 | Claude Fable 5.1 effort high | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 4.8 effort high | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 4.8 effort medium | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 4.8 effort low | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 4.8 thinking off | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 5.5 effort medium | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 5.5 effort low | 100%95% CI 82–100 | 18 |
| 1 | Claude Sonnet 5 effort high | 100%95% CI 82–100 | 18 |
| 1 | Claude Sonnet 5.5 effort high | 100%95% CI 82–100 | 18 |
| 1 | GPT-5.6 Sol effort medium | 100%95% CI 82–100 | 18 |
| 1 | GPT-6 Astra effort medium | 100%95% CI 82–100 | 18 |
| 1 | GPT-6 Astra effort high | 100%95% CI 82–100 | 18 |
| 1 | GPT-6.1 Sol effort high | 100%95% CI 82–100 | 18 |
| 1 | Grok 4.6 effort high | 100%95% CI 82–100 | 18 |
| 1 | Grok 4.7 effort high | 100%95% CI 82–100 | 18 |
| 16 | Claude Haiku 5.5 effort high | 94%95% CI 74–99 | 18 |
| 16 | Claude Opus 5.5 effort high | 94%95% CI 74–99 | 18 |
| 16 | Claude Sonnet 5 effort medium | 94%95% CI 74–99 | 18 |
| 16 | Claude Sonnet 5.5 effort low | 94%95% CI 74–99 | 18 |
| 16 | Claude Sonnet 5.5 effort medium | 94%95% CI 74–99 | 18 |
| 16 | GPT-6 Luna effort medium | 94%95% CI 74–99 | 18 |
| 22 | Claude Sonnet 5 thinking off | 89%95% CI 67–97 | 18 |
| 22 | Gemini 3.8 Flash effort medium | 89%95% CI 67–97 | 18 |
| 24 | Claude Haiku 5.5 effort medium | 78%95% CI 55–91 | 18 |
| 24 | Claude Haiku 5.5 effort low | 78%95% CI 55–91 | 18 |
| 24 | Claude Sonnet 5 effort low | 78%95% CI 55–91 | 18 |
| 27 | DeepSeek V4.1 Flash FP8 · effort high | 61%95% CI 39–80 | 18 |
| 27 | DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high | 61%95% CI 39–80 | 18 |
| 29 | DeepSeek V4.1 Flash Native FP8, FP4 experts · SGLang · effort high | 56%95% CI 34–75 | 18 |
| 29 | GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort low | 56%95% CI 34–75 | 18 |
| 31 | Inkling Small FP8 · effort high | 44%95% CI 25–66 | 18 |
| 32 | GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort high | 33%95% CI 16–56 | 18 |
| 33 | GLM-5.3 Flash EXL3 4.0bpw experts, 4-bit dense · TensorFold · effort high | 28%95% CI 13–51 | 18 |
| 33 | MiMo V2.6 Pro FP8 · thinking on | 28%95% CI 13–51 | 18 |
| 33 | Mistral Large 4 effort high | 28%95% CI 13–51 | 18 |
| 33 | Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium | 28%95% CI 13–51 | 18 |
| 37 | GLM-5.3 Flash FP8 · effort high | 22%95% CI 9–45 | 18 |
| 37 | GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off | 22%95% CI 9–45 | 18 |
| 37 | Nemotron 3 Ultra thinking on | 22%95% CI 9–45 | 18 |
| 37 | Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium | 22%95% CI 9–45 | 18 |
| 41 | Qwen3.8 27B AWQ-INT4 · vLLM · effort medium | 19%95% CI 7–42 | 18 |
| 42 | Claude Haiku 4.5 thinking on | 17%95% CI 6–39 | 18 |
| 42 | Claude Haiku 4.5 thinking off | 17%95% CI 6–39 | 18 |
| 42 | DeepSeek V4 Flash Vision experimental FP8 + FP4 experts · vLLM · effort high | 17%95% CI 6–39 | 18 |
| 42 | DeepSeek V4 Pro FP8 · effort high | 17%95% CI 6–39 | 18 |
| 42 | Gemma 4 26B-A4B NVFP4 · vLLM · thinking on | 17%95% CI 6–39 | 18 |
| 42 | Laguna S 2.1 FP4 · thinking on | 17%95% CI 6–39 | 18 |
| 42 | MiMo V2.6 Flash FP8 + MXFP4 experts · SGLang · thinking on | 17%95% CI 6–39 | 18 |
| 42 | MiMo V2.6 Flash FP8 · thinking on | 17%95% CI 6–39 | 18 |
| 42 | Qwen3.8 27B MLX 8-bit · MTPLX · effort low | 17%95% CI 6–39 | 18 |
| 42 | Qwen3.8 27B MLX 4-bit · MTPLX · effort low | 17%95% CI 6–39 | 18 |
| 42 | Qwen3.8 Flash-Next MLX 4-bit · TensorFold · effort medium | 17%95% CI 6–39 | 18 |
| 42 | Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium | 17%95% CI 6–39 | 18 |
| 54 | Gemma 4 31B QAT W4A16 · vLLM · thinking on | 11%95% CI 3–33 | 18 |
| 54 | Ling 3.0 Flash BF16 · thinking on | 11%95% CI 3–33 | 18 |
| 54 | Nemotron 3.5 Lightning NVFP4 · thinking on | 11%95% CI 3–33 | 18 |
| 54 | Qwen3.6 35B-A3B NVFP4 · vLLM · thinking on | 11%95% CI 3–33 | 18 |
| 54 | Qwen3.8 27B AWQ-INT4 · vLLM · effort low | 11%95% CI 3–33 | 18 |
| 54 | Qwen3.8 27B Splash 4-bit · Splash · effort low | 11%95% CI 3–33 | 18 |
| 54 | Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh | 11%95% CI 3–33 | 18 |
| 54 | Qwen3.8 Flash-Next thinking on | 11%95% CI 3–33 | 18 |
Tasks
What's in the suite
Solve rate is the share of finished runs whose first answer to the task passed (pass@1), a rough guide to how hard models find it. Difficulty is the task author’s label; from results is what the solve rate says: easy when two thirds or more of the finished runs pass it, hard at a third or fewer, once five have graded it.
| Task | Difficulty | From results | Solve rate |
|---|---|---|---|
| Autolaunched shipping quote flow called from Apexflow-shipping-quote-autolaunched | easy | medium | 37% |
| Choose the tool for a reminder relative to a record dateflow-choice-close-date-reminder | easy | easy | 100% |
| Choose the tool for a same-record field updateflow-choice-same-record-update | easy | easy | 100% |
| Escalate breakdown cases with a fast field updateflow-case-breakdown-escalation | easy | medium | 37% |
| Welcome-call task for new customer accountsflow-account-customer-welcome-task | easy | medium | 39% |
| Automation when a record is restored from the Recycle Binflow-choice-undelete-restore | medium | easy | 85% |
| Block closing a case that still has open tasksflow-case-close-open-tasks | medium | hard | 31% |
| Call an invocable Apex credit check from a record-triggered flowflow-account-credit-check-action | medium | hard | 21% |
| Close-date reminder on a scheduled pathflow-opportunity-close-reminder-scheduled | medium | hard | 29% |
| Create a delivery project when an opportunity is wonflow-opportunity-won-project | medium | medium | 34% |
| Flag large opportunity amount swings using $Record__Priorflow-opportunity-amount-swing | medium | medium | 39% |
| Set a case SLA due date/time from its priorityflow-case-sla-due | medium | hard | 18% |
| Cascade an account discount to its open opportunities in bulkflow-account-discount-cascade | hard | hard | 21% |
| Fault path that logs failed shipment creationflow-order-shipment-fault-path | hard | hard | 31% |
| Fix a flow that syncs account addresses to contactsflow-fix-account-address-sync | hard | hard | 26% |
| Invocable Apex contact matcher orchestrated by a flowflow-web-signup-contact-match | hard | hard | 27% |
| Record-Triggered Automation decision guide principlesflow-choice-guide-statements | hard | medium | 46% |
| Reusable onboarding subflow called from a record-triggered flowflow-account-onboarding-subflow | hard | hard | 26% |
Private evaluations
Want this for your team?
Azul Labs runs Forcebench privately for two kinds of Salesforce team.
Choosing a model to run yourself?
A private benchmark of your shortlist
Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.
- Private tasks written from your work only ever run on local models; they are never sent to an API.
- Re-testing after any change of model, quantisation or engine is available.
- Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
Building or buying a Salesforce AI tool?
A private evaluation of your tool
Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.
- It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
- Repeat evaluations after your releases are available.
- Typical turnaround is 2–3 weeks.
Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.
Or email leo@azl.au · azl.au