Suites / fflib
fflib Enterprise Patterns
Apex Enterprise Patterns with the open-source fflib libraries (fflib-apex-common and fflib-apex-mocks): Selector, Domain, Service and Unit of Work layers, the Application factory class, and unit tests with ApexMocks. Code is deployed to a scratch org with the pinned fflib commits installed and must pass hidden Apex tests; test-writing tasks are graded by mutation testing (the model's tests must pass against the correct code and fail against seeded bugs).
How it's graded
Code is deployed as a check-only validation to a scratch org with the pinned fflib libraries installed and must pass hidden Apex tests. ApexMocks test-writing tasks are graded by mutation testing: the model’s tests must pass against the correct code and fail against seeded bugs.
Difficulty mix
Leaderboard
Who's best at fflib Enterprise Patterns
Within three attempts (c@3) on this suite only, with 95% confidence intervals. With 18 tasks the intervals are wide: where two overlap, those configurations are not clearly different. 1 finished configuration isn’t shown yet: its retries haven’t finished. Open in the full leaderboard →
| # | Model and config | Score c@3 · 95% CI | Tasks |
|---|---|---|---|
| 1 | Claude Fable 5.1 effort high | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 5.5 effort high | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 5.5 effort medium | 100%95% CI 82–100 | 18 |
| 1 | Claude Opus 5.5 effort low | 100%95% CI 82–100 | 18 |
| 1 | Gemini 3.8 Flash effort medium | 100%95% CI 82–100 | 18 |
| 1 | GPT-6 Astra effort high | 100%95% CI 82–100 | 18 |
| 1 | GPT-6.1 Sol effort high | 100%95% CI 82–100 | 18 |
| 8 | Claude Opus 4.8 effort high | 94%95% CI 74–99 | 18 |
| 8 | Claude Opus 4.8 effort medium | 94%95% CI 74–99 | 18 |
| 8 | Claude Opus 4.8 effort low | 94%95% CI 74–99 | 18 |
| 8 | Claude Sonnet 5.5 effort low | 94%95% CI 74–99 | 18 |
| 8 | Claude Sonnet 5.5 effort medium | 94%95% CI 74–99 | 18 |
| 8 | GPT-6 Astra effort medium | 94%95% CI 74–99 | 18 |
| 8 | Grok 4.6 effort high | 94%95% CI 74–99 | 18 |
| 8 | Grok 4.7 effort high | 94%95% CI 74–99 | 18 |
| 16 | Claude Opus 4.8 thinking off | 89%95% CI 67–97 | 18 |
| 16 | Claude Sonnet 5.5 effort high | 89%95% CI 67–97 | 18 |
| 18 | Claude Sonnet 5 effort high | 67%95% CI 44–84 | 18 |
| 18 | Claude Sonnet 5 effort medium | 67%95% CI 44–84 | 18 |
| 18 | Claude Sonnet 5 effort low | 67%95% CI 44–84 | 18 |
| 18 | DeepSeek V4.1 Flash FP8 · effort high | 67%95% CI 44–84 | 18 |
| 22 | Claude Haiku 5.5 effort medium | 61%95% CI 39–80 | 18 |
| 22 | Claude Sonnet 5 thinking off | 61%95% CI 39–80 | 18 |
| 22 | GPT-5.6 Sol effort medium | 61%95% CI 39–80 | 18 |
| 22 | GPT-6 Luna effort medium | 61%95% CI 39–80 | 18 |
| 26 | Claude Haiku 5.5 effort high | 56%95% CI 34–75 | 18 |
| 26 | Claude Haiku 5.5 effort low | 56%95% CI 34–75 | 18 |
| 26 | DeepSeek V4 Pro FP8 · effort high | 56%95% CI 34–75 | 18 |
| 29 | DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high | 50%95% CI 29–71 | 18 |
| 29 | DeepSeek V4.1 Flash Native FP8, FP4 experts · SGLang · effort high | 50%95% CI 29–71 | 18 |
| 29 | Mistral Large 4 effort high | 50%95% CI 29–71 | 18 |
| 32 | Claude Haiku 4.5 thinking on | 44%95% CI 25–66 | 18 |
| 32 | Claude Haiku 4.5 thinking off | 44%95% CI 25–66 | 18 |
| 32 | MiMo V2.6 Flash FP8 + MXFP4 experts · SGLang · thinking on | 44%95% CI 25–66 | 18 |
| 35 | GLM-5.3 Flash FP8 · effort high | 39%95% CI 20–61 | 18 |
| 35 | MiMo V2.6 Pro FP8 · thinking on | 39%95% CI 20–61 | 18 |
| 37 | GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort high | 33%95% CI 16–56 | 18 |
| 37 | GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort low | 33%95% CI 16–56 | 18 |
| 37 | Nemotron 3 Ultra thinking on | 33%95% CI 16–56 | 18 |
| 40 | GLM-5.3 Flash EXL3 4.0bpw experts, 4-bit dense · TensorFold · effort high | 28%95% CI 13–51 | 18 |
| 41 | GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off | 22%95% CI 9–45 | 18 |
| 41 | Ling 3.0 Flash BF16 · thinking on | 22%95% CI 9–45 | 18 |
| 41 | MiMo V2.6 Flash FP8 · thinking on | 22%95% CI 9–45 | 18 |
| 41 | Qwen3.8 Flash-Next MLX 4-bit · TensorFold · effort medium | 22%95% CI 9–45 | 18 |
| 45 | DeepSeek V4 Flash Vision experimental FP8 + FP4 experts · vLLM · effort high | 17%95% CI 6–39 | 18 |
| 45 | Gemma 4 31B QAT W4A16 · vLLM · thinking on | 17%95% CI 6–39 | 18 |
| 45 | Inkling Small FP8 · effort high | 17%95% CI 6–39 | 18 |
| 45 | Laguna S 2.1 FP4 · thinking on | 17%95% CI 6–39 | 18 |
| 45 | Qwen3.6 35B-A3B NVFP4 · vLLM · thinking on | 17%95% CI 6–39 | 18 |
| 45 | Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium | 17%95% CI 6–39 | 18 |
| 45 | Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium | 17%95% CI 6–39 | 18 |
| 45 | Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium | 17%95% CI 6–39 | 18 |
| 45 | Qwen3.8 Flash-Next thinking on | 17%95% CI 6–39 | 18 |
| 54 | Qwen3.8 27B MLX 4-bit · MTPLX · effort low | 11%95% CI 3–33 | 18 |
| 54 | Qwen3.8 27B AWQ-INT4 · vLLM · effort medium | 11%95% CI 3–33 | 18 |
| 56 | Gemma 4 26B-A4B NVFP4 · vLLM · thinking on | 6%95% CI 1–26 | 18 |
| 56 | Nemotron 3.5 Lightning NVFP4 · thinking on | 6%95% CI 1–26 | 18 |
| 56 | Qwen3.8 27B Splash 4-bit · Splash · effort low | 6%95% CI 1–26 | 18 |
| 59 | Qwen3.8 27B MLX 8-bit · MTPLX · effort low | 0%95% CI 0–18 | 18 |
| 59 | Qwen3.8 27B AWQ-INT4 · vLLM · effort low | 0%95% CI 0–18 | 18 |
| 59 | Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh | 0%95% CI 0–18 | 18 |
Tasks
What's in the suite
Solve rate is the share of finished runs whose first answer to the task passed (pass@1), a rough guide to how hard models find it. Difficulty is the task author’s label; from results is what the solve rate says: easy when two thirds or more of the finished runs pass it, hard at a third or fewer, once five have graded it.
| Task | Difficulty | From results | Solve rate |
|---|---|---|---|
| ApexMocks unit test with a stubbed selectorfflib-apexmocks-stub-selector | easy | hard | 27% |
| Contact selector with fflib_SObjectSelectorfflib-selector-contacts | easy | medium | 45% |
| Create accounts and contacts in one unit of workfflib-uow-account-contacts | easy | medium | 47% |
| Opportunity domain class with insert defaultsfflib-domain-opportunity-defaults | easy | medium | 58% |
| Static service facade over the Application service factoryfflib-service-facade | easy | easy | 76% |
| ApexMocks test verifying unit of work interactionsfflib-apexmocks-verify-uow | medium | hard | 27% |
| Application class with the four fflib factoriesfflib-application-factories | medium | medium | 50% |
| Custom selector query with its own ordering and limitfflib-selector-query-factory | medium | hard | 31% |
| Domain trigger logic delegating to a service once per triggerfflib-domain-calls-service | medium | medium | 52% |
| Domain unit test with fflib's mock trigger databasefflib-domain-unit-test-mock-db | medium | hard | 32% |
| Insert and update validation in an fflib domain classfflib-domain-validation | medium | hard | 5% |
| Mockable service using the Application unit of work and selectorfflib-service-case-escalation | medium | easy | 69% |
| ApexMocks test for a service's dependency failuresfflib-apexmocks-exception-paths | hard | hard | 8% |
| ApexMocks test for records created and related through the unit of workfflib-apexmocks-argument-captor | hard | hard | 16% |
| ApexMocks test that mocks a domain and verifies call orderfflib-apexmocks-mock-domain | hard | hard | 0% |
| Mockable domain on the fflib_SObjects structurefflib-domain-sobjects-structure | hard | hard | 16% |
| Publish platform events only after a successful unit of work commitfflib-uow-platform-events | hard | hard | 23% |
| User-mode selector with a field set and a child subselectfflib-selector-user-mode-fieldset | hard | hard | 18% |
Private evaluations
Want this for your team?
Azul Labs runs Forcebench privately for two kinds of Salesforce team.
Choosing a model to run yourself?
A private benchmark of your shortlist
Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.
- Private tasks written from your work only ever run on local models; they are never sent to an API.
- Re-testing after any change of model, quantisation or engine is available.
- Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
Building or buying a Salesforce AI tool?
A private evaluation of your tool
Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.
- It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
- Repeat evaluations after your releases are available.
- Typical turnaround is 2–3 weeks.
Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.
Or email leo@azl.au · azl.au