Models / glm-5-3-flash

GLM-5.3 Flash

Z.ai's model, with open weights. Forcebench has run it in 5 configurations on the API and Local boards, plus 6 on the lite subset. API is models reached through a hosted service; Local, models served on Forcebench’s own hardware.

On the API board

Models reached through a hosted service. GLM-5.3 Flash is ranked by its one configuration: the 17th-highest score of 26 models, not clearly different from 5 other models. Scores are within 3 attempts (c@3) with 95% intervals; each configuration is compared with the highest-scoring one by whether their 95% intervals overlap. Every configuration on the board → Cost and speed →

65%
within 3 attempts (c@3), 95% CI 60–69 · high · 17th-highest score of 26 models, not clearly different from 5 other models
3.5k
output tokens per task · median 1.1k, p90 9k
30 s
median per task, every attempt counted, including network and queueing · 90th percentile 457 s
1%
no answer within the 32,768-token budget
$0.0043
per task at the list price of 10 October 2026, every attempt counted
$0.0066
per solved task: the total cost over the tasks solved · cost and speed
Every configuration of GLM-5.3 Flash on the API board
ConfigurationRank all configsWithin 3 attempts c@3 · 95% CITokens mean · median · p90No answerLatency median per task, every attempt countedCost per taskCost per solved task
OpenRouter · FP8 · highhighest score30
65%95% CI 60–69
3.5k1.1k · 9k1%30 s$0.0043$0.0066
Suites, at its highest-scoring configuration
high · OpenRouter · FP8: within 3 attempts (c@3) on each suite with 95% intervals, against the median of the API board's models, each at its highest-scoring configuration.
  • 95% interval
  • API board median
Apex
90%, 95% interval 70–97%
Salesforce APIs
72%, 95% interval 49–88%
CI/CD
82%, 95% interval 59–94%
Salesforce CLI
75%, 95% interval 53–89%
Salesforce docs
12%, 95% interval 3–34%
fflib Enterprise Patterns
39%, 95% interval 20–61%
Flow
22%, 95% interval 9–45%
Governor limits & pushback
79%, 95% interval 57–91%
Lightning Web Components
94%, 95% interval 74–99%
Nonprofit Success Pack
67%, 95% interval 44–84%
Packaging
44%, 95% interval 25–66%
Permissions & access
80%, 95% interval 55–93%
Scratch org definitions
44%, 95% interval 25–66%
SOQL
90%, 95% interval 70–97%
Trigger Actions Framework
78%, 95% interval 55–91%
    Show the numbers
    Per-suite scores of GLM-5.3 Flash's highest-scoring configuration on the API board, with the board's median
    Suitec@3 95% CIAPI medianTasks
    Apex
    90%95% CI 70–97
    		90%20
    Salesforce APIs
    72%95% CI 49–88
    		72%18
    CI/CD
    82%95% CI 59–94
    		88%17
    Salesforce CLI
    75%95% CI 53–89
    		90%20
    Salesforce docs
    12%95% CI 3–34
    		29%17
    fflib Enterprise Patterns
    39%95% CI 20–61
    		61%18
    Flow
    22%95% CI 9–45
    		69%18
    Governor limits & pushback
    79%95% CI 57–91
    		89%19
    Lightning Web Components
    94%95% CI 74–99
    		100%18
    Nonprofit Success Pack
    67%95% CI 44–84
    		81%18
    Packaging
    44%95% CI 25–66
    		69%18
    Permissions & access
    80%95% CI 55–93
    		87%15
    Scratch org definitions
    44%95% CI 25–66
    		50%18
    SOQL
    90%95% CI 70–97
    		100%20
    Trigger Actions Framework
    78%95% CI 55–91
    		86%18

    On the Local board

    Models served on Forcebench’s own hardware. GLM-5.3 Flash is ranked by the highest-scoring of its 4 configurations: the second-highest score of 9 models, not clearly different from 2 other models. Scores are within 3 attempts (c@3) with 95% intervals; each configuration is compared with the highest-scoring one by whether their 95% intervals overlap. Every configuration on the board →

    64%
    within 3 attempts (c@3), 95% CI 59–68 · high · second-highest score of 9 models, not clearly different from 2 other models
    3.4k
    output tokens per task · median 1.1k, p90 10k
    86 s
    median per task, every attempt counted · 90th percentile 1817 s
    2%
    no answer within the 32,768-token budget
    Every configuration
    Within 3 attempts (c@3) with 95% intervals, highest first. Bars whose intervals overlap are not clearly different.
    • 95% interval
    high EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3highest score
    64%, 95% interval 59–68%
    low EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3
    60%, 95% interval 55–66%
    high EXL3 4.0bpw experts, 4-bit dense · TensorFold · 2× NVIDIA DGX Spark (GB10, 128 GB each)
    54%, 95% interval 49–59%
    thinking off EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3
    50%, 95% interval 44–55%
      Show the numbers
      Every configuration of GLM-5.3 Flash on the Local board
      ConfigurationRank all configsWithin 3 attempts c@3 · 95% CIAgainst the highest by 95% interval overlapTokens mean · median · p90No answerLatency median per task, every attempt counted
      EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · highhighest score3
      64%95% CI 59–68
      		—3.4k1.1k · 10k2%86 s
      EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · low4
      60%95% CI 55–66
      		not clearly different733401 · 1.9k0%36 s
      EXL3 4.0bpw experts, 4-bit dense · TensorFold · high · 2× NVIDIA DGX Spark (GB10, 128 GB each)6
      54%95% CI 49–59
      		not clearly different4k1.2k · 10k2%58 s
      EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off10
      50%95% CI 44–55
      		clearly lower2.3k644 · 4.4k0%62 s

      Latency depends on the hardware: compare it only between configurations on the same hardware.

      Suites, at its highest-scoring configuration
      high · EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3: within 3 attempts (c@3) on each suite with 95% intervals, against the median of the Local board's models, each at its highest-scoring configuration.
      • 95% interval
      • Local board median
      Apex
      90%, 95% interval 70–97%
      Salesforce APIs
      56%, 95% interval 34–75%
      CI/CD
      82%, 95% interval 59–94%
      Salesforce CLI
      85%, 95% interval 64–95%
      Salesforce docs
      12%, 95% interval 3–34%
      fflib Enterprise Patterns
      33%, 95% interval 16–56%
      Flow
      33%, 95% interval 16–56%
      Governor limits & pushback
      89%, 95% interval 69–97%
      Lightning Web Components
      94%, 95% interval 74–99%
      Nonprofit Success Pack
      83%, 95% interval 61–94%
      Packaging
      39%, 95% interval 20–61%
      Permissions & access
      67%, 95% interval 42–85%
      Scratch org definitions
      39%, 95% interval 20–61%
      SOQL
      95%, 95% interval 76–99%
      Trigger Actions Framework
      56%, 95% interval 34–75%
        Show the numbers
        Per-suite scores of GLM-5.3 Flash's highest-scoring configuration on the Local board, with the board's median
        Suitec@3 95% CILocal medianTasks
        Apex
        90%95% CI 70–97
        		70%20
        Salesforce APIs
        56%95% CI 34–75
        		44%18
        CI/CD
        82%95% CI 59–94
        		53%17
        Salesforce CLI
        85%95% CI 64–95
        		45%20
        Salesforce docs
        12%95% CI 3–34
        		12%17
        fflib Enterprise Patterns
        33%95% CI 16–56
        		17%18
        Flow
        33%95% CI 16–56
        		17%18
        Governor limits & pushback
        89%95% CI 69–97
        		84%19
        Lightning Web Components
        94%95% CI 74–99
        		89%18
        Nonprofit Success Pack
        83%95% CI 61–94
        		61%18
        Packaging
        39%95% CI 20–61
        		39%18
        Permissions & access
        67%95% CI 42–85
        		60%15
        Scratch org definitions
        39%95% CI 20–61
        		17%18
        SOQL
        95%95% CI 76–99
        		90%20
        Trigger Actions Framework
        56%95% CI 34–75
        		44%18

        By reasoning effort

        The same configuration at several efforts, each against its lowest reasoning effort on the same tasks (a paired 95% interval), and charted against the effort tier and against the output tokens it costs. These scores are pass@1, the first answer alone: a paired comparison needs both efforts' answers to the same tasks, and not every effort has had its retries. Lite-subset sweeps (4 tasks per suite) are never plotted with the full benchmark. The effort study has every model.

        GLM-5.3 Flash by reasoning effort, full benchmark: each effort against the sweep's lowest reasoning effort
        Configuration · effort · full benchmarkpass@1 95% CIAgainst the lowest reasoning effort same tasks, pairedTokens
        EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off
        34%95% CI 29–39
        		no clear difference (±0 pts, 95% CI −6 pts to +5 pts); this data can't rule out a change of up to 6 points2.3k
        EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · lowbaseline
        34%95% CI 29–40
        		—733
        EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · high
        43%95% CI 38–48
        		clearly higher (+9 pts, 95% CI +3 pts to +15 pts)3.4k
        Score by effort tier · full benchmark
        Overall pass@1; vertical bars are 95% confidence intervals.
        GLM-5.3 Flash, full benchmark: pass@1 by reasoning effort tierGLM-5.3 Flash · EXL3 4.0bpw experts, FP8 dense: off 34%, low 34%, high 43%0%25%50%75%100%Thinking offLowHighReasoning effort tierpass@1GLM-5.3 Flash · EXL3 4.0bpw…
        Show the numbers
        ModelEffort (tier)pass@195% CITasksNo answerTokens: meanmedianLatency: median per answer
        GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 densethinking off34%29–392720%2.3k64438 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 denselow34%29–402720%73340123 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 densehigh43%38–482722%3.4k1.1k64 s
        Score against output tokens · full benchmark
        Each point is one effort setting. Further right means more output tokens per task. The numbers are under the chart above.
        GLM-5.3 Flash, full benchmark: pass@1 against mean output tokens per taskGLM-5.3 Flash · EXL3 4.0bpw experts, FP8 dense: 2.3k tokens 34%, 733 tokens 34%, 3.4k tokens 43%0%25%50%75%100%1k2kMean output tokens per task (log scale)pass@1lowthinking offhighGLM-5.3 Flash · EXL3 4.0bpw…
        GLM-5.3 Flash by reasoning effort, lite subset: each effort against the sweep's lowest reasoning effort
        Configuration · effort · lite subsetpass@1 95% CIAgainst the lowest reasoning effort same tasks, pairedTokens
        EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off
        33%95% CI 23–43
        		no clear difference (+1 pt, 95% CI −10 pts to +13 pts); this data can't rule out a change of up to 13 points2.6k
        EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · lowbaseline
        32%95% CI 23–40
        		—710
        EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · max
        35%95% CI 25–45
        		no clear difference (+3 pts, 95% CI −8 pts to +15 pts); this data can't rule out a change of up to 15 points15k
        EXL3 4.0bpw experts, 4-bit dense · TensorFold · thinking off
        23%95% CI 15–32
        		no clear difference (−4 pts, 95% CI −13 pts to +7 pts); this data can't rule out a change of up to 13 points3k
        EXL3 4.0bpw experts, 4-bit dense · TensorFold · lowbaseline
        27%95% CI 17–37
        		—1.5k
        EXL3 4.0bpw experts, 4-bit dense · TensorFold · max
        33%95% CI 25–43
        		no clear difference (+6 pts, 95% CI −3 pts to +17 pts); this data can't rule out a change of up to 17 points16k
        Score by effort tier · lite subset
        Overall pass@1; vertical bars are 95% confidence intervals.
        • GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 dense
        • GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit dense
        GLM-5.3 Flash, lite subset: pass@1 by reasoning effort tierGLM-5.3 Flash · EXL3 4.0bpw experts, FP8 dense: off 33%, low 32%, max 35%; GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit dense: off 23%, low 27%, max 33%0%25%50%75%100%Thinking offLowMaxReasoning effort tierpass@1GLM-5.3 Flash · EXL3 4.0bpw…GLM-5.3 Flash · EXL3 4.0bpw…
        Show the numbers
        ModelEffort (tier)pass@195% CITasksNo answerTokens: meanmedianReasoning tokensLatency: median per answer
        GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 densethinking off33%23–43600%2.6k682—39 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 denselow32%23–40600%710442—25 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 densemax · vendor default35%25–456027%15k12k—688 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit densethinking off23%15–32600%3k860—37 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit denselow27%17–37602%1.5k44645017 s
        GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit densemax · vendor default33%25–436028%16k13k6.5k570 s
        Score against output tokens · lite subset
        Each point is one effort setting. Further right means more output tokens per task. The numbers are under the chart above.
        • GLM-5.3 Flash · EXL3 4.0bpw experts, FP8 dense
        • GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit dense
        GLM-5.3 Flash, lite subset: pass@1 against mean output tokens per taskGLM-5.3 Flash · EXL3 4.0bpw experts, FP8 dense: 2.6k tokens 33%, 710 tokens 32%, 15k tokens 35%; GLM-5.3 Flash · EXL3 4.0bpw experts, 4-bit dense: 3k tokens 23%, 1.5k tokens 27%, 16k tokens 33%0%25%50%75%100%1k2k5k10k20kMean output tokens per task (log scale)pass@1lowthinking offmaxlowthinking offmaxGLM-5.3 Flash · EXL3 4.0bpw…GLM-5.3 Flash · EXL3 4.0bpw…

        By quantisation

        The same effort in different quantisations, each against the highest-precision one on the same tasks (a paired 95% interval), on pass@1, the first answer alone. Where the engine changes too, a difference can't be put down to quantisation alone. The quantisation study has every model.

        Each build at high
        Overall pass@1 with 95% intervals, labelled with quantisation, engine and hardware. Intervals that overlap are not clearly different.
        • 95% interval
        EXL3 4.0bpw experts, FP8 dense vLLM + ExLlamaV3reference
        43%, 95% interval 38–48%
        EXL3 4.0bpw experts, 4-bit dense TensorFold · 2× NVIDIA DGX Spark (GB10, 128 GB each)
        37%, 95% interval 32–42%
          Show the numbers
          Buildpass@195% CITokens
          EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV343%38–483.4k
          EXL3 4.0bpw experts, 4-bit dense · TensorFold · 2× NVIDIA DGX Spark (GB10, 128 GB each)37%32–424k
          GLM-5.3 Flash at high by quantisation
          Quantisation · engine · highpass@1 95% CIAgainst EXL3 4.0bpw experts, FP8 dense same tasks, pairedTokens
          EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3reference
          43%95% CI 38–48
          		—3.4k
          EXL3 4.0bpw experts, 4-bit dense · TensorFold · 2× NVIDIA DGX Spark (GB10, 128 GB each)
          37%95% CI 32–42
          		clearly lower (−6 pts, 95% CI −12 pts to −0.7 pts)4k

          In the studies

          Private evaluations

          Want this for your team?

          Azul Labs runs Forcebench privately for two kinds of Salesforce team.

          Choosing a model to run yourself?

          A private benchmark of your shortlist

          Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

          • Private tasks written from your work only ever run on local models; they are never sent to an API.
          • Re-testing after any change of model, quantisation or engine is available.
          • Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
          Get a private benchmarkHow it works

          Building or buying a Salesforce AI tool?

          A private evaluation of your tool

          Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

          • It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
          • Repeat evaluations after your releases are available.
          • Typical turnaround is 2–3 weeks.
          Ask about an evaluationHow it works

          Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

          Or email leo@azl.au · azl.au