# FinanceBenchmark.ai - Complete Knowledge Base

> Complete dataset, methodology, benchmarks catalog, model rankings, and research articles from FinanceBenchmark.ai for Large Language Models.
> Generated on: 2026-10-20

---

# 1. Methodology & Scoring System

## The FB Composite Score
The FB Composite Score is a weighted average across all financial reasoning benchmarks tracked on the platform:

FB Score = Σ(Benchmark Score × Tier Weight) / Σ(Tier Weight)

### Tier Definitions
- Tier 1 (Weight 30): High-complexity tasks on raw structured financial data, requiring code execution or multi-file forensic verification (e.g., Pave Interchange Fee Benchmark, Finance Agent).
- Tier 2 (Weight 20): Numerical reasoning and document comprehension over corporate filings (e.g., FinQA, ConvFinQA, FinanceBench).
- Tier 3 (Weight 10): Domain NLP and multi-document audit reasoning (e.g., FinBen, MultiHiertt, FinAuditing).

### Attribution Policy
1. Every score references its primary source (arXiv preprint, SSRN paper, or verifiable evaluation harness).
2. No synthetic score estimation or speculative adjustments are permitted.
3. Founding conflict of interest disclosure: FinanceBenchmark was founded by Avae (avae.ai). Avae models are clearly disclosed.

---

# 2. Complete Model Leaderboard

| Rank | Model | Organization | FB Score | Benchmarks Evaluated | License | Context Window |
|:----:|:------|:-------------|:--------:|:--------------------:|:--------|:--------------:|
| 1 | Avae 2.0 | Avae | **97.0** | 2 | Proprietary | N/A |
| 2 | Claude Opus 4.7 | Anthropic | **73.8** | 10 | proprietary | 200K |
| 3 | GPT-5.5 | OpenAI | **72.2** | 12 | proprietary | 128K |
| 4 | GPT-6 Sol | OpenAI | **71.1** | 1 | proprietary | 1M |
| 5 | DeepSeek V4 Pro | DeepSeek | **70.1** | 9 | open-weight | 128K |
| 6 | Gemini 3.1 Pro | Google | **70.0** | 9 | proprietary | 1M |
| 7 | GPT-6 Luna | OpenAI | **70.0** | 1 | proprietary | 1M |
| 8 | Claude Fable 5 | Anthropic | **69.8** | 25 | proprietary | 1M |
| 9 | Kimi K2.6 | Moonshot AI | **67.4** | 9 | open-weight | 128K |
| 10 | Claude Opus 5 | Anthropic | **66.7** | 30 | proprietary | 1M |
| 11 | Claude Fable 5.1 | Anthropic | **66.2** | 5 | proprietary | 1M |
| 12 | GLM-5.1 | Zhipu AI | **65.9** | 7 | open-weight | 128K |
| 13 | Grok 4.6 | xAI | **65.4** | 21 | proprietary | 500K |
| 14 | GPT-5.6 Sol | OpenAI | **65.1** | 29 | proprietary | 1M |
| 15 | Grok 4.3 | xAI | **65.0** | 9 | proprietary | 131K |
| 16 | Gemini 3.8 Flash | Google | **64.8** | 3 | proprietary | 1M |
| 17 | GPT-5.6 Luna | OpenAI | **64.6** | 25 | proprietary | 1M |
| 18 | Claude Sonnet 5 | Anthropic | **64.1** | 25 | proprietary | 1M |
| 19 | DeepSeek V4 Pro (0813) | DeepSeek | **63.7** | 16 | open-weight | 1M |
| 20 | Muse Spark 1.2 | Meta | **63.7** | 23 | proprietary | 1M |
| 21 | Kimi K3 | Moonshot AI | **63.4** | 27 | open-weight | 1M |
| 22 | GLM 5.3 Flash | Zhipu AI | **63.4** | 3 | open-weight | 1M |
| 23 | DeepSeek V4 Flash | DeepSeek | **63.2** | 19 | open-weight | 1M |
| 24 | GPT-5.6 Terra | OpenAI | **63.1** | 25 | proprietary | 1M |
| 25 | Gemini 4 Argon | Google | **63.1** | 3 | proprietary | 1M |
| 26 | Muse Spark 1.1 | Meta | **62.8** | 23 | proprietary | 1M |
| 27 | Qwen 3.8 Max | Alibaba | **62.2** | 25 | open-weight | 1M |
| 28 | Gemini 3.6 Flash | Google | **61.6** | 22 | proprietary | 1M |
| 29 | Gemini 3.7 Flash | Google | **61.2** | 3 | proprietary | 1M |
| 30 | Claude Opus 5.5 | Anthropic | **59.4** | 3 | proprietary | 1M |
| 31 | GPT-6 Astra | OpenAI | **59.3** | 3 | proprietary | 1.1M |
| 32 | MiniMax M2.7 | MiniMax | **58.7** | 8 | open-weight | 1M |
| 33 | Claude Sonnet 5.5 | Anthropic | **58.4** | 3 | proprietary | 1M |
| 34 | Muse Spark 1.3 Max | Meta | **58.0** | 3 | proprietary | 1M |
| 35 | GLM 5.2 | Zhipu AI | **57.5** | 20 | open-weight | 1M |
| 36 | MiniMax M3 | MiniMax | **57.2** | 5 | open-weight | 1M |
| 37 | Grok 4.5 | xAI | **56.8** | 25 | proprietary | 500K |
| 38 | GLM 5.3 | Zhipu AI | **55.3** | 3 | open-weight | 1M |
| 39 | Muse Spark 1.3 | Meta | **55.1** | 3 | proprietary | 1M |
| 40 | DeepSeek V4.1 Flash | DeepSeek | **54.8** | 3 | open-weight | 1M |
| 41 | Gemini 3.5 Flash Lite | Google | **53.5** | 23 | proprietary | 1M |
| 42 | Kimi K2.7 Code | Moonshot AI | **53.5** | 7 | open-weight | 262K |
| 43 | Inkling | Thinking Machines | **52.3** | 26 | open-weight | 256K |
| 44 | Inkling Small | Thinking Machines | **51.5** | 24 | open-weight | 262K |
| 45 | Nemotron 3 Ultra | NVIDIA | **46.3** | 16 | open-weight | 1M |
| 46 | Ling 3.0 Flash | Ant Group | **45.0** | 18 | open-weight | 262K |
| 47 | Qwen 3.7 Plus | Alibaba | **39.5** | 12 | proprietary | 1M |


---

# 3. Complete Benchmark Directory (42 Benchmarks)

## Pave Interchange Fee Benchmark (pave)
- **Description:** Forensic financial verification across 50,000 synthetic transaction records spanning four complexity levels from deterministic aggregation to strategic impact modelling.
- **Domains:** verification, forensic
- **Tier:** 1 (Weight: 30)
- **Task Count:** 24
- **Source:** FinanceBenchmark (2026)
- **Paper:** https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6944801

**Top Model Scores on Pave Interchange Fee Benchmark:**
- Avae 2.0: **100.0%** (2026-05)
- Claude Opus 4.7: **73.0%** (2026-05)
- GPT-5.5: **71.0%** (2026-05)
- Gemini 3.1 Pro: **69.0%** (2026-05)
- Grok 4.3: **68.0%** (2026-05)

## FinanceBench (financebench)
- **Description:** Financial question answering over real earnings documents and financial statements.
- **Domains:** documentQA
- **Tier:** 2 (Weight: 20)
- **Task Count:** 150
- **Source:** arXiv:2311.11944
- **Paper:** https://arxiv.org/abs/2311.11944
- **arXiv:** https://arxiv.org/abs/2311.11944

**Top Model Scores on FinanceBench:**
- GPT-5.5: **91.0%** (2026-04)
- Claude Opus 4.7: **88.0%** (2026-04)
- Gemini 3.1 Pro: **86.0%** (2026-03)
- DeepSeek V4 Pro: **85.0%** (2026-03)
- Grok 4.3: **84.0%** (2026-04)

## FinBen (finben)
- **Description:** Holistic financial benchmark covering 35 tasks across 23 financial datasets.
- **Domains:** multiDomain
- **Tier:** 3 (Weight: 10)
- **Task Count:** 35
- **Source:** arXiv:2402.12659
- **Paper:** https://arxiv.org/abs/2402.12659
- **arXiv:** https://arxiv.org/abs/2402.12659

**Top Model Scores on FinBen:**
- Gemini 3.1 Pro: **85.0%** (2026-02)
- Claude Opus 4.7: **81.0%** (2026-04)
- GPT-5.5: **79.0%** (2026-03)
- DeepSeek V4 Pro: **78.0%** (2026-02)
- Grok 4.3: **77.0%** (2026-03)

## FinQA (finqa)
- **Description:** Numerical reasoning over financial documents requiring multi-step computation.
- **Domains:** numerical
- **Tier:** 2 (Weight: 20)
- **Task Count:** 8281
- **Source:** arXiv:2109.00122
- **Paper:** https://arxiv.org/abs/2109.00122
- **arXiv:** https://arxiv.org/abs/2109.00122

**Top Model Scores on FinQA:**
- DeepSeek V4 Pro: **87.0%** (2026-02)
- Claude Opus 4.7: **84.0%** (2026-03)
- GPT-5.5: **82.0%** (2026-03)
- Gemini 3.1 Pro: **80.0%** (2026-02)
- Grok 4.3: **78.0%** (2026-03)

## ConvFinQA (convfinqa)
- **Description:** Multi-turn conversational financial reasoning over earnings documents.
- **Domains:** documentQA
- **Tier:** 2 (Weight: 20)
- **Task Count:** 3892
- **Source:** arXiv:2210.03849
- **Paper:** https://arxiv.org/abs/2210.03849
- **arXiv:** https://arxiv.org/abs/2210.03849

**Top Model Scores on ConvFinQA:**
- GPT-5.5: **88.0%** (2026-04)
- Claude Opus 4.7: **86.0%** (2026-03)
- DeepSeek V4 Pro: **83.0%** (2026-03)
- Gemini 3.1 Pro: **82.0%** (2026-03)
- Grok 4.3: **79.0%** (2026-04)

## Finance Agent Benchmark (finance-agent)
- **Description:** Core financial analyst tasks and equity research evaluated using the Finance Agent (FAB v2) harness.
- **Domains:** agentTasks
- **Tier:** 1 (Weight: 30)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Finance Agent Benchmark:**
- Gemini 4 Argon: **65.4%** (2026-10)
- Gemini 3.8 Flash: **61.4%** (2026-10)
- Muse Spark 1.2: **60.6%** (2026-10)
- Muse Spark 1.3 Max: **60.0%** (2026-10)
- Gemini 3.7 Flash: **59.0%** (2026-10)

## MultiHiertt (multihiertt)
- **Description:** Numerical reasoning over multi-hierarchical tabular and textual financial data.
- **Domains:** numerical
- **Tier:** 3 (Weight: 10)
- **Task Count:** 1200
- **Source:** arXiv:2206.01347
- **Paper:** https://arxiv.org/abs/2206.01347
- **arXiv:** https://arxiv.org/abs/2206.01347

**Top Model Scores on MultiHiertt:**
- DeepSeek V4 Pro: **82.0%** (2026-02)
- Gemini 3.1 Pro: **80.0%** (2026-02)
- Claude Opus 4.7: **78.0%** (2026-03)
- GPT-5.5: **76.0%** (2026-03)
- Grok 4.3: **74.0%** (2026-03)

## FinAuditing (finauditing)
- **Description:** Multi-document financial audit reasoning requiring cross-document evidence synthesis.
- **Domains:** forensic, compliance
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** arXiv:2510.08886
- **Paper:** https://arxiv.org/abs/2510.08886
- **arXiv:** https://arxiv.org/abs/2510.08886

**Top Model Scores on FinAuditing:**
- Avae 2.0: **88.0%** (2026-05)
- Claude Opus 4.7: **75.0%** (2026-04)
- DeepSeek V4 Pro: **73.0%** (2026-03)
- GPT-5.5: **72.0%** (2026-04)
- Gemini 3.1 Pro: **70.0%** (2026-03)

## VALS Index (vals-index)
- **Description:** Holistic evaluation index measuring agent accuracy, reasoning, coding, and specialized domain proficiency.
- **Domains:** verification, numerical, agentTasks
- **Tier:** 1 (Weight: 25)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on VALS Index:**
- Claude Opus 5: **67.2%** (2026-07)
- Claude Fable 5: **66.0%** (2026-06)
- GPT-5.6 Sol: **63.7%** (2026-07)
- GPT-5.6 Luna: **59.9%** (2026-07)
- Claude Sonnet 5: **59.6%** (2026-06)

## MortgageTax Benchmark (mortgagetax)
- **Description:** Evaluates financial calculations and regulatory tax computations on complex mortgage agreements.
- **Domains:** numerical, forensic
- **Tier:** 2 (Weight: 20)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on MortgageTax Benchmark:**
- Claude Opus 5: **72.1%** (2026-09)
- Claude Fable 5.1: **70.8%** (2026-09)
- Claude Opus 4.7: **70.3%** (2026-09)
- Claude Sonnet 5: **70.0%** (2026-09)
- claude-opus-4-8: **69.9%** (2026-09)

## TaxEval v2 (taxeval-v2)
- **Description:** Comprehensive benchmark for corporate and individual tax code interpretation and compliance reasoning.
- **Domains:** verification, forensic
- **Tier:** 2 (Weight: 20)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on TaxEval v2:**
- Muse Spark 1.2: **80.4%** (2026-09)
- Muse Spark 1.1: **79.7%** (2026-09)
- Claude Fable 5: **76.9%** (2026-09)
- GPT-5.6 Luna: **76.2%** (2026-09)
- GPT-5.6 Terra: **76.2%** (2026-09)

## Code Migration (code-migration)
- **Description:** Assesses code refactoring, legacy framework migration, and syntax translation accuracy.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Code Migration:**
- Claude Opus 5: **57.5%** (2026-07)
- Claude Fable 5: **55.1%** (2026-06)
- GPT-5.6 Sol: **52.9%** (2026-07)
- Grok 4.6: **44.6%** (2026-08)
- GPT-5.6 Luna: **44.5%** (2026-07)

## CyberBench (cyberbench)
- **Description:** Evaluates cybersecurity defense, vulnerability discovery, and exploit mitigation.
- **Domains:** forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on CyberBench:**
- GPT-5.6 Sol: **88.1%** (2026-07)
- GPT-5.6 Luna: **83.9%** (2026-07)
- Kimi K3: **79.0%** (2026-07)
- DeepSeek V4 Flash: **77.5%** (2026-07)
- GLM 5.2: **77.4%** (2026-06)

## Excel Modeling Benchmark (emb)
- **Description:** Evaluating agents on Excel-based financial modeling and spreadsheet valuation tasks (EMB v1.1).
- **Domains:** documentQA, forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Excel Modeling Benchmark:**
- Claude Fable 5.1: **76.7%** (2026-10)
- Claude Sonnet 5.5: **75.9%** (2026-10)
- Claude Opus 5.5: **75.8%** (2026-10)
- Gemini 4 Argon: **74.8%** (2026-10)
- Claude Fable 5: **73.7%** (2026-09)

## Legal Research Bench (legal-research-bench)
- **Description:** Evaluates statutory interpretation, precedent discovery, and legal brief synthesis.
- **Domains:** documentQA, forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Legal Research Bench:**
- Claude Opus 5: **55.3%** (2026-07)
- Claude Fable 5: **49.5%** (2026-06)
- GPT-5.6 Sol: **48.1%** (2026-07)
- Grok 4.6: **48.1%** (2026-08)
- Qwen 3.8 Max: **47.6%** (2026-08)

## MedCode (medcode)
- **Description:** Evaluates medical ontology mapping, ICD billing codes, and clinical software tasks.
- **Domains:** verification
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on MedCode:**
- Claude Opus 5: **63.6%** (2026-07)
- Claude Fable 5: **56.1%** (2026-06)
- Gemini 3.6 Flash: **53.1%** (2026-07)
- Muse Spark 1.2: **49.4%** (2026-08)
- Kimi K3: **48.9%** (2026-07)

## MedScribe (medscribe)
- **Description:** Clinical consultation documentation, transcription accuracy, and medical reporting.
- **Domains:** documentQA
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on MedScribe:**
- Claude Opus 5: **91.0%** (2026-07)
- Muse Spark 1.2: **90.1%** (2026-08)
- Muse Spark 1.1: **88.9%** (2026-07)
- Claude Fable 5: **88.5%** (2026-06)
- Kimi K3: **88.0%** (2026-07)

## ProofBench v1.1 (proofbench-v1-1)
- **Description:** Formal mathematical theorem proving and step-by-step rigorous logical deduction.
- **Domains:** numerical, verification
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on ProofBench v1.1:**
- Claude Opus 5: **99.0%** (2026-07)
- Claude Fable 5: **95.0%** (2026-06)
- Kimi K3: **87.0%** (2026-07)
- GPT-5.6 Sol: **83.0%** (2026-07)
- Claude Sonnet 5: **77.0%** (2026-06)

## SAGE (sage)
- **Description:** Structured Agent Goal Execution across multi-modal tools and interactive environments.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on SAGE:**
- Kimi K3: **54.3%** (2026-07)
- GPT-5.6 Sol: **52.6%** (2026-07)
- Claude Fable 5: **51.9%** (2026-06)
- Qwen 3.8 Max: **51.3%** (2026-08)
- Claude Opus 5: **49.4%** (2026-07)

## Public Benefits Bench (public-benefits-bench)
- **Description:** Evaluates civic policy compliance, public benefits eligibility, and regulatory adjudication.
- **Domains:** verification, forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Public Benefits Bench:**
- Claude Opus 5: **76.9%** (2026-07)
- Claude Fable 5: **70.4%** (2026-06)
- Muse Spark 1.2: **68.5%** (2026-08)
- Kimi K3: **68.3%** (2026-07)
- Qwen 3.8 Max: **67.1%** (2026-08)

## Vibe Code Bench v1.1 (vibe-code-bench-v1-1)
- **Description:** Evaluates end-to-end full-stack frontend/backend generation and interactive software craftsmanship.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Vibe Code Bench v1.1:**
- Claude Fable 5: **90.3%** (2026-06)
- Claude Opus 5: **88.4%** (2026-07)
- Kimi K3: **85.0%** (2026-07)
- DeepSeek V4 Pro (0813): **82.3%** (2026-08)
- Claude Sonnet 5: **81.3%** (2026-06)

## Harvey's Legal Agent (harveys-legal-agent)
- **Description:** Enterprise legal agent evaluation covering contract analysis, clause drafting, and risk assessment.
- **Domains:** agentTasks, forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Harvey's Legal Agent:**
- Muse Spark 1.2: **25.4%** (2026-08)
- Muse Spark 1.1: **20.0%** (2026-07)
- Grok 4.6: **15.8%** (2026-08)
- Grok 4.5: **12.9%** (2026-07)
- Claude Fable 5: **11.3%** (2026-06)

## GPQA Diamond (gpqa-diamond)
- **Description:** Graduate-level Google-proof Q&A benchmark across physics, chemistry, biology, and math.
- **Domains:** numerical, verification
- **Tier:** 2 (Weight: 15)
- **Task Count:** 198
- **Source:** arXiv:2311.12022
- **Paper:** https://arxiv.org/abs/2311.12022
- **arXiv:** https://arxiv.org/abs/2311.12022

**Top Model Scores on GPQA Diamond:**
- GPT-5.6 Sol: **95.2%** (2026-07)
- Grok 4.6: **94.7%** (2026-08)
- Qwen 3.8 Max: **93.7%** (2026-08)
- Claude Opus 5: **93.4%** (2026-07)
- Gemini 3.6 Flash: **93.4%** (2026-07)

## IOI (ioi)
- **Description:** International Olympiad in Informatics algorithmic competitive programming tasks.
- **Domains:** numerical
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** IOI (2026)

**Top Model Scores on IOI:**
- Claude Opus 5: **91.7%** (2026-07)
- GPT-5.6 Sol: **86.7%** (2026-07)
- Qwen 3.8 Max: **73.0%** (2026-08)
- GPT-5.6 Luna: **72.9%** (2026-07)
- Claude Fable 5: **72.3%** (2026-06)

## LiveCodeBench (livecodebench)
- **Description:** Contamination-free competitive programming and code completion benchmark.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** arXiv:2403.07974
- **Paper:** https://arxiv.org/abs/2403.07974
- **arXiv:** https://arxiv.org/abs/2403.07974

**Top Model Scores on LiveCodeBench:**
- Claude Fable 5: **89.8%** (2026-06)
- Claude Opus 5: **89.0%** (2026-07)
- Grok 4.6: **88.2%** (2026-08)
- Gemini 3.6 Flash: **88.0%** (2026-07)
- Qwen 3.8 Max: **87.8%** (2026-08)

## LegalBench (legalbench)
- **Description:** Collaboratively built benchmark for measuring legal reasoning in large language models.
- **Domains:** documentQA, forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** arXiv:2308.11462
- **Paper:** https://arxiv.org/abs/2308.11462
- **arXiv:** https://arxiv.org/abs/2308.11462

**Top Model Scores on LegalBench:**
- Claude Fable 5: **88.6%** (2026-06)
- Claude Opus 5: **87.0%** (2026-07)
- GPT-5.6 Sol: **87.0%** (2026-07)
- Gemini 3.6 Flash: **86.5%** (2026-07)
- Grok 4.6: **86.3%** (2026-08)

## MMLU Pro (mmlu-pro)
- **Description:** Robust multi-task language understanding reasoning benchmark with reasoning-focused questions.
- **Domains:** numerical, verification
- **Tier:** 2 (Weight: 15)
- **Task Count:** 12000
- **Source:** arXiv:2406.01574
- **Paper:** https://arxiv.org/abs/2406.01574
- **arXiv:** https://arxiv.org/abs/2406.01574

**Top Model Scores on MMLU Pro:**
- Claude Opus 5: **91.6%** (2026-07)
- Claude Fable 5: **91.5%** (2026-06)
- Grok 4.6: **89.4%** (2026-08)
- Grok 4.5: **89.2%** (2026-07)
- GPT-5.6 Sol: **89.1%** (2026-07)

## MMMU (mmmu)
- **Description:** Massive Multi-discipline Multimodal Understanding and Reasoning benchmark for expert AGI.
- **Domains:** documentQA, numerical
- **Tier:** 3 (Weight: 10)
- **Task Count:** 11500
- **Source:** arXiv:2311.16502
- **Paper:** https://arxiv.org/abs/2311.16502
- **arXiv:** https://arxiv.org/abs/2311.16502

**Top Model Scores on MMMU:**
- Claude Opus 5: **89.9%** (2026-07)
- Claude Fable 5: **89.3%** (2026-06)
- GPT-5.6 Sol: **88.8%** (2026-07)
- Kimi K3: **88.2%** (2026-07)
- Qwen 3.8 Max: **88.0%** (2026-08)

## ProgramBench (programbench)
- **Description:** Program synthesis and execution verification across complex algorithmic challenges.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on ProgramBench:**
- Claude Opus 5: **3.0%** (2026-07)
- Claude Fable 5: **2.0%** (2026-06)
- Kimi K3: **2.0%** (2026-07)
- GPT-5.6 Sol: **1.5%** (2026-07)
- GLM 5.2: **0.5%** (2026-06)

## SkillsBench (skillsbench)
- **Description:** Measures procedural tool orchestration and software skill invocation.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on SkillsBench:**
- Grok 4.5: **66.0%** (2026-07)
- GPT-5.6 Terra: **60.6%** (2026-07)
- GPT-5.6 Luna: **60.5%** (2026-07)
- Claude Opus 5: **60.4%** (2026-07)
- Muse Spark 1.1: **59.2%** (2026-07)

## SWE-bench (swe-bench)
- **Description:** Software engineering benchmark evaluating resolution of real-world GitHub issues.
- **Domains:** agentTasks
- **Tier:** 2 (Weight: 20)
- **Task Count:** 2294
- **Source:** arXiv:2310.06770
- **Paper:** https://arxiv.org/abs/2310.06770
- **arXiv:** https://arxiv.org/abs/2310.06770

**Top Model Scores on SWE-bench:**
- Claude Opus 5: **97.0%** (2026-07)
- DeepSeek V4 Pro (0813): **96.4%** (2026-08)
- GPT-5.6 Sol: **96.2%** (2026-07)
- Grok 4.6: **95.6%** (2026-08)
- Claude Fable 5: **95.0%** (2026-06)

## Terminal-Bench 2.1 (terminal-bench-2-1)
- **Description:** Interactive Linux terminal command execution, shell debugging, and agent environment handling.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Terminal-Bench 2.1:**
- GPT-5.6 Sol: **85.8%** (2026-07)
- Claude Opus 5: **84.6%** (2026-07)
- Kimi K3: **80.9%** (2026-07)
- Claude Fable 5: **80.5%** (2026-06)
- GPT-5.6 Luna: **79.0%** (2026-07)

## RSI Index (rsi-index)
- **Description:** Recursive Self-Improvement and recursive cognitive task orchestration index.
- **Domains:** agentTasks, numerical
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on RSI Index:**
- Claude Opus 5: **30.9%** (2026-07)
- GPT-5.6 Sol: **28.4%** (2026-07)
- Kimi K3: **22.1%** (2026-07)

## Time Horizon Index (time-horizon-index)
- **Description:** Long-horizon autonomous planning and multi-stage financial forecasting horizon.
- **Domains:** forensic, agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Time Horizon Index:**
- Claude Opus 5: **13.7%** (2026-07)
- GPT-5.6 Sol: **13.0%** (2026-07)
- Grok 4.5: **6.3%** (2026-07)
- Kimi K3: **5.2%** (2026-07)
- Muse Spark 1.1: **1.7%** (2026-07)

## ReverseEngBench (reverseengbench)
- **Description:** Binary analysis, decompilation reasoning, and software reverse engineering.
- **Domains:** forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on ReverseEngBench:**
- GPT-5.6 Sol: **30.5%** (2026-07)
- Claude Opus 5: **12.2%** (2026-07)
- Grok 4.5: **0.8%** (2026-07)
- GLM 5.2: **0.0%** (2026-06)

## Tax Agent Bench (tax-agent-bench)
- **Description:** Evaluating agents on research-grade US corporate tax compliance and advisory questions.
- **Domains:** forensic, agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** VALS (2026)

**Top Model Scores on Tax Agent Bench:**
- GPT-5.6 Sol: **68.0%** (2026-09)
- Gemini 3.8 Flash: **66.8%** (2026-09)
- Qwen 3.8 Max: **66.0%** (2026-09)
- GPT-5.6 Terra: **65.2%** (2026-09)
- GPT-6 Astra: **63.3%** (2026-09)

## Big Finance Bench (big-finance-bench)
- **Description:** Evaluates models on complex financial analysis tasks that require retrieval and multi-step reasoning.
- **Domains:** forensic, documentQA
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** LLM Stats (2026)

## GDPval-MM (gdpval-mm)
- **Description:** Multimodal benchmark evaluating AI performance on economically valuable tasks across documents, spreadsheets, and diagrams.
- **Domains:** documentQA, numerical
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** arXiv:2510.04374
- **Paper:** https://arxiv.org/abs/2510.04374
- **arXiv:** https://arxiv.org/abs/2510.04374

## BankerToolBench (bankertoolbench)
- **Description:** Evaluates models on banking and finance tool-use and API interaction workflows.
- **Domains:** agentTasks
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** LLM Stats (2026)

## GDPval-Rubrics (gdpval-rubrics)
- **Description:** Evaluates AI model performance on rubric-graded economically valuable knowledge work tasks.
- **Domains:** forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** LLM Stats (2026)

## PRBench-Finance (prbench-finance)
- **Description:** Professional reasoning evaluations on complex financial tasks.
- **Domains:** numerical, forensic
- **Tier:** 3 (Weight: 10)
- **Task Count:** Variable
- **Source:** LLM Stats (2026)

## CorpFin (corpfin)
- **Description:** Evaluates understanding and reasoning over long-context credit agreements and complex syndicated loan documentation (CorpFin v2).
- **Domains:** forensic, verification
- **Tier:** 2 (Weight: 20)
- **Task Count:** 134
- **Source:** VALS (2026)

**Top Model Scores on CorpFin:**
- Claude Opus 5: **73.2%** (2026-08)
- Claude Fable 5: **71.8%** (2026-08)
- Kimi K3: **71.6%** (2026-08)
- Muse Spark 1.1: **71.3%** (2026-08)
- Muse Spark 1.2: **70.9%** (2026-08)


---

# 4. Research Publications

## The autumn 2026 frontier wave: Evaluating GPT-6, Claude 5.5, and Gemini 4 on financial reasoning
*Published: 2026-10-18 | Category: Model Evaluation*

The rapid arrival of OpenAI's GPT-6 generation, Anthropic's Claude 5.5 lineup, and Google's Gemini 4 Argon marks a major leap in autonomous agent architectures. Here is how they perform on complex financial verification and forensic accounting.

## A compressed frontier wave

Between late August and mid-October 2026, the artificial intelligence landscape experienced its most compressed succession of frontier releases to date. Google unveiled **Gemini 4 Argon** with 1-million-token output limits and deep multimodal reasoning; Anthropic deployed its **Claude 5.5 series** (Opus 5.5, Sonnet 5.5, and Haiku 5.5) alongside the specialized Fable 5.1 and Mythos 5.1 models; and OpenAI graduated its flagship **GPT-6 generation** (Astra, Sol, and Luna) into global production.

For financial institutions and quantitative audit teams, these releases represent more than benchmark score bumps. They signal a decisive transition from passive question-answering assistants to autonomous, multi-step financial agents capable of tool orchestration, continuous code execution, and long-horizon transaction verification.

## Native reasoning and context scaling

The defining architectural advance across all late-2026 frontier systems is the complete integration of adaptive test-time reasoning into the core model architecture. Rather than treating step-by-step thinking as an external prompting wrapper or post-hoc scratchpad, models like **Claude Opus 5.5** and **GPT-6 Astra** allocate compute dynamically depending on transaction complexity.

When presented with standard corporate earnings filings (such as 10-K or 10-Q reports), these models answer with sub-second latency. However, when tasked with evaluating cascading waterfalls in structured credit agreements or reconciling cross-border interchange billing files, the models initiate extended verification loops—verifying intermediate calculations against known regulatory frameworks before committing to a final report.

## Performance on forensic accounting and verification

Historically, models that scored near-perfect marks on standard benchmarks like FinQA and FinanceBench experienced steep degradation on forensic tasks—such as PAVE Level 3 (cross-scheme anomaly detection) and FinAuditing multi-document discrepancy tracing.

In our October 2026 evaluations, we observed meaningful convergence. **Gemini 4 Argon** demonstrated exceptional resilience on 50,000-record ledgers, leveraging its expansive working memory to trace systemic anomalies across disparate merchant categories. Meanwhile, **Claude Sonnet 5.5** established itself as an extraordinarily cost-effective audit engine, achieving 81.5% on multi-rule interchange classification at a third of frontier inference cost.

## The rise of open-weight specialists

Proprietary labs are no longer the exclusive contenders in institutional finance. DeepSeek's release of **DeepSeek V4.1 Flash** and Alibaba's **Qwen 3.8 Max** demonstrated that open-weight architectures incorporating hybrid sparse attention can compete with leading proprietary models on complex numerical workflows.

For Tier 1 investment banks and regulated payment institutions operating under strict data-residency and air-gapped sovereign cloud mandates, these open-weight models provide viable, high-accuracy alternatives for internal transaction monitoring without third-party API exposure.

## Implications for financial institutions

As frontier models approach saturation on classical financial comprehension tasks, evaluation must pivot toward resilience: how models handle adversarial documents, rule ambiguity, and hallucination containment. Teams building on these models should prioritize verification-first architectures where model outputs are systematically audited against deterministic ground truth.

---

*FinanceBenchmark is the independent evaluation standard for AI in financial services. Full scores and methodology available on financebenchmark.ai.*

---

## The state of financial AI benchmarking in 2026
*Published: 2026-06-14 | Category: State of Financial AI*

A survey of how financial institutions evaluate AI systems today, and why existing benchmarks leave critical gaps in forensic reasoning and verification.

## The evaluation gap in financial AI

Financial institutions are deploying AI systems at an unprecedented pace. Equity research teams use large language models to synthesise earnings calls. Compliance departments rely on them to flag suspicious transactions. Risk teams feed them regulatory filings and expect coherent summaries.

Yet the tools we use to evaluate these systems were not designed for finance. MMLU has a handful of accounting questions. GSM8K tests grade-school arithmetic. Even specialised benchmarks like FinQA, while valuable, focus narrowly on extracting numbers from tables.

The result is a dangerous gap: institutions make deployment decisions based on benchmarks that do not test what matters most - the ability to reason forensically over complex, messy, real-world financial data.

## What financial AI actually needs to do

Consider what a compliance analyst does when reviewing a portfolio of transactions. They cross-reference multiple data sources. They identify patterns across thousands of records. They compute aggregates, detect anomalies, and trace discrepancies through layered financial instruments.

This is not question answering. It is not summarisation. It is forensic discovery - a multi-step reasoning process that requires holding large volumes of structured data in context while applying domain-specific rules.

Current benchmarks test fragments of this workflow. FinanceBench tests whether a model can answer questions about earnings documents. ConvFinQA tests multi-turn reasoning. But none of them test the full pipeline: receive raw data, identify the right questions to ask, execute complex computations, and verify the results against ground truth.

## The landscape today

We track eight financial AI benchmarks at FinanceBenchmark. They span four categories:

**Verification and forensic reasoning** - Benchmarks like Pave and FinAuditing test whether AI systems can perform the kind of deep verification work that auditors and compliance teams need. These are Tier 1 in our weighting because they represent the hardest, most practically relevant tasks.

**Document QA** - FinanceBench and ConvFinQA test comprehension of financial documents. These are well-established benchmarks with broad model coverage.

**Numerical reasoning** - FinQA and MultiHiertt test computational reasoning over financial data. Important, but often testable with simpler methods.

**Agent tasks** - The Finance Agent Benchmark tests end-to-end financial research workflows. This is the newest category and the most representative of how AI will actually be used.

## Key findings from our first evaluation round

Across all nine systems evaluated, several patterns emerge:

**Frontier models cluster tightly on established benchmarks.** The top five models score within 6 points of each other on FinanceBench and ConvFinQA. These benchmarks are approaching saturation for frontier systems.

**Forensic reasoning separates models sharply.** On the Pave benchmark, which requires multi-step verification across 50,000 transaction records, the spread between the best and worst general-purpose model is 11 points. The hardest tasks (Level 3 and Level 4) see even wider gaps.

**Open-weight models are surprisingly competitive.** DeepSeek V4 Pro achieves composite scores within 2 points of Gemini 3.1 Pro, despite being freely available. This matters enormously for financial institutions considering on-premise deployment.

## What comes next

FinanceBenchmark will expand in three directions:

1. **More benchmarks.** We are evaluating several additional financial reasoning benchmarks for inclusion, particularly in regulatory compliance and credit risk assessment.

2. **Deeper evaluation.** For key benchmarks, we will publish detailed error analyses showing exactly where and how models fail.

3. **Temporal tracking.** As models are updated, we will maintain historical scores to track the rate of improvement on financial tasks specifically.

The goal is simple: give financial institutions the data they need to make informed decisions about which AI systems to trust with their most critical workflows.

---

*FinanceBenchmark is the independent evaluation standard for AI in financial services. Founded by Avae. All benchmark scores attributed to their original publishers.*

---

## Why forensic discovery tasks reveal what frontier models still cannot do
*Published: 2026-06-21 | Category: Model Evaluation*

Frontier models excel at question answering but struggle when asked to discover the right questions. Forensic financial tasks expose this gap.

## The difference between answering and discovering

Most AI benchmarks ask models to answer questions. Given a passage and a query, produce the correct response. This paradigm - inherited from the NLP research tradition - measures comprehension. It does not measure investigation.

Forensic financial work is fundamentally investigative. An auditor examining transaction records does not start with a question. They start with data and a mandate: find what is wrong. The questions emerge from the data itself.

This distinction explains why frontier models that score 85+ on document QA benchmarks can score 20-30 points lower on forensic reasoning tasks. They are excellent readers but mediocre investigators.

## What Pave reveals

The Pave Interchange Fee Benchmark tests forensic verification across four complexity levels:

**Level 1 (Deterministic Aggregation):** Sum transactions by category, apply known fee schedules, verify totals. All frontier models score above 89. This is essentially a computation task with clear instructions.

**Level 2 (Rule Application):** Apply complex interchange fee rules that require conditional logic, exception handling, and multi-table joins. Scores drop to the 61-75 range. Models begin making errors in rule interpretation.

**Level 3 (Anomaly Detection):** Identify transactions that have been incorrectly categorised or that violate expected patterns. This requires the model to form hypotheses about what "correct" looks like without being told. Scores drop to 40-52 for the best general-purpose models.

**Level 4 (Strategic Impact Modelling):** Quantify the financial impact of discovered anomalies across different remediation scenarios. This requires combining discovery with forward-looking analysis. Scores range from 56-68.

The pattern is clear: as tasks shift from answering to discovering, model performance degrades significantly.

## Why this matters for financial institutions

Financial institutions do not deploy AI to answer questions about documents they have already read. They deploy AI to find things they have not yet noticed - miscategorised transactions, compliance violations, pricing anomalies, audit exceptions.

If an AI system scores 91 on FinanceBench but 50 on Pave Level 3, it is excellent at confirming what you already know but unreliable at finding what you do not. For compliance and audit use cases, the second capability is the one that matters.

## The investigation gap across models

Our evaluation reveals a consistent pattern across all frontier models: the gap between their question-answering performance and their forensic discovery performance is 15-30 points. No model we tested closes this gap significantly.

Interestingly, the ranking of models shifts between tasks. Claude Opus 4.7 leads on verification tasks overall, but GPT-5.5 leads on document QA. DeepSeek V4 Pro, an open-weight model, outperforms several proprietary models on numerical reasoning despite trailing on forensic tasks.

This suggests that different architectural choices and training approaches produce different capability profiles - and that no single benchmark captures the full picture.

## What would close the gap

Closing the investigation gap likely requires models that can:

1. **Generate hypotheses** from data patterns rather than responding to prompts
2. **Maintain working memory** across large datasets (50,000+ records)
3. **Apply domain rules** with the precision of traditional software
4. **Quantify uncertainty** about their findings

Until frontier models develop these capabilities, specialised systems will continue to outperform them on forensic financial tasks. The question is whether general-purpose models will close the gap through scale and training, or whether financial verification will remain a domain where purpose-built systems are necessary.

---

*This analysis is based on evaluations conducted by FinanceBenchmark across 9 AI systems. Full scores and methodology available on financebenchmark.ai.*

---

## How the FB Composite Score is calculated
*Published: 2026-06-28 | Category: Methodology*

A transparent explanation of the weighting methodology behind the FB Composite Score, including tier definitions and the rationale for each weight.

## What the FB Composite Score represents

The FB Composite Score is a single number that summarises a model's performance across all financial AI benchmarks tracked by FinanceBenchmark. It is a weighted average, not a simple mean, because we believe some financial capabilities matter more than others.

This post explains exactly how the score is calculated, why we chose the weights we did, and what the limitations are.

## The formula

The FB Composite Score is calculated as:

```
FB Score = Σ(benchmark_score × benchmark_weight) / Σ(benchmark_weight)
```

The sum runs over all benchmarks where the model has been evaluated. Models are only compared on the benchmarks they have actually been tested on.

## Tier definitions and weights

We assign each benchmark to one of three tiers based on task complexity and financial relevance:

### Tier 1 - Weight 30

Benchmarks testing reasoning on raw structured financial data, requiring code execution, multi-file joins, or end-to-end agent workflows.

- **Pave Interchange Fee Benchmark** - Forensic verification across 50,000 transaction records
- **Finance Agent Benchmark** - Real-world financial research agent tasks

These receive the highest weight because they most closely approximate what financial institutions actually need AI to do: work with raw data, execute complex multi-step processes, and produce verified results.

### Tier 2 - Weight 20

Benchmarks testing numerical reasoning and document comprehension over financial documents.

- **FinQA** - Multi-step numerical computation
- **ConvFinQA** - Conversational financial reasoning
- **FinanceBench** - Financial question answering

These benchmarks are well-established, widely adopted, and test important capabilities. They receive moderate weight because, while valuable, they test a narrower slice of financial reasoning than Tier 1.

### Tier 3 - Weight 10

Broad financial knowledge and specialised reasoning benchmarks.

- **FinBen** - Multi-task financial NLP
- **MultiHiertt** - Hierarchical tabular reasoning
- **FinAuditing** - Multi-document audit reasoning

These receive the lowest weight not because they are unimportant, but because they either test breadth over depth (FinBen) or cover capabilities partially tested by higher-tier benchmarks.

## Why weighted averages

A simple average would give equal importance to every benchmark. This would mean a model's score on a 35-task NLP benchmark (FinBen) counts as much as its performance on end-to-end financial agent tasks (Finance Agent Benchmark).

We do not believe this reflects the priorities of financial institutions. Compliance teams care more about whether an AI can verify 50,000 transactions than whether it can classify financial sentiment. Our weighting reflects this.

## The limited data asterisk

Models evaluated on fewer than 3 benchmarks display an asterisk (*) next to their composite score. This indicates that the score is based on limited data and may not be representative of the model's overall financial AI capabilities.

For example, a model evaluated only on Pave might score very high (if it's a specialised verification system) or very low (if forensic reasoning is not its strength). Neither score would fairly represent its overall financial AI capability.

## Limitations and transparency

The FB Composite Score has several important limitations:

1. **Weighting is subjective.** Our tier assignments reflect our judgement about what matters most in financial AI. Others may reasonably disagree.

2. **Not all benchmarks are equally rigorous.** Some benchmarks have thousands of test cases; others have dozens. The composite score does not account for benchmark quality.

3. **Missing evaluations create asymmetry.** A model evaluated on all 8 benchmarks is compared against a model evaluated on 3. The weighted average handles this mathematically, but the scores are not perfectly comparable.

4. **Benchmark scores are not deployment recommendations.** A high composite score does not mean a model is suitable for production financial use. It means the model performs well on the specific tasks these benchmarks test.

We publish these limitations because transparency is fundamental to FinanceBenchmark's mission. The composite score is a useful summary statistic. It is not the final word on which AI system a financial institution should deploy.

---

*The complete methodology, including conflict of interest disclosures, is available on the FinanceBenchmark methodology page.*

---