Technology · Community
Will Opus 5 match or exceed GPT-5.6-Sol on the MMLU benchmark?
市场数据
规则
上链语系：英文This market will resolve to "Yes" if Opus 5 achieves an averaged MMLU multiple-choice accuracy score greater than or equal to the averaged MMLU multiple-choice accuracy score reported for GPT-5.6-Sol, in a publicly accessible evaluation conducted at any time after the creation of this market and before January 1, 2027 - 00:00 UTC.
Otherwise this market will resolve to "No."
For the purposes of this market:
- "MMLU" refers to the standard Massive Multitask Language Understanding benchmark (multiple-choice format). The measured quantity is averaged accuracy across MMLU tasks using the same dataset version and evaluation protocol for both models.
- Reported scores must be publicly accessible via official model documentation or publications, public benchmark leaderboards (e.g., PapersWithCode or equivalent), or independent third-party reproducible evaluations. If multiple directly comparable scores exist for a model on the same protocol, the highest publicly reported score will be used for comparison.
- The compared evaluations must be conducted after this market's creation. If no directly comparable MMLU evaluation exists for both Opus 5 and GPT-5.6-Sol by the deadline, this market will resolve to "No."
来源
- Official Opus / OpusAI benchmark reports or model documentation
- Official OpenAI benchmark reports or GPT-5.6-Sol model documentation
- Public benchmark leaderboards (e.g., PapersWithCode MMLU leaderboard) or equivalent
- Independent third-party evaluations published in reputable ML/AI outlets
- Consensus of credible reports from reputable ML benchmarking sites