{"data":{"schema_version":"model_decision_evidence.v1","snapshot_date":"2026-09-25","model":{"model":"gpt-6-astra","direct_answer":"GPT-6 Astra is OpenAI's closed-weight flagship, released on 2026-09-03 as its most capable model, with a 1,050,000-token context window (922,000 input), 128,000 output tokens and text and image input; its tool calling requires the Responses API. It placed first on two real robot tasks in Cortex AI's evaluation and third on AI Coding Daily's coding leaderboard, behind only Claude Opus 5.5, while users report more cost and quota per task than GPT-6 Sol, and one moved a coding project back to DeepSeek V4.1 Flash after $130 of Astra turns.","verified_at":"2026-09-24","model_card":{"context_window":"1,050,000 tokens (up to 922,000 input)","max_output":"128,000 tokens","modalities":"Text and image input; text output","parameter_count":"Not disclosed"},"official_sources":[{"title":"GPT-6 Astra model documentation","publisher":"OpenAI","url":"https://developers.openai.com/api/docs/models/gpt-6-astra","retrieved_at":"2026-09-24"},{"title":"OpenAI API changelog","publisher":"OpenAI","url":"https://developers.openai.com/api/docs/changelog","retrieved_at":"2026-09-24"},{"title":"GPT-6 Astra System Card","publisher":"OpenAI","url":"https://deploymentsafety.openai.com/gpt-6-astra","retrieved_at":"2026-09-24"}],"open_weights":{"status":"not_available","summary":"OpenAI publishes no GPT-6 Astra weights. The model is served through the OpenAI API (Responses, Chat Completions and Batch) and on Amazon Bedrock in us-west-2.","repository":"","licence":"Proprietary hosted model","hardware":"Not applicable","recommended_runtime":"ChinaAPI or the OpenAI API","source_urls":["https://developers.openai.com/api/docs/models/gpt-6-astra","https://developers.openai.com/api/docs/guides/amazon-bedrock"],"verified_at":"2026-09-24","agent_recipe":""},"curated_comparisons":["gpt-6-sol","claude-opus-5-5","claude-fable-5-1","gemini-3.8-flash","deepseek-flash"],"open_weight_alternatives":["deepseek-flash","glm-5.3-flash"],"reviews":[{"id":"cortexai-robot-tasks-2026-09","platform":"Independent benchmark","evidence_type":"independent_suite","title":"Cortex AI: GPT-6 Astra, Gemini 3.8 Flash and Fable 5.1 on two real robot tasks","url":"https://cortexrobot.ai/research/llm-real-world-robotics-evals/","published_at":"2026-09-17","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"Cortex AI Research","handle":"@cortexairobot","profile_url":"https://cortexrobot.ai/","description":"Robotics team that builds real-world robot datasets and runs the MolmoAct2 real-world evaluations"},"task":"Load a sandwich into a microwave and close the door; attach a 5 mm tip to a pipette","method":"Each model controlled a bimanual YAM robot through an overhead and two wrist cameras with depth images, at medium reasoning effort with up to 10 minutes per task; an evaluator scored each attempt with partial credit for completed steps.","finding":"GPT-6 Astra scored highest on both tasks, 80% on the microwave and 70% on the pipette, against 13.3% and 30% for Gemini 3.8 Flash and 6.7% and 20% for Fable 5.1; the lab credits it with retrying until the pipette tip docked and with checking that the microwave door latched.","limitation":"The post does not say how many attempts each model had; scores are partial-credit judgements by the lab's evaluator, and the authors attribute the gap mainly to weaker depth perception in the lower-scoring models.","metrics":[{"label":"Microwave","value":"80%"},{"label":"Pipette","value":"70%"}]},{"id":"x-xueyu-svg-quota-2026-09","platform":"X","evidence_type":"task_test","title":"GPT-6 Astra, Sol and Luna on one SVG animation prompt","url":"https://x.com/xueyu1125/status/2102692499710193960","published_at":"2026-09-23","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"雪瑜","handle":"@xueyu1125","profile_url":"https://x.com/xueyu1125","description":"Programmer and AI-tools builder on X who posted the timings and quota readings"},"task":"Generate an SVG animation of a pelican riding a bicycle, shown in H5","method":"Same prompt on GPT-6 Astra, GPT-6 Sol, GPT-6 Luna and GPT-5.6 Sol; the author recorded time taken and the share of a five-hour subscription quota each used.","finding":"GPT-6 Astra took 6 min 3 s and used 16% of the five-hour quota, against 4 min 42 s and 2% for GPT-6 Sol; the author recommends Sol when quota matters.","limitation":"One prompt; the quota shares are a subscription meter, not API cost, and the post scores no output quality for Astra.","metrics":[{"label":"Time","value":"6 min 3 s"},{"label":"Five-hour quota used","value":"16%"}]},{"id":"reddit-deepseek-after-astra-2026-09","platform":"Reddit","evidence_type":"community_experience","title":"Switching a coding project from GPT-6 Astra to DeepSeek V4.1 Flash","url":"https://www.reddit.com/r/DeepSeek/comments/1wklqt2/i_tried_gpt_astra_turned_to_deepseek_41_flash/","published_at":"2026-09-19","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"OwlZealousideal4779","handle":"u/OwlZealousideal4779","profile_url":"https://www.reddit.com/user/OwlZealousideal4779/","description":"r/DeepSeek member reporting their own coding-agent use"},"task":"The author's own coding project, first on GPT-6 Astra and then on DeepSeek V4.1 Flash","method":"Self-reported spend and outcome of about a dozen GPT-6 Astra turns, followed by the switch to DeepSeek V4.1 Flash.","finding":"About a dozen GPT-6 Astra turns cost the author $130 in a day without meeting the requirements; they switched the project back to DeepSeek V4.1 Flash.","limitation":"One uncontrolled self-report; the post does not describe what the Astra turns were asked to do.","metrics":[{"label":"GPT-6 Astra spend","value":"$130"}]},{"id":"x-bhavy-eiffel-threejs-2026-09","platform":"X","evidence_type":"task_test","title":"Four models build the Eiffel Tower in Three.js","url":"https://x.com/Bhavani_00007/status/2102795814355763277","published_at":"2026-09-23","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"Bhavy","handle":"@Bhavani_00007","profile_url":"https://x.com/Bhavani_00007","description":"X user who ran the same Three.js task on four models and reported cost and time"},"task":"Build the Eiffel Tower in Three.js","method":"Same task on Claude Opus 5.5, GPT-6 Astra, GPT-6 Sol and Kimi K3; the author reported cost and time for each and compared the results by eye.","finding":"GPT-6 Astra cost $7.45 and took 8 minutes; the author preferred Claude Opus 5.5's result ($8.95, 10 minutes) for this task.","limitation":"One task judged by eye by the author; the post does not say how the models were run or how the costs were counted.","metrics":[{"label":"GPT-6 Astra","value":"$7.45 · 8 min"},{"label":"Claude Opus 5.5","value":"$8.95 · 10 min"}]},{"id":"x-bridgebench-lava-lamp-2026-09","platform":"X","evidence_type":"task_test","title":"Four models build a lava lamp from one prompt","url":"https://x.com/bridgebench/status/2102770788319310302","published_at":"2026-09-23","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"Bridgebench","handle":"@bridgebench","profile_url":"https://x.com/bridgebench","description":"BridgeMind's benchmark account, which calls itself the official AI benchmark of the vibe coding movement"},"task":"Build a lava lamp from a single prompt","method":"Same prompt on Claude Opus 5.5, GPT-6 Astra, GPT-6 Sol and GPT-6 Luna; the account reported cost and time, asked readers which did best and commented on their answers in replies.","finding":"GPT-6 Astra cost $0.63 and took 5 min 4 s, between Claude Opus 5.5 ($1.27, 9 min 35 s) and GPT-6 Sol ($0.08, 50 s).","limitation":"One prompt with the quality verdict left mostly to readers; the account is BridgeMind's own benchmark and does not say how the models were accessed.","metrics":[{"label":"GPT-6 Astra","value":"$0.63 · 5m 4s"}]},{"id":"x-dimentary-robot-drawing-2026-09","platform":"X","evidence_type":"task_test","title":"GPT-6 Sol and GPT-6 Astra on a robot drawing benchmark","url":"https://x.com/dimentary/status/2102539444834402314","published_at":"2026-09-22","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"Dmytro Hrybov","handle":"@dimentary","profile_url":"https://x.com/dimentary","description":"ML research engineer working on physical learning and evaluations"},"task":"The author's robot-drawing-on-the-board benchmark","method":"The same benchmark on both models; the author compared output tokens, drawing speed and running cost.","finding":"GPT-6 Astra used about 22k output tokens to Sol's 25k, but Sol's controller drew twice as fast and cost almost five times less to run, though the author had expected stronger spatial reasoning from Sol.","limitation":"One run of a benchmark the post describes only by name, with no score.","metrics":[{"label":"Output tokens","value":"~22k vs ~25k (Sol)"}]},{"id":"aicodingdaily-llm-coding-leaderboard-2026-09","platform":"Independent benchmark","evidence_type":"independent_suite","title":"AI Coding Daily LLM Coding Leaderboard","url":"https://aicodingdaily.com/leaderboard","published_at":"2026-09-24","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"AI Coding Daily","handle":"@AICodingDaily","profile_url":"https://www.youtube.com/@AICodingDaily","description":"Coding-model testing channel and website that runs its own leaderboard and sells premium tutorials and channel advertising"},"task":"Implement and harden production-shaped projects in several stacks, including a Laravel app, a Go service and a React/TypeScript app, from the same prompts","method":"Each model and reasoning setting is a separate row run in a coding harness, with the average cost and time per run. Code quality is graded by a judge model against a published 100-point rubric and scaled to 20 points per project; behavioural reliability runs each of four projects five times and scores the checks every attempt fails.","finding":"GPT-6 Astra at medium reasoning scored 56.03 ($0.69 and 3 min 8 s per run in Codex CLI), third of 47 rows behind both Claude Opus 5.5 settings.","limitation":"One author's benchmark whose code-quality half is graded by a judge model; rows run in different harnesses, the page is updated as models are added (read on 2026-09-24, 47 rows), and the author sells premium tutorials and advertising.","metrics":[{"label":"Points (medium)","value":"56.03"},{"label":"Rank","value":"3 of 47"}]},{"id":"techgogogo-kzhu-llm-benchmark-2026-09","platform":"Independent benchmark","evidence_type":"independent_suite","title":"kzhu's six-question LLM test leaderboard","url":"https://www.techgogogo.com/llm-benchmark/","published_at":"2026-09-24","retrieved_at":"2026-09-24","verified_at":"2026-09-24","author":{"name":"AI产品狙击手 kzhu","handle":"@kevinzhu9305","profile_url":"https://www.youtube.com/@kevinzhu9305","description":"Chinese YouTube reviewer who scores models on a six-question test and publishes every score with per-question notes"},"task":"Six questions scored out of 10: constrained sentence writing, two reasoning puzzles (replaced by pelican and compound-bow SVG animations in version 2.2), a browser OS with a space shooter, a 3D racing game and a Trello-style board","method":"Each row names the harness the model ran in (Pi, Claude Code, WorkBuddy and others) and the date; the author scores every question with a note, and warns that totals are only comparable within one test version.","finding":"GPT-6 Astra scored 58/60 on version 2.1, ahead of Fable 5.1 (54), Gemini 3.8 Flash (49) and Qwen3.8 Max 0902 (47) on the same version: all reasoning answers right and a near-flawless browser OS, with only a collision glitch in the racing game.","limitation":"One author's scores on six questions; rows use different harnesses and test versions (2.1 or 2.2), and a few rows' table total differs by one point from the author's note. This row does not name a harness.","metrics":[{"label":"Score","value":"58 / 60 (v2.1)"}]}]},"ranking":{"name":"gpt-6-astra","display_name":"GPT-6 Astra","vendor":"OpenAI","origin":"intl","category":"text","release_date":"2026-09-03","on_chinaapi":true,"sources":[{"leaderboard":"lmarena_text","metric":"Arena score (Elo)","value":"1477","entry":"gpt-6-astra-max","url":"https://arena.ai/leaderboard/text","retrieved_at":"2026-10-05","interval":"±7","samples":9156},{"leaderboard":"livebench","metric":"Overall","value":"82.2","entry":"GPT-6 Astra Max Effort","url":"https://livebench.ai/","retrieved_at":"2026-10-04"},{"leaderboard":"aa_index","metric":"Intelligence Index","value":"53","entry":"GPT-6 Astra (max)","url":"https://artificialanalysis.ai/leaderboards/models","retrieved_at":"2026-09-21"},{"leaderboard":"vals_index","metric":"Accuracy (GDP-weighted finance, coding, legal and tax)","value":"63.13","entry":"GPT-6 Astra","url":"https://www.vals.ai/benchmarks/vals_index","retrieved_at":"2026-10-04","interval":"±1.19"},{"leaderboard":"epoch_eci","metric":"General ECI","value":"167","entry":"GPT-6 Astra","url":"https://epoch.ai/eci","retrieved_at":"2026-10-04","interval":"(163 - 171)"},{"leaderboard":"lmarena_text_chinese","metric":"Arena score (Elo)","value":"1510","entry":"gpt-6-astra-max","url":"https://arena.ai/leaderboard/text/chinese","retrieved_at":"2026-10-05","interval":"±22","samples":674},{"leaderboard":"lmarena_text_coding","metric":"Arena score (Elo)","value":"1543","entry":"gpt-6-astra-max","url":"https://arena.ai/leaderboard/text/coding","retrieved_at":"2026-10-05","interval":"±13","samples":2135},{"leaderboard":"livebench_coding","metric":"Coding","value":"80.4","entry":"GPT-6 Astra Max Effort","url":"https://livebench.ai/","retrieved_at":"2026-10-04"},{"leaderboard":"livebench_agentic_coding","metric":"Agentic Coding","value":"57.3","entry":"GPT-6 Astra Max Effort","url":"https://livebench.ai/","retrieved_at":"2026-10-04"},{"leaderboard":"aa_speed","metric":"Median output tokens/s","value":"58","entry":"GPT-6 Astra (max)","url":"https://artificialanalysis.ai/leaderboards/models","retrieved_at":"2026-09-22"},{"leaderboard":"aa_gdpval","metric":"Agentic Real-World Work Tasks, (Elo-500)/2000 (%)","value":"52","entry":"GPT-6 Astra (max)","url":"https://artificialanalysis.ai/leaderboards/models","retrieved_at":"2026-09-21"},{"leaderboard":"aa_tbench_v40","metric":"Agentic Coding \u0026 Terminal Use (%)","value":"59","entry":"GPT-6 Astra (max)","url":"https://artificialanalysis.ai/leaderboards/models","retrieved_at":"2026-09-21"},{"leaderboard":"aa_mlcr","metric":"Medical Long Context Reasoning (%)","value":"35.0","entry":"GPT-6 Astra (max)","url":"https://artificialanalysis.ai/evaluations/mlcr-aa","retrieved_at":"2026-09-07"}],"notes":"Released 2026-09-03 per OpenAI's announcement (GPT-6 Astra: A new generation of intelligence). LMArena added gpt-6-astra-max to its text board on 2026-09-11 per its leaderboard changelog; the row stands on 9,156 votes in its 2026-10-02 dataset publish. The Chinese category carried no row for it as of 2026-09-22; in the 2026-10-02 publish it does, on 674 votes. On sale here since 2026-09-05."},"sources":{"aa_gdpval":{"id":"aa_gdpval","name":"Artificial Analysis · GDPval-AA v2.1","operator":"Artificial Analysis","metric":"Agentic Real-World Work Tasks, (Elo-500)/2000 (%)","url":"https://artificialanalysis.ai/leaderboards/models","edition":"GDPval-AA v2.1, live board, read 2026-09-22. 220 agentic task-completion tasks with file outputs, ranked pairwise by a panel of three frontier LLM judges; since v2.1 the Elo scale is anchored to DeepSeek V4.1 Flash (max) at 1600 and fitted with a Crowd-BT model, so its figures are not comparable with the v2 figures this page quoted from its 2026-09-12 read. The column prints clamp((Elo-500)/2000). It carries 10% of the Agents category inside Intelligence Index v4.3.2, so this column is a component of the AA Intelligence Index shown as an individual source, not an independent operator. Rows Artificial Analysis marks deprecated are read with the leaderboard's Status filter set to All (CA-948). The column is hidden until the leaderboard's Intelligence group is expanded","retrieved_at":"2026-09-22","scene":"agent_work"},"aa_image_edit":{"id":"aa_image_edit","name":"Artificial Analysis Editing","operator":"Artificial Analysis","metric":"Elo","url":"https://artificialanalysis.ai/image/leaderboard/editing","edition":"live board; exact row settings retained","retrieved_at":"2026-09-13","task":"image_editing"},"aa_image_t2i":{"id":"aa_image_t2i","name":"Artificial Analysis Text To Image","operator":"Artificial Analysis","metric":"Elo","url":"https://artificialanalysis.ai/image/leaderboard/text-to-image","edition":"live board; exact row settings retained","retrieved_at":"2026-09-13","task":"text_to_image"},"aa_index":{"id":"aa_index","name":"Artificial Analysis Intelligence Index","operator":"Artificial Analysis","metric":"Intelligence Index","url":"https://artificialanalysis.ai/leaderboards/models","edition":"Intelligence Index v4.3.2, read 2026-09-22; its figures are not comparable with the v4.3 figures this page quoted from its 2026-09-12 read. v4.3.1 refreshed the pairwise judge panels of AA-Briefcase and GDPval-AA, and v4.3.2 moved GDPval-AA to v2.1 (Elo scale anchored to DeepSeek V4.1 Flash (max) at 1600, fitted with a Crowd-BT model) and refitted AA-Briefcase v1.1 with Crowd-BT, so rows moved without any model changing. v4.3 had already swapped τ³-Banking out of the index for AutomationBench-AA and Terminal-Bench v2.1 for v4.0. The leaderboard's default Status: Current filter hides rows Artificial Analysis marks deprecated; they still render with Status set to All and are re-read from that view like any other row (CA-948). A trailing * is reproduced as the board prints it: the board's data marks that score as estimated (intelligenceIndexIsEstimated), i.e. not every evaluation in the index had been run for that model","retrieved_at":"2026-09-22"},"aa_mlcr":{"id":"aa_mlcr","name":"Artificial Analysis · MLCR-AA","operator":"Artificial Analysis","metric":"Medical Long Context Reasoning (%)","url":"https://artificialanalysis.ai/evaluations/mlcr-aa","edition":"MLCR-AA, live board, read 2026-09-22. Artificial Analysis marks it a standalone evaluation that is NOT part of Intelligence Index v4.3 — unlike the GDPval-AA v2 and Terminal-Bench v4.0 columns, this one is not a component of the AA Intelligence Index shown as an individual source, only the same operator. The underlying benchmark is Wisedocs' open MLCR (Wisedocs-AI/medical-long-context-reasoning): synthetic medical records of roughly 25,000-64,000 tokens, graded across six tiers from locating a single fact to expert-level clinical synthesis. It measures multi-document reasoning over long records, not medical capability in general. The main leaderboard carries no column for it; the figures were first read off the evaluation page's chart (2026-09-07) and since 2026-09-22 from the per-model pages' payload, which carries every model the chart can show (CA-956)","retrieved_at":"2026-09-22","scene":"medical"},"aa_speed":{"id":"aa_speed","name":"Artificial Analysis · Output Speed","operator":"Artificial Analysis","metric":"Median output tokens/s","url":"https://artificialanalysis.ai/leaderboards/models","edition":"live board; measured by Artificial Analysis against the vendor's own API, not through ChinaAPI. Rows Artificial Analysis marks deprecated are read with the leaderboard's Status filter set to All (CA-948)","retrieved_at":"2026-09-22","scene":"speed"},"aa_tbench_v40":{"id":"aa_tbench_v40","name":"Artificial Analysis · Terminal-Bench v4.0","operator":"Artificial Analysis","metric":"Agentic Coding \u0026 Terminal Use (%)","url":"https://artificialanalysis.ai/leaderboards/models","edition":"Terminal-Bench 4.0 (66 tasks), run by Artificial Analysis with the mini-swe-agent harness, pass@1 averaged over 3 repeats per task; read 2026-09-22. When this column replaced the benchmark's own board on 2026-09-07 the methodology page named the harness as mini-SWE-agent v2.4.6; on 2026-09-21 it and the evaluation page name mini-swe-agent without a version, and every figure quoted before then read back unchanged. One harness for every model, so the column compares models rather than model-plus-best-scaffold — which is why it replaced the benchmark's own board, whose rows are agent × model and covered one Chinese model out of 34 (CA-788, overturning CA-734). It carries 10% of the Coding category inside Intelligence Index v4.3.2, so this column is a component of the AA Intelligence Index shown as an individual source, not an independent operator. Rows Artificial Analysis marks deprecated are read with the leaderboard's Status filter set to All (CA-948). The column is hidden until the leaderboard's Intelligence group is expanded","retrieved_at":"2026-09-22","scene":"agentic"},"aa_video_i2v_audio":{"id":"aa_video_i2v_audio","name":"Artificial Analysis Image To Video · With Audio","operator":"Artificial Analysis","metric":"Elo","url":"https://artificialanalysis.ai/video/leaderboard/image-to-video","edition":"live board; with audio pool; exact row settings retained","retrieved_at":"2026-09-13","task":"image_to_video_audio"},"aa_video_t2v_audio":{"id":"aa_video_t2v_audio","name":"Artificial Analysis Text To Video · With Audio","operator":"Artificial Analysis","metric":"Elo","url":"https://artificialanalysis.ai/video/leaderboard/text-to-video","edition":"live board; with audio pool; exact row settings retained","retrieved_at":"2026-09-13","task":"text_to_video_audio"},"epoch_eci":{"id":"epoch_eci","name":"Epoch Capabilities Index (ECI)","operator":"Epoch AI","metric":"General ECI","url":"https://epoch.ai/eci","edition":"live board","licence":"CC BY 4.0, as labelled on the page (https://creativecommons.org/licenses/by/4.0/)","retrieved_at":"2026-10-04"},"livebench":{"id":"livebench","name":"LiveBench","operator":"LiveBench (sponsored by Abacus.AI)","metric":"Overall","url":"https://livebench.ai/","edition":"LiveBench-2026-06-25","retrieved_at":"2026-10-04"},"livebench_agentic_coding":{"id":"livebench_agentic_coding","name":"LiveBench · Agentic Coding","operator":"LiveBench (sponsored by Abacus.AI)","metric":"Agentic Coding","url":"https://livebench.ai/","edition":"LiveBench-2026-06-25","retrieved_at":"2026-10-04","scene":"agentic"},"livebench_coding":{"id":"livebench_coding","name":"LiveBench · Coding","operator":"LiveBench (sponsored by Abacus.AI)","metric":"Coding","url":"https://livebench.ai/","edition":"LiveBench-2026-06-25","retrieved_at":"2026-10-04","scene":"coding"},"lmarena_i2v":{"id":"lmarena_i2v","name":"LMArena Image-to-Video Arena","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/image-to-video","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (image_to_video, category overall), published 2026-09-22, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","task":"image_to_video","updated_at":"2026-09-22"},"lmarena_image_edit":{"id":"lmarena_image_edit","name":"LMArena Image Edit Arena","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/image-edit","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (image_edit, category overall), published 2026-09-30, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","task":"image_editing","updated_at":"2026-09-30"},"lmarena_t2i":{"id":"lmarena_t2i","name":"LMArena Text-to-Image Arena","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/text-to-image","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (text_to_image, category overall), published 2026-09-24, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","task":"text_to_image","updated_at":"2026-09-24"},"lmarena_t2v":{"id":"lmarena_t2v","name":"LMArena Text-to-Video Arena","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/text-to-video","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (text_to_video, category overall), published 2026-09-22, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","task":"text_to_video","updated_at":"2026-09-22"},"lmarena_text":{"id":"lmarena_text","name":"LMArena Text Arena","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/text","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (text_style_control, category overall), published 2026-10-02, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","updated_at":"2026-10-02"},"lmarena_text_chinese":{"id":"lmarena_text_chinese","name":"LMArena Text Arena · Chinese","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/text/chinese","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (text_style_control, category chinese, prompts in Chinese), published 2026-10-02, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","scene":"chinese","updated_at":"2026-10-02"},"lmarena_text_coding":{"id":"lmarena_text_coding","name":"LMArena Text Arena · Coding","operator":"LMArena (arena.ai)","metric":"Arena score (Elo)","url":"https://arena.ai/leaderboard/text/coding","edition":"Hugging Face dataset lmarena-ai/leaderboard-dataset (text_style_control, category coding), published 2026-10-02, revision 46919c467f7f. Each rating is rounded half-up to a whole number and its interval is rating_upper minus rating, as arena.ai prints them; the dataset carries no Preliminary badge, so each citation records its row's vote count instead, and row names follow the dataset, which can trail a rename on arena.ai","licence":"CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), as labelled on the dataset card (https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset)","retrieved_at":"2026-10-05","scene":"coding","updated_at":"2026-10-02"},"superclue":{"id":"superclue","name":"SuperCLUE 智能指数","operator":"SuperCLUE","metric":"总分 (overall)","url":"https://www.superclueai.com/homepage","edition":"2026-07 evaluation, published 2026-08-06. The homepage reads this edition from its data file (/data/generalboard/2026年7月.xlsx, sheet 总排行榜, last modified 2026-09-11), which also carries rows evaluated after publication, each with its own evaluation publication date (the latest 2026-09-11); the page's Latest Update line refers to a different sub-board","retrieved_at":"2026-10-04"},"vals_index":{"id":"vals_index","name":"Vals Index","operator":"Vals AI","metric":"Accuracy (GDP-weighted finance, coding, legal and tax)","url":"https://www.vals.ai/benchmarks/vals_index","edition":"V2.1, page states updated 2026-10-02; read 2026-10-04. This revision covers finance, coding, legal and tax tasks and is not comparable with the V2 figures previously quoted. The entire cited cohort was re-read, including standard errors; 11 previously cited V2 rows are absent from the V2.1 board. Figures use the default scores including provider fallbacks, as labelled by Vals.","retrieved_at":"2026-10-04"},"vendor_card":{"id":"vendor_card","name":"Vendor's own evaluation card","operator":"the model's own vendor","metric":"varies; the benchmark is named in each row","url":"","edition":"self-reported","self_reported":true,"retrieved_at":"2026-09-22"}}},"success":true}