gpt-6-astra real-task evaluations

Every report keeps its original task, method, finding, limitation, evaluator identity and verification date. A community report is evidence about that run, not a universal score.

  • Cortex AI: GPT-6 Astra, Gemini 3.8 Flash and Fable 5.1 on two real robot tasks

    By Cortex AI Research (@cortexairobot). Robotics team that builds real-world robot datasets and runs the MolmoAct2 real-world evaluations Published 2026-09-17; retrieved 2026-09-24; verified 2026-09-24.

    Task
    Load a sandwich into a microwave and close the door; attach a 5 mm tip to a pipette
    Method
    Each model controlled a bimanual YAM robot through an overhead and two wrist cameras with depth images, at medium reasoning effort with up to 10 minutes per task; an evaluator scored each attempt with partial credit for completed steps.
    Finding
    GPT-6 Astra scored highest on both tasks, 80% on the microwave and 70% on the pipette, against 13.3% and 30% for Gemini 3.8 Flash and 6.7% and 20% for Fable 5.1; the lab credits it with retrying until the pipette tip docked and with checking that the microwave door latched.
    Limitation
    The post does not say how many attempts each model had; scores are partial-credit judgements by the lab's evaluator, and the authors attribute the gap mainly to weaker depth perception in the lower-scoring models.

  • GPT-6 Astra, Sol and Luna on one SVG animation prompt

    By 雪瑜 (@xueyu1125). Programmer and AI-tools builder on X who posted the timings and quota readings Published 2026-09-23; retrieved 2026-09-24; verified 2026-09-24.

    Task
    Generate an SVG animation of a pelican riding a bicycle, shown in H5
    Method
    Same prompt on GPT-6 Astra, GPT-6 Sol, GPT-6 Luna and GPT-5.6 Sol; the author recorded time taken and the share of a five-hour subscription quota each used.
    Finding
    GPT-6 Astra took 6 min 3 s and used 16% of the five-hour quota, against 4 min 42 s and 2% for GPT-6 Sol; the author recommends Sol when quota matters.
    Limitation
    One prompt; the quota shares are a subscription meter, not API cost, and the post scores no output quality for Astra.

  • Switching a coding project from GPT-6 Astra to DeepSeek V4.1 Flash

    By OwlZealousideal4779 (u/OwlZealousideal4779). r/DeepSeek member reporting their own coding-agent use Published 2026-09-19; retrieved 2026-09-24; verified 2026-09-24.

    Task
    The author's own coding project, first on GPT-6 Astra and then on DeepSeek V4.1 Flash
    Method
    Self-reported spend and outcome of about a dozen GPT-6 Astra turns, followed by the switch to DeepSeek V4.1 Flash.
    Finding
    About a dozen GPT-6 Astra turns cost the author $130 in a day without meeting the requirements; they switched the project back to DeepSeek V4.1 Flash.
    Limitation
    One uncontrolled self-report; the post does not describe what the Astra turns were asked to do.

  • Four models build the Eiffel Tower in Three.js

    By Bhavy (@Bhavani_00007). X user who ran the same Three.js task on four models and reported cost and time Published 2026-09-23; retrieved 2026-09-24; verified 2026-09-24.

    Task
    Build the Eiffel Tower in Three.js
    Method
    Same task on Claude Opus 5.5, GPT-6 Astra, GPT-6 Sol and Kimi K3; the author reported cost and time for each and compared the results by eye.
    Finding
    GPT-6 Astra cost $7.45 and took 8 minutes; the author preferred Claude Opus 5.5's result ($8.95, 10 minutes) for this task.
    Limitation
    One task judged by eye by the author; the post does not say how the models were run or how the costs were counted.

  • Four models build a lava lamp from one prompt

    By Bridgebench (@bridgebench). BridgeMind's benchmark account, which calls itself the official AI benchmark of the vibe coding movement Published 2026-09-23; retrieved 2026-09-24; verified 2026-09-24.

    Task
    Build a lava lamp from a single prompt
    Method
    Same prompt on Claude Opus 5.5, GPT-6 Astra, GPT-6 Sol and GPT-6 Luna; the account reported cost and time, asked readers which did best and commented on their answers in replies.
    Finding
    GPT-6 Astra cost $0.63 and took 5 min 4 s, between Claude Opus 5.5 ($1.27, 9 min 35 s) and GPT-6 Sol ($0.08, 50 s).
    Limitation
    One prompt with the quality verdict left mostly to readers; the account is BridgeMind's own benchmark and does not say how the models were accessed.

  • GPT-6 Sol and GPT-6 Astra on a robot drawing benchmark

    By Dmytro Hrybov (@dimentary). ML research engineer working on physical learning and evaluations Published 2026-09-22; retrieved 2026-09-24; verified 2026-09-24.

    Task
    The author's robot-drawing-on-the-board benchmark
    Method
    The same benchmark on both models; the author compared output tokens, drawing speed and running cost.
    Finding
    GPT-6 Astra used about 22k output tokens to Sol's 25k, but Sol's controller drew twice as fast and cost almost five times less to run, though the author had expected stronger spatial reasoning from Sol.
    Limitation
    One run of a benchmark the post describes only by name, with no score.

  • AI Coding Daily LLM Coding Leaderboard

    By AI Coding Daily (@AICodingDaily). Coding-model testing channel and website that runs its own leaderboard and sells premium tutorials and channel advertising Published 2026-09-24; retrieved 2026-09-24; verified 2026-09-24.

    Task
    Implement and harden production-shaped projects in several stacks, including a Laravel app, a Go service and a React/TypeScript app, from the same prompts
    Method
    Each model and reasoning setting is a separate row run in a coding harness, with the average cost and time per run. Code quality is graded by a judge model against a published 100-point rubric and scaled to 20 points per project; behavioural reliability runs each of four projects five times and scores the checks every attempt fails.
    Finding
    GPT-6 Astra at medium reasoning scored 56.03 ($0.69 and 3 min 8 s per run in Codex CLI), third of 47 rows behind both Claude Opus 5.5 settings.
    Limitation
    One author's benchmark whose code-quality half is graded by a judge model; rows run in different harnesses, the page is updated as models are added (read on 2026-09-24, 47 rows), and the author sells premium tutorials and advertising.

  • kzhu's six-question LLM test leaderboard

    By AI产品狙击手 kzhu (@kevinzhu9305). Chinese YouTube reviewer who scores models on a six-question test and publishes every score with per-question notes Published 2026-09-24; retrieved 2026-09-24; verified 2026-09-24.

    Task
    Six questions scored out of 10: constrained sentence writing, two reasoning puzzles (replaced by pelican and compound-bow SVG animations in version 2.2), a browser OS with a space shooter, a 3D racing game and a Trello-style board
    Method
    Each row names the harness the model ran in (Pi, Claude Code, WorkBuddy and others) and the date; the author scores every question with a note, and warns that totals are only comparable within one test version.
    Finding
    GPT-6 Astra scored 58/60 on version 2.1, ahead of Fable 5.1 (54), Gemini 3.8 Flash (49) and Qwen3.8 Max 0902 (47) on the same version: all reasoning answers right and a near-flawless browser OS, with only a collision glitch in the racing game.
    Limitation
    One author's scores on six questions; rows use different harnesses and test versions (2.1 or 2.2), and a few rows' table total differs by one point from the author's note. This row does not name a harness.