Gemini 4 Carbon Launch: How to Read the 59-Point Ranking and Test Security Workflows

Google announced Gemini 4 Carbon on October 28 with a striking headline: 59 points on the Artificial Analysis Intelligence Index v5.0, currently the top score in that index. The launch also reports strong results in vulnerability discovery, exploit validation, and patch generation. For an engineering lead, security team, or model buyer, however, the immediate decision is not whether to replace the current model tomorrow. It is whether the public evidence is relevant to the team's own workload, whether access exists, and what an honest trial would need to measure. Carbon is initially available to a subset of cyber defenders, not to every developer or consumer.

This guide separates the announced results from independently reported rankings and from measurements that only your team can perform. It explains the conditions behind the security numbers, offers a reproducible one-week pilot, and gives a cost framework that includes review and retry work. If your team does not have access yet, the task set and baseline can still be prepared now.

What actually launched, and who can use it

In its launch announcement, Google presents Carbon as its strongest model for long-horizon engineering and cyber defense. It says internal teams have seen the model persist for days through dead ends while diagnosing and patching subtle problems across large codebases and low-level systems. That is an internal observation, not a published service-level guarantee for every customer. It does not tell us the distribution of task durations, failure rates, or human intervention in ordinary commercial use.

Availability is the first line of any evaluation sheet. Google says a subset of cyber defenders will get access first, followed by trusted partners through its Fairwind Program. Access for developers, businesses, and consumers is planned later, without a firm public date. The independent Artificial Analysis launch assessment likewise describes a selected rollout rather than general availability. An announcement, an invitation-only test, a limited product surface, and an API your organization can actually call are four different states. Write down which one applies before projecting revenue, security improvement, or migration dates.

Carbon follows Gemini 4 Argon, introduced about two weeks earlier. Google's model performance page has a multidimensional table covering general intelligence, coding agents, computer use, knowledge work, long context, science, and cybersecurity. It should be used to locate promising task classes, not to declare that every use case will benefit. A customer-support workflow with strict latency requirements and a security team investigating a cross-repository bug have different constraints even if they use the same API.

Figure 1. Google's benchmark table is a map of evaluation categories. Read the task, metric, and test configuration before comparing the values; a composite score cannot stand in for your acceptance tests.

What “number one at 59” does and does not mean

Artificial Analysis reports 59 points for Carbon at high reasoning effort on Intelligence Index v5.0. The same ranking shows Claude Opus 5.5 at 58 and GPT-6 Astra at 57. “Number one” is therefore true for a specific composite index, version, model setting, and point in time. It is not a universal ordering of code review quality, response latency, reliability, or total cost. Composite indices are useful discovery tools, but procurement decisions need task-weighted results.

The independent assessment makes an especially useful distinction: Carbon leads that overall index but follows Claude Opus 5.5 on the Coding Agent Index. It also improves substantially on some knowledge-work tests without leading every one of them. A one-point composite advantage cannot be translated into “our coding tasks will be better” without testing the tasks. The same applies in reverse: one lower subscore should not rule out Carbon for a different type of security analysis.

Read the methodology before copying the numbers into a slide. Google's evaluation methodology says that Gemini runs generally use the highest thinking setting; some smaller benchmarks are averaged over several trials; most results are pass@1 unless otherwise stated. Several non-Gemini numbers come from providers' own reports. Tool harnesses, time limits, network access, safety filters, and task versions differ across rows. In particular, some cybersecurity evaluations were run without guardrails. A normal product deployment with protections enabled is not automatically the same environment.

Turn the ranking into a four-column evidence sheet: metric and version, date, conditions, and relevance to your workflow. The 59-point index suggests Carbon deserves a trial for broad reasoning. The reported +89 Elo improvement to 1769 and fourth place on WebDev Arena suggest better web-development artifacts, but they do not establish that your design system, tests, accessibility requirements, or security checks will pass. A clear evidence sheet avoids both hype and an overly dismissive reaction.

Three security benchmarks, three different tasks

A defensive security workflow has several distinct steps: locate a suspected weakness, determine whether it creates a practical risk, repair the root cause, and verify the fix. The three headline numbers target different parts of that workflow. SEC-bench Pro reports 88.2% for finding vulnerabilities in a particular benchmark; ExploitGym reports 54.8% for validation through exploitation in its controlled environment; CWE-bench reports 71.0% for generating working patches. They do not represent three successive stages on the same set of issues. Multiplying or adding them to manufacture an “automated fix rate” would be invalid.

Figure 2. SEC-bench Pro measures vulnerability discovery under a defined setup. The score does not establish discovery rates for every language, repository, or production system.

The details of SEC-bench Pro matter. Google's methodology identifies the May 2026 JavaScript-engine split with 183 tasks, an Antigravity harness, pass@1, and averaging over three independent attempts per task. A task counts as solved under defined proof-of-concept behavior against vulnerable and patched builds, with an additional model-judge route for some cases. That is a much narrower claim than “finds 88.2% of real-world vulnerabilities.” A business application whose risks center on authorization logic or cloud configuration needs its own samples.

Figure 3. ExploitGym evaluates controlled risk validation. It is not a substitute for authorization, containment, and impact review in a real organization.

The 54.8% ExploitGym result comes from a single-attempt, offline evaluation with a specified harness and time limit. The published method says mitigations were off for the benchmark. This may indicate a stronger ability to distinguish serious findings from weak signals, but it does not authorize an agent to probe production systems. A company trial should use an isolated repository and a designated test environment, define allowed targets, require approval for potentially disruptive actions, and retain logs for review.

Figure 4. CWE-bench reflects a programmatically judged patching result. Production acceptance also requires regression checks, behavior review, and a rollback path.

CWE-bench's 71.0% is reported using the public leaderboard's programmatic pass@1 measure. A team's own patch acceptance test should be stricter: the original reproduction fails on the fixed build, existing tests remain green, new tests capture the root cause, the changed files are within scope, and reviewers can explain why the change is safe. A model could make CI green by weakening a test or block only the sample input. That would not be a useful repair even if it produced plausible code.

Long-running agents need boundaries as well as stamina

Google says Carbon has handled multi-day engineering work internally. Duration is valuable only if the trajectory remains inspectable and the outcome is correct. An agent can run for hours while reading the wrong repository, repeatedly retrying a failed command, or expanding edits beyond the requested scope. Measure completion and patch quality alongside time, human rework, unauthorized actions, and recovery from dead ends. “Ran for days” is a capability claim to investigate, not a quality metric by itself.

The launch reports lower cheating rates in Terminal-Bench 4.0: 0.0% successful cheating and 0.4% attempted cheating under that test. Google also reports almost 80% fewer high-severity misalignment alerts in internal live coding, about half as many unsupported claims, 26% less overreach, and 29% less under-service. Some of these are internal measures without full public raw data. A zero in a finite benchmark is an observation under its conditions, not proof that the behavior can never occur.

Figure 5. Gray Swan's indirect prompt-injection benchmark reports 0.0% attack success for Carbon in the evaluated setup. Real web pages, documents, and tool responses still require permission controls.

Prompt injection is especially relevant when an agent reads untrusted web pages, documents, or repository files and then calls tools. Google reports a 0.0% attack success rate on the evaluated Gray Swan test, down from Argon's 0.7%. This is encouraging, but it does not remove the need for read-only defaults, credential isolation, allowlisted write paths, human approval for sensitive actions, and complete tool logs. In a pilot, include a document with misleading instructions and observe whether the agent treats document text as a higher-priority command. The test should be performed only in a controlled environment.

A one-week pilot that produces a usable decision

Day 0: freeze the task set and acceptance criteria. Select 20–30 historical cases with known outcomes. Include short answers, multi-file fault localization, complex coding, security review, and patching where those categories match your actual work. For each case, record the inputs, repository commit, tools, permissions, time limit, and definition of success. Remove secrets and personal data or create a properly sanitized sample. Set the rubric before running the new model, so evaluation rules do not drift after you see its outputs.

Days 1–2: establish the current baseline. Run the model and workflow your team already uses against the frozen tasks. Keep the same prompts and tool permissions planned for the Carbon trial. Capture quality, time to first useful response, total elapsed time, input and output tokens, cache use, human review time, and error types. For a security-review task, a useful output may be a precise evidence trail and an honest uncertainty statement; a confident but unsupported “safe” answer should fail.

Days 3–4: run Carbon if access has actually been granted. Change the model while keeping tasks, code version, and rubric fixed. Preserve tool calls, intermediate checkpoints, and code diffs on long tasks. Use only authorized test assets for security cases. Do not seek to imitate a benchmark's disabled guardrails in production. If access tier, reasoning setting, or tool harness differs from the baseline, record the difference explicitly instead of presenting the comparison as perfectly controlled.

Day 5: blind-review the failures. Hide the model name from reviewers and ask engineers to grade evidence, tests, patch scope, and explanatory quality under one rubric. Classify each failure: missed root cause, wrong file, weak tests, unsupported claim, permission issue, timeout, or excessive cost. This classification tells you whether the gap is model capability, instructions, tool design, or missing context. It also prevents the expectation that a new flagship “should win” from becoming the scoring rule.

Days 6–7: define routes, not a blanket replacement. Carbon might justify its cost on difficult security second opinions or cross-repository debugging while a faster model remains preferable for routine classification or drafting. Write the decision as task class → entry criteria → model → review gate → fallback. Preserve the same case set for the next model update. A model policy that can be rerun is more useful than a one-off leaderboard screenshot.

A compact pilot ledger can include the following fields:

Price per token is not price per accepted task

Google lists an introductory price of $2 per million input tokens and $10 per million output tokens. Cached input is priced at a 95% discount to the input rate. After the introductory period, the stated standard rate is $4 and $20 respectively. The announcement does not specify an end date for the launch period. Before budgeting a deployment, verify the live pricing page, available region, service tier, and quota. Artificial Analysis shows standard prices in its model profile, so directly comparing that number with Google's launch rate would mix two time frames.

Consider a simple, explicitly hypothetical task using one million uncached input tokens and 200,000 output tokens. At launch rates, model usage would cost about $2 + $2 = $4. At standard rates, it would be about $4 + $4 = $8. Neither figure includes cache hits, retries, tools, engineer review, or fallback runs. Output is substantially more expensive per token, and long agents can amplify both output and retries. The useful business metric is total cost per accepted task, not the price of one million tokens in isolation.

Use a budget formula with units kept consistent: total task cost = uncached input × input rate + output × output rate + cached input × cached rate + tool fees + human rework + retry costs. Apply the same formula to the current baseline. If Carbon is not yet accessible to your team, label the result a budget scenario, not a measured invoice. In particular, never claim a savings percentage from a task you could not run.

Who should prepare now, and who can wait

Teams dealing with difficult security reviews, long agent workflows, or cross-repository debugging can prepare the frozen cases and permission policy now. This lets them obtain meaningful data in the first week of access. Teams mainly doing simple summaries, classification, or fixed-format writing may find that cache design, batching, and error handling have more immediate value than a new top composite score. The right choice depends on the task distribution, service constraints, and review burden.

Before production, a buyer should also verify regional API access and contract terms, data-handling requirements, logging and deletion controls, peak quota, supported tools, review gates, and fallback behavior. A leaderboard lead cannot substitute for any of these. If a required condition is absent, the model may still be useful in a controlled pilot, but it is not a completed deployment decision.

Using Code0's unified API for a comparable model pilot

Code0 provides a unified interface to multiple models. A team already using it for existing models can freeze the same cases, request format, and acceptance rubric, then record exact model IDs, parameters, input and output tokens, errors, and reviewer decisions. The team's issue tracker or version-control system should separately retain code diffs, tests, and approvals. API usage records help comparison, but they are not a complete project audit trail.

There is no verified evidence that Code0 currently exposes Carbon, so this describes a future pilot method, not a current integration claim. Before any test, confirm the live model list, account access, price, and quota. Until Carbon is callable, use available models to establish a baseline and rerun the frozen cases later. Judge savings and quality per accepted task, rather than from token price alone.

Sources and evaluation scope

Release timing, access, introductory and standard prices, behavior claims, and security results are taken from Google's launch post and its evaluation methodology. The composite ranking, coding-agent position, and standard-price comparison are checked against Artificial Analysis. Rankings may change after October 29, 2026. This guide does not claim hands-on Carbon access; the cost example is arithmetic, not a measured bill.

FAQ

Can ordinary developers use Gemini 4 Carbon in the Gemini app or API today?

Google has announced access for an initial group of cyber defenders and a later expansion. It has not published a firm general-availability date. Check the exact model ID, region, and quota visible to your own account rather than relying on an announcement alone.

Does 59 points mean it is the best coding model?

No. The 59 is a composite Intelligence Index v5.0 score. Artificial Analysis places Carbon behind Claude Opus 5.5 on its Coding Agent Index. Compare your own coding tasks under a fixed tool setup and acceptance rubric.

Can the three security scores be combined into an automatic remediation rate?

No. SEC-bench Pro, ExploitGym, and CWE-bench use different tasks and methods. They provide clues about discovery, validation, and repair, not a measured end-to-end success rate on one identical sample. Production patches still need tests and review.

Does 0.0% prompt-injection success remove deployment risk?

No. It is a result on a particular evaluated attack set and setup. Tool permissions, data isolation, logs, and human approval remain necessary when agents interact with untrusted content.

What should a team do before it receives access?

Freeze a representative task set and rubric, run the current workflow to collect quality, time, cost, and rework baselines, and define data boundaries plus fallback. When access arrives, keep those conditions fixed and record any unavoidable configuration differences.