Production-ready agents, Part 13: Choosing and running models for agents

The first question partners ask me about a new agent is often “which model should we use?”. The honest answer is that the model matters less than they think, and in different ways from what the leaderboards suggest. An agent does not need the cleverest model in the world. It needs a model that calls tools correctly, returns output your code can parse, answers fast enough for a person waiting on the other side, costs what you planned, stays available for as long as your release cycle needs, and is allowed to run where your data is allowed to go.

The series so far has used Gemini without making much fuss about it. This part steps back. It covers the model families you will meet, what to check before one goes into production, how to compare candidates on your own questions, why different steps can use different models, and how to live with models that change underneath you. It is about using models. Training and tuning them is Part 14.

[!IMPORTANT] Model facts go stale within months. Every model name, limit, price and date in this post is as of October 2026 and links to the vendor’s own page. Check those pages on the day you decide.

If you prefer to watch first, here is the heart of this part as a short animated explainer.

Video: how to choose, compare and change the model behind every step of an agent, in three minutes. The narration is a synthetic voice; the script and animation are mine.

Where are we in the architecture?

This part adds the other models: how to choose, compare and change the model behind each step. Part 1.25 walks through the whole picture, layer by layer.

where Part 13 sits in the Acme architecture: highlighted boxes are the subject of this part, solid boxes come from earlier parts, faded boxes arrive later

What do I need to know first?

A few terms come up again and again in this part. If you have read Part 1 and Part 11, most will be familiar.

A model family is a line of related models from one vendor, such as Gemini or Claude. A family usually has a large, careful model and smaller, faster ones.
A proprietary model is one you can only use through the vendor’s API (or a cloud that resells it). An open-weight model is one whose weights (the learned numbers that make up the model) you can download and run yourself, under the terms of its licence. Google’s documentation points out that open-weight is not the same as open source: the training data and code are often not published.
A model ID is the exact string you send, such as gemini-3.8-flash. A pinned version is an ID that always means the same model. An alias is a name that can point to a different model over time, such as gemini-flash-latest.
The context window is the most tokens (the small pieces of text a model reads and writes) that fit into one call, input and output together. The maximum output is a separate, usually much smaller, limit.
A thinking (or reasoning) mode lets the model work through a problem before it answers. It usually improves hard answers, and you pay for the thinking tokens.
Function calling (or tool calling) is the model’s ability to pick one of your tools and fill in its arguments. Structured output forces the reply to follow a schema, such as JSON with fixed fields.
Latency has two parts that matter for agents: the time to first token (how long before anything comes back) and the output speed (how fast the rest arrives).
A quota is how much you may send per minute. When you go over it you get a 429 error, which Part 11.5 turned into a fallback ladder.

Which model families are there, and where do they run?

The question is never just “which model?” but “which model, served how?”, because the same family can reach you in very different ways, with different limits, terms and amounts of work for you.

Think about how you can get a meal from a famous chef:

1. Eat at the chef's own restaurant      -> the chef cooks, serves and owns the kitchen
2. Order the chef's dish at a food hall  -> someone else's kitchen cooks it; you just order
3. Buy the chef's recipe and cook it     -> you buy the ingredients, run the oven, wash up

Those are the three ways a model reaches you on Google Cloud. The restaurant is a native model, the food hall is a managed API, and cooking from the recipe is a self-deploy of downloaded weights. The dish can be the same; who does the work, and what can go wrong, is not.

Applied to the models themselves: there are three proprietary families most teams consider (Gemini, Claude and GPT) and five open-weight families that come up most often (Gemma, Llama, Mistral, Qwen and DeepSeek). On Google Cloud, each one reaches you in one of three ways: Gemini natively, a managed API (Google calls it Model as a Service, or MaaS) where someone else runs the servers, or a self-deploy where you run the weights yourself.

How does this apply to Acme? Every model step in the series so far is the first kind: the extract step, the specialists and the writer call Gemini natively, so Google runs the kitchen. Nothing at Acme is self-deployed today; the open-weight section below explains what would change if a step moved to the third kind.

A common mistake is to read “open-weight” as “free” or “less work”. The licence may cost nothing, but in the recipe case you are now paying for the oven, the cook and the washing up.

the model landscape as of October 2026: three proprietary families and five open-weight families, and how each one can be used on Google Cloud: natively, as a managed partner or open model API, or by deploying the weights yourself

The Agent Platform models overview sums up the Google Cloud side: Google models (Gemini, Gemma and others), partner models “such as Anthropic Claude, Grok, and Mistral AI”, and open models “like DeepSeek, Llama, and Qwen”. Partner and open models carry their own terms: each page says the model is “not a Google product” and falls under the “Separate Offerings” terms. Your legal team will want to know that before you pick one.

Here is what I would want to know about each family before using it in an agent.

Gemini (Google)

The current line-up, from Google’s Gemini models page, is Gemini 4 Argon, the flagship reasoning model, released on 13 October 2026; Gemini 3.8 Flash, which Google describes as its most intelligent Flash model, “engineered for long-horizon software engineering, autonomous agents, and complex enterprise workflows”; and Gemini 3.5 Flash-Lite, the fastest and cheapest. On Agent Platform, the Gemini 3.8 Flash page lists a context window of 1,048,576 tokens and a maximum output of 65,536 tokens. The Gemini 4 Argon page lists the same context window, with output up to 1,048,576 tokens.

Things to know for production:

Thinking is on by default. The thinking guide lists a default of medium for both Argon and 3.8 Flash, with levels from low to high, and says that when thinking is on, “response pricing is the sum of output tokens and thinking tokens”. A small routing step rarely needs medium thinking, so set the level per step and watch the thinking token count in your traces.
Launch prices can double. On the Gemini Developer API pricing page, Gemini 3.8 Flash costs USD 0.75 per million input tokens and USD 3.75 per million output tokens through 31 December 2026, rising to USD 1.50 and USD 7.50 from 1 January 2027. Gemini 4 Argon is USD 2.00 and USD 10.00 through 30 November 2026, then USD 4.00 and USD 20.00. Agent Platform has its own pricing page, so check the one for the API you actually call, and put the date in your cost model.
Features differ by model, not just by size. Gemini 4 Argon supports implicit caching only (explicit caching is not supported), and the Agent Platform page lists RAG Engine and tuning as not supported for it. Gemini 3.5 Flash, in contrast, supports supervised fine-tuning. If you rely on a feature, check the model’s page, not the family’s.
The same ID can behave differently on two surfaces. On Agent Platform, gemini-3.5-flash (the model Acme’s extract step uses) is a stable model available until at least 19 May 2027, according to the model versions page. On the Gemini Developer API, the deprecations page says the same ID is no longer selectable and “automatically routes API requests to gemini-3.6-flash”. Same string, different model. This is the best argument I know for running your evals against the exact endpoint you deploy to.

Claude (Anthropic)

Anthropic’s models overview lists Claude Fable 5.1 for “demanding reasoning and long-horizon agentic work”, Claude Opus 5.5, which Anthropic suggests as the starting point for most workloads, Claude Sonnet 5.5 and Claude Haiku 5.5, the fastest, aimed at “classification, extraction, and routing”. All four have a 1M token context window and a 128K token maximum output, and Anthropic gives each one a retirement date of not sooner than September or October 2027. Prices on that page run from USD 10 and USD 50 per million input and output tokens for Fable 5.1 down to “from USD 0.10” and “from USD 0.50” for Haiku 5.5.

Things to know for production:

It runs on Google Cloud as a partner model. The Claude on Google Cloud page describes “fully managed and serverless models as APIs” on Agent Platform, paid as you go or through Provisioned Throughput. There is also an Anthropic-managed option on Google Cloud. They are different offerings with different terms, so be clear which one you mean.
Thinking is adaptive. For Fable 5.1 and Opus 5.5 the overview lists thinking as “Adaptive (always on)”, with a default effort of high and medium respectively. Budget for it.
ADK supports it directly. The ADK Claude page says that in Python on Agent Platform you can “pass a Claude model string directly; ADK’s registry routes it to the Claude wrapper”. Outside Agent Platform, ADK uses its LiteLLM connector with an Anthropic API key.

GPT (OpenAI)

OpenAI’s models page recommends GPT-6 Astra as its flagship, GPT-6.1 Sol to “balance intelligence and cost”, GPT-6.1 Terra for visual reasoning and computer use, and GPT-6 Luna for “cost-sensitive, high-volume workloads”. The GPT-6 Astra page lists a 1,050,000 token context window but a maximum input of 922,000 tokens and a maximum output of 128,000, at USD 10 per million input tokens and USD 50 per million output tokens. GPT-6 Luna is USD 0.10 and USD 0.50.

Things to know for production:

The context window is not the input limit. Astra’s window and its maximum input differ by more than 100,000 tokens. Size your prompts against the input limit.
Long prompts cost more per token. Both pages say prompts “with more than 272K input tokens are priced at 2x input and cache rates and 1.5x output for the full request”. A long history can move a whole call into the higher price band.
GPT is not in Agent Platform’s partner list. As of October 2026 the Agent Platform overview names Claude, Grok and Mistral as partner models, not GPT, so you call OpenAI’s own API (ADK can do this through LiteLLM). OpenAI’s open-weight gpt-oss-120b is different: it is a managed API on Agent Platform, released under Apache 2.0, with a 131,072 token context and General Availability, according to its model page.

Gemma (Google)

Gemma 4 is Google’s open-weight family, “built from the same research and technology used to create the Gemini models”, with text, audio and image input, support for over 140 languages, and a context window of 128K and up to 256K tokens. Gemma 4 is released under the Apache 2.0 licence.

Things to know for production: there is a managed API for Gemma 4 26B A4B IT with function calling and structured output, but its launch stage is Experimental, which is not something I would put a customer-facing agent on. For production you would self-deploy it. The ADK vLLM page uses a Gemma 4 model on a vLLM endpoint as its example.

Llama (Meta)

Meta’s developer site, which llama.com now redirects to, leads with its newer Muse models, including Muse Spark Open, which Meta calls “the most capable US open-weight model to date” (Meta for developers). On Google Cloud, the managed Llama 3.3 70B endpoint was deprecated on 21 July 2026 and retired on 21 October 2026, according to the open model deprecations page, which lists a self-deploy from Model Garden as the alternative. Llama 3.3 uses Meta’s own Llama 3.3 Community License, not a standard open-source licence, so read it before you ship.

Mistral (Mistral AI)

Mistral’s model overview lists a licence next to each model: for example Mistral Small 4 and the Ministral 3 models are Apache 2.0, while Mistral Medium 3.5 has a “Modified MIT” licence. On Google Cloud, the Mistral AI models page offers Mistral Medium 3, Mistral Small 3.1, Mistral OCR and Codestral 2 as managed partner APIs.

Things to know for production: the version on your cloud can lag the vendor’s newest release. Mistral lists Large 4 and Small 4 as featured models, while Agent Platform offers Medium 3 and Small 3.1. That is normal for partner catalogues, and it is why “which model?” really means “which model, on which platform?”.

Qwen (Alibaba)

The Qwen team’s GitHub organisation describes Qwen3.8 as “the large language model series developed by Qwen team”. On Google Cloud, the Qwen models page carries a caution that “all Qwen Model as a Service (MaaS) endpoints are either deprecated or retired”, and points you to a self-deploy from Model Garden. If you want Qwen on Google Cloud today, you are running it yourself.

DeepSeek

On Google Cloud, the DeepSeek models page offers DeepSeek-V4.1-Flash as a managed API, “a multimodal Mixture-of-Experts (MoE) model … built for agentic and coding tasks” with a context window of up to 1M tokens. The older V3.1, V3.2, R1 and OCR managed endpoints were retired on 21 October 2026. DeepSeek’s own API documentation shows a different kind of change: the model name deepseek-flash “is now served by DeepSeek-V4.1-Flash-1017”, and requests to two older names are now served by that model too. A name you did not change now points at a different model.

My view: on Google Cloud in October 2026, the practical shortlist for a production agent is Gemini first, with Claude as the strong second choice through the partner API, and open models when you have a specific reason (more on that below). The retirement wave of 21 October shows why: sixteen managed open model endpoints were deprecated in July and retired three months later, while stable Gemini models come with at least twelve months.

What matters for an agent, beyond leaderboards?

Leaderboards measure how clever a model is; an agent in production fails on how reliable it is. The trap is choosing the model at the top of a public ranking and assuming the rest follows.

Picture hiring a delivery driver. The candidate with the fastest lap time on a race track is impressive, but the job needs other things: reads every address correctly, fills in the delivery note the same way each time, turns up on time at rush hour, costs what you budgeted, holds a licence for the country you deliver in, and will still be working for you next year. A slightly slower driver who does all of that is the better hire.

Map it across: the lap time is the benchmark score; reading addresses is tool calling; the delivery note is structured output; rush hour is latency and quota; the budget is price; the licence is region and terms; next year is the lifecycle.

How does this apply to an agent like Acme’s? Public benchmarks measure general ability. An agent fails for much more boring reasons: a malformed tool call, a field missing from the JSON, a slow first token, a quota hit at 9am on Monday. This is the checklist I use, and where each answer comes from.

what to check for each candidate model before production: tool calling, structured output, limits, thinking, latency, price, quota and consumption option, region, lifecycle and terms, and where each answer comes from: the vendor page or your own eval run

Tool calling. This is the property that matters most and that no model page can tell you. The Gemini function calling guide lists four modes: auto (the model decides), any (it must call a function), none (it may not) and validated, where the “model ensures function schema adherence”. It also supports calling several functions in one turn. The same guide recommends keeping the active set “to 10-20 tools maximum”, which is one more reason for Part 2’s narrow tools.

A simple way to see why this matters: a shop cashier who is supposed to scan every item through the till, but who knows most prices by heart. Most days the remembered price is right. The day it is wrong, nothing in the receipt tells you, because no scan happened. A shop that wants exact totals makes scanning compulsory, and a till that rejects a barcode it cannot read stops a bad entry before it reaches the receipt. Scanning is the tool call, the price from memory is the model answering from its training, compulsory scanning is the any mode, and the till rejecting a bad barcode is schema adherence.

How does this apply to Acme? Here is why the mode matters on the marketing specialist, with an illustrative failure. Maria asks for Verde’s CTR in the UK last month. In auto mode the model may answer from memory with a plausible percentage and call nothing, and the only thing that saves you is the writer’s number check finding a figure that came from no tool. Forcing a call (any) for that step guarantees that query_marketing_measures runs on campaign_performance first. Schema adherence catches a different mistake: a call that sends measures as one string where the tool expects a list never reaches the tool. A model comparison should count both: answers without a tool call, and calls with invalid arguments.

Structured output. Every family above offers some form of schema-constrained output. What differs is how reliably it holds under real prompts. Acme’s writer already checks every number against the tool results (Part 5); a structured output mode makes that check easier, but it does not replace it.

Limits. Three numbers, not one: the context window, the maximum input, and the maximum output. GPT-6 Astra shows why the second one matters. A long conversation plus tool results can reach a limit you never saw in testing; Part 1.5 and Part 8 covered how compaction keeps history small.

Two sums with GPT-6 Astra’s published figures show both traps:

1. A 950,000-token history:
   window 1,050,000   -> fits
   max input 922,000  -> does not fit, so the call fails
2. A 300,000-token prompt (over the 272K line), at USD 10 per million input tokens:
   normal rate:   300,000 x 10 / 1,000,000     = USD 3.00
   long-prompt:   300,000 x 10 x 2 / 1,000,000 = USD 6.00 input, and output billed at 1.5x

The first fails loudly; the second quietly doubles the input bill for the whole request. Both are reasons to let compaction, which is code, keep history well under the input limit.

Thinking. Check the default, the levels you can set, and how thinking tokens are billed. A model that thinks by default can be several times more expensive per call than its price per output token suggests.

Think of a solicitor who bills by the hour. You receive a one-page letter, but the invoice also covers the three hours of research and drafts you never see. Judge the cost by the length of the letter and you will be surprised by the bill. The letter is the visible answer, the hidden research is the thinking tokens, and the hourly rate is the price per output token, because thinking tokens are billed as output.

How does this apply to Acme? The cost() function from the exercise at the end of this post shows it. Take the writer step’s illustrative token counts (6,000 in, of which 4,000 cached, and 400 out) at Gemini 3.8 Flash’s launch prices, and change only the thinking tokens:

thinking = 0       -> USD 0.00330 per question
thinking = 1,200   -> USD 0.00780 per question   (2.4x, from thinking alone)

The 400 visible output tokens are unchanged; 1,200 thinking tokens billed as output cost more than everything else in the call put together. That is why the thinking level is set per step.

Latency. Vendors rarely publish numbers you can compare. Anthropic’s overview gives a relative scale (from “Slower” for Fable 5.1 to “Fastest” for Haiku 5.5), but for real numbers you have to measure: time to first token and total time, at p50 and p95, from the region you deploy in.

Price. Input, output, cached input, and any special rules: launch prices that end on a date, long prompt surcharges, and storage charges for explicit caches. Part 11 showed how to turn these into a cost per question, which is the only price that matters.

Quota and consumption option. The same model can be bought on shared, priority, discounted or reserved capacity, and for a busy agent that choice decides how often it gets a 429. A train ticket is a fair picture: an off-peak ticket is cheaper but you travel when the railway lets you; an anytime ticket gets you on any train, but at rush hour the carriage can be full; a first-class ticket costs more for a better chance of a seat; a season ticket with a reserved seat is paid in advance for a fixed term. Here is the real thing.

On Agent Platform, Standard PayGo runs on shared capacity with usage tiers that rise with your spend over a rolling 30 days, and “doesn’t reserve dedicated capacity” (Standard PayGo). Priority PayGo charges a higher rate per token for “more consistent performance” and lists “agentic workflows and cross-agent interactions” as a use case (Priority PayGo). Flex PayGo is 50% cheaper for latency-tolerant work, is in Preview, and is not supported on newer models such as Gemini 4 Argon (Flex PayGo). Provisioned Throughput is a fixed-term subscription that reserves capacity (Provisioned Throughput). For a user-facing agent with real traffic, the choice between these matters as much as the choice of model. Mapped to the tickets: Flex is off-peak, Standard is anytime, Priority is first class, and Provisioned Throughput is the season ticket with a reserved seat.

How does this apply to Acme? Suppose 9am on Monday brings a spike, and the writer runs on Standard PayGo. Some writer calls come back with HTTP 429. Part 11.5’s fallback ladder retries, then serves those answers from the cheaper fallback model, so users still get answers. Count in your logs what share of answers came from the fallback that morning. If that share is high every Monday, the fix is not in the code: move the writer to Priority PayGo, pay the higher per-token rate, and expect fewer 429s at the peak.

A short guide, from least commitment to most:

Work that can wait (overnight eval runs, batch)  -> Flex PayGo, if the model supports it (Preview)
User-facing, fallback share low at peaks         -> Standard PayGo, with Part 11.5's fallback
User-facing, fallback share high every peak      -> Priority PayGo for that step
Steady, predictable volume you can forecast      -> Provisioned Throughput, a fixed-term commitment

Region and data residency. Gemini 3.8 Flash is listed for the global endpoint and the us and eu multi-regions; Gemini 4 Argon adds europe-west2 and asia-southeast1. If your data must stay in one country, this row can rule a model out before any test. For example, suppose Acme’s UK data had to be processed in europe-west2 (London). Of the two, only Gemini 4 Argon is listed there, so Gemini 3.8 Flash is out before a single eval runs, and the question becomes whether Argon’s quality is worth its higher price for that traffic.

Lifecycle and terms. When was the model released, what stage is it in (Experimental, Preview, GA), when can it retire, and is it a Google product or a “Separate Offering”? The next two sections cover lifecycle in detail.

How do I compare models for my agent?

With your own questions, not someone else’s, and changing only the model. A comparison where the prompt, the tools or the test set also changed tells you nothing about the model.

It is the fair test from a school science lesson. To find out which of two fertilisers grows better tomatoes, you give both groups of plants the same soil, the same water and the same light, change only the fertiliser, grow several plants per group because one plant can be a fluke, and measure more than height: a tall plant with no fruit is not the winner. The fertiliser is the model; soil, water and light are the prompts, tools and golden set; several plants are repeated runs; and height versus fruit is pass rate versus how the failures look.

How does this apply to Acme? Part 10 built a golden set and a held-out set for Acme, with code metrics first and calibrated judges second. Comparing models is the same machinery with one thing changed. This is the procedure I would follow:

Shortlist two or three candidates that pass the checklist rows you cannot negotiate (region, terms, lifecycle, quota).
Change only the model. Same prompts, same tools, same plugins, same golden set. If a candidate needs a different prompt to do well, that is a cost you should count.
Run the full eval set, several times, because models are not deterministic. For Acme that means routing accuracy, tool argument validity, number checks that pass, and the judge scores from Part 10.
Measure what the eval does not: p50 and p95 latency, tokens per question (including thinking tokens), and cost per resolved question using the formula from Part 11.
Look at the failures, not only the scores. A model that fails safely (“I can’t answer that”) is better than one that fails confidently with a wrong number, even at the same pass rate.
Decide per step, which is the next section.

Here is what the output of steps 3 to 6 could look like for Acme’s specialist step, comparing the current gemini-3.5-flash with gemini-3.8-flash. Every number is illustrative, not a result: a toy 20-case golden set, run three times, averaged.

specialist step                  gemini-3.5-flash     gemini-3.8-flash
tool argument validity           96%                  99%
number checks passed             17.7 / 20            18.7 / 20
p95 latency                      baseline             +20%
tokens per question (incl. thinking)  baseline        +60%
cost per resolved question       baseline             +45%

(Routing accuracy is missing on purpose: at Acme the route is decided in code, so it only changes when the extract step’s model changes.)

Read it per step. For the specialists, better tool arguments and one more passing number check might be worth 45% more per resolved question. For the extract step, where both models would score the same, the cheaper one wins. The table is produced by code; the trade-off is a person’s decision.

Step 5 needs its own example, because the pass rate hides it. Take golden case q3_delegation (“What did the stockouts behind the Acme Pure Refill Launch cost us?”, expected 51,331) and two candidate answers that both fail:

Candidate A: "I couldn't get the supply figures for that launch."
             -> fails: 51,331 missing. Safe: the user knows to ask again or ask someone.
Candidate B: "The stockouts cost EUR 15,331 in penalties."
             -> fails: 15,331 appears in no tool result, so the number check replaces the answer.
                Without that check, Maria would have read a wrong figure stated with confidence.

Same score, one failure each, but B is the worse model for this agent. Count confident wrong numbers separately from refusals when you compare.

If the judge is itself a model, keep it fixed while you compare candidates. Part 12’s upgrade checklist makes the same point: a judge calibrated on one model has not been measured on another.

At Acme, this is how a cross-vendor comparison would work: the same method, applied to the evals the router, the specialists and the judges already have on Gemini. One short note: the series shows no Acme results for Claude, GPT or the open models, only the method.

Should every step use the same model?

No, and Acme already doesn’t. Each step has a different job, so each one can have a different model. Match the model to the job, not to the most important job in the system.

A small hospital works the same way. The receptionist takes your details and writes the reason for the visit in one line. A printed rota on the wall says which clinic sees which complaint; nobody needs to think about it. A nurse in that clinic runs the tests. A consultant reads the results and writes the letter to your GP, and if the consultant is off sick, a registrar covers. Nobody pays a consultant to book appointments, and nobody asks the receptionist to diagnose.

The receptionist is the extract step, the rota is the route step (code, no model), the nurses are the specialists, the consultant is the writer, and the registrar is the fallback model. The people who audit the hospital’s letters afterwards are the judges.

How does this apply to Acme? Here is the same layout, one step at a time.

a model per step at Acme: extract runs on a fast Flash model, the route step uses no model at all, the specialists use the Flash model, the writer uses a stronger model with a cheaper fallback, and the judges in the eval gate use a model chosen and calibrated separately

Extract rewrites the question into one clear line. It runs on MODEL, a Flash model chosen for speed and price (Part 5).
Route uses no model at all. Plain code looks up who owns each metric. It is the cheapest and most reliable step in the system.
The specialists call narrow tools and report findings. They also run on a Flash model.
The writer composes the final answer. It runs on WRITER_MODEL, a stronger model, and its output goes through the number check. Part 11.5 gave it a cheaper fallback model for when the primary one fails.
The judges in the eval gate are a separate choice again, calibrated against people (Part 10).

In ADK Python the simplest way to do this is the one we already use: each Agent gets its own model, read from a setting. Other families plug in the same way:

# Illustrative: one model per step, each from a setting (Part 11).
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm   # needs google-adk[extensions]

extract = Agent(name="extract", model=os.getenv("MODEL", "gemini-3.5-flash"), ...)
writer = Agent(name="writer", model=os.getenv("WRITER_MODEL", "gemini-3.1-pro-preview"), ...)

# A Claude model on Agent Platform: pass its model string (ADK's registry picks the wrapper).
# A model ADK does not wrap natively, such as one on your own vLLM server: use LiteLlm.
local = Agent(name="local", model=LiteLlm(model="hosted_vllm/google/gemma-4-E4B-it",
                                          api_base="https://your-vllm-endpoint/v1"), ...)

ADK also has a built-in way to choose a model per request. The model routing page describes RoutedLlm, which can do “model fallback on error, A/B testing between models, and auto-routing by input complexity”. As of October 2026 it is marked Experimental and documented for ADK TypeScript, so in Python I would keep the fixed model per step, plus the fallback callback from Part 11.5.

My view: resist routing every request to a different model to save a few cents. Each extra model is one more thing to evaluate, one more quota, one more retirement date and one more set of terms. Two models (a fast one and a strong one) plus a fallback is enough for most agents.

The litmus test I apply to each step, cheapest answer first:

1. Can plain code do it?                       -> no model (Acme's route step)
2. Does the fast, cheap model pass this step's
   evals at an acceptable latency?             -> the Flash model (extract, specialists)
3. Only then: the stronger model for this step -> the writer, with the number check (Part 5)
4. Whatever you chose in 3                     -> give it a cheaper fallback (Part 11.5)

How do versions, aliases and deprecation work?

A model is a dependency with an expiry date. A pinned version always means the same model; an alias can start meaning a different one without you changing a line. The rules differ by platform, and they are worth knowing exactly.

A library makes the difference easy to picture. If your reading list gives a book’s exact ISBN, you get the same edition every time you borrow it. If it says “whatever is on the bestseller shelf”, you get a different book whenever the shelf changes, even though your list did not. And the library can withdraw an old edition: a good one posts a notice with a date, and after that date the book is simply not on the shelf. The ISBN is a pinned version, the bestseller shelf is an alias such as latest, and the notice is a deprecation followed by a retirement. Acme’s version of the bestseller shelf, and what it would do to the evals, comes after the platform rules below.

the lifecycle of a model on Agent Platform: a stable Gemini model is available for at least twelve months and its retirement date never moves earlier; short-term models get at least 45 days of notice; an alias such as latest can point to a new model; managed open model endpoints can be deprecated and retired within three months

Gemini on Agent Platform. The model versions page splits models into two groups. Most stable models are “available for at least 12 months after release”, and retirement dates “won’t be moved to an earlier date than what is listed”. Short-term models “remain active until a replacement model is released and a retirement date is announced”, with “at least 45 days to migrate” once a date is posted. For example, gemini-3.6-flash is listed as a short-term model retiring on 19 November 2026, with gemini-3.8-flash as the replacement.

Gemini on the Gemini Developer API. The models page describes four kinds of name. Stable names “usually don’t change” and “most production apps should use a specific stable model”. Preview names “will be deprecated with at least 2 weeks notice”. The latest alias “will get hot-swapped with every new release”, with two weeks’ email notice for breaking changes. Experimental names are “not suitable for production use”.

How does this apply to Acme? A note on our own code: a few short examples in Parts 2 and 3 use gemini-flash-latest, an alias, which is fine for a snippet you run once. The production code pins exact IDs through MODEL and WRITER_MODEL. Never let an example’s default reach production.

Here is what goes wrong if one does. Suppose a Part 2 snippet with model="gemini-flash-latest" had been deployed as is. Google hot-swaps the alias to a new release; nobody on the team deploys anything. The next morning q3_delegation comes back worded differently, and a borderline question that used to be answered is now refused. The eval diff against yesterday’s run is the only place it shows up. With MODEL=gemini-3.5-flash, the same morning produces yesterday’s results, and the new model arrives only when someone changes the setting and runs the gate.

Which kind of name to use where, from the rules above:

A snippet you run once                    -> an alias such as latest is fine
A preview model you need now              -> pin its exact name, run the gate, and watch the
                                             deprecation notices (as little as 2 weeks)
Anything users depend on                  -> a pinned, stable ID, read from a setting
Experimental names                        -> never in production

Partner and open models. Anthropic gives each Claude model a “not sooner than” retirement date, and Google keeps a partner model deprecations page. For managed open models, the open model deprecations page says Preview models are available “for at least 45 days” and that after retirement “API requests calling a retired model ID will fail”. The 21 October 2026 retirements show how quickly that can happen.

How to upgrade. Part 12 has the checklist: pin exact IDs, run both eval sets against the new ID, compare scores, cost and latency, recalibrate any judge whose model changed, roll out again from shadow, and keep the previous ID one setting away. Google’s versions page adds the emergency version: if a retirement catches you out, point to the recommended replacement, test the critical paths, and deploy as normal.

When do open-weight models make sense in production?

Open models are not a cheaper Gemini. They are a different deal: more control, more work. The misconception is that downloading free weights makes the model free to run.

Compare taking taxis with owning a delivery van. A taxi costs nothing while you are not using it and you pay per trip; someone else maintains it. A van you own costs the same lease, insurance and servicing every day, including the days it sits in the car park, but it goes where you want, carries what you want and nobody else sees the cargo. The van only beats taxis if you keep it on the road. The taxi is a managed model API paid per token; the van is a self-deployed open model on hardware you pay for by the hour; the cargo nobody sees is your data staying in your project.

How does this apply to Acme? Every Acme step runs on a managed model today, so this is the van question for a future step. Open models make sense when one of these is true:

Data control. The weights run inside your own project and VPC network, which the self-deployed models overview describes as “giving you full control over the deployment environment”.
Cost at very high volume for a narrow task, where a small model on hardware you keep busy can beat a per-token price. This only works if you really keep it busy. With placeholders instead of real prices: a node costs H dollars an hour whether or not it is busy, and serves N questions an hour at full load. At full load a question costs H / N. At 10% load it costs H / (0.1 x N), which is 10 x H / N, ten times as much. Compare that with the managed figure from the script at the end of this post (USD 0.00780 for the writer step at launch prices): self-hosting only wins when your real, average load keeps H / (load x N) below it.
Offline or edge use, where there is no API to call.
Customisation that the managed models do not offer, which Part 14 covers.

And this is what you take on:

who runs what: with a managed model API the provider runs the serving stack, scaling and accelerators, and you own your agent, cost, the model version you pin and the licence terms; a Model Garden self-deploy adds the serving stack and scaling; on GKE with vLLM you own everything, down to the accelerators and patching

There are two ways to self-deploy on Google Cloud. A Model Garden deployment puts the model on an Agent Platform endpoint in your project. GKE with vLLM means running an open-source inference server on your own Kubernetes cluster with GPUs or TPUs; Google has a tutorial for serving Gemma with vLLM on GKE. Either way, ADK connects through LiteLLM, and the ADK vLLM page warns that the server must have tool calling switched on (for vLLM, flags such as --enable-auto-tool-choice and a matching --tool-call-parser). Without it, your agent’s tools will not work.

This is what that looks like in a trace (illustrative):

Server started without tool calling
1. Maria: "What was Verde's CTR in the UK last month?"
2. Marketing specialist: no execute_tool span. The model replies in prose with a CTR figure.
3. Writer's number check: the figure appears in no tool result, so the answer is replaced.

Server restarted with --enable-auto-tool-choice and the matching --tool-call-parser
2. Marketing specialist: execute_tool query_marketing_measures on campaign_performance.
3. The answer uses the figure the tool returned, and the number check passes.

Nothing errors in the first run, which is what makes it dangerous; only the missing tool span and the replaced answer give it away.

My view: for an enterprise agent on Google Cloud, start with a managed model and move a step to an open model only when an eval and a cost report say it is worth the extra operations. When you do, the licence is part of the decision: Gemma 4 and gpt-oss are Apache 2.0, Llama 3.3 has its own community licence, and Mistral lists a licence per model.

The litmus test, in order, cheapest option first:

1. Does a managed model pass the evals, the region and the terms?  -> stay managed, stop here
2. Is there a hard need: data must stay in your network, offline
   or edge use, or a customisation no managed model offers?       -> consider self-deploy
3. Or is the task narrow and high-volume, and will real, average
   load keep H / (load x N) below the managed cost per question?  -> self-deploy can pay
4. Only then: Model Garden first, GKE with vLLM if you need more
   control, with tool calling switched on and the evals re-run

What can go wrong

A silent alias move. A name like gemini-flash-latest or deepseek-flash starts pointing at a new model, and answers change with no deploy on your side. Pin exact IDs.
The same ID, a different model. gemini-3.5-flash routes to gemini-3.6-flash on the Gemini Developer API while it is still itself on Agent Platform. Test against the endpoint you deploy to.
A retirement you missed. Requests to a retired ID fail. Put every retirement date in a calendar and in a test that fails a month before.
A launch price that ends. Gemini 3.8 Flash and Gemini 4 Argon both have prices that double on a published date. Build the date into your cost model.
Thinking tokens you did not budget for. Default thinking levels can multiply the output tokens you pay for. Watch them in traces and set the level per step.
A context limit reached in production. History and tool results grow. Compaction (Part 8) and the input limit, not the advertised window, are what count.
Quota contention. Standard PayGo shares capacity. For a business-critical agent, plan Priority PayGo or Provisioned Throughput, and keep the fallback model from Part 11.5.
An open model server without tool calling. The agent runs, but never calls a tool.

Try it: what does a question really cost on two models?

This script needs no API key. It compares the cost per question of two models for a step, using the token counts you measured in your traces (Part 11) and prices you copy from the pricing pages. The example uses the published Gemini 3.8 Flash prices before and after 1 January 2027, to show what a launch price ending does to a budget.

# cost_per_question.py: compare price sheets for one agent step. No API calls.
def cost(tokens_in, tokens_out, thinking, price_in, price_out, cached_in=0, price_cached=0.0):
    """Prices are USD per million tokens. Thinking tokens are billed as output."""
    fresh_in = tokens_in - cached_in
    return (fresh_in * price_in + cached_in * price_cached
            + (tokens_out + thinking) * price_out) / 1_000_000

# Token counts for Acme's writer step, taken from your own traces (illustrative numbers).
step = dict(tokens_in=6_000, tokens_out=400, thinking=1_200, cached_in=4_000)

sheets = {
    "gemini-3.8-flash, to 31 Dec 2026": dict(price_in=0.75, price_out=3.75, price_cached=0.075),
    "gemini-3.8-flash, from 1 Jan 2027": dict(price_in=1.50, price_out=7.50, price_cached=0.15),
}
for name, p in sheets.items():
    per_q = cost(**step, **p)
    print(f"{name:36} USD {per_q:.5f} per question, USD {per_q * 100_000:,.0f} per 100k questions")

Running it prints:

gemini-3.8-flash, to 31 Dec 2026     USD 0.00780 per question, USD 780 per 100k questions
gemini-3.8-flash, from 1 Jan 2027    USD 0.01560 per question, USD 1,560 per 100k questions

Now add a row for each candidate you are considering, with its own measured token counts. Thinking tokens usually make a bigger difference than the price per token.

What code did we write in this part?

Every part ends with the same map of Acme’s build: which files exist so far, and the code this part added, explained line by line. The map grows with each part.

This part adds no code to Acme’s build. It explains how to choose, compare and change models; the one-model-per-step snippet is marked illustrative and restates the MODEL and WRITER_MODEL settings Acme already has, and cost_per_question.py is a standalone exercise.

Where does it live?

router/                                             the router agent: extract, route, write
  app/agent.py                                      P5,8        workflow; extract reads memory; App with compaction and resumability
  app/app_utils/services.py                         P8          picks the session, artifact and memory services
  app/fast_api_app.py                               P8          Runner with session, artifact and memory services
  app/remote.py                                     P6,7        A2A client over mTLS; metadata carries Sam's token
  app/writer.py                                     P5,8        writer reads memories and saves the conversation to Memory Bank
  scripts/create_memory_bank.py                     P8          creates the Memory Bank and its topics, once
  tests/eval/acb_metrics.py                         P10         code metrics: route_correct, grounded, golden_numbers
  tests/eval/build_datasets.py                      P10         splits source cases into golden and held-out, seed 42
  tests/eval/calibrate.py                           P10         compares judge scores with human labels (kappa)
  tests/eval/datasets/golden_marketing.json         P10         golden cases with expected route, target and numbers
  tests/eval/eval_config.yaml                       P10         eval metrics, including the faithfulness judge
  tests/eval/gate.py                                P10         thresholds per metric; exits 1 when one fails
  tests/unit/test_router.py                         P10         unit tests; routing checked with one row per question type
marketing/                                          the marketing specialist, on BigQuery
  app/agent.py                                      P1,2,3,4    marketing agent with its skill in a SkillToolset
  app/app_utils/a2a.py                              P6          A2A endpoint and agent card (1.0 and 0.3) for marketing
  app/skills/marketing-kpis/SKILL.md                P4          marketing know-how: which tool, definitions, story checks
  app/skills/marketing-kpis/references/tables.md    P4          tables, grain, measures and the join rule, read on demand
  app/tools.py                                      P1,2        three narrow tools: measures, unserved demand, SQL fallback
  tests/unit/test_marketing.py                      P2          unit tests for the tools, with a fake context and database
supply/                                             the supply specialist, on Databricks
  app/agent.py                                      P3,7,8,9    supply agent and App with the confirmed replan tool
  app/app_utils/a2a.py                              P6          A2A endpoint and agent card (1.0 and 0.3) for supply
  app/skills/supply-catalog/SKILL.md                P4          supply know-how, given only to the supply agent
  app/tools.py                                      P3,7,9      supply tools; the replan tool asks a human first
shared/                                             code all three services share
  adk_common.py                                     P3,6,9,11   plugins and redacted JSON logs, now with the usage ledger
  auth_manager.py                                   P7          auth manager callback: token or consent (off by default)
  engines.py                                        P3,7        run_databricks connects as the signed-in user
  factcheck.py                                      P5          ungrounded_numbers: finds figures not in the findings
  registry.py                                       P2          reads semantic.yaml: measure names, computable measures
  routing.py                                        P1          classify: picks the specialist in code, no model
  safety.py                                         P9          per-user token bucket and injection pattern scan
  semantic.yaml                                     P1,2        registry of measures, owners and published formulas
  skill_source.py                                   P4          loads a skill from disk or a pinned registry revision
  sqlguard.py                                       P3          guard_sql read-only allow-list; redact for log lines
  tests/test_cost_report.py                         P11         unit tests for the cost maths
ops/                                                build, deploy and evaluation tooling
  .cloudbuild/eval_gate.yaml                        P10         after staging: eval run, then gate.py
  .cloudbuild/pr_checks.yaml                        P10,11      offline PR checks, cheapest first, ending with the mutants
  env.sh.example                                    P11         settings for every engine: ids, models, plugin switches, OTEL; no secrets
  infra/gateway/setup_gateway.sh                    P12         egress gateway, endpoint registry and IAP rules, in dry-run
  tools/cost_report.py                              P11         prices the ledger and reports cost per resolved question
  tools/mutants/acb_mutate.py                       P10         pytest plugin that injects each mutant's fault
  tools/pricing.json                                P11         list prices per model, kept out of the code
  tools/run_mutants.py                              P10         runs the offline suites once per mutant

+ added in this part   ~ changed in this part   Pn = the parts that touched the file

The series has not compared Claude, GPT or open models on Acme’s eval sets; the comparison method is described, not run.

What did we learn?

The right model for an agent is the one that calls tools reliably, fits your latency, cost, quota, region and lifecycle needs, and passes your own evals. Leaderboards are a starting point at best.
On Google Cloud in October 2026, Gemini runs natively, Claude and Mistral run as managed partner APIs, GPT runs on OpenAI’s own API, and open models run as managed APIs or self-deployed. Several managed open model endpoints were retired on 21 October 2026.
Compare candidates by changing only the model and running the eval sets from Part 10, then cost per question from Part 11.
Different steps can and should use different models: no model for routing, a fast model for small steps, a stronger one for the answer, and a fallback.
Pin exact model IDs, know each model’s retirement rules, and treat a model upgrade as a release.
Open-weight models give you control and take more work. Choose them for a reason an eval and a cost report can confirm.

What comes next?

Before knowledge, one question everyone asks: what does the whole system cost? Production-ready agents, Part 13.5: What does an agent cost to own? prices every line of the bill and every architecture choice. Then Part 14: Knowledge: RAG, long context, fine-tuning and the infrastructure behind them looks at where an agent’s knowledge should live: in the weights, in the context, or behind a tool. We ask whether RAG is still relevant now that context windows reach a million tokens, when tuning a model is really worth it, and what infrastructure each option needs.

Sources

Opinions are my own. Every fact in this post comes from the public documentation linked above.